Contributors may familiarize themselves with Celery itself by going through the First Steps with Celery tutorial.
GWCelery operates on a fork-and-merge development model (see GitLab basics for an introduction).
To contribute to GWCelery development, follow these steps:
Make your changes on a branch.
Open a merge request.
Note that GWCelery uses fast-forward merges.
Where new code should go¶
New code will generally consist of adding Celery tasks.
Tasks are organized by functionality into submodules of
If your new task does not match with one of the existing submodules, please
create a new submodule.
Guidelines for tasks¶
Tasks should be short. When deciding where a new task should go, start from the following loose rules of thumb:
If it’s less than a screenful of code, and related to functionality in an existing module, then put the code in a new task in that module.
If it’s up to a few screenfuls of code, or not related to functionality in an existing module, then try to break it into a few smaller functions or tasks and put it in a new module.
If it’s more than a few screenfuls of code, or adds many additional dependencies, then it should go in a separate package.
Tasks should avoid saving files to disk. Output should be placed directly in GraceDB. Temporary files that are written in
/tmpare OK but should be cleaned up promptly.
There are two extra steps involved in making changes to the dependencies:
The Sphinx-generated documentation (that is to say, this manual) is generally built without most of the dependencies installed. Whenever you add a new package to requirements.txt, you should also add any modules that are imported from that package to the
autodoc_mock_importslist in the Sphinx configuration file, doc/conf.py.
Unit tests and code coverage measurement are run automatically for every branch and for every merge request. New code contributions must have 100% test coverage. Modifications to existing code must not decrease test coverage. To run the unit tests and measure code coverage, run the following commands in the top directory of your local source checkout:
$ poetry install --extras=test $ poetry shell $ pytest --cov --cov-report html
This will save a coverage report that you can view in a web browser as
Most of GWCelery’s unit tests use eager mode, which
causes all tasks to execute immediately and synchronously, even if they are
delay(). This simplifies writing unit tests,
but sacrifices realism: it may mask concurrency bugs that may only occur
when the tasks are executed asynchronously.
It is preferable to write unit tests that use a live worker so that they are subject to realistic, asynchronous task execution. To opt in to using a live worker, simply decorate your test with the live_worker marker, like this:
@pytest.mark.live_worker def test_some_task(): async_result = some_task.delay() result = async_result.get() assert result == 'foobar' # etc.
$ pip install flake8 pep8-naming $ flake8 --show-source .
Documentation strings should be written in the Numpydoc style.
To build the documentation, first, install the extra test dependencies in the Poetry-managed virtual environment by running this command:
$ poetry install --extras=doc
Then, run these commands to build the docs:
$ poetry shell $ make -C doc html
Finally, open the file
doc/_build/html/index.html in your favorite web