Contributing¶
Contributors may familiarize themselves with Celery itself by going through the First Steps with Celery tutorial.
Development model¶
GWCelery operates on a fork-and-merge development model (see GitLab basics for an introduction).
To contribute to GWCelery development, follow these steps:
- Create a personal fork of GWCelery.
- Make your changes on a branch.
- Open a merge request.
Note that GWCelery uses fast-forward merges.
Where new code should go¶
New code will generally consist of adding Celery tasks.
Tasks are organized by functionality into submodules of gwcelery.tasks
.
If your new task does not match with one of the existing submodules, please
create a new submodule.
Guidelines for tasks¶
Tasks should be short. When deciding where a new task should go, start from the following loose rules of thumb:
- If it’s less than a screenful of code, and related to functionality in an existing module, then put the code in a new task in that module.
- If it’s up to a few screenfuls of code, or not related to functionality in an existing module, then try to break it into a few smaller functions or tasks and put it in a new module.
- If it’s more than a few screenfuls of code, or adds many additional dependencies, then it should go in a separate package.
See also the note on Granularity in the Celery manual’s Tips and Best Practices section.
Tasks should avoid saving files to disk. Output should be placed directly in GraceDB. Temporary files that are written in
/tmp
are OK but should be cleaned up promptly.See also the Celery manual’s notes on Data locality and State.
Dependencies should be installable by pip. Dependencies of tasks should be listed in the requirements.txt file so that they are installed automatically when GWCelery is installed with pip.
There are two extra steps involved in making changes to the dependencies:
- The Sphinx-generated documentation (that is to say, this manual) is
generally built without most of the dependencies installed. Whenever you
add a new package to requirements.txt, you should also add any modules
that are imported from that package to the
autodoc_mock_imports
list in the Sphinx configuration file, doc/conf.py. - We use pipenv to make the precise versions of packages reproducible
in our deployment. If you make changes to requirements.txt, then run
pipenv update
and commit the changes to Pipfile.lock.
- The Sphinx-generated documentation (that is to say, this manual) is
generally built without most of the dependencies installed. Whenever you
add a new package to requirements.txt, you should also add any modules
that are imported from that package to the
Unit tests¶
Unit tests and code coverage measurement are run automatically for every branch and for every merge request. New code contributions must have 100% test coverage. Modifications to existing code must not decrease test coverage. To run the unit tests and measure code coverage, run the following commands in the top directory of your local source checkout:
$ pip install pytest-cov
$ python setup.py test --addopts='--cov --cov-report html'
This will save a coverage report that you can view in a web browser as
htmlcov/index.html
.
Code style¶
Code should be written in the PEP 8 style and must pass linting by Flake8. To check code style, run the following commands in the top of your source directory:
$ pip install flake8 pep8-naming
$ flake8 --show-source .
Documentation¶
Documentation strings should be written in the Numpydoc style.
To build the documentation, run the following command in the top of your source directory:
$ python setup.py build_sphinx
Then to view the documentation, open the file build/sphinx/html/index.html
in your favorite web browser.