Developer documentation

Set-up for development in a virtual environment

  1. Checkout the code.

$ git clone git@github.com:neutrons/drtsans.git
$ cd drtsans

  1. Create the development environment.

Pixi is used for managing environments, dependencies, packaging, and task execution.

  • It will use Mantid’s conda package.

  • Pixi is required. Install Pixi with the following command:

    $ curl -fsSL https://pixi.sh/install.sh | bash

  • Create the evironment by running

    $ pixi install

  • Activate the environment by running

    $ pixi shell

  1. Install the code in develop mode.

$ pip install -e .

  1. Try it out. Start python and try

import mantid
import drtsans

Verify you can run the unit tests:

$ python -m pytest tests/unit/
$ python -m pytest tests/integration/

Some unit and integration tests require testing data. Thus these tests can be either run on * SNS analysis cluster or * Local computer with mount to /SNS/ archive. Here is the instruction to mount SNS data directories.

  1. When done, deactivate the virtual environment using

$ exit

Git submodule for test data

To get the current version of the git-lfs associated with the currently checked out branch

$ git submodule update --init

where update checks out the associated refspec and --init tells git to initialize the submodule if it isn’t already. To update the submodule to the latest commit on the branch being tracked

$ git submodule update --remote --merge

where --remote tells git to fetch the latest changes from the upstream repository and --merge tells git to merge the changes into the working tree.

Then, to get the data files associated with the submodule, run

$ cd tests/data/drtsans-data
$ git lfs fetch

See the git-submodules documentation for more detailed information.

Running the tests

The tests for this project are all written using pytest. The build pipeline currently `runs the unit tests and integration tests separately using

$ python -m pytest tests/unit/
$ python -m pytest tests/integration/

This is one of the ways pytest allows for selecting tests. Specifying a directory or file will run all tests within that directory (recursively) or file. Specifying a regular expression using -k will select all tests that match the regular expression independent of where they are defined

$ python -m pytest -k test_samplelogs

To run an individual test within an individual file add :: to the filename to specify the test

$ python -m pytest tests/unit/new/drtsans/tof/eqsans/test_beam_finder.py::test_center_detector

Building the documentation

The site can be built directly using

$ sphinx-build -b html docs/ build/sphinx/html

or

$ pixi run build-docs

Installing the pre-commit hook

To automatically run the pre-commit steps (e.g. linting) when adding a commit, install the git pre-commit hook.

$ pre-commit install

To run pre-commit locally without committing

$ pre-commit run --all

Test Driven Development (TDD)

  • Test driven Development

    drtSANS development follows test-driven development (TDD) process [1]. All software requirements for SANS data reduction shall be converted to test cases before software is fully developed. All software developments are tracked by repeatedly testing the software against all test cases.

  • Unit test

    All methods and modules shall have unit tests implemented. Unit tests are located in repo/tests/unit/new. A unit test shall be created in the corresponding directory to the method or module that it tests against.

    Examples:

  • Integration test

    Integration test will test the combination of Individual modules and methods. Integration tests can be

    • general for all instrument, for instance tests/integration/new/drtsans/test_stitch.py.

    • specific to a suite of similar instruments, for instance tests/integration/new/drtsans/mono/test_transmission.py for all mono-wavelength instruments including Bio-SANS and GP-SANS.

    • specific to an individual instrument, for instance, tests/integration/new/drtsans/mono/gpsans/test_find_beam_center.py for GP-SANS and tests/integration/new/drtsans/tof/eqsans/test_apply_solid_angle.py for EQ-SANS.

  • Testing data location

    Testing data are located on SNS data archive: /SNS/EQSANS/shared/sans-backend/data.

    Testing data for specific instruments have specific locations:

    • EQSANS: /SNS/EQSANS/shared/sans-backend/data/new/ornl/sans/sns/eqsans/

    • Bio-SANS: /SNS/EQSANS/shared/sans-backend/data/new/ornl/sans/hfir/biosans/

    • GP-SANS: /SNS/EQSANS/shared/sans-backend/data/new/ornl/sans/hfir/gpsans/

    Data files are referenced in the tests via the reference_dir pytest fixture. For instance, reference_dir.new.eqsans points to /SNS/EQSANS/shared/sans-backend/data/new/ornl/sans/sns/eqsans/

Required libraries

Package Build and Installation Instructions

Python wheel

$ python -m build --wheel --no-isolation
$ check-wheel-contents dist/drtsans-*.whl

Conda package

# create a conda package
$ pixi run conda-build

Creating Candidate and Production Releases

  • Follow the Software Maturity Model for continous versioning as well as creating release candidates and stable releases.

  • Update file release_notes.rst with major fixes, updates and additions since last stable release.

Standard variable names and what they denote

  • wavelength (was wl) - wavelength

  • sample_det_cent (was s2p) - sample to detector center distance

  • l1 - source to sample distance

  • need definitions of l2, r1, r2, dwl, … These should likely get different names with unambigious meanings

  • see e.g. solid_angle_correction.py for properly documented/named code

Random points about coding conventions

  • All code will follow the pep8 standard

  • Docstrings will use numpy formatting

  • Code that uses Mantid algorithms will have an addional “Mantid algorithms used” section with links using inter-sphinx

  • All internal functions should have an underbar, as is python standard

  • Need more comments in meat of code explaining intent. For example, more clearly named variables (or leave the variables the same but give them better documentation)

  • The development team still hasn’t decided a standard for how errors/exceptions are handled