jetstream

Automated experiment analysis.

Jetstream automatically calculates metrics and applies statistical treatments to collected experiment data for different analysis windows.

For more information, see the documentation.

Local installation

# Create and activate a python virtual environment.
python3 -m venv venv/
source venv/bin/activate
# --no-deps to workaround https://github.com/pypa/pip/issues/9644
pip install --no-deps -r requirements.txt
pip install -e .

Running tests

Install tox (included in requirements.txt)
Run tox from wherever you cloned this repository. (You don't need to install jetstream first.)
- To run integration tests, run tox -e py310-integration.

Linting

ruff check jetstream

ruff can also fix lint issues:

ruff format jetstream

Dependencies

Jetstream uses pip-tools to manage dependencies, along with a script that runs the pip-tools commands. There are two requirements files auto-generated by script/update_deps:

requirements.in: Listing of dependencies and versions generated from the dependencies list in pyproject.toml.
requirements.txt: Auto-generated by pip-tools (pip-compile) from the requirements.in file. Also contains the hashes of each package for verification by pip during installation, and comments showing lineage for each dependency.

Add a new dependency

Add to pyproject.toml list of dependencies.
Generate new requirements files with ./script/update_deps
Install with python -m pip install -r requirements.txt

Update all dependencies

./script/update_deps

Be sure to run pip install --no-deps -r requirements.txt and reinstall jetstream (pip install -e .) afterwards, and test functionality!

Update a single dependency

Prefer to update all dependencies (see above)!
Edit requirements.in

mypy==1.8.0 --> mypy==1.9.0

Regenerate requirements.txt

pip-compile --generate-hashes -o requirements.txt requirements.in
- (Note: this is the last line of script/update_deps)

Install dependencies

pip install --no-deps -r requirements.txt
pip install -e .

Test!