Development Setup#
See also CONTRIBUTING.md and ARCHITECTURE.md
Development is setup for local native and containerized Python coding & testing, and with automatic GitHub Actions for CI + CD. The server tests are like the local ones, except against a wider test matrix of environments.
LFS#
We are starting to use git lfs for data:
# install git lfs: os-specific commands below
git lfs install
git lfs checkout
git lfs: ubuntu#
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs
Docker#
Install#
cd docker && docker compose build && docker compose up -d
For just CPU tests, you can focus on test-cpu
and use the run instructions below:
cd docker && docker compose build test-cpu
Run local tests without rebuild#
Containerized call to pytest
for CPU + GPU modes:
cd docker
# cpu - pandas
./test-cpu-local.sh
# cpu - fast & targeted
WITH_LINT=0 WITH_TYPECHECK=0 WITH_BUILD=0 ./test-cpu-local.sh graphistry/tests/test_hyper_dask.py::TestHypergraphPandas::test_hyper_to_pa_mixed2
# gpu - pandas, cudf, dask, dask_cudf; test only one file
./test-gpu-local.sh graphistry/tests/test_hyper_dask.py
Connector tests (currently neo4j-only): cd docker && WITH_NEO4J=1 ./test-cpu-local.sh
(optional WITH_SUDO=" "
)
Will start a local neo4j (docker) then enable+run tests against it
Docs#
Automatically build via ReadTheDocs from inline definitions.
To manually build, see docs/
.
Ignore files#
You may need to add ignore rules:
flake8: bin/lint.sh
mypi: mypi.ini
sphinx: docs/source/conf.py
Remote#
Some databases like Neptune can be easier via cloud editing, especially within Jupyter:
git clone https://github.com/graphistry/pygraphistry.git
git checkout origin/my_branch
pip install --user -e .
git diff
and
import logging
logging.basicConfig(level=logging.DEBUG)
import graphistry
graphistry.__version__
CI#
GitHub Actions: See .github/workflows
CI runs on every PR and updates them
GPU CI#
GPU CI can be manually triggered by core dev team members:
Push intended changes to protected branches
gpu-public
ormaster
Manually trigger action ci-gpu on one of the above branches
GPU tests can also be run locally via ./docker/test-gpu-local.sh
.
Debugging Tips#
Use the unit tests
use the
logging
module per-file
Publish: Merge, Tag, & Upload#
Manually update CHANGELOG.md
Tag the repository with a new version number. We use semantic version numbers of the form X.Y.Z.
git tag X.Y.Z git push --tags
Confirm the publish Github Action published to pypi, or manually run it for the master branch
Toggle version as active at ReadTheDocs
Merge the desired PR to master and switch to master head (
git checkout master && git pull
)