Testing#

Testing strategy, framework and procedure.

Strategy#

“Use case” notebooks as high-level functional tests.

Requirements:

  • “Use case” notebooks should appear in documentation, at least one per observation type (pointing-type notebooks could be in specific “tests” notebooks).

  • “Use case” notebooks should be tested prior to generate documentation (which shouldn’t a way of testing these notebooks).

  • API usage examples (embedded as docs snippets) should not produce errors when generating documentation.

“Subcutaneous” tests using the new gfinder_cli Python API (tests that doesn’t run against the final end-user interface, but against an interface just below the surface).

Unit tests; priority:

  • GeometryFactory

  • Geometry classes

  • OpportunityDefinition

  • DataStore

  • Opportunity

  • MissionScenario

Framework#

Data store dependency and test data management is key.

What test data should be used? Where should they be located? (Python or data package?)

Tools:

  • pytest

  • nbmake

Procedure#

Notes:

  • Releases required to pass all tests for 1.0+ versions.

  • Alpha versions can be release without having to pass all tests.

Steps:

  1. Delete all data store opportunities

  2. Run pytest

  3. Run nbmake

  4. Delete all docs built notebooks

  5. Generate documentation

Pytest run examples:

pytest --co

pytest tests/func/test_opportunity.py --tb=no

pytest --nbmake --overwrite gfinder-python-cli/notebooks/observation_types/MAJIS_SAT_DISK_SLEW/**/*ipynb
pytest --nbmake --overwrite gfinder-python-cli/notebooks/observation_types/MAJIS_SAT_DISK_SCAN/**/*ipynb
pytest --nbmake --overwrite gfinder-python-cli/notebooks/observation_types/MAJIS_FLYBY_HR/**/*ipynb
pytest --nbmake --overwrite gfinder-python-cli/notebooks/observation_types/MAJIS_SAT_LIMB_SCAN/**/*ipynb