Improve golden master tests#59
Conversation
No longer uses the demo_configuration folder for test configuration.
This includes sorting output to ensure it is deterministic. The 'file' utility has been installed to allow tests to run on the devcontainer.
These are caught in the main() function and printed without traceback to reduce clutter in failed runs.
This removes the need to depend on an image pull in CI.
These were stripped down for the golden master tests, but those are now using separate configuration.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #59 +/- ##
==========================================
+ Coverage 75.51% 83.59% +8.08%
==========================================
Files 25 27 +2
Lines 931 957 +26
==========================================
+ Hits 703 800 +97
+ Misses 228 157 -71 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
This file has been added in preparation for the next test, but given that test two was 'added' and this test is 'updated' it is a little confusing.
Could version 2.0 be deprecated instead?
I see that you are also testing the version order to make sure 2.0 takes precedence over 2.1rc1. Another thought is that you could add this file in the initial deployment and test that behaviour there. Or at least in test 2 where it can be justified as being 'added'.
There was a problem hiding this comment.
updatable will be added from 01-initial, and both new versions will be added in 02-added
Instead, we clearly indicate that the type is a set (and that therefore sorting is necessary).
The 'updatable' module has now been added from the start, and both new versions are added in 02-added. The test description has been updated to reflect these changes.
3 participants
This now exercises multiple stages of the module lifecycle, such as deprecation, removal, restore etc.
This includes the refactor to use a single DeployToolsError, since that was used in the code for patching out the apptainer pull from the lifecycle test. Other code changes (other than applying consistent type hints) have been reserved for the PR that will include the CLI / command testing.
There are a lot of changed files since we have all the different configuration files and generated output. I have tried to name the modules to indicate their purpose, and they have been stripped down from the originals.
You'll probably want to minimise the tests/configs & tests/samples to start rather than read all "Files changed" top-to-bottom.