local test improvments#4037
Open
zhijianli88 wants to merge 6 commits into
Open
Conversation
zhijianli88
force-pushed
the
run-local-test
branch
from
505fa4f to
f1708fd
Compare
zhijianli88
force-pushed
the
run-local-test
branch
from
f1708fd to
534d92c
Compare
zhijianli88
force-pushed
the
run-local-test
branch
from
534d92c to
2ccc978
Compare
The local test runner ./run_local_tests.sh invokes the mantle container with the scripts checkout mounted as /work. When using a git worktree for the scripts directory, git inside the container may see a .git/worktrees path that does not resolve correctly in that environment and fails with: ./run_local_tests.sh ... fatal: not a git repository: ../flatcar/scripts/.git/worktrees/... ... Commit 87e13eb ("ci-automation: Allow git to work on directory owned by other user") added "git config --global --add safe.directory /work" to the test docker invocation to fix an issue where get_git_channel() failed inside the mantle container, which caused AWS publishing to skip the Alpha channel. However, this git config was never necessary for the test path. In _test_run_impl(), both get_git_version() and get_git_channel() are called on the host side before the docker run, and their outputs are stored as files for the container to consume. Since commit 1d6f38a ("ci-automation: Reduce boilerplate in vendor tests"), all vendor testing scripts read git version and channel from these precomputed files via vendor_test.sh rather than running git commands. No vendor script (qemu_uefi, qemu, qemu_update, aws, azure, gce, etc.) executes git inside the mantle container. The git config in the test docker invocation was therefore redundant from the start. Remove it to avoid failures in worktree-based local test setups, while CI behavior remains unchanged since the non-worktree case is unaffected. Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
run_local_tests.sh wraps ci-automation/test.sh for local qemu_uefi and qemu_update testing, but it did not document its arguments or defaults. Add a usage header and a --help option that describe the ARCH, PARALLEL and TEST parameters so that developers can discover how to invoke the script and what behavior to expect without having to read through the implementation. Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
When working with the mantle image it is useful to inspect the set of available kola tests before running any local qemu_uefi or qemu_update runs, but run_local_tests.sh did not provide a way to query them. Add a --list-tests helper option that runs kola list --platform qemu in the configured mantle container and prints the result without starting any tests, allowing developers to quickly discover the test set from the same script they use to run local suites. Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
Stop the retry flow when the vendor test runner does not produce a TAP file. This avoids handing an empty result to test_update_reruns.sh and makes the underlying failure visible earlier in the log. After this patch, when it failed to download the qemu image: $ ./run_local_tests.sh amd64 2 docker.base ================================= Using Mantle docker image 'ghcr.io/flatcar/mantle:git-ca80a2eaee4cc195ae6e17f9202c1d72e729d460' ================================= Running qemu_uefi tests git-ca80a2eaee4cc195ae6e17f9202c1d72e729d460: Pulling from flatcar/mantle Digest: sha256:cd37c1ba21b8051eb2632e171d4f1f9bc7e91708883575343265d3d58940d5df Status: Image is up to date for ghcr.io/flatcar/mantle:git-ca80a2eaee4cc195ae6e17f9202c1d72e729d460 zstd could not be found. unpacking container image may fail. ++++ Running qemu_uefi.sh inside __TESTS__/qemu_uefi ++++ ++++ qemu_uefi.sh: downloading /work/__build__/images/images/amd64-usr/latest/flatcar_production_image.bin for 4593.2.0+4-gbc1fd0010d (amd64) ++++ curl: (22) The requested URL returned error: 404 image_escaped: qemu_uefi.sh return 22, __TESTS__/qemu_uefi/results-run-1.tap ERROR: something went wrong, result is empty: __TESTS__/qemu_uefi/results-run-1.tap ########### All re-runs exhausted (5). Giving up. ########### Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
This reverts the --pull always addition from commit 60a45ef ("ci-automation/test: Always pull mantle image when running tests"). MANTLE_REF points to an immutable tag (ghcr.io/flatcar/mantle:git-<sha>) rather than a mutable tag like :latest, so --pull always only adds an unnecessary registry lookup without any functional effect. Removing it avoids pointless network round-trips and ensures reruns use the same local image for consistent results. Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
When running `./run_local_tests.sh`, a warning is printed if `zstd` is not found in the host environment: $ ./run_local_tests.sh arm64 2 cl.cloudinit.script ... zstd could not be found. unpacking container image may fail. ... This check is performed because `ci-config.env` is sourced by the script. However, the `run_local_tests.sh` workflow itself does not depend on `zstd`. The `zstd` command is required for building Flatcar[1], but that process runs inside the Flatcar SDK container, which is guaranteed to have `zstd` installed. The check on the host running the local tests is therefore redundant and produces confusing noise for developers. This patch removes the check to eliminate the spurious warning. [1] flatcar/mantle#853 (comment) Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
zhijianli88
force-pushed
the
run-local-test
branch
from
2ccc978 to
95fb667
Compare
Contributor
Author
|
Rebase against the latest main branch. |
There was a problem hiding this comment.
Pull request overview
This PR improves the local/CI testing scripts around Mantle/kola to make local runs more ergonomic (help/list options) and to adjust test execution behavior (cleanup + result validation).
Changes:
- Add
--helpand--list-testssupport torun_local_tests.sh. - Clean up prior test artifacts and add an “empty results” guard in
ci-automation/test.sh. - Remove a
zstdpresence warning fromci-automation/ci-config.env.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| run_local_tests.sh | Adds a usage helper and a --list-tests mode for local runs. |
| ci-automation/test.sh | Adjusts test execution container runs and adds local result cleanup/validation. |
| ci-automation/ci-config.env | Removes a startup warning about missing zstd. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+160
to
+165
| --list-tests) | ||
| # List all tests that would be run by default and exit. | ||
| mantle_container="$(cat "sdk_container/.repo/manifests/mantle-container")" | ||
| docker run --rm "${mantle_container}" kola list --platform qemu | ||
| exit 0 | ||
| ;; |
Comment on lines
+160
to
162
| # Remove the old results first | ||
| rm -f ${tests_dir}/*-run-1.* || true | ||
| # Vendor tests may need to know if it is a first run or a rerun |
Comment on lines
+174
to
175
| docker run --rm --name="${container_name}" --privileged --net host -v /dev:/dev \ | ||
| -w /work -v "$PWD":/work "${MANTLE_REF}" \ |
Comment on lines
+187
to
188
| docker run --rm --name="${container_name}" --privileged --net host -v /dev:/dev \ | ||
| -w /work -v "$PWD":/work "${MANTLE_REF}" \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix and improve the run_local_tests.sh and related scripts