From ee55dedb9dbac1c89765c1433d7f209d3560e7e0 Mon Sep 17 00:00:00 2001 From: Eliah Kagan Date: Mon, 5 May 2025 22:26:56 -0400 Subject: [PATCH] Regroup CI doctests and adjust `unit-tests` recipe This seeks to make improvements in four overlapping areas: - The CI `test-fast` job for `windows-latest` had been taking the longest, and integrating PRs would be more efficient if it could be sped up. If it didn't have to build and run doctests, then it would run markedly faster. `test-fast` runs doctests because... - The `unit-tests` recipe in the `justfile`, which is one of the recipes the "full" CI `test` job runs via the `ci-test` recipe, runs a number of `nextest` commands on individual crates (with `-p`, except for testing the top-level `gitoxide` crate, and not with `--workspace`). It also ran doctests, but only on the `gitoxide` top-level crate. But the `gitoxide` crate currently has no doctests, while some `gix-*` crates do. - On CI, we usually prefer `--no-fail-fast`. But it is not always used, because the commands in the `unit-tests` justfile recipe do not use it. `--no-fail-fast` is not always preferred when running tests locally, but... - Both locally and on CI, in most cases that a test fails in a commmand in the `unit-tests` justfile recipe, the effect is that tests have run and results have been reported for multiple `nextest` commands, yet not all the tests specified in the most recent `nextest` command to run. Thus, omitting `--no-fail-fast` may not have the most intuitive effect in `just unit-tests`, even when run locally (even if the user would omit `--no-fail-fast` in individual `cargo nextest` runs carried out manually). This commit makes the following changes: 1. Add `--no-fail-fast` to each of the commands in the `unit-tests` recipe in the `justfile`: the numerous `cargo nextest` commands, as well as the `cargo test` command used to run doctests. 2. Add `--workspace` to the `cargo test` command used to run doctests in the `unit-tests` recipe in the `justfile`, and move it to the end of the recipe. 3. Not to be confused with that `cargo test` command, move the other `cargo test` command used to run doctests in the `ci.yml` workflow (which alredy passed `--workspace`, as its purpose was to run all doctests in all crates) from the `tests-fast` job definition into the `test-32bit-windows-size` job, and rename that latter job `test-32bit-windows-size-doc` accordingly. The rationale for (3) may not be obvious. The idea is: - Running the doctests on only one Unix-like system should be enough, so long as they are run for all crates in the workspace. So the change in the `unit-tests` recipe in the `justfile` makes it so the CI `test` job (which includes a `unit-tests` run) covers doctests sufficiently, *except* for Windows. - Although we should probably keep running doctests regularly on Windows, removing it from `test-fast`, including on Windows, is the simplest way to make the Windows `test-fast` job run faster. (It also makes the job definition clearer, since some of the other steps relate to each other more closely than they do to the step that ran the doctests.) - It should be sufficient to run the doctests in any Windows environment. And it is best to avoid adding a new Windows job just for this, since various other Windows jobs might be added sometime soon (such as for ARM64, native Windows containers, the Git for Windows SDK, MinGit, BusyBox MinGit, and possibly others; some of these may be possible to combine, but likely a few more Windows jobs may be introduced for these, so avoiding adding extra Windows jobs now may make it easier to avoid having too many Windows jobs, in terms of queuing, GHA cache usage, energy usage, and other resources). So if this can be added to another Windows job without causing problems, that is preferable. - The Windows job that took the least amount of time, usually by several minutes, was the `test-32bit-windows-size` job. It is hope that this keeps the benefits of #1556, #1559, and #1654, while improving CI testing performance most of the time. --- .github/workflows/ci.yml | 10 +++--- justfile | 70 ++++++++++++++++++++-------------------- 2 files changed, 40 insertions(+), 40 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d81056d9e4b..7a55e80c820 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -181,8 +181,6 @@ jobs: env: GIX_TEST_CREATE_ARCHIVES_EVEN_ON_CI: '1' run: cargo nextest run --workspace --no-fail-fast - - name: Doctest - run: cargo test --workspace --doc --no-fail-fast - name: Check that tracked archives are up to date run: git diff --exit-code # If this fails, the fix is usually to commit a regenerated archive. @@ -277,7 +275,7 @@ jobs: GIX_TEST_IGNORE_ARCHIVES: '1' run: cargo nextest run --workspace --no-fail-fast - test-32bit-windows-size: + test-32bit-windows-size-doc: runs-on: windows-latest env: @@ -292,8 +290,10 @@ jobs: - uses: taiki-e/install-action@v2 with: tool: nextest - - name: Test (nextest) + - name: Test data structure sizes (nextest) run: cargo nextest run --target $env:TARGET --workspace --no-fail-fast size + - name: Doctest + run: cargo test --workspace --doc --no-fail-fast lint: runs-on: ubuntu-latest @@ -497,7 +497,7 @@ jobs: - test-fast - test-fixtures-windows - test-32bit - - test-32bit-windows-size + - test-32bit-windows-size-doc - lint - cargo-deny - check-packetline diff --git a/justfile b/justfile index f2b0a1e081d..2a9255e07f0 100755 --- a/justfile +++ b/justfile @@ -144,41 +144,41 @@ doc $RUSTDOCFLAGS='-D warnings': # Run all unit tests unit-tests: - cargo nextest run - cargo test --doc - cargo nextest run -p gix-testtools - cargo nextest run -p gix-testtools --features xz - cargo nextest run -p gix-archive --no-default-features - cargo nextest run -p gix-archive --features tar - cargo nextest run -p gix-archive --features tar_gz - cargo nextest run -p gix-archive --features zip - cargo nextest run -p gix-status-tests --features gix-features-parallel - cargo nextest run -p gix-worktree-state-tests --features gix-features-parallel - cargo nextest run -p gix-worktree-tests --features gix-features-parallel - cargo nextest run -p gix-object - cargo nextest run -p gix-object --features verbose-object-parsing-errors - cargo nextest run -p gix-tempfile --features signals - cargo nextest run -p gix-features --all-features - cargo nextest run -p gix-ref-tests --all-features - cargo nextest run -p gix-odb --all-features - cargo nextest run -p gix-odb-tests --features gix-features-parallel - cargo nextest run -p gix-pack --all-features - cargo nextest run -p gix-pack-tests --features all-features - cargo nextest run -p gix-pack-tests --features gix-features-parallel - cargo nextest run -p gix-index-tests --features gix-features-parallel - cargo nextest run -p gix-packetline --features blocking-io,maybe-async/is_sync --test blocking-packetline - cargo nextest run -p gix-packetline --features async-io --test async-packetline - cargo nextest run -p gix-transport --features http-client-curl,maybe-async/is_sync - cargo nextest run -p gix-transport --features http-client-reqwest,maybe-async/is_sync - cargo nextest run -p gix-transport --features async-client - cargo nextest run -p gix-protocol --features blocking-client - cargo nextest run -p gix-protocol --features async-client - cargo nextest run -p gix --no-default-features - cargo nextest run -p gix --no-default-features --features basic,comfort,max-performance-safe - cargo nextest run -p gix --no-default-features --features basic,extras,comfort,need-more-recent-msrv - cargo nextest run -p gix --features async-network-client - cargo nextest run -p gix --features blocking-network-client - cargo nextest run -p gitoxide-core --lib --no-tests=warn + cargo nextest run --no-fail-fast + cargo nextest run -p gix-testtools --no-fail-fast + cargo nextest run -p gix-testtools --features xz --no-fail-fast + cargo nextest run -p gix-archive --no-default-features --no-fail-fast + cargo nextest run -p gix-archive --features tar --no-fail-fast + cargo nextest run -p gix-archive --features tar_gz --no-fail-fast + cargo nextest run -p gix-archive --features zip --no-fail-fast + cargo nextest run -p gix-status-tests --features gix-features-parallel --no-fail-fast + cargo nextest run -p gix-worktree-state-tests --features gix-features-parallel --no-fail-fast + cargo nextest run -p gix-worktree-tests --features gix-features-parallel --no-fail-fast + cargo nextest run -p gix-object --no-fail-fast + cargo nextest run -p gix-object --features verbose-object-parsing-errors --no-fail-fast + cargo nextest run -p gix-tempfile --features signals --no-fail-fast + cargo nextest run -p gix-features --all-features --no-fail-fast + cargo nextest run -p gix-ref-tests --all-features --no-fail-fast + cargo nextest run -p gix-odb --all-features --no-fail-fast + cargo nextest run -p gix-odb-tests --features gix-features-parallel --no-fail-fast + cargo nextest run -p gix-pack --all-features --no-fail-fast + cargo nextest run -p gix-pack-tests --features all-features --no-fail-fast + cargo nextest run -p gix-pack-tests --features gix-features-parallel --no-fail-fast + cargo nextest run -p gix-index-tests --features gix-features-parallel --no-fail-fast + cargo nextest run -p gix-packetline --features blocking-io,maybe-async/is_sync --test blocking-packetline --no-fail-fast + cargo nextest run -p gix-packetline --features async-io --test async-packetline --no-fail-fast + cargo nextest run -p gix-transport --features http-client-curl,maybe-async/is_sync --no-fail-fast + cargo nextest run -p gix-transport --features http-client-reqwest,maybe-async/is_sync --no-fail-fast + cargo nextest run -p gix-transport --features async-client --no-fail-fast + cargo nextest run -p gix-protocol --features blocking-client --no-fail-fast + cargo nextest run -p gix-protocol --features async-client --no-fail-fast + cargo nextest run -p gix --no-default-features --no-fail-fast + cargo nextest run -p gix --no-default-features --features basic,comfort,max-performance-safe --no-fail-fast + cargo nextest run -p gix --no-default-features --features basic,extras,comfort,need-more-recent-msrv --no-fail-fast + cargo nextest run -p gix --features async-network-client --no-fail-fast + cargo nextest run -p gix --features blocking-network-client --no-fail-fast + cargo nextest run -p gitoxide-core --lib --no-tests=warn --no-fail-fast + cargo test --workspace --doc --no-fail-fast # These tests aren't run by default as they are flaky (even locally) unit-tests-flaky: