Introduce test result output file option for libtest

[patskovn]

Libtest currently only writes machine-readable output to stdout, respecting passed --format. This is problematic for CI/build-tools integration and due to possible test results corruption caused by other dependencies writing to stdout. We propose adding a new flag --test-results-output <path>, which writes the chosen --format output to the given file.

Prior attempts to address this gap have not landed: e.g. PRs #96290 and #123365 (to make --logfile use --format) were abandoned, and the testing-devex-team Issue #9 (“Export machine-readable test results to a file”) was closed without merging initially proposed changes.

The recent deprecation of the --logfile flag in libtest was a positive move away from ambiguous reporting behavior toward clearer solutions. However, removing --logfile without introducing a replacement for machine-readable file output left a critical gap unaddressed.

Motivation

Separating test results from other output avoids contamination. If libtest only writes output to stdout, any non-test output (log messages, debug prints, etc.) may corrupt the stream and break parsers. Rust’s println’s are wrapped by libtest, but anything can (and does, in real world) use libc, or have C code using libc that corrupts stdout. There is no possible workaround for the stdout corruption problem.

Also, in practice, projects often resort to external post-processing to filter test output. As one tracking discussion notes, “due to limitations of Rust libtest formatters, Rust developers often use a separate tool to postprocess the test results output”. By writing test results directly to a file, we can guarantee the test results are isolated and parseable, without third-party noise.

Writing results to a file aligns with established patterns: Google Test (GTest) uses a flag like --gtest_output=json:path to produce test reports in a file.

Proposed Solution

We propose introducing a new option, --test-results-output <file>, which directs libtest to write the structured test report to the given file. This flag would be independent of --logfile; it would capture test results in the specified format. Key points:

• Syntax: --test-results-output path/to/results.ext. The path may be relative or absolute; libtest should create or truncate the file at runtime.

• Respecting --format: The output format (JSON, JUnit XML, etc.) is controlled by the existing --format flag. libtest would open the file and use the same formatter logic as for stdout

• Usage Example

cargo test -- -Zunstable-options --test-results-output=tests.json --format=json 

• Unstable Feature: This flag can initially be gated (e.g. behind -Zunstable-options) until its behavior stabilizes

• Error Handling: If the file cannot be written (permissions, etc.), libtest should emit an error to stderr and exit. If multiple binaries produce the same file (or the same test command is executed multiple times), it’s up to the caller to avoid collisions by using a unique file name per invocation or cleaning up the file1

• Exclusivity: Unlike #123365 which refactored libtest so that both stdout and logfile results are written we propose to write only to the file if the argument is passed and do not duplicate outputs. Motivation for that is file outputs are mostly expected to be used by build (and test) tools and not by a user that invokes cargo test from cli and alignment with previous decisions of maintainers.

• Backward Compatibility: This change does not break existing users of --format or --logfile. The change will not alter libtest behavior unless it’s explicitly used.