Introduce file-based declarative test suite support

[nikole-dunixi]

Overview

I'd like to know if the equivalent exists or is in the works to match TestNG's suite XML file format. I asked on Stack Overflow and was directed by @marcphilipp to open a feature request for this. Here are the details from the original SO post:

I've been dynamically generating TestNG XML suites in a project I'm working on. In an effort to take some initiative in migrating to JUnit 5. The TestNG specification is very easy to find, but I cannot find the equivalent for JUnit 5.

All I can find are references that a file or URI can be specified for test discovery. Either by command line option or URI and again under the configuring selectors section.

Naively, I'd like to assume that it's just the fully.qualified.package.ClassName#methodName format because I can find this as an example in the MethodSelector. The UriSelector and FileSelector don't show what the actually expected format of the document is supposed to look like.

Related Issues:

• Introduce support for declarative test suites #744

[sbrannen]

The UriSelector and FileSelector don't show what the actually expected format of the document is supposed to look like.

That's because JUnit Jupiter does not in any way support those two selectors. In addition, the JUnit Platform also does not currently react to the presence of such selectors.

Those selectors only exist for other test engines that want to make use of them, in which case such a test engine would have to document what file formats are supported by that test engine.

[sbrannen]

I'd like to know if the equivalent exists or is in the works to match TestNG's suite XML file format.

The short answer to that is "no". There is currently no support for suite configuration via a file.

However, the fact that you've raised this issue means that it's now "open for discussion". 😉

[sbrannen]

FYI: I've renamed this issue to reflect the scope.

[vlsi]

I have a case for suites: integration testing.
Individual tests are like "prepare configuration, upload configuration, perform test method".
However, "upload configuration" is slow (e.g. it takes minutes), so I work around that by combining the configuration from all the test classes and uploading it once.

Of course, certain configurations can't be combined. For instance, test cases for "invalid configuration should produce reasonable errors" must not be combined with valid configurations otherwise the combined configuration won't even work.

That is why I use suites to split test classes into bigger chunks so I save time on testing (configuration upload is performed just once) with reasonable sacrifice in quality (test data/configurations are mostly independent, so test interference is acceptable for me).

[mpkorstanje]

This looks like a variation of #744 and #2416 but rather then using annotations to create the discovery request an xml file would be used instead.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. Thank you for your contribution.