Sanitizer test regressions (mostly ASAN) #60394
Comments
mgorny
added
compiler-rt:asan
|
I guess it's not a regression, but something changed in glibc. |
|
I thought that it was maybe the kernel but 5.15.90 fails the same (unless it was a kernel change backported to 5.15 ;-)). |
|
I have seen the same set of failures on 2023-01-12 (on top of 6dc85bd) but almost all tests passed on 2023-01-08 (with the same commit). I did not upgrade glibc during that period. |
|
I see the same failures outside the nspawn container, so it's not that either. I've checked install logs for the aforementioned period and I don't see anything obvious that could have caused this — so most likely it was some kernel change that was backported to 5.15. Unfortunately, I don't have the old kernel binpkgs around anymore. |
|
Posted https://reviews.llvm.org/D143322. We may want to file another bug for the remaining 3 issues I get (no idea if mgorny gets them yet). |
|
I can confirm that the patch fixes all test failures for me. |
Without this, if hardening measures like FORTIFY_SOURCE are are in /etc/clang/*.cfg, many sanitizer tests will die before the sanitizer can trap the problem being tested, because e.g. the _chk variants of common functions will abort first. This gets the number of failing tests down from 42->3 for me (and the remaining 3 are unrelated). See: 52ce677 See: 136f778 Closes: llvm/llvm-project#60394 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D143322 (cherry picked from commit 8ab7625)
|
For posterity, the issues were due to Gentoo's See google/sanitizers#247 Added this to my https://maskray.me/blog/2023-01-08-all-about-sanitizer-interceptors |
EugeneZelenko
added
compiler-rt
test-suite
and removed
compiler-rt:asan
|
I am guessing this fix should be picked into the release branch as well? Run the cherry-pick command in that case, reopen until we have decision on that. |
|
Ah, I filed a separate issue for that: #60533. |
|
Ah nice I just merged that one as well. Thanks for the fix! |
|
Thank you! |
Without this, if hardening measures like FORTIFY_SOURCE are are in /etc/clang/*.cfg, many sanitizer tests will die before the sanitizer can trap the problem being tested, because e.g. the _chk variants of common functions will abort first. This gets the number of failing tests down from 42->3 for me (and the remaining 3 are unrelated). See: 52ce677 See: 136f77805fd89cd30e69b3d1204fbf7efedd9a12 Closes: llvm/llvm-project#60394 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D143322
Without this, if hardening measures like FORTIFY_SOURCE are are in /etc/clang/*.cfg, many sanitizer tests will die before the sanitizer can trap the problem being tested, because e.g. the _chk variants of common functions will abort first. This gets the number of failing tests down from 42->3 for me (and the remaining 3 are unrelated). See: 52ce677 See: 136f778 Closes: llvm/llvm-project#60394 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D143322
…or <16 Bug: llvm/llvm-project#60394 Signed-off-by: Sam James <[email protected]>
Without this, if hardening measures like FORTIFY_SOURCE are are in /etc/clang/*.cfg, many sanitizer tests will die before the sanitizer can trap the problem being tested, because e.g. the _chk variants of common functions will abort first. This gets the number of failing tests down from 42->3 for me (and the remaining 3 are unrelated). See: 52ce677 See: 136f778 Closes: llvm/llvm-project#60394 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D143322
Without this, if hardening measures like FORTIFY_SOURCE are are in /etc/clang/*.cfg, many sanitizer tests will die before the sanitizer can trap the problem being tested, because e.g. the _chk variants of common functions will abort first. This gets the number of failing tests down from 42->3 for me (and the remaining 3 are unrelated). See: 52ce677 See: 136f778 Closes: llvm/llvm-project#60394 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D143322
|
Unfortunately, this fix in 8ab7625, flat out disabling any default config files, break other potential use cases. In llvm-mingw, I set a bunch of custom defaults for the mingw cross compilation targets (
I tried switching llvm-mingw over to the third option, in mstorsjo/llvm-mingw@6939aee and mstorsjo/llvm-mingw@4bff029. However, then running the compiler-rt tests fail, because of So for the cases where the system Clang uses a global config that adds an extra option that we want to skip (to make a more "vanilla" compilation), like Gentoo's I'm not entirely sure how to best deal with this. I mean, I guess we could add an option, for when configuring compiler-rt for running tests, for the user to pick whether to override clang config files or not. But this is rather obscure... Or should we have some sort of cmake configure level test, to see whether the compiler still works if we set |
…configs The use of CLANG_NO_DEFAULT_CONFIG in the tests was added because some Linux distributions had a global default config file, that added flags relating to hardening, which interfere with the sanitizer tests. By setting CLANG_NO_DEFAULT_CONFIG, the global default config files that are found are ignored, and the sanitizers get the expected default compiler behaviour. (This was llvm#60394, which was fixed in 8ab7625.) However, some toolchains may rely on default config files for mandatory parts required for functioning at all - setting things like sysroots, -rtlib, -unwindlib, -stdlib, -fuse-ld etc. In such a case we can't forcibly disable any default config, because it will break the otherwise working toolchain. Add a test for whether the compiler works while passing --no-default-config to it. If the option is accepted and the toolchain still works while that is set, set CLANG_NO_DEFAULT_CONFIG while running tests. (This adds a little bit of inconsistency, as we're testing for the command line option, while using the environment variable. However doing compile testing with an environment variable isn't quite as easily doable, and passing an extra command line flag to all compile commands while testing, is a bit clumsy - therefore this inconsistency.)
I posted a PR that tries to fix this now, in #113491. |
…configs The use of CLANG_NO_DEFAULT_CONFIG in the tests was added because some Linux distributions had a global default config file, that added flags relating to hardening, which interfere with the sanitizer tests. By setting CLANG_NO_DEFAULT_CONFIG, the global default config files that are found are ignored, and the sanitizers get the expected default compiler behaviour. (This was llvm#60394, which was fixed in 8ab7625.) However, some toolchains may rely on default config files for mandatory parts required for functioning at all - setting things like sysroots, -rtlib, -unwindlib, -stdlib, -fuse-ld etc. In such a case we can't forcibly disable any default config, because it will break the otherwise working toolchain. Add a test for whether the compiler works while passing --no-default-config to it. If the option is accepted and the toolchain still works while that is set, set CLANG_NO_DEFAULT_CONFIG while running tests. (This adds a little bit of inconsistency, as we're testing for the command line option, while using the environment variable. However doing compile testing with an environment variable isn't quite as easily doable, and passing an extra command line flag to all compile commands while testing, is a bit clumsy - therefore this inconsistency.)
|
I'm sorry that I missed your earlier comment! |
…configs (#113491) The use of CLANG_NO_DEFAULT_CONFIG in the tests was added because some Linux distributions had a global default config file, that added flags relating to hardening, which interfere with the sanitizer tests. By setting CLANG_NO_DEFAULT_CONFIG, the global default config files that are found are ignored, and the sanitizers get the expected default compiler behaviour. (This was #60394, which was fixed in 8ab7625.) However, some toolchains may rely on default config files for mandatory parts required for functioning at all - setting things like sysroots, -rtlib, -unwindlib, -stdlib, -fuse-ld etc. In such a case we can't forcibly disable any default config, because it will break the otherwise working toolchain. Add a test for whether the compiler works while passing --no-default-config to it. If the option is accepted and the toolchain still works while that is set, set CLANG_NO_DEFAULT_CONFIG while running tests. (This adds a little bit of inconsistency, as we're testing for the command line option, while using the environment variable. However doing compile testing with an environment variable isn't quite as easily doable, and passing an extra command line flag to all compile commands while testing, is a bit clumsy - therefore this inconsistency.)
…configs (llvm#113491) The use of CLANG_NO_DEFAULT_CONFIG in the tests was added because some Linux distributions had a global default config file, that added flags relating to hardening, which interfere with the sanitizer tests. By setting CLANG_NO_DEFAULT_CONFIG, the global default config files that are found are ignored, and the sanitizers get the expected default compiler behaviour. (This was llvm#60394, which was fixed in 8ab7625.) However, some toolchains may rely on default config files for mandatory parts required for functioning at all - setting things like sysroots, -rtlib, -unwindlib, -stdlib, -fuse-ld etc. In such a case we can't forcibly disable any default config, because it will break the otherwise working toolchain. Add a test for whether the compiler works while passing --no-default-config to it. If the option is accepted and the toolchain still works while that is set, set CLANG_NO_DEFAULT_CONFIG while running tests. (This adds a little bit of inconsistency, as we're testing for the command line option, while using the environment variable. However doing compile testing with an environment variable isn't quite as easily doable, and passing an extra command line flag to all compile commands while testing, is a bit clumsy - therefore this inconsistency.)
…configs (llvm#113491) The use of CLANG_NO_DEFAULT_CONFIG in the tests was added because some Linux distributions had a global default config file, that added flags relating to hardening, which interfere with the sanitizer tests. By setting CLANG_NO_DEFAULT_CONFIG, the global default config files that are found are ignored, and the sanitizers get the expected default compiler behaviour. (This was llvm#60394, which was fixed in 8ab7625.) However, some toolchains may rely on default config files for mandatory parts required for functioning at all - setting things like sysroots, -rtlib, -unwindlib, -stdlib, -fuse-ld etc. In such a case we can't forcibly disable any default config, because it will break the otherwise working toolchain. Add a test for whether the compiler works while passing --no-default-config to it. If the option is accepted and the toolchain still works while that is set, set CLANG_NO_DEFAULT_CONFIG while running tests. (This adds a little bit of inconsistency, as we're testing for the command line option, while using the environment variable. However doing compile testing with an environment variable isn't quite as easily doable, and passing an extra command line flag to all compile commands while testing, is a bit clumsy - therefore this inconsistency.) (cherry picked from commit a14a83d)
…configs (llvm#113491) The use of CLANG_NO_DEFAULT_CONFIG in the tests was added because some Linux distributions had a global default config file, that added flags relating to hardening, which interfere with the sanitizer tests. By setting CLANG_NO_DEFAULT_CONFIG, the global default config files that are found are ignored, and the sanitizers get the expected default compiler behaviour. (This was llvm#60394, which was fixed in 8ab7625.) However, some toolchains may rely on default config files for mandatory parts required for functioning at all - setting things like sysroots, -rtlib, -unwindlib, -stdlib, -fuse-ld etc. In such a case we can't forcibly disable any default config, because it will break the otherwise working toolchain. Add a test for whether the compiler works while passing --no-default-config to it. If the option is accepted and the toolchain still works while that is set, set CLANG_NO_DEFAULT_CONFIG while running tests. (This adds a little bit of inconsistency, as we're testing for the command line option, while using the environment variable. However doing compile testing with an environment variable isn't quite as easily doable, and passing an extra command line flag to all compile commands while testing, is a bit clumsy - therefore this inconsistency.) (cherry picked from commit a14a83d)
…configs (llvm#113491) The use of CLANG_NO_DEFAULT_CONFIG in the tests was added because some Linux distributions had a global default config file, that added flags relating to hardening, which interfere with the sanitizer tests. By setting CLANG_NO_DEFAULT_CONFIG, the global default config files that are found are ignored, and the sanitizers get the expected default compiler behaviour. (This was llvm#60394, which was fixed in 8ab7625.) However, some toolchains may rely on default config files for mandatory parts required for functioning at all - setting things like sysroots, -rtlib, -unwindlib, -stdlib, -fuse-ld etc. In such a case we can't forcibly disable any default config, because it will break the otherwise working toolchain. Add a test for whether the compiler works while passing --no-default-config to it. If the option is accepted and the toolchain still works while that is set, set CLANG_NO_DEFAULT_CONFIG while running tests. (This adds a little bit of inconsistency, as we're testing for the command line option, while using the environment variable. However doing compile testing with an environment variable isn't quite as easily doable, and passing an extra command line flag to all compile commands while testing, is a bit clumsy - therefore this inconsistency.) (cherry picked from commit a14a83d)
Without this, if hardening measures like FORTIFY_SOURCE are are in /etc/clang/*.cfg, many sanitizer tests will die before the sanitizer can trap the problem being tested, because e.g. the _chk variants of common functions will abort first. This gets the number of failing tests down from 42->3 for me (and the remaining 3 are unrelated). See: 52ce6776cf98e993c6ec04ae54b52e1354fff917 See: 136f77805fd89cd30e69b3d1204fbf7efedd9a12 Closes: llvm/llvm-project#60394 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D143322
I've noticed them around the beginning of January (I'm sorry for not reporting earlier), they don't seem to related to any change in LLVM (i.e. once they started appearing, they happened with older versions as well):
Gentoo Linux amd64
Kernel: 6.1.8-gentoo-dist
glibc: 2.36-r7
systemd: 252.5 (testing is done in nspawn container)
Full log (2.5M): sys-libs:compiler-rt-sanitizers-16.0.0_rc1:20230130-193656.log
The text was updated successfully, but these errors were encountered: