cargo test --package openssl test_ec_key_accessor crashes with 0xc0000005 on arm64 Windows

[saschanaz]

Previous discussion: pyca/cryptography#12867

When I run the test command:

> cargo test --package openssl test_ec_key_accessor
warning: [email protected]: expando.c
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.09s
     Running unittests src\lib.rs (target\debug\deps\openssl-82bd68d2fab20804.exe)

running 1 test
error: test failed, to rerun pass `-p openssl --lib`

Caused by:
  process didn't exit successfully: `D:\rust-openssl\target\debug\deps\openssl-82bd68d2fab20804.exe test_ec_key_accessor` (exit code: 0xc0000005, STATUS_ACCESS_VIOLATION)
note: test exited abnormally; to see the full output pass --nocapture to the harness.

Attaching windbg, it shows a recursive loop on evp_pkey_get_legacy:

(9360.308): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
openssl_82bd68d2fab20804!evp_pkey_get_legacy+0x10:
00007ff7`5d3e1238 f90003ff str         xzr,[sp]

[0x0]   openssl_82bd68d2fab20804!evp_pkey_get_legacy+0x10   0xdea2100000   0x7ff75d3e1234   
[0x1]   openssl_82bd68d2fab20804!evp_pkey_get_legacy+0xc   0xdea2100030   0x7ff75d3e1234   
[0x2]   openssl_82bd68d2fab20804!evp_pkey_get_legacy+0xc   0xdea2100050   0x7ff75d3e1234   
[0x3]   openssl_82bd68d2fab20804!evp_pkey_get_legacy+0xc   0xdea2100070   0x7ff75d3e1234   
[0x4]   openssl_82bd68d2fab20804!evp_pkey_get_legacy+0xc   0xdea2100090   0x7ff75d3e1234   
[0x5]   openssl_82bd68d2fab20804!evp_pkey_get_legacy+0xc   0xdea21000b0   0x7ff75d3e1234   
[0x6]   openssl_82bd68d2fab20804!evp_pkey_get_legacy+0xc   0xdea21000d0   0x7ff75d3e1234   
[0x7]   openssl_82bd68d2fab20804!evp_pkey_get_legacy+0xc   0xdea21000f0   0x7ff75d3e1234   
[0x8]   openssl_82bd68d2fab20804!evp_pkey_get_legacy+0xc   0xdea2100110   0x7ff75d3e1234   
[0x9]   openssl_82bd68d2fab20804!evp_pkey_get_legacy+0xc   0xdea2100130   0x7ff75d3e1234   
(...)

Not very clear where this is even called though. Any help on investigation would be greatly helpful.

[alex]

I wonder if we can reproduce this in CI: https://github.com/sfackler/rust-openssl/blob/master/.github/workflows/ci.yml#L88-L116

[saschanaz]

Good idea, there you go: https://github.com/sfackler/rust-openssl/actions/runs/14978816082/job/42077772096?pr=2407

running 398 tests
test aes::test::ige_vector_1 ... ok
test aes::test::test_wrap_unwrap ... ok
test asn1::tests::asn1_octet_string ... ok
test asn1::tests::bn_cvt ... ok
test asn1::tests::integer_cmp ... ok
error: test failed, to rerun pass `-p openssl --lib`

Caused by:
  process didn't exit successfully: `C:\a\rust-openssl\rust-openssl\target\debug\deps\openssl-c2c0c7ac61ad96c0.exe` (exit code: 0xc0000005, STATUS_ACCESS_VIOLATION)
note: test exited abnormally; to see the full output pass --nocapture to the harness.
Error: Process completed with exit code 1.

[saschanaz]

Oh wait, I found this in the upstream openssl: openssl/openssl#27030

[saschanaz]

When compiling without optimization (/O2) it run successfully without any crash.

Is this something controllable from rust build?

[saschanaz]

Okay, I pointed OPENSSL_DIR to a debug binary and the test does not crash with it.