Stack integration test suite failing for master branch

[borsboom]

General summary/comments

Observed on Linux and macOS (not tested on any other platforms). This blocks the release of stack-1.8.x.

Steps to reproduce

Run:

stack install && stack test --flag stack:integration-tests

Expected

All integration tests pass.

Actual

Some of the integration tests fail:

  test/integration/IntegrationSpec.hs:77: 
  1) 2195-depend-on-exe
       uncaught exception: TestFailure (Exited with ExitFailure 1
       
       stdout:
       
       
       stderr:
       Running: /builds/fpco-mirrors/stack/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/bin/stack build
       Downloading lts-11.6 build plan ...
       Downloaded lts-11.6 build plan.
       Populating index cache ...
       Populated index cache.
       [1 of 2] Compiling Main             ( /tmp/stackhome5455/.stack/setup-exe-src/setup-mPHDZzAJ.hs, /tmp/stackhome5455/.stack/setup-exe-src/setup-mPHDZzAJ.o )
       [2 of 2] Compiling StackSetupShim   ( /tmp/stackhome5455/.stack/setup-exe-src/setup-shim-mPHDZzAJ.hs, /tmp/stackhome5455/.stack/setup-exe-src/setup-shim-mPHDZzAJ.o )
       Linking /tmp/stackhome5455/.stack/setup-exe-cache/x86_64-linux/tmp-Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2 ...
       mtl-2.2.2: download
       mtl-2.2.2: configure
       Cabal-2.0.1.1: download
       mtl-2.2.2: build
       Cabal-2.0.1.1: configure
       Cabal-2.0.1.1: build
       mtl-2.2.2: copy/register
       Cabal-2.0.1.1: copy/register
       happy-1.19.9: download
       happy-1.19.9: configure
       happy-1.19.9: build
       happy-1.19.9: copy/register
       files-0.1.0.0: configure (lib)
       Configuring files-0.1.0.0...
       Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2: Encountered missing dependencies:
       happy -any
       Completed 4 action(s).
       
       --  While building package files-0.1.0.0 using:
             /tmp/stackhome5455/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2 --builddir=.stack-work/dist/x86_64-linux/Cabal-2.0.1.0 configure --with-ghc=/builds/fpco-mirrors/stack/.stack-root/programs/x86_64-linux/ghc-8.2.2/bin/ghc --with-ghc-pkg=/builds/fpco-mirrors/stack/.stack-root/programs/x86_64-linux/ghc-8.2.2/bin/ghc-pkg --user --package-db=clear --package-db=global --package-db=/tmp/stackhome5455/.stack/snapshots/x86_64-linux/lts-11.6/8.2.2/pkgdb --package-db=/tmp/stack-integration-2195-depend-on-exe5455/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/pkgdb --libdir=/tmp/stack-integration-2195-depend-on-exe5455/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/lib --bindir=/tmp/stack-integration-2195-depend-on-exe5455/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/bin --datadir=/tmp/stack-integration-2195-depend-on-exe5455/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/share --libexecdir=/tmp/stack-integration-2195-depend-on-exe5455/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/libexec --sysconfdir=/tmp/stack-integration-2195-depend-on-exe5455/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/etc --docdir=/tmp/stack-integration-2195-depend-on-exe5455/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/doc/files-0.1.0.0 --htmldir=/tmp/stack-integration-2195-depend-on-exe5455/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/doc/files-0.1.0.0 --haddockdir=/tmp/stack-integration-2195-depend-on-exe5455/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/doc/files-0.1.0.0
           Process exited with code: ExitFailure 1
       Main.hs: Expected a warning: 
       "package provides no library"
       CallStack (from HasCallStack):
         error, called at /builds/fpco-mirrors/stack/test/integration/tests/2195-depend-on-exe/Main.hs:11:10 in main:Main
       )

  test/integration/IntegrationSpec.hs:77: 
  2) 3926-ghci-with-sublibraries
       uncaught exception: TestFailure (Exited with ExitFailure 1
       
       stdout:
       
       
       stderr:
       Running: /builds/fpco-mirrors/stack/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/bin/stack clean
       Populating index cache ...
       Populated index cache.
       Copy "src/Lib.v1" to "src/Lib.hs"
       Copy "src-internal/Internal.v1" to "src-internal/Internal.hs"
       Running: /builds/fpco-mirrors/stack/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/bin/stack repl
       Copy "src-internal/Internal.v2" to "src-internal/Internal.hs"
       Copy "src/Lib.v2" to "src/Lib.hs"
       Main.hs: fd:13: hGetChar: end of file
       )

  test/integration/IntegrationSpec.hs:77: 
  3) 3942-solver-error-output
       uncaught exception: TestFailure (Exited with ExitFailure 1
       
       stdout:
       
       
       stderr:
       Running: /builds/fpco-mirrors/stack/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/bin/stack ./script.hs
       Compiler version mismatched, found ghc-8.2.2 (x86_64), but expected minor version match with ghc-8.0.2 (x86_64) (based on resolver setting in /builds/fpco-mirrors/stack/test/integration/tests/3942-solver-error-output/files/test-stack.yml).
       To install the correct GHC into /tmp/stackhome5455/.stack/programs/x86_64-linux/, try running "stack setup" or use the "--install-ghc" flag.
       Main.hs: Expected a warning: 
       "While constructing the build plan, the following exceptions were encountered:"
       CallStack (from HasCallStack):
         error, called at /builds/fpco-mirrors/stack/test/integration/tests/3942-solver-error-output/Main.hs:16:11 in main:Main
       )

Stack version

master branch, as of commit bcbfb1e.

Method of installation

Build from source.

[mihaimaruseac]

The first test failure is my mistake, I was too eager on #4111. I opened #4146 to fix it and will bisect the other in the next week or so (travelling, so not enough time to do it fast :( )

[mihaimaruseac]

Fixed the second test failure in #4149 and I guess the last one is just a matter of changing the resolver at https://github.com/commercialhaskell/stack/pull/4114/files#diff-d4f605d4bd11f79da81131332c55f32aR1 Will do that in around 12-13 hours from now.

[mattaudesse]

I've gone ahead and merged @mihaimaruseac's fixes in #4146, #4149, and #4152 (thanks Mihai).

I'll confirm that all the tests pass once the CI builds have completed then close this out.

[mihaimaruseac]

Thanks @mattaudesse. Did you get a chance to check if CI builds and passes all tests?

[mattaudesse]

Hey @mihaimaruseac - I think we're all set but I ran out of time last night trying to verify the integration tests succeed locally.

Using FreeBSD might be adding some additional variables to the mix in my case so I'm going to re-run in a Debian VM and see how it goes. I have to leave my computer for a few hours but hopefully I'll return to a 100% successful test run.

Or you guys can just close this out if you know it's working - sorry for the delay!

[mattaudesse]

Cool, I'm seeing fully successful integration test runs now - closing.