new-build dependency solver failure with GHC 7.0

[hsenag]

I'm running

The Glorious Glasgow Haskell Compilation System, version 7.0.4
cabal-install version 1.24.0.1
compiled using version 1.24.1.0 of the Cabal library 

This command-line works fine with a clean sandbox:

cabal install HTTP-4000.3.3 --constraint 'time==1.1.2.3' --constraint 'entropy<0.2.2.4' \
        -f-warp-tests --constraint 'HUnit<1.4' --enable-tests

If I unpack HTTP-4000.3.3 and run this command-line in the unpacked folder, it fails in the dependency solver after several minutes:

cabal new-build --constraint 'time==1.1.2.3' --constraint 'entropy<0.2.2.4' \
        -f-warp-tests --constraint 'HUnit<1.4' --enable-tests --max-backjumps=-1

with

Resolving dependencies...
cabal: Could not resolve dependencies:
trying: time-setup.Cabal~>entropy-setup.Cabal-1.20.0.4 (dependency of time-1.1.2.3)
trying: time-setup.time~>entropy-setup.time-1.1.2.3 (dependency of time-1.1.2.3)
Dependency tree exhaustively searched.

I'm not sure how to interpret the error message - the first line suggests it's trying to link the Cabal versions used for time and entropy's setup programs, but I'm not sure what the second line is about at all.

[ezyang]

One thing that pops out to me: GHC 7.0.4 is really old. When I run your repro steps with cabal-install HEAD, I get this:

ezyang@sabre:~/Dev/labs/T4154/HTTP-4000.3.3$ cabal new-build --constraint 'time==1.1.2.3' --constraint 'entropy<0.2.2.4'         -f-warp-tests --constraint 'HUnit<1.4' --enable-tests --max-backjumps=-1 -w ghc-7.0.4
Resolving dependencies...
cabal: Could not resolve dependencies:
trying: HTTP-4000.3.3 (user goal)
trying: random-1.0.0.3/installed-751... (dependency of test-framework-0.8.1.1)
next goal: time (dependency of HTTP-4000.3.3)
rejecting: time-1.2.0.3/installed-349..., time-1.7, time-1.6.0.1, time-1.6,
time-1.5.0.1, time-1.5, time-1.4.2, time-1.4.1, time-1.4.0.2, time-1.4.0.1,
time-1.4, time-1.3, time-1.2.0.5, time-1.2.0.4, time-1.2.0.3, time-1.2.0.2,
time-1.2.0.1, time-1.2, time-1.1.4, time-1.1.3, time-1.1.2.4 (constraint from
command line flag requires ==1.1.2.3)
rejecting: time-1.1.2.3 (conflict: random => time==1.2.0.3/installed-349...)
rejecting: time-1.1.2.2, time-1.1.2.1, time-1.1.2.0, time-1.0 (constraint from
command line flag requires ==1.1.2.3)
Dependency tree exhaustively searched.

[ezyang]

I think the problem is that your time constraint is forcing the dependency resolution for the setup script to not pick the installed version of Cabal, and then a pile of constraints cause things to fail (I think sandboxes are less clever when it comes to setup dependencies). So what you would like is a way to specify the constraint, but only for the main dep solving and not setup deps. I don't know how to do that. See also #3502

[hsenag]

I don't understand the error you get from cabal-install HEAD: can't it just install a new random?

FWIW I'm using an old GHC and time to test that HTTP continues to work with its lower bounds. If I can't make this work I'll just bump them, but I thought it was worth investigating/understanding because the error didn't seem specific to using very old software.

Thanks for looking at this, in any case.

[grayjay]

I think there are two more bugs here. new-build adds default setup dependencies to packages that have build-type: Custom. One of the defaults is time (

cabal/cabal-install/Distribution/Client/ProjectPlanning.hs

Lines 2503 to 2505 in 32cae60

	[ "array", "base", "binary", "bytestring", "containers"
	, "deepseq", "directory", "filepath", "old-time", "pretty"
	, "process", "time", "transformers" ]

), though time-1.1.2.3 also has build-type Custom. This creates a cycle. Then the solver filters the full log incorrectly and doesn't show the most important line. Here is the end of the -v3 log, with the message about cyclic dependencies on the last line:

[109] trying: time-setup.Cabal~>entropy-setup.Cabal-1.20.0.4 (dependency of time-1.1.2.3)
[110] trying: time-setup.Cabal-1.20.0.4:!test
[111] trying: time-setup.template-haskell~>entropy-setup.template-haskell-2.5.0.0/installed-957... (dependency of time-1.1.2.3)
[112] trying: time-setup.ghc-prim~>entropy-setup.ghc-prim-0.2.0.0/installed-d9d... (dependency of time-1.1.2.3)
[113] trying: time-setup.unix~>entropy-setup.unix-2.4.2.0/installed-58b... (dependency of time-1.1.2.3)
[114] trying: time-setup.transformers~>entropy-setup.transformers-0.5.2.0 (dependency of time-1.1.2.3)
[115] trying: time-setup.time~>entropy-setup.time-1.1.2.3 (dependency of time-1.1.2.3)
[116] trying: time-setup.time-1.1.2.3:+split-base
[117] trying: time-setup.old-locale~>entropy-setup.old-locale-1.0.0.2/installed-161... (dependency of time-setup.time-1.1.2.3:+split-base)
[118] trying: time-setup.process~>entropy-setup.process-1.0.1.5/installed-da4... (dependency of time-1.1.2.3)
[119] trying: time-setup.pretty~>entropy-setup.pretty-1.0.1.2/installed-f8d... (dependency of time-1.1.2.3)
[120] trying: time-setup.old-time~>entropy-setup.old-time-1.0.0.6/installed-7f1... (dependency of time-1.1.2.3)
[121] trying: time-setup.filepath~>entropy-setup.filepath-1.2.0.0/installed-b4f... (dependency of time-1.1.2.3)
[122] trying: time-setup.directory~>entropy-setup.directory-1.1.0.0/installed-393... (dependency of time-1.1.2.3)
[123] next goal: time-setup.deepseq (dependency of time-1.1.2.3)
[123] rejecting: time-setup.deepseq~>deepseq-1.4.2.0 (dependencies not linked: cannot merge {*entropy-setup.deepseq-1.3.0.2,time-setup.deepseq-1.3.0.2} and {*deepseq-1.4.2.0}; conflict set: deepseq, entropy-setup.deepseq, entropy-setup.time, time-setup.Cabal, time-setup.deepseq, time-setup.time)
[123] trying: time-setup.deepseq~>entropy-setup.deepseq-1.3.0.2
[124] trying: time-setup.containers~>entropy-setup.containers-0.4.0.0/installed-b48... (dependency of time-1.1.2.3)
[125] trying: time-setup.bytestring~>entropy-setup.bytestring-0.9.1.10/installed-6aa... (dependency of time-1.1.2.3)
[126] trying: time-setup.binary~>entropy-setup.binary-0.8.2.1 (dependency of time-1.1.2.3)
[127] trying: time-setup.binary-0.8.2.1:!bench
[128] trying: time-setup.binary-0.8.2.1:!test
[129] next goal: time-setup.array (dependency of time-1.1.2.3)
[129] rejecting: time-setup.array~>entropy-setup.array-0.3.0.2/installed-143... (cyclic dependencies; conflict set: time-setup.Cabal, time-setup.time)

Even the full log is a little misleading, because the solver doesn't complain about the cycle until after it has chosen all of the dependencies. One solution would be to check for cycles at every step.

I'm not sure what we should do about the default setup dependencies.

/cc @dcoutts

EDIT: I ran cabal with --max-backjumps=0, which is why the log stopped after the first error.

[hsenag]

@grayjay thanks for the explanation - which also points to a simpler repro.

I see the same problem either building time-1.1.2.3 itself, or a fresh cabal project that depends on time, both with the --constraint time==1.1.2.3' specification. Specifying a dependency on time==1.1.2.3` in the fresh cabal project itself is fine as we might expect, as it doesn't constrain the setup dependency.

[grayjay]

Yes, when time's setup script depends on the installed time, it isn't considered a cycle.

I just realized that the log actually refers to a cycle between Cabal and time. That seems harder to fix, because it's a real cycle: time has a setup dependency on Cabal, and Cabal has a regular dependency on time. The solver currently doesn't detect cycles involving just one package: #4161

[hsenag]

Isn't this a general bootstrapping problem? For example if I unpack Cabal itself locally, and try to new-build it with a constraint on the same version (e.g. Cabal==1.24.1.0) it looks like the solver also fails.

I think as @ezyang says we need a way to specify the constraint without constraining setup deps. In fact I'd say that should be the default, and we'd need something explicit to constrain setup deps. Otherwise you're always risking confusion if you try to constrain something that happens to be a dep of Cabal.

[grayjay]

I think as @ezyang says we need a way to specify the constraint without constraining setup deps. In fact I'd say that should be the default, and we'd need something explicit to constrain setup deps. Otherwise you're always risking confusion if you try to constrain something that happens to be a dep of Cabal.

I think it makes sense to only constrain the main dependencies by default, to avoid interfering with bootstrapping.