incr.comp.: Add new DepGraph implementation.

[michaelwoerister]

This commits does a few things:

1. It adds the new dep-graph implementation -- in addition to the old one. This way we can start testing the new implementation without switching all tests at once.
2. It persists the new dep-graph (which includes query result fingerprints) to the incr. comp. caching directory and also loads this data.
3. It removes support for loading fingerprints of metadata imported from other crates (except for when running autotests). This is not needed anymore with red/green. It could provide a performance advantage but that's yet to be determined. For now, as red/green is not fully implemented yet, the cross-crate incremental tests are disabled.

Note, this PR is based on top of soon-to-be-merged #44696 and only the last 4 commits are new:

- incr.comp.: Initial implemenation of append-only dep-graph. (c90147c)
- incr.comp.: Do some various cleanup. (8ce20c5)
- incr.comp.: Serialize and deserialize new DepGraph. (0e13c1a)
- incr.comp.: Remove support for loading metadata fingerprints. (270a134)
EDIT 2:
- incr.comp.: Make #[rustc_dirty/clean] test for fingerprint equality ... (d8f7ff9)

(EDIT: GH displays the commits in the wrong order for some reason)

Also note that this PR is expected to certainly result in performance regressions in the incr. comp. test cases, since we are adding quite a few things (a whole additional dep-graph, for example) without removing anything. End-to-end performance measurements will only make sense again after red/green is enabled and all the legacy tracking has been turned off.

EDIT 2: Pushed another commit that makes the #[rustc_dirty]/#[rustc_clean] based autotests compared query result fingerprints instead of testing DepNode existence.

[rust-highfive]

r? @eddyb

(rust_highfive has picked a reviewer for you, use r? to override)

[michaelwoerister]

r? @nikomatsakis

[nikomatsakis]

Why have both the debug assert and the bug!?

Also, what is going on here =) -- that is, when exactly are these getting allocated? This is not obvious to me.

[nikomatsakis]

OK, I skimmed the commits. It all looks good to me. I left a question of something I don't understand, but r=me (I'm sure you can explain it to me async =)

[michaelwoerister]

Why have both the debug assert and the bug!?

Oh, that seems to be an oversight.

Also, what is going on here =) -- that is, when exactly are these getting allocated? This is not obvious to me.

Nodes are only allocated via DepGraph::with_task and DepGraph::with_anon_task. For input nodes this is done eagerly at the moment, during HIR mapping (e.g.) and in load_dep_graph (e.g.). With full red/green this could be switched to lazy node allocation for all things that correspond to a query. Not for Hir nodes though, unless we make them query-based too. It does not make much of difference though. Both approaches are clean, IMO.

[bors]

☔ The latest upstream changes (presumably #44696) made this pull request unmergeable. Please resolve the merge conflicts.

[michaelwoerister]

@bors r=nikomatsakis

#44696 has landed and redundant debug_assert! removed. Let's give bors something to do over the weekend.

[bors]

📌 Commit 89aec1e has been approved by nikomatsakis

[bors]

⌛ Testing commit 89aec1e with merge acb73db...

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing acb73db to master...