Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid expensive version calculations during build plan construction #2062

Merged
merged 1 commit into from
Apr 27, 2016

Conversation

sjakobi
Copy link
Member

@sjakobi sjakobi commented Apr 25, 2016

For no-op builds of the these package I see speedups of about 17%. For the stack project it's only around 8%.

@sjakobi sjakobi changed the title Load package caches only when necessary during build plan construction Avoid expensive version calculations during build plan construction Apr 25, 2016
=> ((PackageName -> IO (Set Version)) -> m a)
-> m a
withLoadPackageVersions inner =
join $ runOnce $ do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I think there's a misunderstanding here about runOnce. join . runOnce is essentially id. The action returned by runOnce lets us ask for the value. That action can be run multiple times, but the computation only gets run once (hence the name).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I realized that the runOnce wouldn't help and removed it in my second commit, but I hadn't thought that it was quite so bad!

I'll just squash my two commits and hope that the combination makes more sense.

@mgsloan
Copy link
Contributor

mgsloan commented Apr 25, 2016

Thanks for working on this, I think there's a bit more work to do here (see comment)

This change delays searching the package caches for versions to
the point where the versions are actually used. As the versions
are only used for error messages in the case of version conflicts,
this change results in a noticeable speedup for no-op builds.

While this change does not remove the call to getPackageCaches as
originally intended, the call should not have any performance impact
as the package caches have already been loaded and cached in the
withLoadPackage function that wraps constructPlan.
@sjakobi sjakobi force-pushed the faster-constructplan branch from 8cde223 to a89cd5c Compare April 26, 2016 07:49
@sjakobi
Copy link
Member Author

sjakobi commented Apr 26, 2016

Thanks for working on this, I think there's a bit more work to do here (see comment)

Please take a look at the squashed commit.

@mgsloan
Copy link
Contributor

mgsloan commented Apr 27, 2016

Gotcha! LGTM!

The main point of #1892 is about avoiding loading the cache, not about the expensive computation. This is a good change too, though, so merging

@mgsloan mgsloan merged commit e6536a0 into commercialhaskell:master Apr 27, 2016
@sjakobi sjakobi deleted the faster-constructplan branch April 27, 2016 02:46
@sjakobi
Copy link
Member Author

sjakobi commented Apr 27, 2016

Thanks for the review and the merge!

The main point of #1892 is about avoiding loading the cache, not about the expensive computation.

Do you agree with my point that the call to getPackageCaches in constructPlan is entirely harmless because withLoadPackage (or more specifically withCabalLoader) has already performed the deserialization and stored the caches in the IORef?

In that case I believe the core issue of #1892 is already resolved.

@mgsloan
Copy link
Contributor

mgsloan commented Apr 27, 2016

Ah, I see where some of the confusion around runOnce may have come from - bde3a4#commitcomment-17280506

I have made a change such that withCabalLoader only loads the cache if it actually needs to. The case we're targeting here is where the user hasn't changed any dependencies, and is just doing a rebuild where the local packages are either unmodified or changed in some way. In this case, we should not need the hackage index.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants