Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Excessive RAM usage while “Updating package index” #3586

Closed
Blaisorblade opened this issue Nov 18, 2017 · 3 comments
Closed

Excessive RAM usage while “Updating package index” #3586

Blaisorblade opened this issue Nov 18, 2017 · 3 comments
Assignees
@snoyberg
Labels
type: performance
Milestone
P2: Should

Comments

@Blaisorblade
Copy link
Collaborator

While running stack install —dry-run lhs2tex on a rather fresh stack install, I noticed: stack using up to ~700 MB RAM, between Updating package index hackage (mirrored at <... S3 URL ...>) and Populated index cache.. RAM usage seemed to be growing linearly with time.

This might be relevant for small machines (say, ARM targets): while installing lhs2tex, surprisingly, GHC (8.0.2) seems to have used less RAM.

@snoyberg answered:

That does sound like a problem. I'm guessing memory fragmentation from usage of bytestring, but it's just a guess. I think it's worth investigating/reporting

This was with Stack 1.5.1 installed from the website (through curl ... | sh) on a 2008 iMac with 2GB RAM, OS X 10.10.
@mgsloan mgsloan added this to the P2: Should milestone Nov 19, 2017
@snoyberg snoyberg self-assigned this Nov 19, 2017
snoyberg added a commit that referenced this issue Nov 29, 2017
@snoyberg
Don't parse cabal files twice
5308723 
This improves on the previous warning hack to keep a cache of parsed
GenericPackageDescriptions, and avoid rerunning hpack.

There are some TODOs added in this commit. One further point of concern:
should we opt-out of caching the results of parsing index files? I'm
imagining that when loading a snapshot, this may result in a lot of
memory usage. (Then again, this may already be the case, see #3586.)
@snoyberg snoyberg mentioned this issue Nov 29, 2017
Merged
2 tasks
@decentral1se
Copy link
Member

So, with #3615 down the hatch, could there be some improvement here?

@snoyberg
Copy link
Contributor

It's still on my radar, but I'm planning on some Stackage infrastructure improvements before I hit this. If someone else wants to take a stab, go for it.

@snoyberg
Copy link
Contributor

This will be addressed by my work on the pantry branch; caching all of Hackage took less than 4mb of resident memory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@snoyberg snoyberg

Labels
type: performance
Projects
None yet
Milestone
P2: Should
Development

No branches or pull requests
4 participants
@mgsloan @snoyberg @Blaisorblade @decentral1se