Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stratosphere takes too much RAM to compile #117

Closed
jdreaver opened this issue Mar 2, 2019 · 5 comments
Closed

Stratosphere takes too much RAM to compile #117

jdreaver opened this issue Mar 2, 2019 · 5 comments

Comments

@jdreaver
Copy link
Contributor

jdreaver commented Mar 2, 2019

We are starting to hit the 4 GB limit of CircleCI on newer GHC versions. In particular, compiling the big Resources module is gnarly. I can think of a few options:

  1. Pay for a larger build machine. We are on the Freckle github org, so maybe we can just use a larger build machine and it will go to our company bill?
  2. Get rid of the big ResourceProperties sum type. We used to want that so FromJSON was easier, but I highly doubt anyone even uses FromJSON. Even so, we could just make ResourceProperties opaque and have each resource produce a ResourceProperties product type itself.
  3. Split up stratosphere? If we do that, Resources.hs will still probably take a ton of RAM unless we do option 2. This option seems highly nontrivial.
  4. Get rid of FromJSON entirely since I doubt anyone uses it. That might make a dent in RAM.
@utdemir
Copy link

utdemir commented Mar 3, 2019

Relevant: #95

@andreyk0
Copy link

andreyk0 commented Mar 7, 2019

We are bumping into this too a bit, option (2) sounds good!

@Znack
Copy link
Contributor

Znack commented Mar 11, 2019

If we get rid of FromJSON we can also consider to implement #80 in new ways :)

@jdreaver
Copy link
Contributor Author

Yeah when I compile on my laptop, even with --fast, stack barely uses memory until we compile Stratosphere.Resources. Then we go above 4.5 GB. I'm leaning towards at removing FromJSON and the big resource sum type.

Thanks for the feedback everyone!

@jdreaver
Copy link
Contributor Author

Fixed via #118 🎉

Simply removing FromJSON fixed everything. I didn't have to touch ResourceProperties at all (although I'm not a fan of that sum type even having to exist...).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants