Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make /admin lighter #542

Closed
matschaffer opened this issue Apr 10, 2019 · 5 comments
Closed

Make /admin lighter #542

matschaffer opened this issue Apr 10, 2019 · 5 comments
Assignees

Comments

@matschaffer
Copy link
Contributor

We have railsadmin set up at /admin but it tries to count measurements when you load it (very slow, too expensive to load on dev envs).

It's possible #529 might also fix this if we do an ActiveRecord count cache and RailsAdmin uses that by default.

Either way, we can't count the measurement table for ever admin page load. Too expensive in general.

@matschaffer matschaffer added Help Wanted A good place to get started. These issues should require minimal knowledge of safecast's domain. project:api labels Apr 10, 2019
@matschaffer
Copy link
Contributor Author

Looks like this is largely due to railsadmin attempting to count large tables.

One query I caught was SELECT COUNT(*) FROM "bgeigie_logs"; which the db says looks like this:

safecast=> explain analyze SELECT COUNT(*) FROM "bgeigie_logs";
                                                             QUERY PLAN

-------------------------------------------------------------------------------------------------------------------------
------------
 Aggregate  (cost=3223832.62..3223832.63 rows=1 width=0) (actual time=142717.206..142717.206 rows=1 loops=1)
   ->  Seq Scan on bgeigie_logs  (cost=0.00..3013346.50 rows=84194450 width=0) (actual time=0.007..123746.505 rows=841585
42 loops=1)
 Planning time: 0.589 ms
 Execution time: 142717.299 ms
(4 rows)

railsadminteam/rails_admin#2968 mentions a limited_pagination setting we may be able to place on our larger tables to avoid trying to perfectly number all pages in the display.

https://wiki.postgresql.org/wiki/Count_estimate is also another interesting technique, but I suspect it'd be hard to wedge into rails admin.

But it does seem to offer a row estimate that's < 1% off from the actual row count:

safecast=> SELECT COUNT(*) FROM "bgeigie_logs";
  count
----------
 84164952
(1 row)

safecast=> explain SELECT COUNT(*) FROM "bgeigie_logs";
                                  QUERY PLAN
-------------------------------------------------------------------------------
 Aggregate  (cost=3224082.05..3224082.06 rows=1 width=0)
   ->  Seq Scan on bgeigie_logs  (cost=0.00..3013579.64 rows=84200964 width=0)
(2 rows)

@matschaffer
Copy link
Contributor Author

We'll need to upgrade rails admin to at least https://github.com/sferik/rails_admin/blob/master/CHANGELOG.md#130---2018-02-18 for that, but probably time for an upgrade (still on 0.8.1)

@matschaffer
Copy link
Contributor Author

Once we're upgraded it looks like the config is probably this:

  config.model 'BgeigieLog' do
    list do
      limited_pagination
    end
  end

  config.model 'Measurement' do
    list do
      limited_pagination
    end
  end

@robouden
Copy link
Contributor

robouden commented May 4, 2019

This is already implemented?

@matschaffer
Copy link
Contributor Author

Not yet, just gathering details. Looks like if we do the above we can probably get /admin working without bogging down the DB

@eitoball eitoball self-assigned this May 23, 2019
@matschaffer matschaffer removed the Help Wanted A good place to get started. These issues should require minimal knowledge of safecast's domain. label Jun 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants