Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patch for load times with pg_type loading in AR #14642

Merged
merged 3 commits into from
Feb 6, 2019
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Development

### Bug fixes / enhancements
- Add base URL to lockout redirection in static pages ([#14617](https://github.com/CartoDB/cartodb/pull/14617))
- Improve the in_database operations fixing some rails behaviors that were problematic for us ([#14642]https://github.com/CartoDB/cartodb/pull/14642)
- Makes maps listing go faster with related tables (user db size cache issue, #14165)
- Do not redirect to /login by default when error is unknown in network interceptor ([#14616](https://github.com/CartoDB/cartodb/pull/14616))
- Update CARTO.js to v4.1.10
Expand Down
38 changes: 38 additions & 0 deletions config/initializers/zz_patch_activerecord_type_loading.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Activerecord every time it mades a connection brings all the `pg_type`
# data into a cache in order to do the relation between pg types and rails
# types
ethervoid marked this conversation as resolved.
Show resolved Hide resolved
#
# What is the problem?
#
# Well in our users database we have 300K type. AR picks ~142K which leads to
# spent ~1.3s picking the data and ~0.5s in processing those records so we end
ethervoid marked this conversation as resolved.
Show resolved Hide resolved
# spending about ~2s or more without needed in all our in_database operations
ethervoid marked this conversation as resolved.
Show resolved Hide resolved
#
# How this fix works?
ethervoid marked this conversation as resolved.
Show resolved Hide resolved
#
# Well, we filter all the table types (table, analysis tables, overviews) because
# in our app they aren't going to be used and in case they're need, Rails is going
ethervoid marked this conversation as resolved.
Show resolved Hide resolved
# to query the database for it, so no problem.
module ActiveRecord
ethervoid marked this conversation as resolved.
Show resolved Hide resolved
module ConnectionAdapters
module PostgreSQL
module OID # :nodoc:
class TypeMapInitializer # :nodoc:
def query_conditions_for_initial_load(type_map)
known_type_names = type_map.keys.map { |n| "'#{n}'" }
known_type_types = %w('r' 'e' 'd')
ethervoid marked this conversation as resolved.
Show resolved Hide resolved
<<-SQL % [known_type_names.join(", "), known_type_types.join(", ")]
LEFT JOIN pg_type as tt ON (tt.typtype = 'c' AND tt.typarray = t.oid AND tt.typinput = 'record_in(cstring,oid,integer)'::regprocedure)
WHERE
tt.oid is null
AND (t.typname IN (%s)
ethervoid marked this conversation as resolved.
Show resolved Hide resolved
OR t.typtype IN (%s)
ethervoid marked this conversation as resolved.
Show resolved Hide resolved
OR t.typinput = 'array_in(cstring,oid,integer)'::regprocedure
OR t.typelem != 0)
SQL
end
end
end
end
end
end