Skip to content

Commit

Permalink
Improve the records processing time
Browse files Browse the repository at this point in the history
Now it iterates over the same enumerable multiple times without need
which in low records size object is not a big deal but in our case
it takes a considerable amount of time to do it:

Records count: 207
  0.000000   0.010000   0.010000 (  0.002049)
  0.000000   0.000000   0.000000 (  0.002214)
  0.000000   0.000000   0.000000 (  0.001979)
  0.010000   0.000000   0.010000 (  0.002516)
  0.000000   0.000000   0.000000 (  0.002142)
Records count: 142191
  2.050000   0.180000   2.230000 (  2.224884)
  2.090000   0.090000   2.180000 (  2.176774)
  2.050000   0.080000   2.130000 (  2.124951)
  2.140000   0.100000   2.240000 (  2.249662)
  2.040000   0.100000   2.140000 (  2.157886)

  ### NEW METHOD
Records count: 207
  0.010000   0.000000   0.010000 (  0.001821)
  0.010000   0.000000   0.010000 (  0.001999)
  0.000000   0.000000   0.000000 (  0.001983)
  0.000000   0.000000   0.000000 (  0.001906)
  0.000000   0.000000   0.000000 (  0.001829)
Records count: 142191
  1.050000   0.000000   1.050000 (  1.056097)
  1.040000   0.000000   1.040000 (  1.034397)
  1.020000   0.010000   1.030000 (  1.018424)
  1.100000   0.000000   1.100000 (  1.103647)
  1.030000   0.000000   1.030000 (  1.032034)

I've also check the objects allocation and are the same so its worth to
change how this is done
  • Loading branch information
ethervoid committed Feb 1, 2019
1 parent b108c1f commit 15f5b49
Showing 1 changed file with 20 additions and 0 deletions.
20 changes: 20 additions & 0 deletions config/initializers/zz_patch_activerecord_type_loading.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,26 @@ module ConnectionAdapters
module PostgreSQL
module OID # :nodoc:
class TypeMapInitializer # :nodoc:
def run(records)
records.each do |row|
if @store.key? row['oid'].to_i
continue
elsif @store.key? row['typname']
register_mapped_type(row)
elsif row['typtype'] == 'r'.freeze
register_range_type(row)
elsif row['typtype'] == 'e'.freeze
register_enum_type(row)
elsif row['typtype'] == 'd'.freeze
register_domain_type(row)
elsif row['typinput'] == 'array_in'.freeze
register_array_type(row)
elsif row['typelem'].to_i != 0
register_composite_type(row)
end
end
end

def query_conditions_for_initial_load(type_map)
known_type_names = type_map.keys.map { |n| "'#{n}'" }
known_type_types = %w('r' 'e' 'd')
Expand Down

0 comments on commit 15f5b49

Please sign in to comment.