Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Relation and conditions to fix mutating scopes issues #268

Merged
merged 3 commits into from
Mar 30, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,11 +211,11 @@ Country#name= # => sets the name
The ActiveHash::Base.all method functions like an in-memory data store. You can save your records as ActiveHash::Relation object by using standard ActiveRecord create and save methods:
```ruby
Country.all
=> #<ActiveHash::Relation:0x00007f861e043bb0 @klass=Country, @all_records=[], @query_hash={}, @records_dirty=false>
=> #<ActiveHash::Relation:0x00007f861e043bb0 @klass=Country, @all_records=[], @conditions=[..], @records_dirty=false>
Country.create
=> #<Country:0x00007f861b7abce8 @attributes={:id=>1}>
Country.all
=> #<ActiveHash::Relation:0x00007f861b7b3628 @klass=Country, @all_records=[#<Country:0x00007f861b7abce8 @attributes={:id=>1}>], @query_hash={}, @records_dirty=false>
=> #<ActiveHash::Relation:0x00007f861b7b3628 @klass=Country, @all_records=[#<Country:0x00007f861b7abce8 @attributes={:id=>1}>], @conditions=[..], @records_dirty=false>
country = Country.new
=> #<Country:0x00007f861e059938 @attributes={}>
country.new_record?
Expand All @@ -225,7 +225,7 @@ country.save
country.new_record?
# => false
Country.all
=> #<ActiveHash::Relation:0x00007f861e0ca610 @klass=Country, @all_records=[#<Country:0x00007f861b7abce8 @attributes={:id=>1}>, #<Country:0x00007f861e059938 @attributes={:id=>2}>], @query_hash={}, @records_dirty=false>
=> #<ActiveHash::Relation:0x00007f861e0ca610 @klass=Country, @all_records=[#<Country:0x00007f861b7abce8 @attributes={:id=>1}>, #<Country:0x00007f861e059938 @attributes={:id=>2}>], @conditions=[..], @records_dirty=false>
```
Notice that when adding records to the collection, it will auto-increment the id for you by default. If you use string ids, it will not auto-increment the id. Available methods are:
```
Expand Down
2 changes: 2 additions & 0 deletions lib/active_hash.rb
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@

require 'active_hash/base'
require 'active_hash/relation'
require 'active_hash/condition'
require 'active_hash/conditions'
require 'active_file/multiple_files'
require 'active_file/hash_and_array_files'
require 'active_file/base'
Expand Down
46 changes: 3 additions & 43 deletions lib/active_hash/base.rb
Original file line number Diff line number Diff line change
Expand Up @@ -21,50 +21,8 @@ class FileTypeMismatchError < StandardError
end

class Base

class_attribute :_data, :dirty, :default_attributes, :scopes

class WhereChain
def initialize(scope)
@scope = scope
@records = @scope.all
end

def not(options)
return @scope if options.blank?

# use index if searching by id
if options.key?(:id) || options.key?("id")
ids = @scope.pluck(:id) - Array.wrap(options.delete(:id) || options.delete("id"))
candidates = ids.map { |id| @scope.find_by_id(id) }.compact
end

filtered_records = (candidates || @records || []).reject do |record|
options.present? && match_options?(record, options)
end

ActiveHash::Relation.new(@scope.klass, filtered_records, {})
end

def match_options?(record, options)
options.all? do |col, match|
if match.kind_of?(Array)
match.any? { |v| normalize(v) == normalize(record[col]) }
else
normalize(record[col]) == normalize(match)
end
end
end

private :match_options?

def normalize(v)
v.respond_to?(:to_sym) ? v.to_sym : v
end

private :normalize
end

if Object.const_defined?(:ActiveModel)
extend ActiveModel::Naming
include ActiveModel::Conversion
Expand Down Expand Up @@ -205,7 +163,9 @@ def create!(attributes = {})
end

def all(options = {})
ActiveHash::Relation.new(self, @records || [], options[:conditions] || {})
relation = ActiveHash::Relation.new(self, @records || [])
relation = relation.where!(options[:conditions]) if options[:conditions]
relation
Comment on lines +166 to +168
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a reason to change this block?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose the Relation could be initialized with the conditions directly.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The initializer has:

self.conditions = Conditions.wrap(conditions || [])

I wondered if there was a reason to not use that.
The new code passes a nil (so it gets and empty array)
and the where! then adds to that conditions.

Is this to ensure we don't modify the options[:conditions]?

I kinda like the original way better but if this has a reason then 👍

end

delegate :where, :find, :find_by, :find_by!, :find_by_id, :count, :pluck, :ids, :pick, :first, :last, :order, to: :all
Expand Down
44 changes: 44 additions & 0 deletions lib/active_hash/condition.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
class ActiveHash::Relation::Condition
attr_reader :constraints, :inverted

def initialize(constraints)
@constraints = constraints
@inverted = false
end

def invert!
@inverted = !inverted

self
end

def matches?(record)
match = begin
return true unless constraints

expectation_method = inverted ? :any? : :all?

constraints.send(expectation_method) do |attribute, expected|
value = record.public_send(attribute)

matches_value?(value, expected)
end
end

inverted ? !match : match
end

private

def matches_value?(value, comparison)
return comparison.any? { |v| matches_value?(value, v) } if comparison.is_a?(Array)
return comparison.include?(value) if comparison.is_a?(Range)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you squash the third commit into this commit.

no reason to introduce a bug to then fix it in a later commit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but if this PR is eventually merged via a single squashed commit, I think it's good to keep the individual commit in this PR for future clarity.

Happy to squash if you guys plan merging via merge-commits and not squash and merge strategy.

return comparison.match?(value) if comparison.is_a?(Regexp)

normalize(value) == normalize(comparison)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you talk more on this?

The first thing I tested was:

(1.0).to_s == 1.to_s

But cases like this work great (which are probably more relevant since this will fix the very common find_by(id: "4"):

"4".to_s == "4".to_s

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes; this is to make ActiveHash behave similar to how ActiveRecord behaves for querying.

ActiveRecord (or rather the DB) does a similar typecasting when querying:

# AR
User.find("2").id # => 2
User.find_by(id: "2").id # => 2
User.find_by(username: :john).username # => "john"

The normalize with typecasting to string here makes ActiveHash behave similar:

# AH
City.find("2").id # => 2
City.find_by(id: "2").id # => 2
City.find_by(country_code: :gb).country_code # => "gb"

.. without this normalization, no records would be returned in the examples above for AH, while it would for AR.

end

def normalize(value)
value.respond_to?(:to_s) ? value.to_s : value
end
end
21 changes: 21 additions & 0 deletions lib/active_hash/conditions.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
class ActiveHash::Relation::Conditions
attr_reader :conditions

delegate_missing_to :conditions
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trying to wrap my mind around this one.
(the future commit does a little more explaining)

I guess real active record does something similar any time you run methods on relations.

Can you just put a comment to explain this a little more?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. This class is an delegator to an array of conditions. It delegates methods to the array of conditions, but also implements #matches? which is a short-hand for array_of_conditions.all? { |c| c.matches?(record) } and makes it a bit cleaner to work with in the Relation.

It also adds .wrap which is similar to Array.wrap to 'wrap' an array of conditions (or return itself if already wrapped).

Happy to add comments to the code, if you believe this would make it clearer.


def initialize(conditions = [])
@conditions = conditions
end

def matches?(record)
conditions.all? do |condition|
condition.matches?(record)
end
end

def self.wrap(conditions)
return conditions if conditions.is_a?(self)

new(conditions)
end
end
161 changes: 84 additions & 77 deletions lib/active_hash/relation.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,24 +7,80 @@ class Relation
delegate :empty?, :length, :first, :second, :third, :last, to: :records
delegate :sample, to: :records

def initialize(klass, all_records, query_hash = nil)
attr_reader :conditions, :order_values, :klass, :all_records

def initialize(klass, all_records, conditions = nil, order_values = nil)
self.klass = klass
self.all_records = all_records
self.query_hash = query_hash
self.records_dirty = false
self.conditions = Conditions.wrap(conditions || [])
self.order_values = order_values || []
end

def where(conditions_hash = :chain)
return WhereChain.new(self) if conditions_hash == :chain

spawn.where!(conditions_hash)
end

class WhereChain
attr_reader :relation

def initialize(relation)
@relation = relation
end

def not(conditions_hash)
relation.conditions << Condition.new(conditions_hash).invert!
relation
end
end

def order(*options)
spawn.order!(*options)
end

def reorder(*options)
spawn.reorder!(*options)
end

def where!(conditions_hash, inverted = false)
self.conditions << Condition.new(conditions_hash)
self
end

def where(query_hash = :chain)
return ActiveHash::Base::WhereChain.new(self) if query_hash == :chain
def spawn
self.class.new(klass, all_records, conditions, order_values)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this ok to not dup conditions or order_calues?

I think all_records is fine since this is not modifying that.


thinking out loud (and not a request):
Also feels like we should be able to avoid all this passing around all_records.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking out loud (and not a request): Also feels like we should be able to avoid all this passing around all_records.

Passing around all_records is to support invert_where and other unscoping methods.

In theory, it could probably be possible to re-load all the records from the main class when a relation is unscoped, but I have a feeling that it would make the behavior of calling reload on a chained relation behave unexpected.

Let me know if this is something I should look into.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks. if it ain't broke...

end

def order!(*options)
check_if_method_has_arguments!(:order, options)
self.order_values += preprocess_order_args(options)
self
end

def reorder!(*options)
check_if_method_has_arguments!(:order, options)

self.order_values = preprocess_order_args(options)
@records = apply_order_values(records, order_values)

self.records_dirty = true unless query_hash.nil? || query_hash.keys.empty?
self.query_hash.merge!(query_hash || {})
self
end

def records
@records ||= begin
filtered_records = apply_conditions(all_records, conditions)
ordered_records = apply_order_values(filtered_records, order_values) # rubocop:disable Lint/UselessAssignment
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems it would have been easier to remove ordered_records than add the rubocop line

Is this for future debugging or something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, probably had a breakpoint here at one point to inspect :-)

end
end

def reload
@records = nil # Reset records
self
end

def all(options = {})
if options.has_key?(:conditions)
if options.key?(:conditions)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You did not write this code, so probably ignore this but...

is there a reason to have the conditional here instead of an ||?

where(options[:conditions] || {})

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There would be a difference when passing an explicit false-y (nil, false, ..) value as the :condition, right?

But not sure what effect it has for the finders though.

where(options[:conditions])
else
where({})
Expand Down Expand Up @@ -58,10 +114,11 @@ def find(id = nil, *args, &block)
end

def find_by_id(id)
return where(id: id).first if query_hash.present?

index = klass.send(:record_index)[id.to_s] # TODO: Make index in Base publicly readable instead of using send?
index and records[index]
return unless index

record = all_records[index]
record if conditions.matches?(record)
end

def count
Expand All @@ -84,86 +141,32 @@ def pick(*column_names)
pluck(*column_names).first
end

def reload
@records = filter_all_records_by_query_hash
end

def order(*options)
check_if_method_has_arguments!(:order, options)
relation = where({})
return relation if options.blank?

processed_args = preprocess_order_args(options)
candidates = relation.dup

order_by_args!(candidates, processed_args)

candidates
end

def to_ary
records.dup
end

def method_missing(method_name, *args)
return super unless self.klass.scopes.key?(method_name)
return super unless klass.scopes.key?(method_name)

instance_exec(*args, &self.klass.scopes[method_name])
instance_exec(*args, &klass.scopes[method_name])
end

attr_reader :query_hash, :klass, :all_records, :records_dirty

private

attr_writer :query_hash, :klass, :all_records, :records_dirty

def records
if !defined?(@records) || @records.nil? || records_dirty
reload
else
@records
end
def respond_to_missing?(method_name, include_private = false)
klass.scopes.key?(method_name) || super
end

def filter_all_records_by_query_hash
self.records_dirty = false
return all_records if query_hash.blank?

# use index if searching by id
if query_hash.key?(:id) || query_hash.key?("id")
ids = (query_hash.delete(:id) || query_hash.delete("id"))
ids = range_to_array(ids) if ids.is_a?(Range)
candidates = Array.wrap(ids).map { |id| klass.find_by_id(id) }.compact
end
private

return candidates if query_hash.blank?
attr_writer :conditions, :order_values, :klass, :all_records

(candidates || all_records || []).select do |record|
match_options?(record, query_hash)
end
end
def apply_conditions(records, conditions)
return records if conditions.blank?

def match_options?(record, options)
options.all? do |col, match|
if match.kind_of?(Array)
match.any? { |v| normalize(v) == normalize(record[col]) }
else
normalize(match) === normalize(record[col])
end
records.select do |record|
conditions.matches?(record)
end
end

def normalize(v)
v.respond_to?(:to_sym) ? v.to_sym : v
end

def range_to_array(range)
return range.to_a unless range.end.nil?

e = records.last[:id]
(range.begin..e).to_a
end

def check_if_method_has_arguments!(method_name, args)
return unless args.blank?

Expand All @@ -179,7 +182,9 @@ def preprocess_order_args(order_args)
ary.map! { |e| e.split(/\W+/) }.reverse!
end

def order_by_args!(candidates, args)
def apply_order_values(records, args)
ordered_records = records.dup

args.each do |arg|
field, dir = if arg.is_a?(Hash)
arg.to_a.flatten.map(&:to_sym)
Expand All @@ -189,14 +194,16 @@ def order_by_args!(candidates, args)
arg.to_sym
end

candidates.sort! do |a, b|
ordered_records.sort! do |a, b|
if dir.present? && dir.to_sym.upcase.equal?(:DESC)
b[field] <=> a[field]
else
a[field] <=> b[field]
end
end
end

ordered_records
end
end
end
Loading