Mongoid support?

[tetherit]

It would be great if this could support Mongoid.

[radar]

Given that Ransack is built on top of ARel and that ARel only works with relational databases, I don't see how we could add Mongoid support without dramatically changing everything.

[brianp]

Not to mention Mongoid doesn't support things like association searching etc. There just incompatible systems (right now at least)

[kristianmandrup]

It would be quite feasible to port at least some (the main parts) of the Ransack functionality to non-relational data mappers such as mongoid. I think association searching can be done in mongoid sth like this?:

Post.where('category.name': name) or Post.where('category.level'.gte: level)

The search form could simply build up the hash to the where clause. See: http://mongoid.org/en/mongoid/docs/querying.html

Band.
  where(:founded.gte => "1980-1-1").
  in(name: [ "Tool", "Deftones" ]).
  union.
  in(name: [ "Melvins" ])

With each chained method on a criteria, a newly cloned criteria is returned with the new query added. This is so that with scoping or exposures, for example, the original queries are not modified and reusable.

Most of Mongoid's criteria has been extracted into it's own gem, Origin, which Mongoid now depends on for most of it's API. A complete list of commands can be found in Selection with Origin and Options with Origin.

http://mongoid.org/en/origin/docs/selection.html

http://mongoid.org/en/origin/docs/options.html

Would be a fun to do a POC. Anyone?

[brianp]

Your example:

Post.where('category.name': name)

Will only work for embedded documents. Not related documents.

Although for non related documents I think your method could work.

[kristianmandrup]

Yeah :) and I think you would rarely want/need to search on relations. It usually makes more sense to search on embedded docs or perhaps eve parent doc. Items such as category and tags are often embedded directly as attributes for speed concerns. Conclusion: Would/could work fine for most real-life cases I believe...

On another note, I wonder if it would makes sense with a ransack-tire integration or similar? Would be much faster/more efficient to search on a pre-index data set I believe? Not sure about the limitations though.

[brianp]

a POF without tire would be a good spot to start still. Maybe keep it in mind but worry about making it faster later.

We use lots of related documents in our apps but I don't think it's any reason not to try. Having something that acts on embedded documents only is still more then having nothing at all.

I'd assume you originally looked at using ransack for a particular purpose. Could we extract your use case as the basis to start something?

Edit: Just realized you weren't the OP. So where to start?

[kristianmandrup]

About the relationships. I think one way to solve this, would be to hack into :after_save, determine which of the updated attributes are relations, and then for the ones of interest (as defined by some new class level macro), update some search specific fields on the object related to the relationship.

mongoid_doc :post do
  field :name, type: String

  belongs_to :authors, class_name: 'User'
  has_many :reviewers, class_name: 'User'

  enable_ransack # includes Ransack macro module

  # Ransack macro module makes the macro #search_field available
  # creates embedded doc called 'search_reviewers' 
  # of class SearchReview (unless class already exists)
  # with fields name and rating
  # this embedded doc will be auto-updated on after-save
  search_field :reviewers do
    index :name, :rating
  end
end

Behind the scenes... (handled by mongoid-ransack)

after_save do
  ransack_attributes
end

def ransack_attributes
  attributes.each do |att|
    if search_attributes.include?(att) && relationship?(att)
      update_search_field att
    end
end

I think this would work beautifully :)

Perhaps even with longer relationship chains?

  search_field :reviewers do
    index :name, :rating

    search_field :boss, for: %w{name}    
  end

What do you think?

I don't have time at hand to start this solution right now. I think a way to start would be to make a fork of ransack (in order to reuse what makes sense), then strip out all the Arel related code and rework it using Mongoid Criterias, some meta-magic and macros ;)

In fact, I think it would make good sense to extract the ransack form builder into a separate gem: ransack-form-mongoid and ransack-mongoid? Just an idea...

A good starting point would be to implement this macro code in order to dynamically build the embedded docs from relations for use in searching. I have some macro code for generating classes that you can find fx in my imperator-ext repo under class_factory.rb and method_factory.rb. Have a look. Then look into the mongoid code and docs in order to see how to get the model meta data needed.

[kristianmandrup]

I've started a mongoid fork here.

https://github.com/kristianmandrup/ransack-mongoid

[ernie]

Assuming you guys feel what you want to do could be accomplished inside an adapter, a specialized gem shouldn't be necessary. I'll admit I'm a bit skeptical, given the differences between an RDBMS and a document store, but would be exciting to have someone implement a new adapter, and would be happy to accept a pull request.

The trick, as Ryan pointed out, is that a lot of the core concepts are ARel-centric. That being said, the building and executing of queries based on incoming search params is confined to the adapters.

[kristianmandrup]

In my ransack-mongoid fork I have so far created a very early stage mongoid adapter (files/folder structure) - removed Arel specific code. I think we should start off without supporting relations, just simply building the mongoid search DSL from the params[:q]. Then gradually add more complex functionality as we progress.

[kristianmandrup]

I've done a lot of work on it this afternoon. Mostly what is needed is implementation of context.rb for the mongoid adapter.
The rest of the code is only marginally linked to Arel. Looking good...

Would be much appreciated if someone could help me understand context.rb and perhaps help me implement it or provide some guidance! Also, where exactly is the search clause being built? Looks like configuration.rb, in the #add_predicate method?
I've added a negation option in constants.rb see http://mongoid.org/en/origin/docs/selection.html#negation

queryable.not.where(name: "Dave")
queryable.selector #=> { "$not" => { "name" => "Dave" }}

So I need to insert a call to #not before the call to #where (or whatever operation) when negation: true. How?
Thanks :)

[tetherit]

Thank you for doing all this work, didn't expect such immediate response :) - I'm only really playing with MongoDB and Mongoid right now and it sure makes it easier not having to deal with migrations. I've come across ransack in a railscasts episode so it's great to see it will soon be usable with mongoid, even if the functionality will be limited at first. Thank you again!

[ernie]

@kristianmandrup the clause itself is built by the evaluate method of the context. The default implementation uses a visitor that visits the Ransack objects and emits ARel nodes, but that's not necessary.

I'm glad you've been able to make progress. I specifically did as much as I could (within certain unfortunate time constraints, since I had an actual need for this code, hence the less-than-stellar spec coverage :( ) to separate concerns in the Ransack code, to enable this sort of thing via appropriate adapters. If you succeed in writing a new adapter for Ransack, it'd be a good proof of that design goal being met. :)

[kristianmandrup]

I am really strughling with the context of the adapter. Could you please refactor it to make it more clear, possibly dividing it into some smaller helpers (classes?) and documenting the main methods... possibly making clear what is core functionality and what is optional "extras"? joins/relations fx?

Some of the if tests would benefit from "extract_method" refactor pattern IMO.

Very cryptic otherwise...

elsif (segments = str.split(/_/)).size > 1# huh?

while !found_assoc && remainder.unshift(segments.pop) && segments.size > 0 do # huh?

if Class === obj && ::ActiveRecord::Base > obj# huh?

Besides some of these tests are duplicated in at least 2 methods, another reason to extract_method.

Also some of the methods are very "long", possibly breaking Single Responsibility I would think. I rarely have methods of more than 3-5 lines myself :P

Cheers!

[ernie]

@kristianmandrup As I mentioned, I had some time constraints on this code that led to some of the lack of documentation you see here.

I don't, personally, find the lines you referenced terribly confusing, however, and extract method would require the passing of a bunch of variables to the extracted method, as well as mutating the input (in the second case), which I find distasteful.

To explain the three things you referenced:

The general algorithm for mapping an "attribute" name to an association/associations and attribute is recursive -- the logic is as follows:

1.If the full string name of the method maps directly to an attribute of the class being used as the current base, this is the attribute being searched on, stop there.
2. Otherwise, if the string contains underscores, it's possible that the attribute is meant to reference an attribute of an association. As such, split on the underscores and begin stripping away one trailing piece at a time into the "remainder", to arrive at the most specific association name that would match an association from the current base. This way, we match, for example, a hm:t association named user_articles before instead matching user and then articles.
3. Once/if an association is found, use this association's class as the new base class, and seek for an attribute on this class, starting at step 1.

In the third if statement you referenced, we're checking if the object is a descendant of AR::Base if it's a Class.

[ernie]

Regarding the API expected of a context, the public methods as implemented are the expected API, and the private ones are simply implementation details. As you can see from table_for, there is, as @radar mentioned earlier, an expected dependence on a relational DB, however, you could return, instead, any object that implements the [] method to create a searchable attribute.

[kristianmandrup]

Hi ernie,

Thanks for the tips!

I think it would be nice with two context classes, one building on top of the other (or some derivation thereof).

The base_context.rb could be a really simple one without support for associations. This would be the one I would first try to extend for use with mongoid.

The associations_context.rb would be one which also handles associations like you mention.

Then it would be much clearer exactly what functionality is 'core' and so on. Also the join (relational db?) specific methods could be extracted into a separate module or sth. I prefer a much stricter division of concerns, as it makes it easier to understand and extend the code IMO. Thanks again :)

I will try to refactor the code so I am better able to understand it.

[ernie]

It should be possible for you to implement things like attribute_method? in a non-relationally-relevant way from a duck-typing standpoint, without resorting to inheritance or DI to accomplish.

On Aug 13, 2012, at 10:06 AM, Kristian Mandrup [email protected] wrote:

Hi ernie,

Thanks for the tips!

I think it would be nice with two context classes, one building on top of the other (or some derivation thereof).

The base_context.rb could be a really simple one without support for associations. This would be the one I would first try to extend for use with mongoid.

The associations_context.rb would be one which also handles associations like you mention.

Then it would be much clearer exactly what functionality is 'core' and so on. Also the join (relational db?) specific methods could be extracted into a separate module or sth. I prefer a much stricter division of concerns, as it makes it easier to understand and extend the code IMO. Thanks again :)

I will try to refactor the code so I am better able to understand it.

—
Reply to this email directly or view it on GitHub.

[kristianmandrup]

I've started a refactoring effort at

git://github.com/kristianmandrup/ransack-refactor.git

Much nicer IMO, much easier to understand. Now I "just" need to make the specs pass again ;-P

[johnnyshields]

@kristianmandrup did you ever get anywhere on this? I'd be glad to lend a hand, especially for spec writing if you need it.

[kristianmandrup]

You can clone my ransack-mongoid repo and work from there. Definitely, a good start would be to rework the specs, then gradually make them pass. I did a lot of work on refactoring in order to make sense of it all. In the original project there is a lot of duplication and many of the methods are way too large with too many local variables being pushed around that you can make sense of it, IMO.

[johnnyshields]

OK just to be confirm, I should start work from ransack-mongoid rather than ransack-refactor?

[kristianmandrup]

ah no, just use the latest branch. I forgot which. Mostl likely ransack-refactor?

[kristianmandrup]

So did you get started on this or did you give up already?

[johnnyshields]

I'll get started in 2 weeks or so--haven't even looked at it yet. Have a product launch I need to wrap up first. I've converted 3 or 4 other gems for Mongoid, I'd like to contribute if I can.

Just curious, I assume you needed search/sorting for Mongoid--what did you end up doing for your project? Roll your own?

[johnnyshields]

Still on my radar, not started yet though.

[kristianmandrup]

I rolled my own solution. It basically wraps the whole sorting "thing" in a nice DSL, with very flexible configuration. Then then I can do the UI part completely as I like. I found that coupling the UI to this sort of solution (auto-generating UI) wasn't such a great idea, at least for my use case which had a very dynamic/responsive sorting interface using JS and ajax ;) I might open source all or part of that solution in a few months time...

[johnnyshields]

Sounds cool. I'm also on the fence as to whether ransack is the right approach. Haven't had the chance to look in detail. I wrote my whole app heavily in rails, but as I learn more I find JS/ajax is the way forward and in the end I'll probably strip out 20-30% of the server code to redo it on the client side.

[kristianmandrup]

I agree!!! Use EmberJS ;)

https://github.com/kristianmandrup/ember-railsapi

My little demo, integrating a nice stack of modern technologies... the old Rails way feel sooooo 2000s... ;)

[johnnyshields]

An update: like @kristianmandrup I found it was easier to write my own lightweight search layer than port Mongoid. So I probably will not get around to doing this... will take a step back in case anyone else wants to pick this up.

[phstc]

I've just released ransack_mongo v0.0.1. This gem works with mongo-ruby-driver and Mongoid.

I don't see how we could add Mongoid support without dramatically changing everything.

I agree with @radar's mention above. So I extracted some logic from Ransack and created a Mongo compatible gem.

Ransack Mongo isn't coupled with Rails or ActiveRecord, the gem basically converts query params into Mongo queries.

[tetherit]

Very nice, thank you for that, great work!

[Zhomart]

Hi, I've added support for mongoid, but without associations. https://github.com/Zhomart/ransack/tree/mongoid

Here what I've done:

• moved active_record, arel codes into adapters/active_record
• created adapters/mongoid
• created Attribute class that imitates Arel::Attributes::Attribute
• added specs for mongoid
• disabled joins for mongoid. there are no specs for joins
• added rake tasks
  • $ rake runs active_record specs. default
  • $ DB=mongodb rake runs mongoid specs

I needed ransack for gem active_admin. I updated gem active_admin-mongoid https://github.com/Zhomart/activeadmin-mongoid/tree/ransack-mongoid. Now I am using this gem in my projects. And they work well. Even mongoid's relation based searching works.

[johnnyshields]

@Zhomart would you be able to raise a PR?

[Zhomart]

sure. it is #407

UPDATE: .travis, Gemfile and Rakefile changed to support mongo