belongs_to should default to required: true

[dhh]

Almost every belongs_to declaration seems to be a required association. It's rare that you allow your foreign key to be nil. So let's have required: true be the default and required: false be the optional turn-off.

[simi]

I can take this, but I'm afraid this can cause a lot of harm.

[dhh]

Rocking. I think the main issue will be thinking about whether this has any backwards incompatibility issues. Like if you upgrade an app that already declares validates_presence_of or something. I don’t think it should be a problem, since the validation should just overwrite in that case, I believe. But something to be on the lookout for. 👍

On Dec 28, 2014, at 8:01 PM, Josef Šimánek [email protected] wrote:

I can take this.

—
Reply to this email directly or view it on GitHub #18233 (comment).

[simi]

My fears were fulfilled :(

For this naive implementation (actually it should be changed somewhere in the reflection):

diff --git a/activerecord/lib/active_record/associations/builder/belongs_to.rb b/activerecord/lib/active_record/associations/builder/belongs_to.rb
index 954ea38..e0edc28 100644
--- a/activerecord/lib/active_record/associations/builder/belongs_to.rb
+++ b/activerecord/lib/active_record/associations/builder/belongs_to.rb
@@ -23,6 +23,11 @@ module ActiveRecord::Associations::Builder
       add_counter_cache_methods mixin
     end

+    def self.define_validations(model, reflection)
+      reflection.options[:required] ||= true
+      super
+    end
+
     private

     def self.add_counter_cache_methods(mixin)

I'm getting (because unsaved objects) a lot failing tests:

4305 runs, 10485 assertions, 117 failures, 508 errors, 2 skips

Since belongs_to is heavily used in test suite:

grep -r 'belongs_to' test/ | wc -l
505

it is not good idea to add require: false everywhere or construct complex relations in setup phases in tests IMHO.

[al2o3cr]

@simi - the implementation given there is incorrect, since ||= will replace an explicit false with true :(

If we make this the default, we should include a warning that it alone isn't sufficient to maintain referential integrity - it suffers from the same sort of race condition as validates_uniqueness_of. Example:

Thread A                               | Thread B
p = Post.new                           |
p.blog_id = 5                          |
p.save                                 |
  BEGIN                                |
    validation verifies that blog with |
      id=5 exists                      |
                                       |
                                       | Blog.find(5).destroy
    INSERT INTO posts etc.             |
  COMMIT                               |

p.save returns true, but the record is now orphaned since the corresponding row
in the blogs table no longer exists.

I'm not personally a huge fan of options where the default is true, since it leaves "nil is different from false" rakes around for people to step on (as above :) ). Maybe required should eventually be inverted and replaced with optional? Associations that wanted to opt-out of the validation could declare optional: true.

[simi]

@al2o3cr sure, this is really bad implementation. But since there is no usage of required: false in whole test suite, it should not be problem for my arguments.

optional sounds good, but I'm still not sure if this is good default behavior.

[dhh]

I like optional: true 👍

On Dec 29, 2014, at 5:32, Josef Šimánek [email protected] wrote:

@al2o3cr sure, this is really bad implementation. But since there is no usage of required: false in whole test suite, it should not be problem for my arguments.

optional sounds good, but I'm still not sure if this is good default behavior.

—
Reply to this email directly or view it on GitHub.

[dhh]

Matt, this is to give you the best error message in the majority of the cases. You’ll still need to back it up with a unique constraint in the DB to prevent what you mention. But such an error would raise an exception, not a validation error. So the less that needs to happen, the better.

On Dec 29, 2014, at 5:23, Matt Jones [email protected] wrote:

@simi - the implementation given there is incorrect, since ||= will replace an explicit false with true :(

If we make this the default, we should include a warning that it alone isn't sufficient to maintain referential integrity - it suffers from the same sort of race condition as validates_uniqueness_of. Example:

Thread A | Thread B
p = Post.new |
p.blog_id = 5 |
p.save |
BEGIN |
validation verifies that blog with |
id=5 exists |
|
| Blog.find(5).destroy
INSERT INTO posts etc. |
COMMIT |

p.save returns true, but the record is now orphaned since the corresponding row
in the blogs table no longer exists.

I'm not personally a huge fan of options where the default is true, since it leaves "nil is different from false" rakes around for people to step on (as above :) ). Maybe required should eventually be inverted and replaced with optional? Associations that wanted to opt-out of the validation could declare optional: true.

—
Reply to this email directly or view it on GitHub.

[simi]

I think it will be better to introduce new relation based on belongs_to just with another defaults. I'm not sure about the name. Let me think about it for a while.

[sgrif]

Since we're dropping support for Ruby 1.9, why not just use keyword arguments to set the default?

[sgrif]

This would also constitute a major breaking change for most apps, so we should probably have a deprecation cycle.

[chancancode]

We would need to pair this with a corresponding change in migration generator, correct? We might want to wait till we figured out what we to do with that (hopefully soon)?

[dhh]

This is not worth introducing another name than belongs_to for. For migration purposes, we can have a global config switch like we do in so many other cases. config.active_record.belongs_to_validates_presence = true or whatever.

On Dec 29, 2014, at 10:37, Godfrey Chan [email protected] wrote:

We would need to pair this with a corresponding change in migration generator, correct? We might want to wait till we figured out what we to do with that (hopefully soon)?

—
Reply to this email directly or view it on GitHub.

[simi]

Global switch sounds good. But what about AR test suite? Should that switch it be turned off for test suite or should be optional: true or required: false added everywhere? I think it is not good idea to force creating of complex db objects for AR test suite since it will slow it down a lot IMHO.

[dhh]

We can turn that global switch off for the suite, except for where we are explicitly testing it.

On Dec 29, 2014, at 10:52, Josef Šimánek [email protected] wrote:

Global switch sounds good. But what about AR test suite? Should that switch it be turned off for test suite or should be optional: true or required: false added everywhere? I think it is not good idea to force creating of complex db objects for AR test suite since it will slow it down a lot IMHO.

—
Reply to this email directly or view it on GitHub.

[matthewd]

A global switch that changes the meaning of a line of code like this is a very bad idea... which is why I really don't think we do do it in so many other cases.

Global switches are exactly how we end up stranding large applications on older versions.

[dhh]

We have global switches that set the defaults all over the place. In fact, I think this is key to allowing migrations to progress forward. It allows them to upgrade without submitting themselves to keeping up with every default change at the moment of upgrading. The alternative is just to say “this has changed”, and thus requiring to make all the changes to accommodate that change the moment you upgrade. We’ve just seen how laborious that in the Rails code base itself.

On Dec 30, 2014, at 6:27 AM, Matthew Draper [email protected] wrote:

A global switch that changes the meaning of a line of code like this is a very bad idea... which is why I really don't think we do do it in so many other cases.

Global switches are exactly how we end up stranding large applications on older versions.

—
Reply to this email directly or view it on GitHub #18233 (comment).

[dhh]

@simi still interested in finishing this up?

[simi]

Sure. Is the global switch good way how to implement this? Should it be turned on for new applications?

[dhh]

Awesome. Yes and yes.

[dhh]

@simi Any update on this?

[simi]

@dhh I spent some time with this and I think @al2o3cr is right here. We should deprecate required: true and add optional: true.

@sgrif Keyword arguments make sense here also, but it is not trivial change and all association macros should be updated also.

[dhh]

Agree, optional: true is the way to go 👍.

On Feb 13, 2015, at 16:23, Josef Šimánek [email protected] wrote:

@dhh I spent some time with this and I think @al2o3cr is right here. We should deprecate required: true and add optional: true.

@sgrif Keyword arguments make sense here also, but it is not trivial change and all association macros should be updated also.

—
Reply to this email directly or view it on GitHub.

[egilburg]

How feasible would it be do determine requirement based on db schema?

We are (or should be) encouraging good DB design for maintaining referential integrity instead of solely relying on validations which are not as consistent (e.g. model changes in future, application level bugs, batch jobs, performance-sensitive low-level updates, etc.

As such, if db table has null: false on a belongs_to field, it implies required:true on model, otherwise not. WDYT?

This would also significantly reduce migration path for users, as I can't imagine a lot of apps who prefer to not have required: true on validation when their schema requires it and would throw db error if trying to save (even with a soft save)

[dhh]

Not keen on tying this together. Lots of schemas out there that aren’t likely to change soon, and this change still makes sense for them.

On Feb 13, 2015, at 7:40 PM, Eugene Gilburg [email protected] wrote:

How feasible would it be do determine requirement based on db schema?

We are (or should be) encouraging good DB design for maintaining referential integrity instead of solely relying on validations which are not as consistent (e.g. model changes in future, application level bugs, batch jobs, performance-sensitive low-level updates, etc.

As such, if db table has null: false on a belongs_to field, it implies required:true on model, otherwise not. WDYT?

—
Reply to this email directly or view it on GitHub #18233 (comment).

[simi]

@egilburg I think this can be added to gems like schema_validations.

[bwheeler96]

I realize I'm a little late to the party on this one, but what about the concern that now rails associations either fall into a "required" category, or a "not required" category?

It seems like this change is now adding opinions to how I should model my data, whereas previously rails just gave me the tools to create whatever unopinionated data model I wanted. Would it not make more sense to relegate this functionality to a gem or a new method, something along the lines of child_of :model.

Otherwise how are we deciding which associations are "required" and which are "not required". Has one sounds like it should be required (and belongs_to is almost always the complement of has_one). Also if possible it would be nice to see a deprecation warning included with the validation failure message

[dhh]

Party is over months ago. Rails is opinionated software. We always had conventions about how your data should be modeled and what the defaults were. This matches more of the use, more of the time.

[dhh]

http://rubyonrails.org/doctrine/#convention-over-configuration is a helpful guide on how these decisions are made and will continue to be made.

[sandeepravi]

This seems to be causing an issue with accepts_nested_attributes_for . If I want to create the child object along with the parent, it seems to throwing an error in the child model that the parent has to exist.

This seems to be a pretty standard thing to do, right? Is there any workaround for this? Or am I missing something?

[dhh]

@sandeepravi I'd suggest adding required: false to child models in that case. Not a big fan of accepts_nested_attributes_for anyway and we should figure out a long-term path away from that. But saying required: false will return things to how they were for you.

[sandeepravi]

@dhh , yeah right now I've made it optional so it seems to work. But I would rather keep it required and not use accepts_nested_attributes.

Yeah, nested_attributes in multiple places can get messed up quite soon. You generally prefer to have form objects or just plain POROs?

[dhh]

I like aggregation through regular Ruby objects. For example, we have a Signup model that's just a Ruby object orchestrating the build process. So I'd give that a go!

[sandeepravi]

yeah, it seems the simplest option as well.

Cool, Thanks!

[ghost]

@dhh Is there a recommended method for cases where you still need to validate the presence of the child objects on the parent object? I am using optional: true but that makes my validation tests pass even though the parents objects can be created without any of their "required" child objects.

[bluengreen]

@ClaudioCarmeli - a year of silence :(

Exactly why rails is losing popularity. Brilliant idea - lets change default behavior that has been core for 12 years. Not the way to win the hearts and minds and expand adoption. Also not much in the way of solutions for fixing the issues with nested attributes using accepts_nested_attributes_for. Granted I try not to use them, but there are a ton of legacy projects from the glory days of jquery that still do. I mean, that is, other than touch every association. Also found that the global config to return to the previous behavior, doesn't seem to work for gems that add belongs_to associations. Dunno maybe it was just bewilderment and the lack of sleep staying up late trying to find and "fix" this problem and there may be other solutions. But must say - just another stupid "rails way" idea.