UtilityFunction should not complain method that works inside an interator

[isaiah]

The UtilityFunction smell could be misleading, e.g.

# Just for Fun
class Fun
  attr_accessor :arr

  def bar
    arr.each do |a_variable|
      # This is an intermediate state, should not be communicated as
      # an instance variable, as reek suggests
      b_variable = process(a_variable)
      puts b_variable
    end
  end

  private

  def process(x_variable)
    x_variable + 1
  end
end

reek fun.rb
=> fun.rb -- 1 warning:
=>   [16]:Fun#process doesn't depend on instance state (UtilityFunction)

It is useful most of the time, but for method that process an intermediate state, create a instance variable is not the right way.

[chastell]

I’d argue that Fun#process is a UtilityFunction, but I think we should consider accepting private UtilityFunctions; they’re usually added for readability.

[isaiah]

It is, but instance state doesn't apply here. Actually it's huge NO.

[troessner]

I'm open to changing this because I agree that the current set up of UtilityFunction is too strict.
How about making it part of this smell's configuration if we alert on private UtilityFunction and disable it by default?
That way stricter settings are still possible.

[mvz]

... for method that process an intermediate state, create a instance variable is not the right way.

I think here there is some confusion in the description reek uses: Reek is not trying to suggest changing the method to use an instance variable. In fact, the smell description doesn't try to suggest anything, but one way to resolve this is to move the implementation of process to a_variable's class.

How about making it part of this smell's configuration if we alert on private UtilityFunction and disable it by default?

I'm not too sure: I think UtilityFuntions are in fact often private and not flagging them means we miss out on opportunities to move methods to classes where they belong more naturally.

[andyw8]

@mvz makes a good point. I'd like to see some real-world examples of where this warning would be considered invalid.

[troessner]

@andyw8 what @chastell said above:

I’d argue that Fun#process is a UtilityFunction, but I think we should consider accepting private UtilityFunctions; they’re usually added for readability.

is something I see quite often unfortunately.

[andyw8]

Well let's say that process was a more complex method, and x_variable was a fully-fledged object rather than just a count:

  def process(x_variable)
    if x_variable.code == :foo
      "foo"
    elsif x_variable.code == :bar
      "bar"
    else
      raise "something went wrong"
    end
  end

In that case I think the better action would be to extract this out into a method defined in x_variable's class.

Trying to automatically determine if the method is 'complex enough' to justify this would no doubt be difficult, but I feel that by disabling the warning for private methods, we'd be discarding something which acts as a helpful prompt to refactor in many cases.

[troessner]

Trying to automatically determine if the method is 'complex enough' to justify this would no doubt be difficult, but I feel that by disabling the warning for private methods, we'd be discarding something which acts as a helpful prompt to refactor in many cases.

Agreed - that's why I'd make it configurable ;)

[tuexss]

From the example provided, I'd say this is a misuse of .each do - it looks like avoiding .map, .reduce or .inject, and to do that you need this ugly little helper. Proper use of .each does not need intermediate states to accumulate something across each invocation of the block.

[chastell]

Hmmm. In general moving it to the x_variable’s class would be better (so we should flag it), except when it happends to be a core type (which I’d rather not reopen for this).

We could argue that you shouldn’t use core types too much (as that’s primitive obsession), which I agree with in general, but in practice I do remember factoring out private utility functions to process some data. Factoring out a Process.call(x_variable) private module is just really wrapping the same logic in Process.call() rather than process() – formally probably purer (it blows the smell away!), but not really changes anything other than wrapping the logic in four lines – module + def + end + end – rather than two, def + end.

ArtDecomp::CircuitPresenter::WiresPresenter::WirePresenter::PinPresenter does this and I’ll probably stick with it, but I’m not 100% sure it’s worth it. ;)

[isaiah]

Let's put it the other way, please tell me how the code suggested by reek is acceptable.

# Just for Fun
class Fun
  attr_accessor :arr

  def bar
    arr.each do |a_variable|
      @x_variable = avariable
      b_variable = process
      puts b_variable
    end
  end

  private

  def process
    @x_variable + 1
  end
end

I saw this code in real code base, and it's there basically to conform reek.

[andyw8]

i agree that's a bad approach, but Reek isn't suggesting you do that. There are a number of actions that could have been taken in response to this warning. Choosing the good one requires experience and understanding of OOP – satisfying Reek won't magically turn bad code into good code.

Perhaps the warning could be improved by linking to a deeper discussion of why extracting the behaviour into another object could make sense in some situations.

[isaiah]

Yes, for experienced developers this is not a problem, but for new developers who take the warnings as principle, it does them more harm than good. Maybe besides suggesting turn the parameter passing into instance variable, there could be other suggestions.

[troessner]

Factoring out a Process.call(x_variable) private module is just really wrapping the same logic in Process.call() rather than process() – formally probably purer (it blows the smell away!), but not really changes anything other than wrapping the logic in four lines – module + def + end + end – rather than two, def + end.

Couldn't agree more. Unfortunately, this is exactly what happens in quite a few projects.

Perhaps the warning could be improved by linking to a deeper discussion of why extracting the behaviour into another object could make sense in some situations.

and

Maybe besides suggesting turn the parameter passing into instance variable, there could be other suggestions.

Love the idea!

Do you guys have suggestions for something we can link to? If not, we should come up with something on our own.

However I still believe that we should make it configurable if we enable / disable UtilityFunction for private methods - @mvz, @chastell what are your thoughts on this?

[mvz]

@isaiah:

Yes, for experienced developers this is not a problem, but for new developers who take the warnings as principle, it does them more harm than good.

I think this may be a problem with reek in general: It detects code smells but it doesn't try to tell you what the solution should be. As it stands, it is not really suitable for beginners who don't have a more experienced developer at hand to help guide them toward good OO design.

We can probably do a better job emphasising that the goal should not be to make your code smell free, but more to be aware of problems that may hinder further development and that you may want to tackle if and when you're touching that particular piece of code.

Apart from that, we can provide some sample solutions for the smells in the documentation.

Maybe besides suggesting turn the parameter passing into instance variable, there could be other suggestions.

I'm still not sure where reek suggests this. Can you point that out to me please?

@troessner:

I still believe that we should make it configurable if we enable / disable UtilityFunction for private methods.

I think I'd like to see some more real-world examples before deciding on this. I think most cases of UitilityFunction may be private methods, but I don't know what the conclusion should be. Some threshold on method length may be more appropriate. I'll go over smells in my own code as well and see what the situation is.

[mvz]

I'll go over smells in my own code as well

Preliminary results of that: One real bug in UtilityFunction found (it does not consider super), one possible improvement, and several cases where it at least points at code that needs to be rethought.

It now seems to me that the primary point of UtilityFunction is to ask "Why is this code here and not elsewhere?"

[chastell]

For the docs: I think in general reek would benefit with ‘possible ways to adress this’ section for most (each?) smells.

For UtilityFunction: I’m tentatively for making it configurable to whitelist private methods; at the same time I’ll probably keep them smelly in my pet projects, as fixing FeatureEnvy and UtilityFunction smells was an OOP eye-opener for me.

[troessner]

I'm still not sure where reek suggests this. Can you point that out to me please?

I assume because of the warning message:

[16]:Fun#process doesn't depend on instance state (UtilityFunction)

which implies to use instance variables. The problem is that it doesn't say "btw, dawg, with instance state i don't necessarily mean the current object but maybe another as well".

I think we can improve this a LOT by just adjusting our warning message accordingly. I just can't think of anything concise 😆

Ideas?

[isaiah]

@chastell The problem remains, is the style that pass intermediate state by instance variable a smell to reek? If it is should we warn that as well? If you actually think that's a good style, it is a REAL eyes opener for me.

[chastell]

is the style that pass intermediate state by instance variable a smell to reek?

No. :)

If it is should we warn that as well?

I personally don’t like mutable objects at all, so in my personal projects ivars are set in the constructor and ~never change. I really don’t like setting ivars outside of constructors.

That said I doubt we could easily distinguish ‘this ivar is set only to make a private UtilityFunction depend on state’ from typical Ruby object mutations.

If you actually think that's a good style, it is a REAL eyes opener for me.

😆 No worries – as I wrote, IMHO good style in general would be to move the method to the object that it gets as a param, but there are a lot of edge cases:

• if x_variable is an Integer (or any other core class) I’d rather people not reopen Integer just to add Integer#process and move the logic there,
• if x_variable is an instance of a custom class (say, XClass), then moving the logic to XClass#process that’s only ever used by Fun#bar sometimes is a good thing to do, but sometimes feels like an overkill,
• adding a private module that would be then called Process.call(x_variable) just to side-step the UtilityFunction smell doesn’t really change anything other than adding two more lines (module + end, see above),
• I really like my value object classes not to have too much behaviour (and definitely no presentation behaviour, maybe outside of #inspect or #to_s), so in the ArtDecomp::CircuitPresenter::WiresPresenter::WirePresenter::PinPresenter case when I didn’t want to have small UtilityFunctions on CircuitPresenter I had to create separate presenters for every bit along the way – which is a purer design, but seems to border on absurd. ;)

[troessner]

If you actually think that's a good style, it is a REAL eyes opener for me.

Jeez, don't be such a grumpy cat! Nobody think that is good style ;)

No worries – as I wrote, IMHO good style in general would be to move the method to the object that it gets as a param

Yes, exactly. Like we already discussed above, the problem is that most people naturally don't immediately jump to the conclusion that this method should exist on another object.

adding a private module that would be then called Process.call(x_variable) just to side-step the UtilityFunction smell doesn’t really change anything other than adding two more lines (module + end, see above),

Agree with all your edge cases. We should probably tweak UtilityFunction ;)

[hwo411]

What I suggested to Timo about a month ago - is to whitelist private and protected methods for UtilityFunction. My point is that class provides a public interface which should be checked, but in implementation details classes sometimes need such helper methods which were extracted from another methods to make them more readable.
In my opinion UtilityFunction warning shouldn't be shown for private/protected methods, I think there can only be a hint(but I think reek has no such definition) like "hey, take a look on this method, you may need to extract it to a separate module if such functionality is used in some other peace of code".

[troessner]

I agree with @hwo411. A lot of people complain about this rightfully and I believe we are too rigid in this regard.

Lets make this configurable and give people the chance to opt out if it. People who want to stay strict can just ignore this setting.
@mvz && @chastell lets give it a shot, please.

I'd come up with a pull request for this in the next days unless somebody in this thread would like to take over ;)

[martinciu]

IMO it's worth to add @chastell's hints to https://github.com/troessner/reek/blob/master/docs/Utility-Function.md

For me private utility functions should be highlighted by default with configure to turn it off.

[mvz]

@hwo411 wrote:

In my opinion UtilityFunction warning shouldn't be shown for private/protected methods, I think there can only be a hint(but I think reek has no such definition) like "hey, take a look on this method, you may need to extract it to a separate module if such functionality is used in some other peace of code".

Basically, all of Reek's smell warnings are of that nature. We really need to stress this more in the docs.

@troessner:

Lets make this configurable and give people the chance to opt out if it.

I agree. If the alternative is that people turn of UtilityFunction entirely this is a good middle ground.

@martinciu:

For me private utility functions should be highlighted by default with configure to turn it off.

Agreed.

[hwo411]

@mvz usually, when a developer, especially junior developer, see a warning, he/she either treats it as an error or ignores it. If you say that it's a hint, he'll take it as advice and will think about it, rather than trying to ignore or remove it. It's just a psychological difference, not essential one, however hint is a light version of warning as for me.

[mvz]

@hwo411 developers should consider all of reek's messages as hints. Maybe we should just say 'smells' instead of 'warnings'.

[troessner]

General fyi: I'll come up with a PR the next days.

@hwo411 developers should consider all of reek's messages as hints. Maybe we should just say 'smells' instead of 'warnings'.

Agreed. I guess we should hire somebody from marketing to fix our wording ;)

[hwo411]

@troessner the problem is that we have travis configured to make a build red if it sees warnings from reek(and the same problem will arise if you follow reek-driven development) and we need either to put this warnings to ignore list or to fix, but some hints can be useful without making build red and some warnings are actually the things which indicate that something is wrong with the code. So there are cases where we need a message but don't need to treat is an error automatically. Probably ignoring is the right way to handle such cases but I see 2 problems in ignoring:

1. When a method/class changes, ignoring remains the same, so we need to maintain ignoring manually. If in some case a second warning, which is real, appears in this method, it won't be shown if the first one is ignored(even if it's a correct ignore for the first one). Or e.g. if we decided to ignore some warning and for some reason in next commit we fix it, the next warning of such type will be silently ignored.
2. Ignoring doesn't enforce comments and they also need to be maintained manually, so we also can lose the reason of ignoring.
   That's why I think reek needs some kind of hints, which show messages but doesn't enforce errorness of the checks. And also ignoring probably needs some rethinking based on the points above.

[mvz]

@hwo411 I think ignoring smells in code comments is what you should use for your case.

Be sure to specify which smell you're ignoring, so new smells will still show up, like so (example from https://github.com/troessner/reek/blob/master/docs/Smell-Suppression.md):

# This method smells of :reek:NestedIterators
def smelly_method foo
  foo.each {|bar| bar.each {|baz| baz.qux}}
end

If another smell starts occuring in this method, it will still be reported by reek.

We just added ingoring comments to all of Reek precisely to be able to use it in Travis. You can see how we do this in reek's code.

[troessner]

We merged #705 and #698 which should fix this issue.
Release 3.4.0 on it's way here: #706
Closing this one.

[troessner]

@isaiah @nTraum @lipanski @hwo411 @RaVbaker @tomash reek 3.4 is released which makes UtiltiyFunction configurable for non-public methods (#698), feel free to update as necessary ;)