EndOfLine cop broken for UTF-8 files

[marten]

Example file (app/controllers/epd/reports_controller.rb):

# encoding: UTF-8
class Epd::ReportsController < EpdAreaController
  'terecht bij uw ROM-coördinator.'
end

Output from Rubocop:

Inspecting 1 file
Scanning /Users/marten/rgoc/roqua/app/controllers/epd/reports_controller.rb
For /Users/marten/rgoc/roqua/app/controllers/epd: configuration from /Users/marten/rgoc/roqua/.rubocop.yml
Inheriting configuration from /Users/marten/rgoc/roqua/rubocop-todo.yml
Default configuration from /Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/config/default.yml
Inheriting configuration from /Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/config/enabled.yml
Inheriting configuration from /Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/config/disabled.yml
An error occurred while EndOfLine cop was inspecting /Users/marten/rgoc/roqua/app/controllers/epd/reports_controller.rb.
invalid byte sequence in US-ASCII
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cop/style/end_of_line.rb:13:in `block in investigate'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cop/style/end_of_line.rb:12:in `each'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cop/style/end_of_line.rb:12:in `each_with_index'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cop/style/end_of_line.rb:12:in `investigate'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cop/commissioner.rb:82:in `block (2 levels) in invoke_cops_callback'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cop/commissioner.rb:88:in `with_cop_error_handling'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cop/commissioner.rb:81:in `block in invoke_cops_callback'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cop/commissioner.rb:79:in `each'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cop/commissioner.rb:79:in `invoke_cops_callback'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cop/commissioner.rb:55:in `investigate'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cop/team.rb:42:in `inspect_file'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cli.rb:138:in `inspect_file'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cli.rb:77:in `block in process_files'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cli.rb:71:in `each'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cli.rb:71:in `process_files'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/lib/rubocop/cli.rb:32:in `run'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/bin/rubocop:14:in `block in <top (required)>'
/Users/marten/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/benchmark.rb:296:in `realtime'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/gems/rubocop-0.15.0/bin/rubocop:13:in `<top (required)>'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/bin/rubocop:19:in `load'
/Users/marten/.rvm/gems/ruby-2.0.0-p247/bin/rubocop:19:in `<main>'
.

1 file inspected, no offences detected

1 error occurred:
An error occurred while EndOfLine cop was inspecting /Users/marten/rgoc/roqua/app/controllers/epd/reports_controller.rb.
Errors are usually caused by RuboCop bugs.
Please, report your problems to RuboCop's issue tracker.
Mention the following information in the issue report:
0.15.0 (using Parser 2.0.0, running on ruby 2.0.0 x86_64-darwin12.3.0)
Finished in 0.013773 seconds

[sameera207]

Hi @marten , I just tried it with following example and it works for me. Only difference I can see is, you are using a
config file called . rubocop-todo.yml.

  # encoding: UTF-8
  class Admin::DashbordsController < ApplicationController
     ' same sample text'
  end

• rubocop - 0.15.0
• rails 4.0,1
• ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64-darwin12.3.0]
• on mac

or probably since I'm very new to rubocop, I might be missing something 😕

[marten]

@sameera207 I think you might simply not see the issue because you don't have any UTF-8 characters like ö or é in your example file. But it's possible that something is wrong with my setup. Could you try inserting an o-umlaut in your "same sample text"?

[bbatsov]

The example source code doesn't trigger an exception for me. @jonas054 can you try to reproduce this as well?

[lee-dohm]

@bbatsov I was unable to reproduce the error using the sample code as well.

• RuboCop: b713ba0
• Ruby: ruby 2.0.0p353 (2013-11-22 revision 43784) [x86_64-darwin13.0.0]
• Parser: 2.0.0
• OS: OS X 10.9

Output from rubocop --debug test.rb:

Inspecting 1 file
Scanning /Volumes/Data/Users/Lee/Source/etherjournal/test.rb
For /Volumes/Data/Users/Lee/Source/etherjournal: configuration from /Volumes/Data/Users/Lee/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/bundler/gems/rubocop-b713ba0f078c/config/default.yml
Inheriting configuration from /Volumes/Data/Users/Lee/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/bundler/gems/rubocop-b713ba0f078c/config/enabled.yml
Inheriting configuration from /Volumes/Data/Users/Lee/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/bundler/gems/rubocop-b713ba0f078c/config/disabled.yml
AllCops/Excludes configuration from /Volumes/Data/Users/Lee/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/bundler/gems/rubocop-b713ba0f078c/.rubocop.yml
Inheriting configuration from /Volumes/Data/Users/Lee/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/bundler/gems/rubocop-b713ba0f078c/config/default.yml
Inheriting configuration from /Volumes/Data/Users/Lee/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/bundler/gems/rubocop-b713ba0f078c/config/enabled.yml
Inheriting configuration from /Volumes/Data/Users/Lee/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/bundler/gems/rubocop-b713ba0f078c/config/disabled.yml
Default configuration from /Volumes/Data/Users/Lee/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/bundler/gems/rubocop-b713ba0f078c/config/default.yml
Inheriting configuration from /Volumes/Data/Users/Lee/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/bundler/gems/rubocop-b713ba0f078c/config/enabled.yml
Inheriting configuration from /Volumes/Data/Users/Lee/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/bundler/gems/rubocop-b713ba0f078c/config/disabled.yml
C

Offences:

test.rb:2:1: C: Documentation: Missing top-level class documentation comment.
class Epd::ReportsController < EpdAreaController
^^^^^

1 file inspected, 1 offence detected
Finished in 0.026551 seconds

[sameera207]

@marten , I tried with the same code that you have

# encoding: UTF-8
class Admin::DashbordsController < ApplicationController
  'terecht bij uw ROM-coördinator.'
end

and still working fine for me :).

25 files inspected, 34 offences detecte

[jonas054]

I was finally able to reproduce the error! The key is to set the environment variable LANG:

LANG=C rubocop -d /tmp/ex.rb

I thought at first that this means there's a bug in RuboCop, but maybe that's not the case. What's your setting of the LANG variable, @marten?

[jonas054]

I'm back to thinking that this is a bug in RuboCop. The special thing about EndOfLine is that it reads the source code again with IO.read, thus missing out on the encoding recognition done in Parser. I'll try to fix it.

[marten]

Awesome! Thanks a lot!