Debug bundler #5159

jmarrec · 2024-04-19T12:50:01Z

Pull request overview

Pull Request Author

Model API Changes / Additions
Any new or modified fields have been implemented in the EnergyPlus ForwardTranslator (and ReverseTranslator as appropriate)
Model API methods are tested (in src/model/test)
EnergyPlus ForwardTranslator Tests (in src/energyplus/Test)
If a new object or method, added a test in NREL/OpenStudio-resources: Add Link
If needed, added VersionTranslation rules for the objects (src/osversion/VersionTranslator.cpp)
Verified that C# bindings built fine on Windows, partial classes used as needed, etc.
All new and existing tests passes
If methods have been deprecated, update rest of code to use the new methods

Labels:

If change to an IDD file, add the label IDDChange
If breaking existing API, add the label APIChange
If deemed ready, add label Pull Request - Ready for CI so that CI builds your PR

Review Checklist

This will not be exhaustively relevant to every PR.

Perform a Code Review on GitHub
Code Style, strip trailing whitespace, etc.
All related changes have been implemented: model changes, model tests, FT changes, FT tests, VersionTranslation, OS App
Labeling is ok
If defect, verify by running develop branch and reproducing defect, then running PR and reproducing fix
If feature, test running new feature, try creative ways to break it
CI status: all green or justified

jmarrec · 2024-04-19T12:51:30Z

ruby/bindings/InitRubyBindings.cpp

+# require 'logger'
+# $logger = Logger.new(STDOUT)
+# $logger.level = Logger::DEBUG #WARN
+# Mimic ruby Logger, but forward to OpenStudio, so we don't have to worry about setting the LogLevel
+class ConvenienceLogger
+
+  attr_accessor :level
+
+  def initialize(log_level)
+    @level = log_level
+  end
+
+  def trace?
+    @level <= OpenStudio::Trace
+  end
+
+  def debug(msg)
+    OpenStudio::logFree(OpenStudio::Debug, "ruby", msg)
+  end
+
+  def info(msg)
+    OpenStudio::logFree(OpenStudio::Info, "ruby", msg)
+  end
+
+  def warn(msg)
+    OpenStudio::logFree(OpenStudio::Warn, "ruby", msg)
+  end
+
+  def error(msg)
+    OpenStudio::logFree(OpenStudio::Error, "ruby", msg)
+  end
+
+  def fatal(msg)
+    OpenStudio::logFree(OpenStudio::Fatal, "ruby", msg)
+  end
+
+end
+$logger = ConvenienceLogger.new(OpenStudio::Logger.instance.standardOutLogger.logLevel.get)


A clever way to keep the "$logger" code below working, while respecting the user's choice in terms of log level. PReviosuly the C++ CLI was lacking the ability to debug this stuff because the loglevel was set in stone to WARN.

jmarrec · 2024-04-19T12:51:39Z

ruby/bindings/InitRubyBindings.cpp

+if $logger.trace?
+  ENV['DEBUG_RESOLVER'] = "1"
+end


Even better than the ruby cli

jmarrec · 2024-04-19T12:54:49Z

ruby/bindings/InitRubyBindings.cpp

+        rescue Gem::LoadError => e
          # There may be conflicts between the bundle and the embedded gems,
          # those will be logged here
+          exception_msg = "Error activating gem #{spec.spec_file}: #{e.class}: #{e.message}\nTraceback:\n"
+          exception_msg += e.backtrace.join("\n")
+          STDERR.puts exception_msg


No matter what I do, I can't get the failure like I do in the ruby cli.

cd OpenStudio/src/cli/test/bundle ruby --version ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] bundle install --path ./test_gems

C++

$ $os_build_rel/Products/openstudio --bundle Gemfile.lock --bundle_path './test_gems' --loglevel Debug test.rb [...] [ruby] <-2> Adding dependency on tbd '~> 3.4.1' [ruby] <-2> Adding dependency on topolys '~> 0.6.2' [ruby] <-2> Adding dependency on unicode-display_width '~> 2.5.0' [ruby] <-2> Adding dependency on uuid '~> 2.3.9' [ruby] <-2> Adding dependency on yamler '~> 0.1.0' [ruby] <-2> Adding dependency on zliby '~> 0.0.5' [ruby] <-2> Processing 136 activation requests [ruby] <-2> Activating Bundler gem :/ruby/3.2.0/specifications/bundler-2.5.5.gemspec Failed to execute '/home/julien/Software/Others/OpenStudio/src/cli/test/bundle/test.rb'

Classic

$ $os_build_rel/Products/openstudio classic --bundle Gemfile.lock --bundle_path './test_gems' --verbose test.rb [...] D, [2024-04-19T14:52:29.102995 #1611746] DEBUG -- : Processing 136 activation requests D, [2024-04-19T14:52:29.119140 #1611746] DEBUG -- : Activating gem :/ruby/3.2.0/specifications/bundler-2.5.5.gemspec Error executing argv: ["--bundle", "Gemfile.lock", "--bundle_path", "./test_gems", "--verbose", "test.rb"] Error: [!] There was an error parsing `Gemfile.lock`: syntax error, unexpected ':', expecting end-of-input - remote: http://rubygems.org/ ^ . Bundler cannot continue. # from /home/julien/Software/Others/OpenStudio/src/cli/test/bundle/Gemfile.lock:2 # ------------------------------------------- # GEM > remote: http://rubygems.org/ # specs: # ------------------------------------------- in :/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/dsl.rb:49:in `instance_eval' :/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/dsl.rb:49:in `eval_gemfile' :/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/dsl.rb:12:in `evaluate' :/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/definition.rb:38:in `build' :/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler.rb:326:in `definition' :/openstudio_cli.rb:645:in `parse_main_args' :/openstudio_cli.rb:791:in `execute' :/openstudio_cli.rb:1994:in `<main>' eval:233:in `eval' eval:233:in `require_embedded_absolute' eval:218:in `block in require_embedded' eval:212:in `each' eval:212:in `require_embedded' eval:171:in `require' eval:3:in `<main>'

@kbenne that error makes zero sense to me.

@kbenne

…g messages This was missing compared to the CLI. @kbenne

…with this

… but not for trying to use --bundle

…it::TestCase/Minitest::Test/g' @`

jmarrec · 2024-04-19T13:32:46Z

Seems like we try to tell Bundler to load a Gemfile.lock when it exepcts a Gemfile instead

jmarrec · 2024-04-19T13:34:37Z

I've modified dsl.rb locally in build/openstudio-gems which corresponds to https://github.com/rubygems/rubygems/blob/63bec47bb84105aec729bc12be02770d3f5b48e6/bundler/lib/bundler/dsl.rb#L44-L54

    def eval_gemfile(gemfile, contents = nil)
      expanded_gemfile_path = Pathname.new(gemfile).expand_path(@gemfile&.parent)
      original_gemfile = @gemfile
      @gemfile = expanded_gemfile_path
      @gemfiles << expanded_gemfile_path
      puts "contents=#{contents}"
      contents ||= Bundler.read_file(@gemfile.to_s)
      puts "gemfile=#{gemfile}"
      puts "@gemfile=#{@gemfile.to_s}"
      puts "expanded_gemfile_path=#{expanded_gemfile_path}"
      puts "contents=#{contents}"
      instance_eval(contents, gemfile.to_s, 1)

When I pass the "Gemfile" to the CLI (not the Gemfile.lock) I still have an error

$os_build_rel/Products/openstudio classic --bundle Gemfile --bundle_path './test_gems' --verbose test.rb
D, [2024-04-19T15:30:20.040158 #1628991] DEBUG -- : Activating gem :/ruby/3.2.0/specifications/bundler-2.5.5.gemspec
contents=
gemfile=/home/julien/Software/Others/OpenStudio/src/cli/test/bundle/Gemfile
@gemfile=/home/julien/Software/Others/OpenStudio/src/cli/test/bundle/Gemfile
expanded_gemfile_path=/home/julien/Software/Others/OpenStudio/src/cli/test/bundle/Gemfile
contents=source 'http://rubygems.org'

gem 'openstudio-workflow', '2.2.0' # This is compatible with bundler 2.1 (OS 3.x), and already older than currently used at this time
gem 'tilt', '2.0.8'
Error executing argv: ["--bundle", "Gemfile", "--bundle_path", "./test_gems", "--verbose", "test.rb"]
Error: 
[!] There was an error parsing `Gemfile`: nil is not a class/module. Bundler cannot continue.

 #  from /home/julien/Software/Others/OpenStudio/src/cli/test/bundle/Gemfile:1
 #  -------------------------------------------
 >  source 'http://rubygems.org'
 #  
 #  -------------------------------------------
 in :/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/source/rubygems.rb:24:in `initialize'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/source_list.rb:12:in `new'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/source_list.rb:12:in `global_rubygems_source'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/source_list.rb:72:in `add_global_rubygems_remote'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/dsl.rb:180:in `source'
/home/julien/Software/Others/OpenStudio/src/cli/test/bundle/Gemfile:1:in `eval_gemfile'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/dsl.rb:54:in `instance_eval'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/dsl.rb:54:in `eval_gemfile'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/dsl.rb:12:in `evaluate'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/definition.rb:38:in `build'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler.rb:326:in `definition'
:/openstudio_cli.rb:645:in `parse_main_args'
:/openstudio_cli.rb:791:in `execute'
:/openstudio_cli.rb:1994:in `<main>'
eval:233:in `eval'
eval:233:in `require_embedded_absolute'
eval:218:in `block in require_embedded'
eval:212:in `each'
eval:212:in `require_embedded'
eval:171:in `require'
eval:3:in `<main>'

…dings

jmarrec · 2024-04-19T13:56:19Z

Wait, so in that bit where we try to activate the bundler spec, we call bundler itself? Isn't that strange or something?

jmarrec · 2024-04-19T13:58:30Z

So the error happens here:

OpenStudio/ruby/bindings/InitRubyBindings.cpp

Line 765 in 8b0f3a9

spec.activate

D, [2024-04-19T15:58:11.967007 #1633316] DEBUG -- : Activating gem :/ruby/3.2.0/specifications/bundler-2.5.5.gemspec
contents=
gemfile=/home/julien/Software/Others/OpenStudio/src/cli/test/bundle/Gemfile
@gemfile=/home/julien/Software/Others/OpenStudio/src/cli/test/bundle/Gemfile
expanded_gemfile_path=/home/julien/Software/Others/OpenStudio/src/cli/test/bundle/Gemfile
contents=source "https://rubygems.org"

# gem 'openstudio-workflow', '2.4.0' # This is compatible with bundler 2.5 / Ruby 3.2 (OS 3.8+), but **NOT** already older than currently used at this time
gem 'tilt', '2.0.8'
Error executing argv: ["--bundle", "Gemfile", "--bundle_path", "./test_gems", "--verbose", "test.rb"]
Error: 
[!] There was an error parsing `Gemfile`: nil is not a class/module. Bundler cannot continue.

 #  from /home/julien/Software/Others/OpenStudio/src/cli/test/bundle/Gemfile:1
 #  -------------------------------------------
 >  source "https://rubygems.org"
 #  
 #  -------------------------------------------
 in :/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/source/rubygems.rb:24:in `initialize'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/source_list.rb:12:in `new'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/source_list.rb:12:in `global_rubygems_source'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/source_list.rb:72:in `add_global_rubygems_remote'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/dsl.rb:180:in `source'
+/home/julien/Software/Others/OpenStudio/src/cli/test/bundle/Gemfile:1:in `eval_gemfile'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/dsl.rb:54:in `instance_eval'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/dsl.rb:54:in `eval_gemfile'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/dsl.rb:12:in `evaluate'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler/definition.rb:38:in `build'
:/ruby/3.2.0/gems/bundler-2.5.5/lib/bundler.rb:326:in `definition'
:/openstudio_cli.rb:645:in `parse_main_args'
:/openstudio_cli.rb:791:in `execute'
:/openstudio_cli.rb:1994:in `<main>'
eval:233:in `eval'
eval:233:in `require_embedded_absolute'
eval:218:in `block in require_embedded'
eval:212:in `each'
eval:212:in `require_embedded'
eval:171:in `require'
eval:3:in `<main>'

…'t one we can use

ci-commercialbuildings · 2024-04-19T14:47:39Z

CI Results for cd134b6:

Ubuntu 20.04:
- OpenStudio-3.8.0-beta+cd134b6ef8-Ubuntu-20.04-x86_64.deb
- OpenStudio-3.8.0-beta+cd134b6ef8-Ubuntu-20.04-x86_64.tar.gz
Ubuntu 22.04:
- OpenStudio-3.8.0-beta+cd134b6ef8-Ubuntu-22.04-x86_64.deb
- OpenStudio-3.8.0-beta+cd134b6ef8-Ubuntu-22.04-x86_64.tar.gz
Darwin:
- OpenStudio-3.8.0-beta+cd134b6ef8-Darwin-x86_64.dmg
- OpenStudio-3.8.0-beta+cd134b6ef8-Darwin-x86_64.tar.gz
Windows:
Code Coverage:

ref #5161

jmarrec added the Developer Issue label Apr 19, 2024

jmarrec requested a review from kbenne April 19, 2024 12:50

jmarrec self-assigned this Apr 19, 2024

jmarrec commented Apr 19, 2024

View reviewed changes

jmarrec added 4 commits April 19, 2024 15:31

Improve quality of life in InitRubyBindings by correctly logging debu…

f80fb9c

…g messages This was missing compared to the CLI. @kbenne

I can't get the C++ cli to throw the full error same as classic even …

7e40de8

…with this

Try flushing in UpdateCommand. It works fine for a script that fails,…

7373780

… but not for trying to use --bundle

`rg "MiniTest::Unit::TestCase" -l | xargs -I @ sed -i 's/MiniTest::Un…

98a95ba

…it::TestCase/Minitest::Test/g' @`

jmarrec force-pushed the debug_bundler branch from 63f2037 to 98a95ba Compare April 19, 2024 13:32

jmarrec added the Pull Request - Ready for CI This pull request if finalized and is ready for continuous integration verification prior to merge. label Apr 19, 2024

Base automatically changed from 5147_modernize_measure_templates to develop April 19, 2024 13:41

Adjust a test to filter out the new debug messages due to InitRubyBin…

8b0f3a9

…dings

bundle (most basic test) : Disabling pulling workflow since there isn…

cd134b6

…'t one we can use

Fix bundler for Ruby v3.2.2

d28943a

ref #5161

kbenne approved these changes Apr 22, 2024

View reviewed changes

kbenne merged commit d2f4b05 into develop Apr 22, 2024
2 checks passed

kbenne deleted the debug_bundler branch April 22, 2024 18:21

jmarrec mentioned this pull request Apr 25, 2024

measure.xml default bool values reported as 0/1 on develop #5167

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Debug bundler #5159

Debug bundler #5159

jmarrec commented Apr 19, 2024

jmarrec Apr 19, 2024

jmarrec Apr 19, 2024

jmarrec Apr 19, 2024

jmarrec Apr 19, 2024

jmarrec commented Apr 19, 2024

jmarrec commented Apr 19, 2024

jmarrec commented Apr 19, 2024

jmarrec commented Apr 19, 2024 •

edited

Loading

ci-commercialbuildings commented Apr 19, 2024 •

edited

Loading

Debug bundler #5159

Debug bundler #5159

Conversation

jmarrec commented Apr 19, 2024

Pull request overview

Pull Request Author

Review Checklist

jmarrec Apr 19, 2024

Choose a reason for hiding this comment

jmarrec Apr 19, 2024

Choose a reason for hiding this comment

jmarrec Apr 19, 2024

Choose a reason for hiding this comment

jmarrec Apr 19, 2024

Choose a reason for hiding this comment

jmarrec commented Apr 19, 2024

jmarrec commented Apr 19, 2024

jmarrec commented Apr 19, 2024

jmarrec commented Apr 19, 2024 • edited Loading

ci-commercialbuildings commented Apr 19, 2024 • edited Loading

jmarrec commented Apr 19, 2024 •

edited

Loading

ci-commercialbuildings commented Apr 19, 2024 •

edited

Loading