Screenshot testing.

[jubishop]

I'd love to add a feature that lets the user save and then compare against "golden" (i.e. correct and expected) screenshot images of what their website looks like at different stages of a testing flow. It'd go something like:

• Define a folder that holds screenshots (with some reasonable default, maybe goldens/)
• At any point that you want to ensure the browser is showing what it should, call a method like flow.current_screenshot.should match_screenshot("...."). The comparison can actually be done with direct bytes of files, the routine in Ruby in my code right now works like this:

          @page.driver.save_screenshot(tmp_file(filename), { full: true })

          unless File.exist?(golden_file(filename))
            warn("Golden not found for for: #{filename}".red)
            Goldens.mark_failure(Failure.new(name: filename,
                                             golden: golden_file(filename),
                                             new: tmp_file(filename)))
            return
          end

          golden_bytes = File.read(golden_file(filename), mode: 'rb')
          new_bytes = File.read(tmp_file(filename), mode: 'rb')
          return if golden_bytes == new_bytes

          warn("Golden match failed for: #{filename}".red)
          Goldens.mark_failure(Failure.new(name: filename,
                                           golden: golden_file(filename),
                                           new: tmp_file(filename)))

          raise ::RSpec::Expectations::ExpectationNotMetError,
                "#{filename} does not match"

• If they don't match, the test not only marks the failure but offers you some convenient and easy way to view the differences and accept the new screenshot as your golden if you've changed your UI and expected this change. The framework would then copy the new screenshot over the old one in the goldens/ folder. In my implementation I launch a little localhost web server and launch the browser to my hosted page showing all the "failed" golden images alongside their new ones, with button to "Accept New" if you want. The entire implementation is here https://github.com/jubishop/tony-test/tree/master/lib/tony/goldens

Thoughts on how much of this sort of functionality, (if any), could make sense for me to add into LuckyFlow? and/or if it should be some sort of extension for the Awesome page?

Also any thoughts on exactly how this should be implemented and look following Lucky convention and philosophy?

[jubishop]

Here's the code that launches the server at the end of the RSpec suite run https://github.com/jubishop/tony-test/blob/master/lib/tony/goldens/goldens.rb

And note that the Server instance just returns immediately with no action if there weren't any failures: https://github.com/jubishop/tony-test/blob/master/lib/tony/goldens/server.rb