Testing Infrastrucutre

[trek]

I can't tell where (git log -- returns nothing for that file), but at some point after 0.9.3 the Rails test case integration code was removed.

I can't find where the equivalent code is in the current revision, so this breaks controller unit test like

assert_serializer :photos 

when trying to update, even though that behavior has not changed. Am I just not seeing where this code was moved to?

[joaomdmoura]

Hi @trek, actually the 0.10.x version wrote from scratch, base on 0.8.0 structure.
But indeed would be awesome to bring that to 0.10.0. I'm totally up for it.
I'll keep this open and flagged in case someone wants to do it, otherwise I might do it once I finish the deserialization feature.

[trek]

If you happen to know the general direction, point me in it and I'll do the work. It wasn't as simple as just restoring the file :p

[joaomdmoura]

hey @trek, awesome, I think a could start would be something similar to what the 0.9.3 had. Once we have it we can evolve it or add new features, but I think it would be a good starting point.

[bolshakov]

Hi, @trek

I was the author of assert_serializer and currently I don't use it anymore and I think It's not very useful.
I'm not against assert_serializer, but there is a much better approach for testing rails controllers with serializers. Each API endpoint I describe with JSON Schema. This is not waste of time! You can reuse your schemas on documentation and on tests. This is very convenient since your response always validated against JSON Schema from the documentation!

For validating schema I have a simple RSpec matcher:

require 'json-schema'

# Json schema matcher
# Schema files should be placed at spec/support/fixtures/
#
# @example validate json against bundles.json schema
#   expect(json).to match_json_schema('bundles/show')
#
# @example it's also possible to validate test response
#   get :show, id: 664
#   is_expected.to match_json_schema('bundles/show')
#
RSpec::Matchers.define :match_json_schema do |against|
  match do |object_to_be_validated|
    file_name = Rails.root.join('json_schemas', "#{against}.json").to_s

    fail "Could not find schema file #{file_name}" unless File.exist?(file_name)

    @errors = validate(file_name, object_to_be_validated)

    return true if @errors.empty?
  end

  failure_message do
    @errors.join("\n")
  end

  private

  def validate(file_name, response_or_json)
    json = if response_or_json.is_a?(ActionController::TestResponse)
                 JSON.parse(response_or_json.body).with_indifferent_access
               else
                 response_or_json
               end

    JSON::Validator.fully_validate(file_name, json)
  end
end

I'm sure someone already build gem for this.

So my controller specs looks like so:

RSpec.describe V1::MoviesController, type: :controller do
  let!(:movie) { create(:movie, :with_images) }

  context '#index' do
    subject do
      get :index
    end

    it { is_expected.to have_http_status :ok }
    it { is_expected.to match_json_schema('movies/index') }
  end
end

I do not test serializer implementation details here, only ensure that response is valid against provided schema.

I also unit-test serializers separately, checking all edge-cases and ensure it's valid against JSON Schema too.

[bolshakov]

If you're curious about rendering JSON Schema in docs, look at awesome library docson. It can generates something like this http://lbovet.github.io/docson/index.html#/docson/examples/example.json

[trek]

I'm not against assert_serializer, but there is a much better approach for testing rails controllers with serializers. Each API endpoint I describe with JSON Schema. This is not waste of time! You can reuse your schemas on documentation and on tests. This is very convenient since your response always validated against JSON Schema from the documentation!

Comparing to JSON Schema is partially how we unit test the serialization itself, but this approach turns your controller unit tests into more of an integration tests. Similar to how tests would behave with integrate_views/render_views turned on in RSpec.

I've found this approach to end up being mostly annoying, especially as you're mid-development of the next version of a public API. You change the schema and get, from a unit test perspective, a bunch of false negatives that you need to chase down.

Preferably in this situation only the unit tests for serialization and any actual integration tests would report failures.

[joaomdmoura]

Great @bolshakov, I like json-schema, and indeed it seems a good approach. I'm deeply interested into documentation stuff right now 😄
@trek I would not discard the implementation itself, as you said, it still pretty useful. You could start creating a new using with a proposal of it as reference point.

[bolshakov]

@trek look at discussion about deprecating assert_template in Rails. It contains interesting thoughts about testing controllers.

@joaomdmoura if you have some ideas about documenting API, please don't hesitate to share it with us :)

[joaomdmoura]

😁 @bolshakov let's chat on IRC or over email someday. But I'll open an issue to discuss it once I finish deserialization 😄

[maurogeorge]

Hi guys,

I made a PR to bring back the assert_serializer #1099

@bolshakov the gem is json-schema-rspec and here has a post about it too.

I really like the idea of @bolshakov I think we can have both helpers the assert_serializer and maybe a assert_json_schema this make sense to you guys?

[trek]

Closing in favor of #1099

[joaomdmoura]

moving to #1099