Discussion about some features

[LinusU]

Ping @netmilk

I think that dredd is a really cool product, and I would like to use it for pretty much every api that I build from now on. However, there is a few things that I don't know how to best accomplish in dredd at the moment and that is what I would like to open a discussion on.

I have a quick and dirty implementation of what I want in LinusU/mocha-api-blueprint, don't be fooled by the mocha in the name, that is only a minor part.

I have also created an example of how it is used at LinusU/mocha-api-blueprint-example. This is the beautiful specifications and the strict testing that I want to accomplish.

The first thing that I would like to mention is the response body. I've added a feature where I can specify the value as {variableName} and have it accept any value and store it for later. As you can see this makes it possible to test that I can make a POST request and then GET that same resource back.

This also allows me to use assert.deepEqual() for testing the values instead of guessing a JSON-schema for the response. It will make sure that the api actually stores the correct values.

The parameters in the uriTemplates will now fetch the previously stored value if applicable. Please run the tests in mocha-api-blueprint-example to see this in action. The api will return random ids every time but yet it will work beautifully.

That was the most important feature, but I have also integrated very tightly with mocha and I think it would be great if it was very easy to integrate into any existing test frameworks. I think that this would lower the barrier for other people to start using dredd, since it would fit right into their existing setup.

I think that a nice way of doing this is to expose all the transactions that needs to be run to the user, divided by category, and then let the user run them and return the response. This also ties in with the next thing; letting the user send the requests.

Now, I think that this part should be optional; but I think it's brilliant to let the user run the transaction and return the result, instead of having lots of hooks. It gives the user the most control, and it makes for cleaner code.

We could just have a default handler that does the normal thing.

I realize that this is a lot of very brave changes but what I really want to have is a discussion where we can see how much of this would be feasible to have in dredd, and how much that I should keep outside.

It's really two issues, the one that allows multiple and stricter tests, and the one that allows tighter integration with existing test frameworks.

Happy coding 🍻

[LinusU]

ping @kuba-kubula

[netmilk]

Hi @LinusU,

this is a reply for your question in #151. I think it fits more here than to that issue.

I'm so sorry it took me this long time to respond you and thank you for pinging me. 😏 It's super awesome how you are thinking about Dredd and how to connect it with Mocha. It's very helpful to have a feedback from a experienced JS developer like you.

About your points:

1. Deep property testing

Best practice for doing this is to use chai in Dredd's hooks and write an imperative assertion on properties and values which you want to validate. You can easily write a function for dynamically generate assertion for all properties by traversing the object. Dredd is able to catch Chai assertion errors (e.g. assert.deepEqualwhich you mentioned above) as a reportable errors the same way as Mocha is doing it. Think about hook functions like about Mocha's it blocks.

2. Sending requests imperatively

JSON bodies for request and expected response come from the blueprint as text. They are always serialized to string, so you can't avoid body deserialization if you want to validate the object structure or change it before or after execution.

I'd like to stress that Dredd is suppose to be language agnostic testing framework for HTTP APIs and their documentation. Think about Dredd as about integration framework. Avoiding the HTTP stack does not make lot of sense, because APIs can be split across

The signature for the hooks functions is exactly the same how you wrote above: (transaction, done) Please take a look at the transaction object documentation, there are accessible both request and expected response objects. You can do anything with these objects before and after the transaction is fired. It's only an upside-down approach in comparison to what you propose, but it does absolutely the same in my opinion.

3. Integration with existing test suite

Thank you very much for stressing how important is to lowering the barrier for users with existing test suites. Developer experience and steep learning curve has to be first class citizen in Dredd. This is the reason why dredd init was introduced and lot of documentation was added in the past.

My strong belief is that Dredd is completely different test framework than Mocha and it should not be part of Mocha itself. It's a different tool for running and testing HTTP transactions. Dredd is intended to be language agnostic with future support for hooks written in Ruby, Python etc...

It's something very similar to jMeter or other testing tools. They are part of the overall test suite and they are not integrated into main Mocha. You can wrap them in some grunt or gulp task. And pipe the exit status to tthe CI or you can create a simple shell script wrapping all test suites together.

Please let me know what you think. I'd love to discuss it further if you want. As I said I totally welcome any feedback and new points of view. If you want, we can schedule a call regarding this, just let me know when it works for you at [email protected].

[netmilk]

Hey @LinusU I anything I can help you with? As I mentioned I'd love to discuss it further to make everything mentioned above more convinent. I'm happy to have a call with you anytime it fits to you regarding this. Just ping me.

[LinusU]

Hey @netmilk, I've been pulled over to another project for at least a week more so I don't have anything in my head. The next thing on the roadmap is this again so I'll be back shortly.

I'm going to take another stab at implementing it they way you suggested in your post and get back with my comments. It would be great to have a call, but lets take that when I'm sitting with it 👍

[honzajavorek]

@netmilk @LinusU Any updates on this?

[honzajavorek]

@netmilk @LinusU Feel free to re-open or get to a new issue with more ideas! Closing this now for inactivity.