Testing of RESTful API in context of continuous integration

• Description
• Requirements
• Acknowledgements
• Abstract EN
• Abstract CS
• Table of Content
• Bibliography and References

Description

This is the text of master thesis at FIT CTU in Prague written by Martin Štekl. Feel free to contact me on email or follow me on Twitter.

Original requirements can be found as a Gist.

Project related to this text can be found in separate repository.

Requirements

Study the basics of testing, different methods, levels and types of testing, processes related to software testing and describe the strengths and weaknesses of various methods, levels and types of tests. Find and compare different tools that focus on testing RESTful API in the context of continuous integration or the description of the RESTful API. Based on these findings describe the differences and similarities in methods of testing code itself for regular SW and methods for testing RESTful API.

Study the syntax of the API Blueprint language for describing RESTful API and study testing tool Dredd. Compare current testing possibilities using this tool to previously described methods and then propose possible changes and modifications to extend the capabilities of this tool.

Choose some of the proposed modifications and implement them. Propose implemented changes as Pull Requests on server GitHub for merge in main repositories of this tool.

Table of Content

1. Introduction
2. Software testing in general
   1. Testing methods
      1. Black-Box testing
      2. White-Box testing
      3. Grey-Box testing
   2. Testing levels
      1. Unit testing
      2. Integration testing
      3. System testing
      4. Acceptance testing
   3. Testing types
      1. Compatibility testing
      2. Smoke and sanity testing
      3. Regression testing
      4. Acceptance testing
      5. Destructive testing
      6. Performance testing
      7. Accessibility testing
      8. Security testing
   4. Testing process
      1. Continuous integration
   5. Differences between RESTful API testing and software testing in general
3. RESTful API description and testing tools
   1. Specification of important parameters for comparison
   2. Basic tools description
      1. SOAP UI
      2. REST assured
      3. frisby.js
      4. RAML
      5. Runscope Radar
      6. Dredd
   3. Tools comparison
4. Details about API Blueprint and Dredd
   1. API Blueprint sytax basics
      1. Resources
      2. Actions
      3. Request and Response
      4. Models
   2. Dredd testing process
      1. Test configuration and running
      2. CI server integration
      3. Validations
5. Creation of scenario testing tool
   1. Requirements
   2. Implementation possibilities
   3. Implementation of proof of concept for scenario testing tool using Cucumber
   4. Improvements and refactoring of the scenario testing tool
   5. Testing of scenario testing tool
6. Conclusion
7. Installation and an example of usage
8. Bibliography

Bibliography and References

• [1]: R. Fielding. Architectural Styles and the Design of Network-based Software Architectures. 2000. (http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm)
• [2]: API Blueprint Language Specification. (https://github.com/apiaryio/api-blueprint/blob/master/API%20Blueprint%20Specification.md)
• [3]: JSON Schema. (http://json-schema.org/)
• [4]: Trello. (https://trello.com/)
• [5]: GitHub. (https://github.com/)
• [6]: Cucumber. (https://github.com/cucumber/cucumber-js)
• [7]: SOAP UI. (http://www.soapui.org/)
• [8]: REST-assured. (https://code.google.com/p/rest-assured/wiki/GettingStarted)
• [9]: Postman. (http://www.getpostman.com/)
• [10]: frisby.js. (http://frisbyjs.com/)
• [11]: RAML. (http://raml.org/)
• [12]: Runscope Radar. (https://www.runscope.com/)
• [13]: Dredd. (https://github.com/apiaryio/dredd)
• [14]: R. Fielding; J. Gettys; J. Mogul; H. Frystyk; L. Masinter; P. Leach; T. Berners-Lee. Hypertext Transfer Protocol -- HTTP/1.1. 1999. (http://www.w3.org/Protocols/rfc2616/rfc2616.html)
• [15]: Dredd hooks. 2014. (https://github.com/apiaryio/dredd/wiki/Writing-Hooks)
• [16]: Dredd ALL transactions hook. 2014. (apiaryio/dredd#63)
• [17]: Gavel. (https://www.relishapp.com/apiary/gavel/docs)
• [18]: Amanda. (https://github.com/apiaryio/Amanda)
• [19]: CoffeeScript. (http://coffeescript.org)
• [20]: Apiary. (http://apiary.io)
• [21]: Selenium. (http://docs.seleniumhq.org)
• [22]: Balanced API. (https://github.com/balanced/balanced-api)
• [23]: Buster.js. (http://docs.busterjs.org/en/v0.7.x)
• [24]: Jasmine. (http://jasmine.github.io)
• [25]: Chai. (http://chaijs.com)
• [26]: Mocha. (http://mochajs.org/)
• [27]: Gherkin. (https://github.com/cucumber/gherkin)
• [28]: Protagonist. (https://github.com/apiaryio/protagonist)
• [29]: Node.js. (http://nodejs.org)
• [30]: express. (http://expressjs.com)
• [31]: Nock. (https://github.com/pgte/nock)
• [32]: T. Berners-Lee; R. Fielding; L. Masinter. Uniform Resource Identifier (URI): Generic Syntax. 2005. (http://www.ietf.org/rfc/rfc3986.txt)
• [33]: NPM. (https://www.npmjs.org)
• [34]: Google Code. (https://code.google.com)
• [35]: jUnit. (http://junit.org)
• [36]: API Blueprint. (http://apiblueprint.org)
• [37]: M. Štekl. Cucumber 4 API Blueprint. 2014. (https://github.com/stekycz/cucumber-4-api-blueprint)
• [38]: MIT license. (http://opensource.org/licenses/MIT)