diff --git a/content/06.quality_criteria.md b/content/06.quality_criteria.md index 0bb8e73..c720a56 100644 --- a/content/06.quality_criteria.md +++ b/content/06.quality_criteria.md @@ -12,7 +12,10 @@ new contributions in order to validate them before being added to the software c Consequently, software components are more eligible for deployment in production infrastructures, reducing the likelihood of service disruption. -## 4.1. Code Accessibility [QC.Acc] + +## 4.1. Code management + +### 4.1.1. Code Accessibility [QC.Acc] * **[QC.Acc01]** Following the open-source model, the source code being produced **MUST** be open and publicly available to promote the adoption and augment the visibility of the software @@ -26,7 +29,71 @@ reducing the likelihood of service disruption. * **[QC.Acc03]** Source code produced within the scope of a broader development project **SHOULD** reside in a common organization of a version control repository hosting service. -## 4.2. Licensing [QC.Lic] +### 4.1.2. Code Workflow [QC.Wor] + +A change-based approach is accomplished with a branching model. + +* **[QC.Wor01]** The main branch in the source code repository **MUST** maintain a working state + version of the software component. + + * **[QC.Wor01.1]** Main branch **SHOULD** be protected to disallow force pushing, thus preventing + untested and un-reviewed source code from entering the production-ready version. + + * **[QC.Wor01.2]** New features **SHOULD** only be merged in the main branch whenever the SQA + criteria is fulfilled. + +* **[QC.Wor02]** New changes in the source code **MUST** be placed in individual branches. + + * **[QC.Wor02.1]** It is **RECOMMENDED** to agree on a branch nomenclature, usually by prefixing, + to differentiate change types (e.g. feature, release, fix). + +* **[QC.Wor03]** The existence of a secondary long-term branch that contains the changes for the + next release is **RECOMMENDED**. + + * **[QC.Wor03.1]** Next release changes **SHOULD** come from the individual branches. + + * **[QC.Wor03.2]** Once ready for release, changes in the secondary long-term branch are merged + into the main branch and versioned. At that point in time, main and secondary branches are + aligned. This step **SHOULD** mark a production release. + +### 4.1.3. Code Management [QC.Man] + +* **[QC.Man01]** An issue tracking system facilitates structured software development. Leveraging + issues to track down both new enhancements and defects (bugs, documentation typos) is + **RECOMMENDED**. + + * **[QC.Man01.1]** In addition to monitoring the internal development, issues are the best means + for supporting users. External users **SHOULD** be able to create issues based on the + operational performance of the software. + + * **[QC.Man01.2]** The description of an issue **SHOULD** be concise and state clearly the + problem. It is **RECOMMENDED** to add any reference to the actual problem. In the case of bugs, + the issue **SHOULD** be accompanied by the relevant debug information. + * The usage of templates for the issue description is **RECOMMENDED**. + +* **[QC.Man02]** In social coding environments, pull or merge requests represent the cornerstone of + collaboration. A pull or merge request provides a place for review and discussion of the changes + proposed to be part of an existing version of the code. + + * **[QC.Man02.1]** Pull/Merge requests **SHOULD** be used for every change in the codebase. + + * **[QC.Man02.2]** A software project **SHOULD** be open to external collaboration through + pull/merge requests. + + * **[QC.Man02.3]** A pull/merge request description **SHOULD** be concise and state clearly its + purpose (e.g. if it is fixing an observed bug or adding a new feature). + + * **[QC.Man02.4]** The usage of templates for the pull/merge request's description is + **RECOMMENDED**. + + * **[QC.Man02.5]** It is **RECOMMENDED** to use pull/merge requests to address open issues. + + * **[QC.Man02.6]** The pull/merge request description **SHOULD** make reference to the identifiers + of the issues it is fixing (to eventually close them, either manually or automatically). + +## 4.2. Files and documents + +### 4.2.1. Licensing [QC.Lic] * **[QC.Lic01]** As open-source software, source code **MUST** adhere to an open-source license to be freely used, modified and distributed by others. Non-licensed software is exclusive copyright @@ -41,7 +108,91 @@ reducing the likelihood of service disruption. * **[QC.Lic02.1]** **RECOMMENDED** licenses are listed in the Open Source Initiative portal under the Popular Licenses category [@https://opensource.org/licenses]. -## 4.3. Code Style [QC.Sty] +### 4.2.2. Code metadata [QC.Met] + +Metadata for the software component provides a way to achieve its full identification, thus making +software citation viable [@doi:10.7717/peerj-cs.86]. It allows the assignment of a Digital Object +Identifier (DOI) and is key towards preservation, discovery, reuse, and attribution of the software +component. + +* **[QC.Met01]** A metadata file **SHOULD** exist along side the code, under its VCS. The metadata + file **SHOULD** be updated when needed, as is the case of a new version. + +### 4.2.3. Documentation [QC.Doc] + +* **[QC.Doc01]** Documentation **MUST** be treated as code. + + * **[QC.Doc01.1]** Version controlled, it **MAY** reside in the same repository where the source + code lies. + +* **[QC.Doc02]** Documentation **MUST** use plain text format using a markup language, such as + Markdown or reStructuredText. + + * **[QC.Doc02.1]** It is **RECOMMENDED** that all software components delivered by the same + project agree on a common markup language. + +* **[QC.Doc03]** Documentation **MUST** be online and available in a documentation repository. + + * **[QC.Doc03.1]** Documentation **SHOULD** be rendered automatically. + +* **[QC.Doc04]** Documentation **MUST** be updated on new software versions involving any + substantial or minimal change in the behavior of the application. + +* **[QC.Doc05]** Documentation **MUST** be updated whenever reported as inaccurate or unclear. + +* **[QC.Doc06]** Documentation **MUST** be produced according to the target audience, varying + according to the software component specification. The identified types of documentation and their + RECOMMENDED content are: + + * **[QC.Doc06.1]** README file **MUST** be present: + + * One-paragraph description of the application. + * A *"Getting Started"* step-by-step description on how to get a development environment running + (prerequisites, installation). + * Automated test execution how-to. + * Links to the external documentation below (production deployment, user guides). + * Versioning specification. + * Author list and contacts. + * Acknowledgments. + + * **[QC.Doc06.2]** CONTRIBUTING file **MUST** be present in order to communicate how external + parties can contribute to the code. + + * **[QC.Doc06.3]** A code of conduct (usually defined in a CODE_OF_CONDUCT file) **MUST** be + present in order to establish the positive social attitudes expected within the community of + code contributors. + + * **[QC.Doc06.4]** LICENSE file **MUST** be present, License information with detailed description. + + * **[QC.Doc06.5]** Developer + + * Private API documentation. + * Structure and interfaces. + * Build documentation. + + * **[QC.Doc06.6]** Deployment and Administration + + * Installation and configuration guides. + * Service Reference Card, with the following RECOMMENDED content: + * Brief functional description. + * List of processes or daemons. + * Init scripts and options. + * List of configuration files, location and example or template. + * Log files location and other useful audit information. + * List of ports. + * Service state information. + * List of cron jobs. + * Security information. + * FAQs and troubleshooting. + + * **[QC.Doc06.7]** User + + * Public API documentation. + * Command Line Interface (CLI) reference. + +## 4.3. Static testing - White box testing + +### 4.3.1. Code Style [QC.Sty] Code style requirements pursue the correct maintenance of the source code by the common agreement of a series of style conventions. These vary based on the programming language being used. @@ -67,17 +218,7 @@ a series of style conventions. These vary based on the programming language bein * **[QC.Sty04]** Code style compliance testing **MUST** be automated and **MUST** be triggered for each candidate change in the source code. -## 4.4. Code metadata [QC.Met] - -Metadata for the software component provides a way to achieve its full identification, thus making -software citation viable [@doi:10.7717/peerj-cs.86]. It allows the assignment of a Digital Object -Identifier (DOI) and is key towards preservation, discovery, reuse, and attribution of the software -component. - -* **[QC.Met01]** A metadata file **SHOULD** exist along side the code, under its VCS. The metadata - file **SHOULD** be updated when needed, as is the case of a new version. - -## 4.5. Unit Testing [QC.Uni] +### 4.3.2. Unit Testing [QC.Uni] Unit testing evaluates all the possible flows in the internal design of the code, so that its behavior becomes apparent. It is a key type of testing for early detection of failures in the @@ -100,7 +241,7 @@ development cycle. * **[QC.Uni04.1]** When working on automated testing, the use of testing doubles is **RECOMMENDED** to mimic a simplistic behavior of objects and procedures (c.f. section 4.6.). -## 4.6. Test Harness [QC.Har] +### 4.3.3. Test Harness [QC.Har] In software development, a test harness [@https://searchsoftwarequality.techtarget.com/definition/test-harness], is a collection of software @@ -144,7 +285,7 @@ As such the following criteria is defined for Test Harness: * **[QC.Har04]** Test Doubles and regression, **MUST** be checked on change basis. -## 4.7. Test-Driven Development [QC.Tdd] +### 4.3.4. Test-Driven Development [QC.Tdd] Test-Driven Development [@isbn:9780321146533], is a software development process relying on software requirements being converted to test cases before software is fully developed, and tracking all @@ -154,78 +295,7 @@ software being developed first and test cases created later. * **[QC.Tdd01]** Software requirements **SHOULD** be converted to test cases, and these test cases **SHOULD** be checked automatically. -## 4.8. Documentation [QC.Doc] - -* **[QC.Doc01]** Documentation **MUST** be treated as code. - - * **[QC.Doc01.1]** Version controlled, it **MAY** reside in the same repository where the source - code lies. - -* **[QC.Doc02]** Documentation **MUST** use plain text format using a markup language, such as - Markdown or reStructuredText. - - * **[QC.Doc02.1]** It is **RECOMMENDED** that all software components delivered by the same - project agree on a common markup language. - -* **[QC.Doc03]** Documentation **MUST** be online and available in a documentation repository. - - * **[QC.Doc03.1]** Documentation **SHOULD** be rendered automatically. - -* **[QC.Doc04]** Documentation **MUST** be updated on new software versions involving any - substantial or minimal change in the behavior of the application. - -* **[QC.Doc05]** Documentation **MUST** be updated whenever reported as inaccurate or unclear. - -* **[QC.Doc06]** Documentation **MUST** be produced according to the target audience, varying - according to the software component specification. The identified types of documentation and their - RECOMMENDED content are: - - * **[QC.Doc06.1]** README file **MUST** be present: - - * One-paragraph description of the application. - * A *"Getting Started"* step-by-step description on how to get a development environment running - (prerequisites, installation). - * Automated test execution how-to. - * Links to the external documentation below (production deployment, user guides). - * Versioning specification. - * Author list and contacts. - * Acknowledgments. - - * **[QC.Doc06.2]** CONTRIBUTING file **MUST** be present in order to communicate how external parties can - contribute to the code. - - * **[QC.Doc06.3]** A code of conduct (usually defined in a CODE_OF_CONDUCT file) **MUST** be present in - order to establish the positive social attitudes expected within the community of code contributors. - - * **[QC.Doc06.4]** LICENSE file **MUST** be present, License information with detailed description. - - * **[QC.Doc06.5]** Developer - - * Private API documentation. - * Structure and interfaces. - * Build documentation. - - * **[QC.Doc06.6]** Deployment and Administration - - * Installation and configuration guides. - * Service Reference Card, with the following RECOMMENDED content: - * Brief functional description. - * List of processes or daemons. - * Init scripts and options. - * List of configuration files, location and example or template. - * Log files location and other useful audit information. - * List of ports. - * Service state information. - * List of cron jobs. - * Security information. - * FAQs and troubleshooting. - - * **[QC.Doc06.7]** User - - * Public API documentation. - * Command Line Interface (CLI) reference. - -## 4.9. Security [QC.Sec] +### 4.3.5. Security [QC.Sec] Security assessment is essential for any production **Software**. An effective implementation of the security requirements applies to every stage in the Software Development Life Cycle (SDLC), @@ -252,74 +322,14 @@ especially effective at the source code level. * **[QC.Sec04]** World-writable files or directories **MUST NOT** be present in the product’s configuration or logging locations. -## 4.10. Code Workflow [QC.Wor] - -A change-based approach is accomplished with a branching model. - -* **[QC.Wor01]** The main branch in the source code repository **MUST** maintain a working state - version of the software component. - - * **[QC.Wor01.1]** Main branch **SHOULD** be protected to disallow force pushing, thus preventing - untested and un-reviewed source code from entering the production-ready version. - - * **[QC.Wor01.2]** New features **SHOULD** only be merged in the main branch whenever the SQA - criteria is fulfilled. +## 4.4. Code release -* **[QC.Wor02]** New changes in the source code **MUST** be placed in individual branches. - - * **[QC.Wor02.1]** It is **RECOMMENDED** to agree on a branch nomenclature, usually by prefixing, - to differentiate change types (e.g. feature, release, fix). - -* **[QC.Wor03]** The existence of a secondary long-term branch that contains the changes for the - next release is **RECOMMENDED**. - - * **[QC.Wor03.1]** Next release changes **SHOULD** come from the individual branches. - - * **[QC.Wor03.2]** Once ready for release, changes in the secondary long-term branch are merged - into the main branch and versioned. At that point in time, main and secondary branches are - aligned. This step **SHOULD** mark a production release. - -## 4.11. Semantic Versioning [QC.Ver] +### 4.4.1. Semantic Versioning [QC.Ver] * **[QC.Ver01]** Semantic Versioning [@https://semver.org] specification is **RECOMMENDED** for tagging the production releases. -## 4.12. Code Management [QC.Man] - -* **[QC.Man01]** An issue tracking system facilitates structured software development. Leveraging - issues to track down both new enhancements and defects (bugs, documentation typos) is - **RECOMMENDED**. - - * **[QC.Man01.1]** In addition to monitoring the internal development, issues are the best - means for supporting users. External users **SHOULD** be able to create issues based on the - operational performance of the software. - - * **[QC.Man01.2]** The description of an issue **SHOULD** be concise and state clearly the - problem. It is **RECOMMENDED** to add any reference to the actual problem. In the case of bugs, - the issue **SHOULD** be accompanied by the relevant debug information. - * The usage of templates for the issue description is **RECOMMENDED**. - -* **[QC.Man02]** In social coding environments, pull or merge requests represent the cornerstone of - collaboration. A pull or merge request provides a place for review and discussion of the changes - proposed to be part of an existing version of the code. - - * **[QC.Man02.1]** Pull/Merge requests **SHOULD** be used for every change in the codebase. - - * **[QC.Man02.2]** A software project **SHOULD** be open to external collaboration through - pull/merge requests. - - * **[QC.Man02.3]** A pull/merge request description **SHOULD** be concise and state clearly its - purpose (e.g. if it is fixing an observed bug or adding a new feature). - - * **[QC.Man02.4]** The usage of templates for the pull/merge request's description is - **RECOMMENDED**. - - * **[QC.Man02.5]** It is **RECOMMENDED** to use pull/merge requests to address open issues. - - * **[QC.Man02.6]** The pull/merge request description **SHOULD** make reference to the identifiers - of the issues it is fixing (to eventually close them, either manually or automatically). - -## 4.13. Code Review [QC.Rev] +### 4.4.2. Code Review [QC.Rev] Code review implies the informal, non-automated, peer review of any change in the source code [@https://owasp.org/www-project-code-review-guide/migrated_content]. It appears as the last step in @@ -367,7 +377,9 @@ set of change-based tests. * **[QC.Rev05]** Code reviews **SHOULD** assess the inherent security risk of the changes, ensuring that the security model has not been downgraded or compromised by the changes. -## 4.14. Automated Delivery [QC.Del] +## 4.5. Delivery + +### 4.5.1. Automated Delivery [QC.Del] Automated delivery comprises the build of Software into an artifact, its upload/registration into a public repository of such artifacts and notification of the success of the process. @@ -384,10 +396,12 @@ public repository of such artifacts and notification of the success of the proce * **[QC.Del03]** Upon success of the previous (**QC.Del02**) process, a notification **MUST** be sent to pre-defined parties such as the main developer or team. -## 4.15. Automated Deployment [QC.Dep] +## 4.6. Deployment + +### 4.6.1. Automated Deployment [QC.Dep] -* **[QC.Dep01]** Production-ready code **MUST** be deployed as a workable system with the minimal user - or system administrator interaction leveraging software configuration management (SCM) tools. +* **[QC.Dep01]** Production-ready code **MUST** be deployed as a workable system with the minimal + user or system administrator interaction leveraging software configuration management (SCM) tools. * **[QC.Dep02]** A SCM module is treated as code.