Add finer-grain reference tests for translation phase (#103)

### Motivation We have a reference test, which we've been making use of to validate end-to-end generation from OpenAPI document matches handwritten Swift files. This started out life as the canonical Petstore example from the OpenAPI specification but has since been augmented in various ways to test out some edge cases. This approach isn't scaling too well as we have an all-or-nothing test. We also don't have great test coverage over the translation phase of the generator pipeline, which translates between the parsed OpenAPI document to an intermediate representation. ### Modifications Add test harnesses for finer grain reference tests that exercise smaller sections of the translation phase. It should allow us to test a small OpenAPI document renders to _equivalent_ Swift source (stripping comments and reformatting). ### Result Hopefully a more modular and maintainable testing strategy as we try to support more of the OpenAPI specification. ### Test Plan This PR _is_ tests. --------- Signed-off-by: Si Beaumont <[email protected]>
apple · Jul 14, 2023 · 914560c · 914560c
1 parent 8c5258b
commit 914560c
Show file tree

Hide file tree

Showing 3 changed files with 813 additions and 31 deletions.
diff --git a/...neratorReferenceTests/ReferenceTest.swift → ...erenceTests/FileBasedReferenceTests.swift b/...neratorReferenceTests/ReferenceTest.swift → ...erenceTests/FileBasedReferenceTests.swift
@@ -33,7 +33,7 @@ extension TestConfig {
 }
 
 /// Tests that the generator produces Swift files that match a reference.
-class ReferenceTests: XCTestCase {
+class FileBasedReferenceTests: XCTestCase {
     override func setUp() {
         super.setUp()
         continueAfterFailure = false
@@ -82,10 +82,8 @@ class ReferenceTests: XCTestCase {
         )
 
         // Run the requested generator invocation
-        let diagnostics: any DiagnosticCollector = strictDiagnosticsCollector
         let generatorPipeline = self.makeGeneratorPipeline(
-            config: referenceTest.asConfig,
-            diagnostics: diagnostics
+            config: referenceTest.asConfig
         )
         let generatedOutputSource = try generatorPipeline.run(input)
 
@@ -143,32 +141,8 @@ class ReferenceTests: XCTestCase {
     }
 }
 
-struct StrictDiagnosticsCollector: DiagnosticCollector {
-    var test: XCTestCase
-
-    func emit(_ diagnostic: Diagnostic) {
-        print("Test emitted diagnostic: \(diagnostic.description)")
-        switch diagnostic.severity {
-        case .note:
-            // no need to fail, just print
-            break
-        case .warning, .error:
-            XCTFail("Failing with a diagnostic: \(diagnostic.description)")
-        }
-    }
-}
-
-extension ReferenceTests {
-
-    var strictDiagnosticsCollector: any DiagnosticCollector {
-        StrictDiagnosticsCollector(test: self)
-    }
-
-    private func makeGeneratorPipeline(
-        config: Config,
-        diagnostics: any DiagnosticCollector
-    ) -> GeneratorPipeline {
-
+extension FileBasedReferenceTests {
+    private func makeGeneratorPipeline(config: Config) -> GeneratorPipeline {
         let parser = YamsParser()
         let translator = MultiplexTranslator()
         let renderer = TextBasedRenderer()
@@ -183,7 +157,7 @@ extension ReferenceTests {
                 return newFile
             },
             config: config,
-            diagnostics: diagnostics
+            diagnostics: XCTestDiagnosticCollector(test: self)
         )
     }