Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(webAPI): Make all lists nullable in OpenAPI schema #1359

Merged

Conversation

oskogstad
Copy link
Collaborator

@oskogstad oskogstad commented Oct 29, 2024

Description

Related Issue(s)

Verification

  • Your code builds clean without any errors or warnings
  • Manual testing done (required)
  • Relevant automated test added (if you find this hard, leave it and we'll help out)

Documentation

  • Documentation is updated (either in docs-directory, Altinnpedia or a separate linked PR in altinn-studio-docs., if applicable)

Summary by CodeRabbit

  • New Features

    • Updated API specifications to allow properties to be nullable, enhancing flexibility in handling optional data.
    • Introduced a method to make array-type properties nullable in OpenAPI document schemas.
  • Bug Fixes

    • Updated serialization logic to include empty collections during data serialization, affecting the output of serialized data.
  • Tests

    • Enhanced assertion logic in integration tests to require closer matches for DateTimeOffset types.
    • Modified dialog retrieval tests to no longer exclude missing members in assertions, improving accuracy in test validations.
    • Refined response handling in dialog search tests for better clarity and accuracy in expectations.

Copy link
Contributor

coderabbitai bot commented Oct 29, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

The pull request modifies the serialization behavior in the Program class by removing the IgnoreEmptyCollections method, which previously filtered out empty collections during serialization. This change allows empty collections to be included in serialized data. Additionally, the DialogApplication class in the integration tests has been updated to alter assertion behavior and enhance MassTransit configuration. The GetDialogTests class also sees changes to assertion logic regarding missing members. Overall, the focus is on refining serialization and testing configurations without changing existing control flows.

Changes

File Change Summary
src/Digdir.Domain.Dialogporten.WebApi/Program.cs Removed IgnoreEmptyCollections method; updated serializer configuration to include specific converters without ignoring empty collections.
tests/Digdir.Domain.Dialogporten.Application.Integration.Tests/Common/DialogApplication.cs Updated AssertionOptions.AssertEquivalencyUsing to include missing members; enhanced ConfigureServicesWithMassTransitTestHarness for dynamic consumer configuration.
tests/Digdir.Domain.Dialogporten.Application.Integration.Tests/Features/V1/ServiceOwner/Dialogs/Queries/GetDialogTests.cs Removed .ExcludingMissingMembers() from assertions in Get_ReturnsSimpleDialog_WhenDialogExists and Get_ReturnsDialog_WhenDialogExists.
tests/k6/tests/serviceowner/dialogSearch.js Introduced response variable for JSON response handling; updated checks for the "items" property to ensure accurate validation.
docs/schema/V1/swagger.verified.json Added nullable attribute to various properties across multiple schemas to allow explicit null values.
src/Digdir.Domain.Dialogporten.WebApi/OpenApiDocumentExtensions.cs Added MakeCollectionsNullable method to handle nullable collections in OpenAPI document schemas.

Possibly related PRs

Suggested reviewers

  • MagnusSandgren

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@oskogstad oskogstad force-pushed the feat/remove-filtering-of-empty-collections-when-serializing branch from 316478a to 33cf27a Compare November 1, 2024 08:47
@oskogstad oskogstad changed the title [DRAFT] feat(webAPI): Remove filtering of empty collections when serializing response feat(webAPI): Remove filtering of empty collections when serializing response Nov 1, 2024
@oskogstad oskogstad marked this pull request as ready for review November 1, 2024 09:04
@oskogstad oskogstad requested a review from a team as a code owner November 1, 2024 09:04
arealmaas
arealmaas previously approved these changes Nov 1, 2024
@elsand
Copy link
Member

elsand commented Nov 1, 2024

#1355 (comment)

Hm, hvis vi gjør om en List til å være nullable, gir det da ikke mening å initialisere den til null i stedet for [], slik at den da blir fjernet med Serializer.Options.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull som vi allerede bruker? Eller mener du noe annet?

@oskogstad

elsand
elsand previously approved these changes Nov 1, 2024
Copy link
Collaborator

@MagnusSandgren MagnusSandgren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

sonarqubecloud bot commented Nov 6, 2024

@oskogstad oskogstad changed the title feat(webAPI): Remove filtering of empty collections when serializing response feat(webAPI): Make all lists nullable in OpenAPI schema Nov 6, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (4)
src/Digdir.Domain.Dialogporten.WebApi/OpenApiDocumentExtensions.cs (1)

84-90: Add XML documentation for consistency.

Other public methods in this file have XML documentation explaining their purpose and behavior. Consider adding documentation to maintain consistency.

+    /// <summary>
+    /// Makes all collection properties in the OpenAPI document nullable to support serialization of empty collections.
+    /// </summary>
+    /// <param name="openApiDocument">The OpenAPI document to process.</param>
     public static void MakeCollectionsNullable(this OpenApiDocument openApiDocument)

Also, consider adding null checks for defensive programming:

     public static void MakeCollectionsNullable(this OpenApiDocument openApiDocument)
     {
+        ArgumentNullException.ThrowIfNull(openApiDocument);
+        ArgumentNullException.ThrowIfNull(openApiDocument.Components?.Schemas);
+
         foreach (var schema in openApiDocument.Components.Schemas.Values)
src/Digdir.Domain.Dialogporten.WebApi/Program.cs (3)

Line range hint 166-173: Update serializer configuration to remove empty collection filtering.

The current configuration still includes the IgnoreEmptyCollections modifier which contradicts the PR's objective of allowing empty collections in the response.

Apply this diff to update the serializer configuration:

 x.Serializer.Options.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull;
-            // Do not serialize empty collections
-            x.Serializer.Options.TypeInfoResolver = new DefaultJsonTypeInfoResolver
-            {
-                Modifiers = { IgnoreEmptyCollections }
-            };

Line range hint 214-223: Remove unused IgnoreEmptyCollections method.

This method is no longer used after updating the serializer configuration and should be removed to maintain code cleanliness.

Remove the entire method:

-static void IgnoreEmptyCollections(JsonTypeInfo typeInfo)
-{
-    foreach (var property in typeInfo.Properties)
-    {
-        if (property.PropertyType.IsAssignableTo(typeof(ICollection)))
-        {
-            property.ShouldSerialize = (_, val) => val is ICollection collection && collection.Count > 0;
-        }
-    }
-}

Line range hint 166-166: Consider using null instead of empty arrays.

As discussed in PR comments, since DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull is already configured, consider initializing empty collections as null instead of empty arrays ([]) in your models. This would achieve the same effect of omitting empty collections from serialization while being more memory efficient.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 08f5c50 and a3675ea.

📒 Files selected for processing (4)
  • docs/schema/V1/swagger.verified.json (74 hunks)
  • src/Digdir.Domain.Dialogporten.WebApi/OpenApiDocumentExtensions.cs (1 hunks)
  • src/Digdir.Domain.Dialogporten.WebApi/Program.cs (1 hunks)
  • tests/k6/tests/serviceowner/dialogSearch.js (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/k6/tests/serviceowner/dialogSearch.js
🔇 Additional comments (4)
src/Digdir.Domain.Dialogporten.WebApi/OpenApiDocumentExtensions.cs (2)

83-106: Verify coordination with JSON serialization settings.

Based on the PR discussion, consider if this OpenAPI documentation change should be coordinated with corresponding changes to the JSON serialization configuration (e.g., JsonIgnoreCondition.WhenWritingNull). This would ensure consistency between the API documentation and actual behavior.

✅ Verification successful

OpenAPI nullable collections align with JSON serialization configuration

The OpenAPI documentation changes are consistent with the existing JSON serialization configuration. The codebase already:

  • Sets DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull in WebApi
  • Includes logic to ignore empty collections
  • Uses WhenWritingDefault for cloud events

This means the nullable collections in OpenAPI spec correctly reflect the actual runtime behavior.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Look for JSON serialization configuration

# Test: Search for JsonIgnoreCondition configuration
rg -l "JsonIgnoreCondition" 

# Test: Search for serialization setup in Program.cs or similar startup files
fd -e cs -x rg -l "AddJsonOptions|ConfigureJsonOptions"

Length of output: 468


Script:

#!/bin/bash
# Let's check the actual JSON serialization configuration in the identified files

# Check Program.cs for JSON configuration
echo "=== WebApi Program.cs JSON Configuration ==="
rg -A 5 -B 5 "JsonIgnoreCondition|AddJsonOptions|ConfigureJsonOptions" src/Digdir.Domain.Dialogporten.WebApi/Program.cs

echo "\n=== SerializerOptions.cs Configuration ==="
cat src/Digdir.Domain.Dialogporten.Infrastructure/Common/Serialization/SerializerOptions.cs

Length of output: 2757


92-106: Consider handling nested objects recursively.

The current implementation only processes array properties at the top level of each schema. Consider recursively processing nested objects to ensure all array properties throughout the schema hierarchy are made nullable.

     private static void MakeCollectionsNullable(JsonSchema schema)
     {
         if (schema.Properties == null)
         {
             return;
         }

-        foreach (var property in schema.Properties.Values)
+        foreach (var property in schema.Properties.Values)
         {
             if (property.Type.HasFlag(JsonObjectType.Array))
             {
                 property.IsNullableRaw = true;
             }
+            // Recursively process nested objects
+            if (property.Type.HasFlag(JsonObjectType.Object))
+            {
+                MakeCollectionsNullable(property);
+            }
         }
     }

Also, let's verify the impact of these changes on the OpenAPI specification:

src/Digdir.Domain.Dialogporten.WebApi/Program.cs (1)

197-197: LGTM: Making collections nullable aligns with PR objectives.

The addition of MakeCollectionsNullable aligns with the PR's goal of modifying collection serialization behavior.

docs/schema/V1/swagger.verified.json (1)

Line range hint 1-7540: LGTM! Schema is well-documented and consistent

The OpenAPI specification is well-structured with:

  • Clear descriptions for all properties
  • Good examples
  • Proper type definitions
  • Consistent nullable array handling

@@ -211,6 +211,7 @@
"items": {
"$ref": "#/components/schemas/V1EndUserDialogsQueriesSearch_Dialog"
},
"nullable": true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Breaking Change: Arrays are now nullable across all DTOs

The change consistently makes all array properties nullable across the API contract. While this is a valid approach to distinguish between "no data" (null) and "empty set" ([]), this is a breaking change that requires client updates.

Consider the following recommendations:

  1. Add a migration guide for clients explaining:

    • The change from empty arrays to null
    • How to handle both null and empty arrays
    • Examples of proper null checking
  2. Consider adding a version suffix to the API endpoints (e.g., /v2) to maintain backward compatibility during migration.

  3. Update API documentation to explicitly mention this change in array handling.

Also applies to: 243-243, 265-265, 326-326, 359-359, 612-612, 620-620, 628-628, 679-679, 725-725, 758-758, 784-784, 866-866, 947-947, 960-960, 1052-1052, 1120-1120, 1190-1190, 1203-1203, 1416-1416, 1467-1467, 1600-1600, 1613-1613, 1716-1716, 1791-1791, 1804-1804, 1907-1907, 2021-2021, 2041-2041, 2099-2099, 2213-2213, 2299-2299, 2365-2365, 2373-2373, 2470-2470, 2478-2478, 2486-2486, 2532-2532, 2568-2568, 2598-2598, 2664-2664, 2694-2694, 2762-2762, 2777-2777, 2879-2879, 2965-2965, 3045-3045, 3060-3060, 3164-3164, 3172-3172, 3180-3180, 3220-3220, 3234-3234, 3250-3250, 3317-3317, 3347-3347, 3415-3415, 3430-3430, 3658-3658, 3666-3666, 3674-3674, 3726-3726, 3780-3780, 3813-3813, 3846-3846, 3928-3928, 4009-4009, 4022-4022, 4115-4115, 4182-4182, 4253-4253, 4266-4266, 4495-4495, 4552-4552, 4680-4680, 4748-4748, 4761-4761, 4864-4864, 4935-4935, 4948-4948, 5051-5051, 5121-5121

@oskogstad oskogstad merged commit 920d749 into main Nov 6, 2024
25 checks passed
@oskogstad oskogstad deleted the feat/remove-filtering-of-empty-collections-when-serializing branch November 6, 2024 11:44
arealmaas pushed a commit that referenced this pull request Nov 6, 2024
🤖 I have created a release *beep* *boop*
---


##
[1.29.0](v1.28.3...v1.29.0)
(2024-11-06)


### Features

* **webAPI:** Make all lists nullable in OpenAPI schema
([#1359](#1359))
([920d749](920d749))


### Bug Fixes

* **graphql:** ensure gql has maskinporten environment set
([#1408](#1408))
([152417a](152417a))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants