Feature Test Summary For JSON Binding (JSONB) 3.0 for Jakarta EE 10

[KyleAure]

Test Strategy

Describe the test strategy & approach for this feature, and describe how the approach verifies the functions delivered by this feature.

Repackage and automate the TCK tests provided by the jsonb API project to ensure we are running all applicable tests that have already been created to verify that the jsonb implementation, yasson, we are shipping passes all required tests for certification.

Re-ran the existing com.ibm.ws.jsonb_fat test suite using Jakarta EE 10 features.

Wrote a new test suite, io.openliberty.jakarta.jsonb.internal_fat , to test new Jakarta EE 10 features

List of FAT projects affected

io.openliberty.jakarta.jsonb.3.0_fat_tck

Regression and New Function Testing

Package Name	Purpose
ee.jakarta.tck.json.bind.api.annotation	Tests annotations such as @JsonbNillable and @JsonbProperty work
ee.jakarta.tck.json.bind.api.builder	Tests the JsonbBuilder api methods
ee.jakarta.tck.json.bind.api.config	Tests the JsonbConfig api methods
ee.jakarta.tck.json.bind.api.exception	Negative tests to ensure correct exceptions are thrown
ee.jakarta.tck.json.bind.api.jsonb	Tests the Jsonb api methods
ee.jakarta.tck.json.bind.api.jsonbadapter	Test the withAdapaters() config api
ee.jakarta.tck.json.bind.cdi.customizedmapping.adapters	Assert that CDI injection is supported in adapters
ee.jakarta.tck.json.bind.cdi.customizedmapping.serializers	Assert that CDI injection is supported in serializers and deserializers
ee.jakarta.tck.json.bind.customizedmapping.adapters	End to end adapters tests
ee.jakarta.tck.json.bind.customizedmapping.binarydata	Test binary data is encoded correctly
ee.jakarta.tck.json.bind.customizedmapping.dateformat	Test date formats are honored and applied correctly
ee.jakarta.tck.json.bind.customizedmapping.ijson	Test stick I-JSON serialization compliance
ee.jakarta.tck.json.bind.customizedmapping.instantiation	Test required and optional jsonb objects are created
ee.jakarta.tck.json.bind.customizedmapping.nullhandling	End to end JsonbNillable tests
ee.jakarta.tck.json.bind.customizedmapping.numberformat	Test JsonbNumberFormat annotations
ee.jakarta.tck.json.bind.customizedmapping.propertynames	Test JsonbTransient fields
ee.jakarta.tck.json.bind.customizedmapping.propertyorder	Test property order strategies
ee.jakarta.tck.json.bind.customizedmapping.serializers	Test custom serializers using withSerializers
ee.jakarta.tck.json.bind.customizedmapping.visibility	Test custom visability strategies with withPropertyVisibilityStrategy
ee.jakarta.tck.json.bind.defaultmapping.arrays	Test default array serialization
ee.jakarta.tck.json.bind.defaultmapping.attributeorder	Test default attrribute order
ee.jakarta.tck.json.bind.defaultmapping.basictypes	Test default primitive type serialization
ee.jakarta.tck.json.bind.defaultmapping.bignumbers	Test default BigNumber type serialization
ee.jakarta.tck.json.bind.defaultmapping.classes	Test default Class type serialization
ee.jakarta.tck.json.bind.defaultmapping.collections	Test default Collection type serialization
ee.jakarta.tck.json.bind.defaultmapping.dates	Test default Date type serialization
ee.jakarta.tck.json.bind.defaultmapping.enums	Test default ENUM type serialization
ee.jakarta.tck.json.bind.defaultmapping.generics	Test default java generic type serialization
ee.jakarta.tck.json.bind.defaultmapping.identifiers	Test default IDs based on type
ee.jakarta.tck.json.bind.defaultmapping.ignore	Test ignore unknown attributes
ee.jakarta.tck.json.bind.defaultmapping.interfaces	Test deserialize to interface type
ee.jakarta.tck.json.bind.defaultmapping.jsonptypes	Test JsonObject is handled correctly
ee.jakarta.tck.json.bind.defaultmapping.nullvalue	Test null values are ignored by default
ee.jakarta.tck.json.bind.defaultmapping.polymorphictypes	Test polymorphic type serialization
ee.jakarta.tck.json.bind.defaultmapping.specifictypes	Test random specific type serialization
ee.jakarta.tck.json.bind.defaultmapping.uniqueness	Test duplicate property names are rejected
ee.jakarta.tck.json.bind.defaultmapping.untyped	Test predicable iteration order for collections
ee.jakarta.tck.json.bind.signaturetest.jsonb	Test correct signatures

io.openliberty.jakarta.jsonb.internal_fat

New Functionality

Test Name	Issue	Description
JsonBTestServlet#testCreatorParametersOptional	jakartaee/jsonb-api#121	JsonbCreator parameters are optional by default.
JsonBTestServlet#testCreatorParametersRequired	jakartaee/jsonb-api#121	JsonbCreator parameters are required when withCreatorParametersRequired is specified.
JsonBTestServlet#testCreatorWithTypeAdapter	jakartaee/jsonb-api#71	JsonbTypeAdapter can be used on JsonbCreator parameters
JsonBTestServlet#testCreatorWithTypeDeserializer	jakartaee/jsonb-api#71	JsonbTypeDeserializer can be used on JsonbCreator parameters.
JsonBTestServlet#testNillableMethodsAndFields	jakartaee/jsonb-api#52	JsonbNillable is allowed on methods and fields and determines if a JSON null value is written versus omitting the property entirely.
JsonBTestServlet#testNullJsonValue	jakartaee/jsonb-api#181	JSON null deserializes as JsonValue.NULL rather than Java null.
JsonBTestServlet#testPolymorphism	jakartaee/jsonb-api#147	Tests polymorphic types in JSON-B.
JsonBTestServlet#testCreatorParametersRequired	NONE	jsonbContainer feature is used to add a fake JSON-B provider that switches the default for CREATOR_PARAMETERS_REQUIRED to true.
JsonBContainerTestServlet#testNillableMethodsAndFields	NONE	JsonbNillable is allowed on methods and fields and determines if a JSON null value is written versus omitting the property entirely. The jsonbContainer is used to add a fake JSON-B provider that switches the default PROPERTY_NAMING_STRATEGY to LOWER_CASE_WITH_DASHES.
JsonBContainerTestServlet#testNullJsonValue	jakartaee/jsonb-api#181	JSON null deserializes as JsonValue.NULL rather than Java null.
JsonBContainerTestServlet#testPolymorphism	jakartaee/jsonb-api#147	Tests polymorphic types in JSON-B.

com.ibm.ws.jsonb_fat

Regression Testing - Container features

Test Class	Description	What is tested
JSONBContainerTest	Uses jsonbContainer + jsonp features with a bell pointing to a thrid party implementation of Jsonb	Tests jsonb running in a servlet and CDI bean. Ensures the correct jsonb provider is used. Tests previous release features
JSONBInAppTest	Uses jsonbContainer + jsonpContainer features with a bell pointing to yasson(RI) and a thrid party implementation of jsonp	Tests jsonb running in a servlet and CDI bean. Ensures the correct jsonb provider is used. Tests previous release features.
JSONBTest	Uses jsonb and jsonp features	Tests jsonb running in a servlet and CDI bean. Ensures the correct jsonb provider is used. Tests previous release features.
JSONPContainerTest	Uses jsonpContainer feature with a bell pointing to a thrid party implementation of jsonp	Tests jsonp running in a servlet. Ensures the correct jsonp provider is used. Tests previous release features.
JsonUserFeatureTest	Uses jsonbContainer + jsonpContainer features with a bell pointing to yasson(RI) and a thrid party implementation of jsonp along with user features that use jsonb and jsonp during their activate methods	Tests jsonb and jsonp were able to be used during OSGi service method via injection.

Test strategy

• What functionality is new or modified by this feature?

  • Allow @JsonbNillable at field and method level, and then deprecate nillable=true/false on @JsonbProperty
  • Allow @JsonbTypeDeserializer and @JsonbTypeAdapter to be specified at parameter level
  • Introduce @JsonbRequired to designate which @JsonbCreator fields are optional vs required.
  • When deserializing a JsonValue typed attribute, a null value should be serialized as JsonValue.NULL rather than null
  • Polymorphic serialization/deserialization

• What are the positive and negative tests for that functionality? (Tell me the specific scenarios you tested. What kind of tests do you have for when everything ends up working (positive tests)? What about tests that verify we fail gracefully when things go wrong (negative tests)?

  • Previously outlined in io.openliberty.jakarta.jsonb.internal_fat section

• What manual tests are there (if any)? (Note: Automated testing is expected for all features with manual testing considered an exception to the rule.)

  • There are no manual tests

Confidence Level

Collectively as a team you need to assess your confidence in the testing delivered based on the values below. This should be done as a team and not an individual to ensure more eyes are on it and that pressures to deliver quickly are absorbed by the team as a whole.

Please indicate your confidence in the testing (up to and including FAT) delivered with this feature by selecting one of these values:

0 - No automated testing delivered

1 - We have minimal automated coverage of the feature including golden paths. There is a relatively high risk that defects or issues could be found in this feature.

2 - We have delivered a reasonable automated coverage of the golden paths of this feature but are aware of gaps and extra testing that could be done here. Error/outlying scenarios are not really covered. There are likely risks that issues may exist in the golden paths

3 - We have delivered all automated testing we believe is needed for the golden paths of this feature and minimal coverage of the error/outlying scenarios. There is a risk when the feature is used outside the golden paths however we are confident on the golden path. Note: This may still be a valid end state for a feature... things like Beta features may well suffice at this level.

4 - We have delivered all automated testing we believe is needed for the golden paths of this feature and have good coverage of the error/outlying scenarios. While more testing of the error/outlying scenarios could be added we believe there is minimal risk here and the cost of providing these is considered higher than the benefit they would provide.

5 - We have delivered all automated testing we believe is needed for this feature. The testing covers all golden path cases as well as all the error/outlying scenarios that make sense. We are not aware of any gaps in the testing at this time. No manual testing is required to verify this feature.

Based on your answer above, for any answer other than a 4 or 5 please provide details of what drove your answer. Please be aware, it may be perfectly reasonable in some scenarios to deliver with any value above. We may accept no automated testing is needed for some features, we may be happy with low levels of testing on samples for instance so please don't feel the need to drive to a 5. We need your honest assessment as a team and the reasoning for why you believe shipping at that level is valid. What are the gaps, what is the risk etc. Please also provide links to the follow on work that is needed to close the gaps (should you deem it needed)

Voting Results: 5
Voting Log: https://ibm-cloud.slack.com/archives/C31DXH4GJ/p1653420724727189

[ayoho]

LGTM 👍