use microProfile-7.0 feature, /openapi/ui does not show openapi documentation

[tam512]

Describe the bug

Use Open Liberty 24.0.0.9-beta (wlp-1.0.92.cl240820240729-1903) and java.version = 17.0.11
Use the following features

	<featureManager>
		<platform>jakartaee-11.0</platform>
		<feature>appSecurity</feature>
		<feature>beanValidation</feature>
		<feature>cdi</feature>
		<feature>enterpriseBeansLite</feature>
		<feature>faces</feature>
		<feature>pages</feature>
		<feature>persistence</feature>
		<feature>restfulWS</feature>
		<feature>websocket</feature>
		<feature>xmlWS</feature>
		<feature>jndi-1.0</feature>
		<feature>ldapRegistry-3.0</feature>
		<feature>localConnector-1.0</feature>
		<feature>microProfile-7.0</feature>
	  </featureManager>

Liberty installed the following features:

The server installed the following features: [appSecurity-6.0, cdi-4.1, distributedMap-1.0, enterpriseBeansLite-4.0, expressionLanguage-6.0, faces-4.1, federatedRegistry-1.0, jdbc-4.3, jndi-1.0, json-1.0, jsonp-2.1, ldapRegistry-3.0, localConnector-1.0, mpConfig-3.1, mpHealth-4.0, mpOpenAPI-4.0, pages-4.0, persistence-3.2, persistenceContainer-3.2, restfulWS-4.0, restfulWSClient-4.0, servlet-6.1, ssl-1.0, transportSecurity-1.0, validation-3.1, websocket-2.2, xmlBinding-4.0, xmlWS-4.0].

1. In server.xml, take out <variable name="mp.openapi.extensions.liberty.merged.include" value="all"/>, viewing http://localhost:9081/openapi/ui/ , I got No operations defined in spec!. Below are messages in the log, so maybe the feature to show all apps by default is not ready in this build.

[8/7/24, 13:21:06:358 CDT] 0000004d liberty.microprofile.openapi20.internal.ApplicationProcessor I CWWKO1660I: The application SocialSphere-mp was processed and an OpenAPI document was produced.
[8/7/24, 13:21:06:360 CDT] 0000004d liberty.microprofile.openapi20.internal.MergeDisabledAlerter I CWWKO1663I: Combining OpenAPI documentation from multiple modules is disabled. Only WebModule [SocialSphere-mp/SocialSphereApp-SocialSphereAdminWeb-1.0-SNAPSHOT] is used to generate OpenAPI documentation. To enable merging of OpenAPI documentation from all modules, set config property mp.openapi.extensions.liberty.merged.include = all.

2. In server.xml, I included 2 apps as following, but I got the same No operations defined in spec!

<mpOpenAPI>
	<include>SocialSphere-mp/SocialSphereApp-AllergyForecastMPOpenAPI-1.0-SNAPSHOT</include>
	<include>SocialSphere-mp/SocialSphereApp-SocialSphereWebService-1.0-SNAPSHOT</include>
</mpOpenAPI>

It is working ok if I include only 1 app at a time with the following configuration:

<variable name="mp.openapi.extensions.liberty.merged.include" value="SocialSphere-mp/SocialSphereApp-SocialSphereWebService-1.0-SNAPSHOT"/>

or

<variable name="mp.openapi.extensions.liberty.merged.include" value="SocialSphere-mp/SocialSphereApp-AllergyForecastMPOpenAPI-1.0-SNAPSHOT"/>

Application is at https://github.ibm.com/was-svt/SocialSphere-mpOpenAPI/tree/mp-7.0

[tam512]

@Azquelt I see 28861 is closed, so I'm trying my app again with build openliberty-24.0.0.12-beta-cl241120240930-0302.zip

I did the following tests

1. Test with <variable name="mp.openapi.extensions.liberty.merged.include" value="all"/> in server.xml, I can see all 3 apps (Weather, Allergy, Words)

2. Removed <variable name="mp.openapi.extensions.liberty.merged.include" value="all"/> from server.xml, I only see on the 1 app (Weather app) listed

• Message in the log:

[10/1/24, 14:01:55:031 CDT] 0000003c liberty.microprofile.openapi20.internal.ApplicationProcessor I CWWKO1660I: The application SocialSphere-mp was processed and an OpenAPI document was produced.
[10/1/24, 14:01:55:033 CDT] 0000003c liberty.microprofile.openapi20.internal.MergeDisabledAlerter I CWWKO1663I: Combining OpenAPI documentation from multiple modules is disabled. Only WebModule [SocialSphere-mp/SocialSphereApp-SocialSphereWebService-1.0-SNAPSHOT] is used to generate OpenAPI documentation. To enable merging of OpenAPI documentation from all modules, set config property mp.openapi.extensions.liberty.merged.include = all.

3. I also tried adding the following to server.xml, but I still see only 1 Weather app listed

	<mpOpenAPI>
		<include>all</include>
	</mpOpenAPI>

I see the same message in the log as test 2 above

[Azquelt]

Thank you for the update.

The logic to do the merge was completed in #28861 (as can be seen from the fact the MP Config properties to control merging now work).

However, there is still work required before everything you're trying here will work:

• server.xml configuration to specify which modules are included Add MP OpenAPI server.xml config to include or exclude apps (2.0+) #28857
• including all modules by default Document all modules by default on MP OpenAPI 4.0 #29793

There's also a few more capabilities that aren't yet implemented:

• server.xml configuration to set the info section Allow MP OpenAPI server.xml info section to be set with server.xml (2.0+) #28858
• server.xml configuration to set the OpenAPI version used for output Allow OpenAPI document version to be set with server.xml for MP OpenAPI 4.0 #28859

[tam512]

Test with build openliberty-25.0.0.1-beta-cl241220241107-1102.zip

I re-test the 3 settings as in #29035 (comment)
I see the following error in the log

[ERROR ] CWWKO1661E: An error occurred when processing application SocialSphere-mp and an OpenAPI document was not produced. The error was: io.smallrye.openapi.runtime.OpenApiRuntimeException: Failed to read YAML stream.

Full log:

[INFO] Listening for transport dt_socket at address: 7777
[INFO] Launching defaultServer (Open Liberty 25.0.0.1-beta/wlp-1.0.96.cl241220241107-1102) on Eclipse OpenJ9 VM, version 17.0.11+9 (en_US)
[INFO] CWWKE0953W: This version of Open Liberty is an unsupported early release version.
[INFO] [AUDIT   ] CWWKE0001I: The server defaultServer has been launched.
[INFO] CWWKM2010I: Searching for CWWKF0011I: in /Users/tam/Documents/WAS-SVT/2024-work/App_Development/SocialSphere-mpOpenAPI/SocialSphereOpenAPIEAR/target/liberty/wlp/usr/servers/defaultServer/logs/messages.log. This search will timeout after 90 seconds.
[INFO] [WARNING ] CWWKG0033W: The value [LDAPSSLSettings] specified for the reference attribute [sslRef] was not found in the configuration.
[INFO] [AUDIT   ] CWWKZ0058I: Monitoring dropins for applications.
[INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://tam-macbook3.attlocal.net:9081/openapi/
[INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://tam-macbook3.attlocal.net:9081/jwt/
[INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://tam-macbook3.attlocal.net:9081/health/
[INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://tam-macbook3.attlocal.net:9081/openapi/ui/
[INFO] [ERROR   ] CWWKO1661E: An error occurred when processing application SocialSphere-mp and an OpenAPI document was not produced. The error was: io.smallrye.openapi.runtime.OpenApiRuntimeException: Failed to read YAML stream.
[INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://tam-macbook3.attlocal.net:9081/SocialSphereApp-SocialSphereAdminWeb-1.0-SNAPSHOT/
[INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://tam-macbook3.attlocal.net:9081/SocialSphereWebService/
[INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://tam-macbook3.attlocal.net:9081/SocialSphere/
[INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://tam-macbook3.attlocal.net:9081/SocialSphereWebSocketChatRoom/
[INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://tam-macbook3.attlocal.net:9081/SocialSphereApp-AllergyForecastMPOpenAPI-1.0-SNAPSHOT/
[INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://tam-macbook3.attlocal.net:9081/SocialSphereEverydayWords/
[INFO] [AUDIT   ] CWWKZ0001I: Application SocialSphere-mp started in 19.149 seconds.
[INFO] [AUDIT   ] CWWKF0012I: The server installed the following features: [appSecurity-6.0, cdi-4.1, distributedMap-1.0, enterpriseBeansLite-4.0, expressionLanguage-6.0, faces-4.1, federatedRegistry-1.0, jdbc-4.3, jndi-1.0, json-1.0, jsonb-3.0, jsonp-2.1, jwt-1.0, ldapRegistry-3.0, localConnector-1.0, microProfile-7.0, monitor-1.0, mpConfig-3.1, mpFaultTolerance-4.1, mpHealth-4.0, mpJwt-2.1, mpOpenAPI-4.0, mpRestClient-4.0, mpTelemetry-2.0, pages-4.0, persistence-3.2, persistenceContainer-3.2, restfulWS-4.0, restfulWSClient-4.0, servlet-6.1, ssl-1.0, transportSecurity-1.0, validation-3.1, websocket-2.2, xmlBinding-4.0, xmlWS-4.0].
[INFO] [AUDIT   ] CWWKF0011I: The defaultServer server is ready to run a smarter planet. The defaultServer server started in 25.698 seconds.
[INFO] CWWKM2015I: Match number: 1 is [11/7/24, 21:46:44:209 CST] 00000035 com.ibm.ws.kernel.feature.internal.FeatureManager            A CWWKF0011I: The defaultServer server is ready to run a smarter planet. The defaultServer server started in 25.698 seconds..
[INFO] ************************************************************************
[INFO] *    Liberty is running in dev mode.
[INFO] *        Automatic generation of features: [ Off ]
[INFO] *        h - see the help menu for available actions, type 'h' and press Enter.
[INFO] *        q - stop the server and quit dev mode, press Ctrl-C or type 'q' and press Enter.
[INFO] *        
[INFO] *    Liberty server port information:
[INFO] *        Liberty server HTTP port: [ 9081 ]
[INFO] *        Liberty debug port: [ 7777 ]
[INFO] ************************************************************************
[INFO] SocialSphereAdminWeb source compilation was successful.
[INFO] SocialSphereConfig source compilation was successful.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: /Users/tam/Documents/WAS-SVT/2024-work/App_Development/SocialSphere-mpOpenAPI/SocialSphereEJB/ejbModule/com/svt/social/sphere/ejb/Ticker.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
[INFO] SocialSphereEJB source compilation was successful.
Note: /Users/tam/Documents/WAS-SVT/2024-work/App_Development/SocialSphere-mpOpenAPI/SocialSphereMarket/ejbModule/com/svt/social/sphere/market/Advertise.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
[INFO] SocialSphereMarket source compilation was successful.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
[INFO] SocialSphereWebService source compilation was successful.
Note: /Users/tam/Documents/WAS-SVT/2024-work/App_Development/SocialSphere-mpOpenAPI/SocialSphereWeb/src/com/svt/social/sphere/servlet/Talk.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
[INFO] SocialSphereWeb source compilation was successful.
[INFO] SocialSphereWebSocketChatRoom source compilation was successful.
[INFO] SocialSphereEverydayWords source compilation was successful.
[INFO] AllergyForecastMPOpenAPI source compilation was successful.

Run http://localhost:9081/openapi/ui and I see only 2 apps instead of 3. It does not show Allergy Forecast app.

[Azquelt]

Deploying the app locally, I see from the FFDC that it's complaining about tabs in the AllergyForecastMPOpenAPI/WebContent/META-INF/openapi.json file.

I'm investigating why this is causing the problem, but could you try converting the tabs to spaces to see if there are any more problems?

[Azquelt]

I've raised an upstream bug for the parsing problem and am working on a fix: smallrye/smallrye-open-api#2059

Locally, replacing tab with four spaces does allow me to deploy the static file successfully so you should be able to complete the rest of your testing.

[tam512]

I tried to replace tab with 4 spaces on the 2nd line in AllergyForecastMPOpenAPI/WebContent/META-INF/openapi.json file. Then it throws the same error on the next line. There are 172 lines in this file and most of them using tabs. I will wait for the fix for this to get in and will test this again

[Azquelt]

@tam512 please just do a find and replace for tab with four spaces. I can provide a PR to your repo if that's easier.

[Azquelt]

@tam512 https://github.ibm.com/was-svt/SocialSphere-mpOpenAPI/pull/6

[tam512]

@Azquelt yes please save the original openapi.json with tabs and create a new one with spaces and provide a PR in my repo. If I have time to circle back to MP test by the end of today I will give it a quick test. Thanks

[Azquelt]

@tam512 thanks. PR above has been updated to create a new file rather than replacing the existing one.

[tam512]

I tested with openapi-with-spaces.json and I can see all 3 apps ok with and without <variable ../>

[tam512]

Testcases work ok on build 25.0.0.1-beta-cl241220241110-1902.zip