Improvements to allow future deployment of multiple instances

[fxprunayre]

Various experiments were done in GeoNetwork 3 to try to deploy multiple instances and the main issue was the Lucene index. Moving to Elasticsearch in version 4 allows to easily share the index and setup a cluster of Elasticsearch nodes if needed.

With https://metadata.vlaanderen.be/ team, we analyzed other limitations related to a multiple instances setup and fixed the main issue. Without altering a single instance mode, we describe below the changes made, provide a docker configuration for testing it and list remaining known limitations that we can improve in the future.

This PR combine the work done in the following PR for easier testing:

• Improvements to allow future deployment of multiple instances #6990
• Harvester / Add property to turn off scheduling #7217
• Introducing scheduled refresh of harvesterjobs. #7333

For testing the easiest is to use the docker configuration https://github.com/geonetwork/docker-geonetwork/tree/update-gn-4.4.0/4.4.0#clustering-experimental to easily start the main node and replicas.

Docker configuration is done in geonetwork/docker-geonetwork#107

Changes

HTML cache for formatter and WRO4J

This allows to have multiple instances starting up but having their own separate HTML cache. If not, H2 database are locked.

Caused by: org.h2.mvstore.MVStoreException: The file is locked: /.../WEB-INF/data/data/resources/htmlcache/wro4j-cache.mv.db [2.1.212/7]
	at org.h2.mvstore.DataUtils.newMVStoreException(DataUtils.java:1004) ~[h2-2.1.212.jar:2.1.212]

geonetwork.htmlcache.dir can be used to customize the location. Another workaround is to use LRU cache instead

cacheStrategy=disk-memory

replaced by

cacheStrategy=lru

to have in memory cache (but you can't benefit from the pre-built cache).

Schema initialization

On startup, GeoNetwork publish each schema plugin XSDs to /data/resources/xml/schemas folder. If more that one
instance starts at the same time, they can fails to copy the XSD. Adding a new data directory variables for the schemas -Dgeonetwork.schemapublication.dir=.

Admin / Site info

Admin console information panel provides relevant information to check node configuration (hostname, harvester configuration):

Harvester

Add configuration to only run scheduled harvesting tasks in one instance which will reload harvester configuration on a regular basis.

Java 11 fixes

• Fix error on sending mail under Jetty

{"message":"NestedServletException","code":"runtime_exception",
"description":"Handler dispatch failed; nested exception is
 java.lang.NoClassDefFoundError: javax/activation/DataSource"}

Remaining known limitations

• Harvester / Scheduler needs to be refreshed when the database harvester configuration is modified (the harvesting node refresh the schedule every 2 minutes as a stopgap solution)
• Settings / When saving application settings, some modules need to be updated (not really blocker because those settings usually don't change often - restart or save settings on each node as a workaround)
  • log file,
  • DOI configuration,
  • proxy configuration (use Java environment variable instead of database configuration)
• Thesaurus / Local thesaurus modified in one node are not updated on others.

Future improvements

• Define a way to exchange messages between node to be able to fix the above limitations (https://trac.osgeo.org/geonetwork/wiki/LoadBalanceable)
• Editing session in parallel on same record - locking system to not allow multiple editing session

[sonarqubecloud]

SonarCloud Quality Gate failed.   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
8 Code Smells

0.0% Coverage
0.0% Duplication

The version of Java (11.0.20.1) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

Catch issues before they fail your Quality Gate with our IDE extension SonarLint