Problem during fulltextsearch:index "Root mapping definition has unsupported parameters:"

[fillg1]

Hi folks
I tried to update my existing nextcloud to elasticsearch 8, but I got a problem during reindexing, even after a fulltextsearch:reset

My versions
Nextcloud Hub 6 27.1.3
Full text search 27.0.2
Full text search - Elasticsearch Platform 27.0.4
Full text search - Files 27.0.1
Full text search - Files - Tesseract OCR 27.0.0
Elasticsearch 8.8.2

php occ fulltextsearch:test seems to work, I don't see any errors in the output
``
.Testing your current setup:
Creating mocked content provider. ok
Testing mocked provider: get indexable documents. (2 items) ok
Loading search platform. (Elasticsearch) ok
Testing search platform. ok
Locking process ok
Removing test. ok
Pausing 3 seconds 1 2 3 ok
Initializing index mapping. ok
Indexing generated documents. ok
Pausing 3 seconds 1 2 3 ok
Retreiving content from a big index (license). (size: 32386) ok
Comparing document with source. ok
Searching basic keywords:

• 'test' (result: 1, expected: ["simple"]) ok
• 'document is a simple test' (result: 2, expected: ["simple","license"]) ok
• '"document is a test"' (result: 0, expected: []) ok
• '"document is a simple test"' (result: 1, expected: ["simple"]) ok
• 'document is a simple -test' (result: 1, expected: ["license"]) ok
• 'document is a simple +test' (result: 1, expected: ["simple"]) ok
• '-document is a simple test' (result: 0, expected: []) ok
• 'document is a simple +test +testing' (result: 1, expected: ["simple"]) ok
• 'document is a simple +test -testing' (result: 0, expected: []) ok
• 'document is a +simple -test -testing' (result: 0, expected: []) ok
• '+document is a simple -test -testing' (result: 1, expected: ["license"]) ok
• 'document is a +simple -license +testing' (result: 1, expected: ["simple"]) ok
  Updating documents access. ok
  Pausing 3 seconds 1 2 3 ok
  Searching with group access rights:
• 'license' - [] - (result: 0, expected: []) ok
• 'license' - ["group_1"] - (result: 1, expected: ["license"]) ok
• 'license' - ["group_1","Group_2"] - (result: 1, expected: ["license"]) ok
• 'license' - ["group_3","Group_2"] - (result: 1, expected: ["license"]) ok
• 'license' - ["group_3"] - (result: 0, expected: []) ok
  Searching with share rights:
• 'license' - notuser - (result: 0, expected: []) ok
• 'license' - User number_2 - (result: 1, expected: ["license"]) ok
• 'license' - User3 - (result: 1, expected: ["license"]) ok
• 'license' - User@4 - (result: 1, expected: ["license"]) ok
  Removing test. ok
  Unlocking process ok
  ``

But when I try to reindex all my documents with fulltextsearch:index, and the first PDF file is "hit", I'm getting an error from elasticsearch

org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping: Root mapping definition has unsupported parameters: [standard : {dynamic=true, properties={owner={type=keyword}, groups={type=keyword}, circles={type=keyword}, metatags={type=keyword}, source={type=keyword}, title={copy_to=combined, analyzer=keyword, term_vector=with_positions_offsets, type=text}, content={copy_to=combined, analyzer=analyzer, term_vector=with_positions_offsets, type=text}, users={type=keyword}, tags={type=keyword}, provider={type=keyword}, subtags={type=keyword}, links={type=keyword}, combined={analyzer=analyzer, term_vector=with_positions_offsets, type=text}, hash={type=keyword}}}] at org.elasticsearch.index.mapper.MapperService.parseMapping(MapperService.java:385) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:361) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.updateIndexMappingsAndBuildSortOrder(MetadataCreateIndexService.java:1339) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.lambda$applyCreateIndexWithTemporaryService$2(MetadataCreateIndexService.java:472) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.indices.IndicesService.withTempIndexService(IndicesService.java:694) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexWithTemporaryService(MetadataCreateIndexService.java:470) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV1Templates(MetadataCreateIndexService.java:593) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:420) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService$1.execute(MetadataCreateIndexService.java:299) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService$UnbatchedExecutor.execute(MasterService.java:550) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService.innerExecuteTasks(MasterService.java:1039) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:1004) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService.executeAndPublishBatch(MasterService.java:232) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.lambda$run$2(MasterService.java:1622) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.action.ActionListener.run(ActionListener.java:357) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.run(MasterService.java:1619) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService$5.lambda$doRun$0(MasterService.java:1237) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.action.ActionListener.run(ActionListener.java:357) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService$5.doRun(MasterService.java:1216) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.8.2.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:840) ~[?:?]

Any idea what is going wrong here?

[linnaea]

fulltextsearch_elasticsearch/lib/Service/IndexMappingService.php

Line 241 in 2283022

'standard' => [

Root cause is here, mapping type removal that should've been done during ES7.

There are probably some PRs that needs to be reverted, like #237, otherwise fixing the root cause will cause user/group/circle filtering to always return nothing.

Also I find the acceptance of #246 quite baffling.

https://github.com/nextcloud/fulltextsearch_elasticsearch/pull/246/files#diff-9d002463e43412b20501d12daf6ad95fe0c6cdfd8e17cb3627a57e4c66a05d9d

This changed the behaviour during index creation from "translate some errors" to "log a warning and pretend nothing happens". Shouldn't have been accepted.

[snowphone]

I'm also facing the same issue (NC 30.0.1, ES 8.5.1). There's no workarounds to fix this bug?