Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When loading configuration from data config file, variable substitution does not work for "solr.core.instanceDir" #65

Open
szhou1998 opened this issue Jan 20, 2024 · 0 comments

Comments

@szhou1998
Copy link

I'm running into an issue where variable substitution is not working properly when loading data config from an XML file. I have an entity element with attribute url="${solr.core.instanceDir}/workDir/claimcontact.xml". When running full-import, I get this error:

"level":"ERROR","loggerName":"org.apache.solr.handler.dataimport.DocBuilder",
"Exception while processing: claimcontact document : SolrInputDocument(fields: [])"
"org.apache.solr.handler.dataimport.DataImportHandlerException: 
java.security.AccessControlException: access denied ("java.io.FilePermission" "/workDir/claimcontact.xml" "read")"

"java.security.AccessControlContext","method":"checkPermission","file":"AccessControlContext.java","line":472
"java.security.AccessController","method":"checkPermission","file":"AccessController.java","line":897
"java.lang.SecurityManager","method":"checkPermission","file":"SecurityManager.java","line":322
"java.lang.SecurityManager","method":"checkRead","file":"SecurityManager.java","line":661
"java.io.File","method":"isFile","file":"File.java","line":889
"org.apache.solr.handler.dataimport.FileDataSource","method":"getFile","file":"FileDataSource.java","line":119
"org.apache.solr.handler.dataimport.FileDataSource","method":"getData","file":"FileDataSource.java","line":86
"org.apache.solr.handler.dataimport.FileDataSource","method":"getData","file":"FileDataSource.java","line":48
"org.apache.solr.handler.dataimport.XPathEntityProcessor","method":"initQuery","file":"XPathEntityProcessor.java","line":291
"org.apache.solr.handler.dataimport.DataImportHandlerException","method":"wrapAndThrow","file":"DataImportHandlerException.java","line":61
"org.apache.solr.handler.dataimport.XPathEntityProcessor","method":"initQuery","file":"XPathEntityProcessor.java","line":294
"org.apache.solr.handler.dataimport.XPathEntityProcessor","method":"fetchNextRow","file":"XPathEntityProcessor.java","line":232
"org.apache.solr.handler.dataimport.XPathEntityProcessor","method":"nextRow","file":"XPathEntityProcessor.java","line":212
"org.apache.solr.handler.dataimport.EntityProcessorWrapper","method":"nextRow","file":"EntityProcessorWrapper.java","line":267
"org.apache.solr.handler.dataimport.DocBuilder","method":"buildDocument","file":"DocBuilder.java","line":494
"org.apache.solr.handler.dataimport.DocBuilder","method":"buildDocument","file":"DocBuilder.java","line":433
"org.apache.solr.handler.dataimport.DocBuilder","method":"doFullDump","file":"DocBuilder.java","line":348
"org.apache.solr.handler.dataimport.DocBuilder","method":"execute","file":"DocBuilder.java","line":236
"org.apache.solr.handler.dataimport.DataImporter","method":"doFullImport","file":"DataImporter.java","line":424
"org.apache.solr.handler.dataimport.DataImporter","method":"runCmd","file":"DataImporter.java","line":483
"org.apache.solr.handler.dataimport.DataImporter","method":"lambda$runAsync$0","file":"DataImporter.java","line":466

As you can see, the solr.core.instanceDir variable is not being substituted correctly. I believe the issue is with this bit of code in DocBuilder:
https://github.com/SearchScale/dataimporthandler/blob/branch_9x/src/main/java/org/apache/solr/handler/dataimport/DocBuilder.java#L116-L120

In Solr 8, a change was made to get the properties from CoreDescriptor substitutableProperties: apache/lucene-solr@17ae79b#diff-9eadac54fe1ace540cb05b6711791be03a9289a7d6f16615fb4efa4ee31641f9

CoreDescriptor substitutableProperties has solr.core.instanceDir as a property: https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/core/CoreDescriptor.java#L269

However, note that in the Solr 9 DocBuilder change in this repo, the Solr 8 "before" code did not have this change: ec730d5#diff-88595e01694c11981cecf4b71fdc54969870ff49737208856685388eb3bb0ee5

Should this be changed to new VariableResolver(dataImporter.getCore().getCoreDescriptor().getSubstitutableProperties());?

@szhou1998 szhou1998 changed the title When loading configuration from Data Config file, variable substitution does not work for "solr.core.instanceDir" When loading configuration from data config file, variable substitution does not work for "solr.core.instanceDir" Jan 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant