How to apply mitigation for CVE-2021-44228

[plumdog]

I believe from apache/solr#454 (comment) and https://apache.github.io/solr-operator/docs/solr-cloud/solr-cloud-crd.html#custom-solrxml that to mitigate, I need to set:

spec:
  solrOpts: '-Dlog4j2.formatMsgNoLookups=true'

on a SolrCloud resource.

Is anyone able to verify my thought process here that this is a valid mitigation?

Ultimately, I suppose solr-operator should run with this set by default.

Edit: also see https://solr.apache.org/security.html#apache-solr-affected-by-apache-log4j-cve-2021-44228

[HoustonPutman]

That is the correct way to mitigate the CVE.

And you are correct, it's a good option to have by default.

[HoustonPutman]

Pinning this, so that others can see the mitigation step if they come looking for information.

[plumdog]

@HoustonPutman another thing I have just thought of: should we also be concerned about the Zookeeper that is created for a SolrCloud?

Edit: or rather, that may be created for a SolrCloud, if spec.zookeeperRef.provided is provided with config. I can't see anything in there about passing options to Java like I can with solrOpts for Solr.

Edit2: I have got as far as finding https://zookeeper.apache.org/security.html, which doesn't mention anything. And my Googling finds nothing useful looking, eg https://www.google.com/search?q=zookeeper+log4j&tbs=qdr:w

[sylus]

@plumdog were u able to find anything about zookeeper?

Also when this is set in solropts, is there a way for me to confirm it is passed? I thought I would see it in the Solr UI somewhere.

[sylus]

I'm not sure whether this effects zookeeper but I think you first would need access to the solr pod so first mitigation suffices?

[plumdog]

Have opened pravega/zookeeper-operator#422 to try to work out how zookeeper-operator is impacted and how to mitigate.

[plumdog]

Ok, from pravega/zookeeper-operator#422 (comment), believe Zookeeper not impacted, so I think the mitigation above is sufficient.

[sylus]

Thanks @plumdog really happy with how quickly solr-operator responded to this. Awesome work!

[nosvalds]

Does anyone know if the SolrPrometheusExporter resource is also affected? This line from the newspost on the Solr website made me think it could be:

The vulnerability in the Prometheus Exporter Contrib can be mitigated by any of the following:

• Upgrade to Solr 8.11.1 or greater (when available), which will include an updated version of the log4j2 dependency.
• Manually update the version of log4j2 on your runtime classpath and restart your Solr application.
• Edit your solr-exporter script to include: JAVA_OPTS="$JAVA_OPTS -Dlog4j2.formatMsgNoLookups=true"
• Follow any of the other mitgations listed at https://logging.apache.org/log4j/2.x/security.html

It doesn't look like spec.solrOpts or spec.javaOptsis available on the SolrPrometheusExporter.

[HoustonPutman]

@nosvalds from here: apache/solr-site#55, the consensus is that the Solr Prometheus Exporter is not actually susceptible to this CVE

[nosvalds]

@sylus

Also when this is set in solropts, is there a way for me to confirm it is passed? I thought I would see it in the Solr UI somewhere.

You should see it on the Dashboard tab in the JVM > Args section:

[plumdog]

Can someone with more Java config confidence than me verify that I should not be concerned that I, as in @nosvalds screenshot above have:

-Dlog4j.configurationFile=/var/solr/log4j2.xml
-Dlog4j2.formatMsgNoLookups=true

Is it correct that one config item starts log4j. and the other log4j2.?

[madrob]

Yes, the new property should be log4j2.* https://github.com/carterkozak/logging-log4j2/blob/release-2.x/log4j-core/src/main/java/org/apache/logging/log4j/core/util/Constants.java#L63

[HoustonPutman]

It should be mentioned that the official Docker images (_/solr) have been updated to have this fix included by default. If you have imagePullPolicy: Always set and are using these images, then you can restart your pods, and they will be good to go.

[sylus]

@HoustonPutman @plumdog i hate to be bearer of bad news :(

https://nvd.nist.gov/vuln/detail/CVE-2021-45046

A new CVE issued without a score and previous mitigation won't be enough :(

[sylus]

Although this was mentioned in the mailing list:

Re: Log4j < 2.15.0 may still be vulnerable even if -Dlog4j2.formatMsgNoLookups=true is set
The MDC Patterns used by solr are for the collection, shard, replica, core
and node names, and a potential trace id. All of those are restricted to
alphanumeric, no special characters like $ or { needed for the injection.
And trying to access a collection that didn’t exist Returns 404 without
logging.
Upgrading is always going to be more complete, but I think we’re still ok
for now, at least until the next iteration of this attack surfaces.

[janhoy]

It should be mentioned that the official Docker images (_/solr) have been updated to have this fix included by default. If you have imagePullPolicy: Always set and are using these images, then you can restart your pods, and they will be good to go.

All supported docker tags have been updated, i.e. all latest patch versions of 8.x as well as 7.7, 6.6 and 5.5. But not 7.6, 7.5, 6.5, 5.1, 8.10.0 etc. So please make sure you are on a supported tag. See https://issues.apache.org/jira/browse/SOLR-15850 for more.