Slow restart of Solr Pod init_container cp-solr-xml chown

[smoldenhauer-ish]

Solr Operator 0.6.0
In our deployments a restart of Solr Pods will take minutes up to half an hour.
The init container cp-solr-xml with the command
cp /tmp/solr.xml /tmp-config/solr.xml && chown -R 8983:8983
/var/solr/data/backup-restore/local-collection-backups-1
esp. the recursive chown seems to be the cause. We mitigated it a bit by cleaning up the backup volume regularly. But there are deployments with a larger number of collections (~400) and then it is still slow.
...
The backup repository is a azurefile-csi storage.

mailing list discussion: https://lists.apache.org/thread/tlxg38v6y81dhykycrgqw77mm2xdoqbr

[smoldenhauer-ish]

basic idea is to skip the chown if the backup directory is already writable for the solr user.

[HoustonPutman]

Yeah, sounds like a good improvement! Looks like you are testing out some options, so make a PR when you are ready.

[smoldenhauer-ish]

The changed statement is working now. I'm currently waiting for a test with forked custom image (0.6.0 + fix) in production deployment.

[HoustonPutman]

Awesome, make the PR when your tests are done! We will probably start the process for the v0.7.0 release late next week, and it would be great to have this included.

[HoustonPutman]

TODO: We should think about just using the fsGroupChangePolicy available in Kubernetes v1.23

[smoldenhauer-ish]

I saw that fsGroupChangePolicy = onRootMismatch , but wasn't sure how to apply that to the Solr backup PVC
It was kind of inspiration for the bugfix.
If that would work, I guess the cp-solr-xml will be back to only copy the solr.xml

[HoustonPutman]

Yeah reading more through it, it seems like the default functionality is "Always", so its weird that it wouldn't be working in the first place. I guess we need to keep the logic in regardless.