Issue using any urllib3>=1.26.16

[nodje]

Using elastic-apm for ansible callback community.general.elastic works fine up until urllib3==1.26.15.

Anything above ends up with error message: Closing the transport connection timed out

Tested with
1.26.16
1.26.17
2.0.5
2.0.6

[basepi]

Can you give me more details on how the agent is being used? Did you add it yourself or is it an integration provided by ansible?

[nodje]

So we use elastic-apm==6.18.0 through the community.general.elastic callback plugin (itself coming from ansible main collection community.general==7.3.0 and using ansible-core==2.15.4).

cf. https://github.com/ansible-collections/community.general/blob/7.3.0/plugins/callback/elastic.py

To identify this bug we did a manual test, we install it ourselves in a venv with all required ansible binaries.

you activate the plugin in an ansible.cfg file:

[defaults]
callbacks_enabled = community.general.elastic

then you can test it with a simple debug task in a playbook:
ELASTIC_APM_API_KEY=XXXXX ELASTIC_APM_SERVER_URL=http://my-apm-server.local:8200 ELASTIC_APMSERVICE_NAME=service ansible-playbook -i localhost, playbook.yml

It either works if urllib3<=1.26.15 or issue a Closing the transport connection timed out with any superior version.

[basepi]

@v1v It looks like you wrote this. Can you take a look? Perhaps we need an explicit apm_cli.close()? This is the diff from 1.26.16. The only thing that sticks out is perhaps the way PoolManagers are closed in this change?

[v1v]

Can you take a look?

Sure, I'll try to reproduce this locally and see whether apm_ci.close() can help in this case. I'll keep you posted with my findings.

[karmi]

I'm getting security warnings for urllib3==1.26.15, any news regarding the issue?

[iurisilvio]

I reproduced the issue in https://github.com/iurisilvio/elastic-apm-1912, with tox to create different configurations and validate the bug.

I created the issue and a PR in ansible-collections repo.

from ansible_collections.community.general.plugins.callback.elastic import ElasticSource

source = ElasticSource("")
apm_cli = source.init_apm_client("http://localhost:8200", "service", None, None, None)
# NOT CLOSING

$ python -Wd unclosed.py
sys:1: ResourceWarning: unclosed <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 58904), raddr=('127.0.0.1', 8200)>

from ansible_collections.community.general.plugins.callback.elastic import ElasticSource

source = ElasticSource("")
apm_cli = source.init_apm_client("http://localhost:8200", "service", None, None, None)
apm_cli.close()

$ python -Wd closed.py

[iurisilvio]

It is not a bug in elastic-apm, but docs don't say anything about .close(), it is not clear it is required.

https://www.elastic.co/guide/en/apm/agent/python/current/api.html

[nodje]

thanks, found it: ansible-collections/community.general#7518 (comment)