OTLP export spam exception stack trace on console when endpoint is unavailable

[kchow1985]

Describe the bug
When a python app is auto instrumented with opentelementry-instrument, if the exporter endpoint becomes unavailable then it will continuously spam the following messages even after the endpoint becomes available again (i.e. sidecar collector is back up and running)

Transient error StatusCode.UNAVAILABLE encountered while exporting metrics, retrying in Nones.
Exception while exporting metrics an integer is required (got type NoneType)
Traceback (most recent call last):
  File "C:\git\data-integration-processor\.venv\lib\site-packages\opentelemetry\exporter\otlp\proto\grpc\exporter.py", line 306, in _export
    self._client.Export(
  File "C:\git\someproject\.venv\lib\site-packages\grpc\_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "C:\git\someproject\.venv\lib\site-packages\grpc\_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "failed to connect to all addresses; last error: UNAVAILABLE: WSA Error"
        debug_error_string = "UNKNOWN:Failed to pick subchannel {created_time:"2022-09-28T18:44:29.602414264+00:00", children:[UNKNOWN:failed to connect to all addresses; last error: UNAVAILABLE: WSA Error {grpc_status:14, created_time:"2022-09-28T18:44:29.60241115+00:00"}]}"
>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\git\someproject\.venv\lib\site-packages\opentelemetry\sdk\metrics\_internal\export\__init__.py", line 482, in _receive_metrics
    self._exporter.export(metrics_data, timeout_millis=timeout_millis)
  File "C:\git\someproject\.venv\lib\site-packages\opentelemetry\exporter\otlp\proto\grpc\metric_exporter\__init__.py", line 248, in export
    return self._export(metrics_data)
  File "C:\git\someproject\.venv\lib\site-packages\opentelemetry\exporter\otlp\proto\grpc\exporter.py", line 346, in _export
    sleep(delay)
TypeError: an integer is required (got type NoneType)

Looking at the code, the call on _export line 299 for the exporter/otlp/exporter.py file invokes expo() but looking at the backoff library (_wait_gen.py, line 22) the expo() call initially yields None which cause this error.

It means that once the endpoint is gone then the application must be restarted to get rid of this error.

What version did you use?
Version: 0.33b

What config did you use?
No config used

Environment
OS: (e.g., "Windows 10")
Python v3.8
backoff library: v2.1.2
oltp exporter: v1.12.0 (using grpc)

[kchow1985]

Realize this is not the right repo to post this but the python contrib one

[CaueP]

Hey @kchow1985 have you found a solution to this issue or opened another issue? I searched for the same title on python contrib repo, but couldn't find it.

[CaueP]

Nevermind, looks like it was fixed by open-telemetry/opentelemetry-python#2980