asyncio instrumentation does not await generator coroutines

[rrjjvv]

Describe your environment

OS: Ubuntu 22.04.4 LTS
Python version: Python 3.12.3
Package version: 0.47b0 (and main)

What happened?

My production app works as expected when not instrumented. After adding asyncio instrumentation, various components of our app simply "didn't run". No explicit errors are thrown, but for a concrete example, messages published via a redis pubsub were not delivered since there were no subscribers. After disabling the instrumentation, subscriptions occurred.

I found a commonality between all the code that "didn't run", which was that they all involved methods that yielded.

Steps to Reproduce

main.py:

import asyncio
from opentelemetry.instrumentation.asyncio import AsyncioInstrumentor

AsyncioInstrumentor().instrument()

async def foo():
    print('thanks for calling')
    return 42

async def bar():
    yield await foo()

async def main():
    ret = await asyncio.create_task(anext(bar()))
    print(ret)
    
if __name__ == '__main__':
    asyncio.run(main())

Expected Result

$ python3 main.py 
thanks for calling
42

Actual Result

$ python3 main.py 
<async_generator_asend object at 0x7b0fa554ae80>

Additional context

Our first usage of this package was 0.45b0, which we had to disable due to #2340. It appears the emphasis was on 'don't crash'; the first fix simply did an early return. It was noticed during review, and after addressing, it was noted (#2521 (comment)) that it "somehow worked" whether the coro was returned or not.

The original reproducer (#2340 (comment)) is nearly identical to my reproducer. The only meaningful difference is that the original reproducer (nor the test case) did not actually look at the results, whereas mine (and my production app) does.

Would you like to implement a fix?

Yes