Introduce easy api for starting tasks for remote participants

[keepingitneil]

No description provided.

[changeset-bot]

🦋 Changeset detected

Latest commit: a95cd86

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
livekit-agents	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[davidzhao]

I like the ergonomics of ctx.add_participant_task

[davidzhao]

woohoo!

[davidzhao]

this confused me a bit in terms of the sequencing (i.e. intention is to run them serially vs override). Does it make sense to enforce serial dispatch for two unrelated callbacks?

imagine this use cases:

async def task1(p: rtc.RemoteParticipant):
    for frame in p.audio_stream:
       pass

async def task2(p: rtc.RemoteParticipant):
    for frame in p.video_stream:
       pass

ctx.add_participant_task(task1)
ctx.add_participant_task(task2)

[keepingitneil]

Hrmm, I suppose we could actually key off of the (identity,callback) tuple and say that each different callback runs in parallel but only one of each per participant runs at a time.

[davidzhao]

a couple of other thoughts/questions:

1. should this be in rtc? it doesn't look like any job specific functions are used. i.e. why not room.add_participant_task?
2. lifecycle-wise, how does it end when either local or remote participant becomes disconnected? is there an easy way for the user to tell? or maybe the answer is just to listen to participant_disconnected
3. is there a way to limit only the first participant?

[keepingitneil]

a couple of other thoughts/questions:

1. should this be in rtc? it doesn't look like any job specific functions are used. i.e. why not room.add_participant_task?
2. lifecycle-wise, how does it end when either local or remote participant becomes disconnected? is there an easy way for the user to tell? or maybe the answer is just to listen to participant_disconnected
3. is there a way to limit only the first participant?

1. I leaned toward agents because it has some overlap participant_(dis)connected callbacks but behaves differently. I worry it could be confused as two different ways to do the same thing (even though they are different). A very rough symmetry argument could be made with other client libs: this feels like convenience on top of the core client lib, similar to the hooks you'd find in components-js.

For 2. it's just up to the users to end the task. So, for example, a task iterating audio frames would simply exit when the participant disconnects.

3. Hrmm no not really, do you think it's needed? The user would probably know and be able to filter out participants they don't want. Of course, they could also use a closure-scoped variable to keep track of it if they really only wanted the first participant without caring about any other filtering criteria.

[theomonnom]

nit, add a taskname

like "participant_entrypoint"

[theomonnom]

let's allow to put directly the callback instead of using * ?

[davidzhao]

nice!