[WIP] rfc21: document job reattach events

[chu11]

This documents what I did in flux-framework/flux-core#3947

I don't know if this should really be considered a mergeable candidate, but mostly wanted to just document what is in that PR and/or potentially lead to discussion. The terminology "reattach" is of course debatable.

[garlick]

What do the events signify, and do they need to be in the main eventlog or should the shells log disconnect/reconnect events in their eventlog?

[chu11]

What do the events signify, and do they need to be in the main eventlog or should the shells log disconnect/reconnect events in their eventlog?

At the moment they are informational. So perhaps they don't even need to be documented? Or don't even need to be in the eventlog? The only reason I put them in the eventlog is that we want to know when a job has been "reattached", because there's info that could be lost.

For example, we normally calculate runtime by calculating t_cleanup - t_run timestamps. But in a "reattach" case, this runtime calculation may not be accurate b/c a job may have finished before we reattached and output the cleanup state to the eventlog. (Edit: I have no idea how to solve in the simulated case. In the systemd case longer out will probably have extra information and can do something to estimate a more accurate runtime)

[chu11]

closing, for the purposes of flux-framework/flux-core#3947 decided to prefix eventlog events with debug as we design the longer term solution