[RFC] Session RFC PR to move to Stage 2 (Draft)

[dainperkins]

Updated verbiage in description and promoted to stage 2 (draft)

Markdown preview of this RFC

TODO

• Finish populating the source data section
  • DB Connection
  • Web Session
  • Cloud Admin Session
• Subject matter experts weighed in on technical utility of field definitions

[ebeahan]

ECS team will update to the date when PR is merged as stage 2.

[ebeahan]

We can also link the Stage 2 PR at the bottom now that the PR's opened.

[ebeahan]

Thanks @dainperkins for opening up the PR for advancing Session.

Based on the discussion in the ECS committee meeting last month, @jonathan-buttner will act as sponsor with @andrewvc @drewpost (cc @cyrille-leclerc ) acting as stakeholders. Would you all be able to review the proposal and weigh in?

[jonathan-buttner]

@ferullo @rw-access @joe-desimone @jrmolin We're working on adding some session fields to the ECS core schema. I think this could be useful for the session events that the endpoint sends. I don't think we ever nailed down the exact fields that the endpoint would send in our original discussion here: https://github.com/elastic/endpoint-app-team/issues/258 Are there certain changes we'd like to include to cover the information that the endpoint would will send?

Full RFC is here: https://github.com/elastic/ecs/blob/master/rfcs/text/0004-session.md

[rw-access]

thanks for the ping @jonathan-buttner.
one thought I have -- session.type seems to be higher level than session.kind in the hierarchy, but the opposite is true for event.* Do you think it would make sense to be consistent in the terminology?

[rw-access]

@gabriellandau or @wburgess1 may also have some insights to relevant metadata from an OS perspective for logon sessions. I think it's good that the field set starts out more minimal, with the things that are universally understood when the word "session" is invoked

[rw-access]

Another question: (sorry for all the spam)

Do you think this warrants a new event.category? What do you have in mind for the category with the examples you have? And how would you categorize a windows logon session (4624 logon event through a 4634 or 4647 logoff)?

[dainperkins]

@rw-access
kind vs. type - good catch, thanks! I'll update that one to better reflect the typical hierarchy of e.g. event.kind/type.

Looking at the 4624 logon types I think there's definitely potential to add a few examples that I missed in the initial stages. I'll add the windows examples & the others I have to a gsheet mapping to make sure the coverage is there and post here for review

/d

[dainperkins]

@rw-access in looking at the kind/type - how does the following re-org look to you?

Field	Description
session.kind:	system, application, network
session.type:	local, remote, virtual, wired, wireless, vpn

I think it covers all the bases, tho I feel like application is under represented in type... and I'm on the fence about a specific session.kind for "cloud" or "admin" (thinking of gcp console or specific admin interface sessions for instance - e.g. ece portal or palo admin interface, its a web app, but is that descriptive enough?)

The rest of the fields probably need revisiting against some additional real world examples as I suspect I'm missing some discrete session stuff, and possibly replicating other field sets needlessly.

On a side note I'd love to know how I manage to keep closing things by accident

[ebeahan]

Looks like this was closed on accident 😄

[cyrille-leclerc]

@dainperkins shall we add an Observability member as reviewer of this PR to ensure we observability continue to contribute to this "session" RFC?
cc @andrewvc , @drewpost
reference #879

[dainperkins]

@cyrille-leclerc I think thats a very good idea - while I tried to make sure I covered the session fields that seemed to make sense from a number of perspectives, having someone with an APM / Observability view would be a huge benefit

[rw-access]

@dainperkins I think that your kind/type distinction makes sense to me. I think type could even be an array.
my guess is that the "Close and comment" is just too close to the "Comment" button

[webmat]

Thanks Dain.

Let's formalize a few easy tasks that should get done in this PR:

• In the "People" section at the bottom
  • Add Cyrille / Observability as a stakeholder
  • Add Andrew Cholakian as a subject matter expert. I do want this to work for web sessions if possible.
  • Add Ross Wolf as a subject matter expert for the endpoint POV

• Add link "Stage 2: [RFC] Session RFC PR to move to Stage 2 (Draft) #935" in the references at the end

The YAML field definitions is looking great. Please take it out of the Markdown file and instead put it in a subdirectory for RFC 0004, at rfcs/text/0004/session.yml.

In doing so, merging this stage 2 PR will make it possible for folks to generate ECS artifacts that include these fields with something like python scripts/generator.py --include rfcs/text/0004/ (see USAGE.md for details).

I've tried the above command locally, and I've noticed we're missing short definitions for fields "id", "name" and "authorization", so please add those as well.

---

That's it for the virtual paperwork 😂

In terms of specifics on the subject of sessions, here's what I'd like to see added in this PR.

• My understanding is that both session.kind and session.type are meant to be values from a pre-determined list. Let's list all acceptable values in the "Usage" section. We can use your initial list as a starting point, and finalize the list with the other subject matter experts.
• When you get around to introducing concrete examples, one thing I'd like to see represented is how each kind of sessions should populate all of the session fields. If some attributes are not pertinent for some kinds of sessions, we should start fleshing this out

[joshdevins]

Might be good to have someone from Enterprise Search on this. We're in the middle of defining some tracking events and sessions/session IDs has come up.

/cc @RossAllenBell

[RossAllenBell]

Might be good to have someone from Enterprise Search on this. We're in the middle of defining some tracking events and sessions/session IDs has come up.

/cc @RossAllenBell

Thanks @joshdevins!

Yes, we're going to be tracking user search sessions in Workplace Search soon. If I can help in any way by providing more use case context, please let me know.

[dainperkins]

@RossAllenBell - how well do you see the RFC session fields fitting into what you plan on tracking (in concert with the rest of the ECS space as well...)

Do you have any examples I can dig thru?

[RossAllenBell]

@RossAllenBell - how well do you see the RFC session fields fitting into what you plan on tracking (in concert with the rest of the ECS space as well...)

Do you have any examples I can dig thru?

@dainperkins I expect Workplace Search's use case would primarily make use of the session.id field, as we attempt to track analytics data across a user's session when using search in Workplace Search. While we would have data to populate the other fields proposed in this RFC, they would likely always be the same value and potentially not worth storing.

The Workplace Search Analytics project brief: https://docs.google.com/document/d/1k5ttaeVjgnsWJTSJ1s3d0wv2dIV_DVVvKPSBcLfpeNc

An example of data we track today that I would like to add session.id to: https://docs.google.com/document/d/1QedgCC8EQLJdPfKzIvxcoCJXbTdfvYT5N_JKXyQWFOE/edit#heading=h.btqtgkfgroil

[webmat]

Just dropping a quick note here that I agree with @rw-access' suggestion that we should consider adding event.category "session". I think this would make a lot of sense.

[dainperkins]

Effectively the break out is for visualizing sessions of different kinds (no pun intended) with type management being of particular importance in a siem use case.

In a nutshell -
Kind is a generic category indicating what type of session (logging into a system, gaining network access, or application access) While most of the type labels could be derived from e.g. protocol information, it seemed like the trade off to not have to search on e.g. "session.kind: system & network.protocol: https" or similar would be worth it

If you check the ECS event fields:

.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event.
.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization

so it does follow the idiom... not that I'd say no to better categorization terms, but it does seem to scan overall

[andrewvc]

Thanks for the background @dainperkins , I wasn't aware we had prior art here. That said, reading through the category fields docs they don't seem like a good fit here. They seem explicitly designed for events. If you look at the docs for kind I would think that all sessions would have either state or event as types.

That said, the docs for ECS category fields are very confusing IMHO, and replicating them would seem to me to only replicate that confusion.

Here's a thought, given the generic nature of this, we could name these things context.group and context.name, so you'd get:

{
  "context": {
     "group": "network",
     "name": "wireguard"
  }
}

The specific words of context, group and name are unimportant except for the fact that name is obviously more specific than group, and nesting them within an object makes their relationship as a hierarchy more obvious.

[dainperkins]

I was just referring to the use of kind and type nested under event as a categorization mechanism for sessions as well. I can see the point of differentiating the field names under session, however I also think it makes sense to re-use existing hierarchical categorization schemes within ECS (kind, type, category, etc.) for ease of use.

I'll give some more thought to the session categorization, and take a look at whatever other feedback comes.

[ebeahan]

That said, the docs for ECS category fields are very confusing IMHO

@andrewvc I don't want to deviate this conversation, but if you have any specific feedback, feel free to reach out. 😃

[ebeahan]

Since a bit of time has past since this PR was originally open, I'll call out that event.type: session was added as part of ECS 1.8.0. Do we see a possible alternative in pairing event.type: session with some of the other event.type allowed values to achieve similar categorization as proposed here by using session.kind, and session.type?

Examples:

• Authenticated session: event.type: [ "authentication", "session" ]
• Network session: event.type: [ "network", "session" ]
• Web session: event.type: [ "web", "session" ]

[dainperkins]

Maybe its time to go for a good core base rather than a perfect complete implementation...

a) provide basic session fields (name/id) to use with event.type: session
b) take some time to evaluate the above suggestions for discriminating between the various types of sessions users might want to use or report on

Thoughts?

[ebeahan]

Maybe its time to go for a good core base rather than a perfect complete implementation...

++ It's much easier to add later than take away 😄 .

I would like to see some mapped examples. Having some examples of these fields populated with real-world values would help anchor the conversation.

[bryce-b]

Mobile agents have definite need for this! This RFC looks very applicable, but should an additional kind be added for mobile?

[github-actions]

This PR is stale because it has been open for 60 days with no activity.

[djptek]

A session is a sequence or family of events

I think it may be simpler to leave event.type: session under event and not involve it further at this stage.

Looking at each of the proposed fields for session:

• session.kind - the example values for this field might need to encompass every possibility of parent kinds for families of event.kind. I'd suggest to provide the cited examples and word this as expected, rather than allowed values
• session.type - same issue as session.kind
• session. authorization - LGTM
• session.name - LGTM
• session.id - LGTM

I'd favour making all fields optional

[dainperkins]

Has there been any feedback from the session viewer team? Sneha might have some feedback based on session viewer (minimally worth a discussion if there's overlap). (@snehsach19)

[qcorporation]

@mitodrummer @m-sample Just giving you two visibility from a session viewer perspective

[mitodrummer]

@mitodrummer @m-sample Just giving you two visibility from a session viewer perspective

All good from my end. Session viewer's sessions are linux centric. We use a nested process fieldset to group our process events by session_leader, entry_leader etc.... see: https://www.elastic.co/guide/en/ecs/current/ecs-process.html#ecs-process-nestings

Sounds like this RFC is geared towards a more generic handling of sessions.

[m-sample]

Has there been any feedback from the session viewer team? Sneha might have some feedback based on session viewer (minimally worth a discussion if there's overlap).

There is definitely overlap with the session view ECS RFC (https://github.com/elastic/ecs/blob/main/rfcs/text/0030-linux-event-model.md).

On Linux, the term "session" means a very specific thing - a set of processes that are descendants of a session leader process that all share the same SID. This session leader process is distinguished by its session-id/SID being the same as its PID. Linux Sessions are not just humans logging in - they are used for services as well (ie things started by init that do not represent remote access, nor do they have an authn or authz phase)

Also, the concept of "local" sessions can be fuzzy - for example, terminals and console/usb-kbd sessions may seem like "local" - e.g. a directly attached terminal via serial cable. In some cases apparent serial terminals are actually remote/network sessions coming over the IPMI serial from the LAN etc where the IPMI BMC is designed to look like a serial device to the host server but the data coming over it may be from a NIC/remote terminal. For cases like this we chose to record the session as having a specific serial TTY number (so that a reviewer could use their knowledge that /dev/ttyS2 is the IPMI magic serial port)

We introduced the entry.meta.type and entry.meta.source which have overlap here (https://github.com/elastic/ecs/blob/main/rfcs/text/0030/process.yml). Rather than general session types that can have semantic challenges (see IPMI 'local' session discussion above) we identified session types by the service that created them on the server: init(e.g systemd), sshd, ssm, kubelet, teleport, terminal, console. (note that teleport may use the SSH protocol but is distinguished from sshd, and other implementations of SSH) We also use the presence of a TTY to identify likely human-interactive sessions (which has security review value and Rule value).

@mitodrummer I'm sure I've missed other areas of overlap. What did I miss?

[mitodrummer]

Ya, I think had this fieldset been available at the time of our RFC, we may have considered using session.type to track the entry type. session.id could have been used for sid, but it seems our widened model approach using the process fieldset covers more ground (w context), since we track two types of sessions, entry_leader and session_leader.

[m-sample]

it seems our widened model approach using the process fieldset covers more ground (w context), since we track two types of sessions, entry_leader and session_leader.

Good point. On Linux, a single remote access by a human may have 2 (or even more) actual Linux sessions associated with it. This is typically from people running screen or tmux in their entry session to create a child session that can be re-attached to on a subsequent remote access (commonly done to prevent losing ones work/running-processes if the ssh TCP conn dies)

[djptek]

Looking at the original intention of the session RFC this was envisioned to support:

• physical user sessions at a desktop
• various kinds of network sessions, including browser sessions

so it's totally appropriate that the process RFC has a widened model.

[github-actions]

This PR is stale because it has been open for 60 days with no activity.

[rsk0]

Is there intention to continue trying to develop this RFC for generalized sessions? It seems like the right field set for our use of web application session.

[github-actions]

This PR is stale because it has been open for 60 days with no activity.

[buzzdeee]

any intention to continue this one?

[github-actions]

Hi!

We just realized that we haven't looked into this PR in a while. We're
sorry!

We're labeling this PR as Stale to make it hit our filters and
make sure we get back to it as soon as possible. In the meantime, it'd
be extremely helpful if you could take a look at it as well and confirm its
relevance. A simple comment with a nice emoji will be enough :+1.

Thank you for your contribution!