Add service to glossary

[svrnm]

Fixes #2050

Changes

Add a definition for service and service instance to the glossary.

[tigrannajaryan]

I think it would be more suitable to place the definition of concepts described by semantic conventions in the semantic convention document itself.

[svrnm]

I think it would be more suitable to place the definition of concepts described by semantic conventions in the semantic convention document itself.

I am OK with moving it there, I just picked the glossary because for me as someone who is not yet deeply into all the bits and pieces of the spec, this would be the place to look for definitions of commonly used terms (like in a math paper where you have a section "Definitions" first)

[carlosalberto]

@tigrannajaryan @svrnm was it decided where this should be added? Otherwise, LGTM for the actual content.

[tigrannajaryan]

I still think https://github.com/open-telemetry/opentelemetry-specification/blob/main/semantic_conventions/resource/service.yaml is a better place for Service definition. Not a strong opinion, would like other @open-telemetry/specs-approvers to chime in.

[yurishkuro]

We could link from semconv file into glossary

[svrnm]

@yurishkuro @tigrannajaryan any updates on this? Should I updated my PR to link from the semconv into the glossary then?

[yurishkuro]

Yes please

[svrnm]

done.

[jkwatson]

I personally don't understand what this definition of service includes or doesn't include. We should make sure it does not include "client applications" like web pages or mobile apps, as developers of those programs do not think of them as "services" and will be confused if we refer to them that way.

[yurishkuro]

We should make sure it does not include "client applications"

Why should we? A mobile app is not an "application" as defined here, because it typically does not function without a backend. I don't have any issues with considering a mobile app a "service". I think it's irrelevant what different people think what "service" means (e.g. in J2EE it was typical to refer to certain internal interfaces as services), what's relevant is how we define it here (which is pretty aligned with what people think a service is).

[jkwatson]

We should make sure it does not include "client applications"

Why should we? A mobile app is not an "application" as defined here, because it typically does not function without a backend. I don't have any issues with considering a mobile app a "service". I think it's irrelevant what different people think what "service" means (e.g. in J2EE it was typical to refer to certain internal interfaces as services), what's relevant is how we define it here (which is pretty aligned with what people think a service is).

It is definitely not aligned with what mobile developers think a service is, or what web developers think a service is. A service is a backend thing that the mobile app calls. If we define semantics that don't make sense to one more segments of our end users, we will end up building APIs that are confusing to those users.

[jonatan-ivanov]

I agree with @jkwatson but I'm not a native english speaker so please weight this accordingly:

Based on my experience, service usually means something that can serve its clients for many people (e.g.: a webservice that can serve client requests).

Based on this, the following applications usually are not considered as services:

• Desktop/Mobile/other Client apps (e.g.: browsers, games, IDEs)
• Batch data processors (e.g.: ETLs)
• Scheduled jobs
• Web crawlers, automated bots

[svrnm]

As a former web developer I agree with @jkwatson and @jonatan-ivanov that service is not how I would have referred to my frontend application, although it's not something I would have hard feelings about.

However, as someone who is building a backend for observability, I prefer a common mandatory attribute that helps me to identify groups of telemetry sources which are doing the same thing (instances of backend services, end-user apps, scheduled jobs, bots, database), because I don't have to bother about the type of thing to identify it properly. To give two examples:

• let's say you have spans in your jaeger/zipkin and now you have to think first, if you should search by app.name or service.name or job.name or bot.name, db.name to find the things you're looking for.
• same is true for the poor person building the visualisation of a trace/log/metric..., who needs to check for the type first to get the right name to put it into the UI ...

I raised the issue and created this PR to verify that "everything is a service" and with that service.name is that common mandatory attribute, so we can be sure that it is always there.

I see three options:

• Everyone lives with calling that thing which is sending telemetry a "service", accepting that it is not 100% accurate for all kinds of software.
• Different kinds of software get their specific name, so there is no common mandatory attribute and people implementing the backends need to look out for that.
• Replace "service" with something more generic, which probably leads to a breaking change of the spec?

[jkwatson]

@svrnm The reason why I believe we shouldn't use "service" to refer to client applications is because the shape of the data is very, very different from traditional services. It usually requires a completely different telemetry pipeline, as the volume of data is far, far larger than from traditional services. And, the analysis that you do on that data is significantly different than for traditional services.

Hence, it very rarely makes sense to be building common dashboards for both RUM and service use-cases, and the queries that you would build should, inherently, be different for the different types.

[jonatan-ivanov]

I think having a specific name for different kind of software (app.name, service.name, job.name, bot.name, etc.) is not a good idea it would make things hard and increase confusion by a lot.

But calling them applications (application.name) and making it possible to identify groups (application.group/application.group.name) sounds more straightforward to me.
I don't have strong feelings, and I don't want to go into bikeshedding just wanted to leave some feedback/opinion here because I think it is really relevant what users think and this has an effect on the UX.

[svrnm]

While I understand the different needs in the telemetry pipeline due to the volume and origin of data (thousands to millions of untrusted devices), I can not follow why the data is very, very different from traditional services and why it makes rarely sense to build common dashboards. I created a bunch of those dashboards in the past and my naive perspective always was that it is all just software. I am happy to get my view changed :-)

Nevertheless, my goal is to have clarity on what a service is (and what it is not) and if it can be used for uniquely identifying a group of software thingies doing the same thing.

I would prefer to have a unique name (service, application, telemetrySource, entity, ...) just for the sake of simplicity on the implementation side (see above) and specification side (e.g. if I look at #2115, I see a copy&paste defintion of .name, .version, .namespace, the introduction of a .short_name making me wonder if we need the same for service, ...)

However, if the consensus is that "service" does not cover client applications, I can update my code and also add a definition for "app" as in #2115

[yurishkuro]

I think "service" is a perfectly fine term to describe different kinds of workloads. I worked with developers generating telemetry from mobile apps who had no problem understanding that it is just a convention and putting the name of the app into service.name field, then using it in the UI to search for their telemetry. What matters is that we have a uniform way of referring to the name of the software component. I think "source" would be semantically much more confusing and imprecise.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[iNikem]

If we feel that service is not self-explanatory, then we cannot define it via application, which itself is not defined.

[svrnm]

That's a good point. A proposal how service defines the word application:

A service is a component of a larger distributed application. A service logically represents a repeatable business activity with a specified outcome. A service exchanges information with other services via some form of inter-process communication mechanism, such as network or shared storage. A group of services are forming an application.

Note: we have to pay attention with the word "application" since it is heavily overloaded, see the conversation around service vs frontend app(lication). Distributed Application would make this more clear, but as per your other comment, this would not make sense for an application only having a single service.

[bogdandrutu]

Let's not correlate at all with "application" to avoid confusion as you suggested:

A service logically represents a repeatable business activity with a specified outcome. A service exchanges information with other services via some form of inter-process communication mechanism, such as network or shared storage.

[jkwatson]

I don't understand what "logically represents a repeatable business activity with a specified outcome" means. That doesn't sound like a service to me. Also, why does a service need to exchange information with other services? That seems like something a service might do, but it's certainly not a requirement.

[svrnm]

"logically represents a repeatable business activity with a specified outcome"
See #2111 (comment): this comes from the service definition of SOA. What alternative would you suggest?

Also, why does a service need to exchange information with other services? That seems like something a service might do, but it's certainly not a requirement.
Would adding a "may", i.e. "A service may exchange..." help?

[jkwatson]

Higher level answer provided as a comment on the PR itself, for broader visibility.

[iNikem]

Is distributed important here? If my application is not distributed, then I don't have any services by this definition?

[svrnm]

See my note above, if we remove distributed it can be confused much more easily with a frontend application. Maybe there is a better term.

[bogdandrutu]

See my note, let's remove any reference to "application" for the moment.

[bogdandrutu]

@jkwatson

I think it's a bad decision; I think the spec can be easily tweaked to fix the "service" requirement by enabling SDKs to allow disabling of the default service.name attribute. "We made a mistake now we can't fix it" doesn't seem like a good justification to me.

Are you saying the decision to have one for both cases or keeping the name?

[jkwatson]

@jkwatson

I think it's a bad decision; I think the spec can be easily tweaked to fix the "service" requirement by enabling SDKs to allow disabling of the default service.name attribute. "We made a mistake now we can't fix it" doesn't seem like a good justification to me.

Are you saying the decision to have one for both cases or keeping the name?

If a decision was reached to define a service as "anything that emits telemetry" and the justification was "we made a mistake and we can't fix it", then that would be a bad decision.

[jkwatson]

Let me explain my over-arching problem with this PR (or any PR that attempted to do what this one is doing).

First, this PR defines a "service" as "A service logically represents a repeatable business activity with a specified outcome.". This is trivially shown to be both too broad, and too narrow. For example:

a) Clearly not a service: me walking to the corner store to buy a six-pack of beer. But, it falls under this definition.
b) Clearly (I think) a service: the thing running on my windows machine that records the temperature of the CPU once per second. This does not fall under this definition because there is no "business" involved here.

In the end, I firmly believe that what constitutes a "service" lies in the eye of the beholder. Whatever definition we come up with will end up either being too broad, or too narrow. We should not be trying to define this at all, in my opinion. We should leave this up to the author of the service to call their thing a service. Or NOT call it a service if they don't think of it as such.

If the end-goal of this PR is to define a service as "anything that emits OpenTelemetry signals", then let's call a spade a spade and just write that. I think this is a near-useless definition, as I have stated, but it is one possible outcome from this discussion.

The other option is to abandon "service" as the "one true namespace for things that emit telemetry" and create a new one, that is obviously inclusive (aka telemetry source). And, keep the existing service namespace for people to describe the things that they consider services. And, design new source-specific namespaces for the things that people don't consider services.

[tigrannajaryan]

In the end, I firmly believe that what constitutes a "service" lies in the eye of the beholder. Whatever definition we come up with will end up either being too broad, or too narrow. We should not be trying to define this at all, in my opinion. We should leave this up to the author of the service to call their thing a service. Or NOT call it a service if they don't think of it as such.

+1.

If the end-goal of this PR is to define a service as "anything that emits OpenTelemetry signals", then let's call a spade a spade and just write that. I think this is a near-useless definition, as I have stated, but it is one possible outcome from this discussion.

I would define it slightly differently though: "anything that uses OpenTelemetry SDK to emit telemetry". The distinction is important: there are other emitters of OpenTelemetry signals which have nothing to do with SDK (e.g. the Collector which can emit telemetry on behalf of variety of other entities, e.g. k8s pods or nodes, hosts, etc - which clearly are NOT Services themselves, but may or may not contain one or more Services).

I share your doubts that this is a great approach though. It appears to be influenced by the fact that some existing backends make the Service a required concept on the wire (Zipkin?) and thus we also decided to make it a required concept for our SDK. I do not think the SDK should be designed around limitations of particular backends, instead those limitations must be a concern of a particular exporter that deals with that backend. In my opinion the Service should NOT be a required entity of recorded telemetry, at best it may be the default entity, but the user of the SDK should be able to specify a different entity.

I do not know if the Service is the right term to use for client-side apps. May be it is. But it is clear that there are some other telemetry sources that no matter how much you try to shoehorn cannot be called services. If the goal is to call every telemetry source a Service then I disagree that it is a good idea.

IMO the following is what we need to do:

• We don't need to define the concept of Service more precisely than the semantic conventions already do. The Service does not need to be some central concept that require a special entry in the glossary.
• Delete the SDK requirement that the service.name is a required attribute.
• Exporters for formats where service name is a required attribute need to deal with it. Define some default value, make it an error if the service name is not defined, define fallback rules, etc. Do whatever is most reasonable for the particular exporter.
• Allow users to decide if whatever they are modelling can be rightfully called a Service. If it can then tell them to use existing service.* semantic conventions. Client-side application Workgroups needs to make a decision whether what they model can be called a Service.
• Allow end users and Otel Workgroups to define their own entities (telemetry sources) and semantic conventions for such entities and give them domain-specific names. Don't try to introduce a "capture all" kind of entity name that everybody needs to use (no telemetry.source.name conventions).

[carlosalberto]

@tigrannajaryan I like the summary and the steps overall. Let's discuss this next Tuesday in the Specs SIG, so we can get more feedback on this (and finally unblock the effort - one way or another).

[svrnm]

If the end-goal of this PR is to define a service as "anything that emits OpenTelemetry signals", [...]

@jkwatson: the purpose of this PR and the related issue (#2050) is to create clarity if a service is that "thing that uses OpenTelemetry SDK to emit telemetry" or not.

The discussions we have at multiple places, makes it obvious that this clarity is urgently needed. From those discussions I am with you that "service" should not be that magical thing.

We don't need to define the concept of Service more precisely than the semantic conventions already do. The Service does not need to be some central concept that require a special entry in the glossary.

I don't see a definition of the concept of a service in the semantic conventions. However, if the service is not going to be that important anymore, it's also not that relevant to have a clear definition of it. But, it would be good to have some hints or examples what a service could be, like the specs has them for other resources like "Compute Unit" or "Compute Instance"

@tigrannajaryan I like the summary and the steps overall. Let's discuss this next Tuesday in the Specs SIG, so we can get more feedback on this (and finally unblock the effort - one way or another).

+1

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[svrnm]

@carlosalberto, @tigrannajaryan, @jkwatson, @tedsuo following up on this topic: is there any update from the discussions with the client side telemetry SIG?

To be honest, I am not really sure if I fully understood the discussion around using schema_url to distinguish client/server-side telemetry. However, I was wondering if a change like the following was suggested before, that, I think, could fix the issue (changes are in bold):

Semantic Attributes with SDK-provided Default Value

These are the One and only one of the following attributes MUST be provided by the SDK as specified in the Resource SDK specification:

service.name
app.name

This solution would have the following advantages:

• it is simple: there are no immediate consequences of this change.
• it is backward compatible: current implementations don't have to change anything, especially those who are not relevant for "non-services"
• it is extensible: if there will ever be yet another domain (network devices? IoT devices?) they can just ask to have their attribute added as option
• it leaves the details to the domain experts: Requirements that are specific to service or app are then described in detail in their resource specification:
  • e.g. service.name makes the triplet service.namespace,service.name,service.instance.id globally unique,
  • e.g. the client side telemetry group can come up with their own subsequent requirements for what other things are mandatory if you have an app
• it is flexible: the part I understood from the schema_url discussion is that with that some attributes are available for the domain and some for the other, leaving you at a difficult place if you build something akward:
  • e.g. I had a customer building a service that used a headless browser internally to create PDFs, is this a service or an app?

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[tigrannajaryan]

@svrnm I don't know if any additional discussions happened. So far I have not seen any new information since I posted my comment above, so my position remains unchanged.

As for schema_url-based approach, it is a possibility that we can consider for the future, but I think it is non-trivial and will complicate even more. I would advise to set that aside for now. It will require some structural changes to how schemas work today which I can look into at, but it will take time to figure out and come to an agreement. I would not make that a prerequisite to resolving this issue.

[svrnm]

@svrnm I don't know if any additional discussions happened.

There have been some discussions on one of the last client telemetry SIG meeting, but there was also no final conclusion. We all agreed that there should be some place (probably not this ticket, or any of the other that has been opened on that), where all requirements are collected. I am not sure if that exists already.

@tedsuo, @jkwatson can maybe add some more details to that.

As for schema_url-based approach, it is a possibility that we can consider for the future, but I think it is non-trivial and will complicate even more. [...]

I fully agree with that. As said I couldn't follow that discussion and from the pieces I understood I found it overly complicated.

If it helps I am happy to close this PR and the related issue for now.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[github-actions]

Closed as inactive. Feel free to reopen if this PR is still being worked on.