Baseline: why is Edge included but something like Samsung Internet isn't?

[romainmenke]

In the current definition of Baseline which is latest major + 1, Edge is always just Chrome.
Edge releases are also timed closely together with Chrome.

So a natural heuristic for developers is to assume Edge support when there is Chrome support.
There might be days or even a few weeks where this isn't true, but as a heuristic it will work well enough.

The same however isn't true for Samsung Internet.
Samsung releases skip many Blink versions and roll out more slowly.

However Samsung Internet does have significant market share on mobile (±5%) https://gs.statcounter.com/browser-market-share/mobile/worldwide

Is it possible that Samsung Internet was omitted because most developers don't really consider Samsung Internet when thinking about cross browser support?

That would be a self-reinforcing cycle.

Developers overlook Samsung Internet despite it's market share, so it's omitted from Baseline, because it is omitted from Baseline developers keep overlooking it.

---

Samsung Internet is both an example and a concrete case here.
The same might be true for other browsers.

[jhvanderschee]

'Latest major + 1' seems like a poor requirement. 95% support in real browser usage seems more relevant to me. We could discuss about 95% (should it be 98%?). The current requirement leads to promotion of features with less than 60% real browser support (like the new viewport units for mobile).

Because Edge is in and Samsung Internet is out, it seems like a tool by and for big tech, not actual developers.

[ddbeck]

cc: @atopal some interesting research-related questions here. I tried summarizing them in the list at the end of the comment.

Thanks for your feedback and ideas on this.

First, to answer the question in the subject of the issue: why is Edge included? Because web developers, in surveys and interviews, reported that they cared about Edge support. For more context, a significant factor for choosing the Baseline browser set was about developer expectations (see #132).

As for Samsung Internet, I don't recall web developers reporting (in the surveys and interviews I saw) that Samsung Internet was a support requirement for their work. That said, I don't recall any particular mention of Samsung Internet in the research: it'd probably be a good idea to double-check our work in this area.

Finally, if I understand correctly, there's some speculation that web developers don't care about Samsung Internet because developers overlook the browser (i.e., they're not actually considering Samsung Internet's relevance to their work, one way or the other). I think that's something future research could probably answer.

Thus, some possible lines of inquiry here:

• In the survey data and interviews used to help choose the Baseline browser set, did web developers have opportunities to mention Samsung Internet as a browser of interest? (If not, then we ought to go back and re-run some of that research.)
• If the web developers did have the opportunity to mention it, did they choose it as a required browser often? How did compare or contrast to Chrome, Edge, Firefox, and Safari?
• Do web developers know about Samsung Internet, when they do or do not choose it as a support requirement? (i.e., if they don't select it, do they do so intentionally.)

I'll respond to a couple of narrower points in a follow-up comment.

[romainmenke]

As for Samsung Internet, I don't recall web developers reporting (in the surveys and interviews I saw) that Samsung Internet was a support requirement for their work. That said, I don't recall any particular mention of Samsung Internet in the research: it'd probably be a good idea to double-check our work in this area.

Also important to capture the distinction between :

• developers really don't need to support Samsung Internet
• developers do need to support it, but they are unaware and largely get away with not knowing because Samsung Internet is not that far behind Chrome

"not that far behind Chrome" is very important in the context of Baseline where anything older than the last two majors becomes unsupported.

Developers that adopt Baseline and only support the latest two majors of Chrome, Edge, Firefox and Safari might rapidly discover that they do in fact need to support Samsung Internet after all.

[ddbeck]

To respond to a couple of specific points:

@romainmenke

So a natural heuristic for developers is to assume Edge support when there is Chrome support.

This is natural if you already know that the browsers share Chromium as underlying technology. Unfortunately, this is less widely understood than I'd like. Similar issues apply to understanding of the other browser engines too. Absent a significant (and successful) education effort, we're probably stuck identifying actual browsers, even if they're closely related ones.

capture the distinction between […]

Yes, that's exactly what I'm getting at with that third item in the to do list: is this a deliberate choice on the part of developers or an artifact of the way previous research was conducted?

@jhvanderschee

95% support in real browser usage seems more relevant to me

I'd be curious to know more about why that is. How did you come to 95%? Is that a preference or hard requirement? When, if ever, do you depart from that threshold? (I'm not doubting you here, but I am curious to know the context behind it.)

You might want to check out the discussion on #174 as well. There's some interesting complexity around usage stats (in terms of stability, trust, and the disconnect between global usage stats and the constraints that developers report).

[jhvanderschee]

Developers, like me, would LIKE to develop for only the last few versions (and maybe we often also do)... However, once a client complains we are suddenly forced to support an old Safari version or even IE11. Asking developers what they are supporting will give you an image of what they develop for until they are forced otherwise...

Developers are often not the decision makers. Their bosses, clients and support teams are. These decision makers will want to support as many users as possible to get an maximum market share/reach. Sometimes you can get away with 90%, sometimes only with 98%.

The 95% (I would love to discuss) is based on the level of real world support at which I (business owner AND programmer) am usually not getting any complaints. However, I am based in the EU. I think that companies in the US can face a law suit if they build a website that is not accessible to old browsers = poor people, due to the accessability guidelines/laws. Taking risk at this point can be very costly and can lead to bad publicity (thus is NOT safe).

That is why I think it is SAFE to use 98% real world support. I say this not just from a technical, but also from a business point of view.

[atopal]

I'm adding one of the surveys we ran here: web-platform-dx/developer-research#25

This is indeed a result of the principle that it should reflect developer expectations. Samsung Internet is cited on par with Internet Explorer as "needs to be supported in browser before I can use a feature". That's why it was not included in the initial core browser set.

Developers that adopt Baseline and only support the latest two majors of Chrome, Edge, Firefox and Safari might rapidly discover that they do in fact need to support Samsung Internet after all.

Yeah, this is a good point. Of course it would only be an issue in cases where Chrome is the last one to implement a specific feature. I'm not sure that it will keep developers from using those features, but you're right that it might impact people's perception of how reliable the Baseline designation is

[romainmenke]

Of course it would only be an issue in cases where Chrome is the last one to implement a specific feature.

This is incorrect.

This would be an issue each time something becomes part of Baseline in the window between the Chrome release + 1 and the next update to Samsung Internet.

Maybe Firefox is the last to implement, maybe Chrome.

---

My broader point here is that the current definition is reenforcing existing blindspots that developers have and that new blind spots are created.

This is not good.

[jhvanderschee]

From Baseline features you can use today:

In this post, I want to highlight some features that are now available to all major browser engines for the past two major versions. This is the cut-off point where we feel that the majority of developers will feel a feature is safe to use, and is the feature set we're calling Baseline.

In the article above the dialog element is promoted. I took a closer look at the data. The dialog element is not supported in Internet Explorer and only in the last version of Android Webview. The people who want to support Android Webview might not be the same as the ones that want to support Samsung Internet or Internet Explorer. If they are not overlapping they account for 34% of all developers, if they are for at least 20%. Reality will be somewhere in the middle.

The assumption (or this 'feeling') about this majority of developers might be correct, but it also means that you are setting a WRONG baseline for the rest of the developers. This is what you see reflected in this discussion. Being 'right' in most cases cannot be the actual goal of Baseline, especially when that is from a developers point of view, and not from a business point of view (as argued before)... it should be valid for (almost) everyone.

[ddbeck]

A whopping 40% feel they NEED to support the current AND previous version of IE!

(the following responds to an earlier version of the previous comment, since edited, see discussion below)

@jhvanderschee This is not correct. The questions were presented sequentially, so respondents were only asked about current/previous/older versions (question 9), if they had chosen the browser in the previous question about browsers in general (question 5). 14% of respondents said they required support for IE and, of those respondents, some percentage said they needed something older than IE11. Back of the envelope, it's closer to 8% (though based on a rather small number of respondents, so I doubt that it's a particularly rigorous figure).

[jhvanderschee]

@ddbeck Thank you. Sorry... my bad... I corrected it and updated my post. I did not read the fine print, although I participated in the result, so I should have known. :-)

Still... setting a wrong baseline in the eyes of apporoximately 20% of all developers might be a bit tricky. Rooting for real world coverage!

[romainmenke]

@jhvanderschee I think this is definitely worthy of further discussion, but maybe it is a bit off topic for this issue?

I don't think the exact numbers or even the definition of "safe to use" is directly relevant to my concern here.

My fear is that by omitting Samsung Internet in Baseline you create a situation where it is less likely that Samsung Internet will be taken into consideration by developers in the future.

If developers truly don't need to support Samsung Internet or if support for it comes automagically from having support for Chrome then that is fine. But I am not convinced this is the case.

I think it is more likely that developers are now too conservative with adopting new technologies and that targeting and testing on an old Chrome version ensures that they also support Samsung Internet. Even when they never even considered support for Samsung Internet, it is just a natural consequence of being conservative with new features.

Baseline however promotes adopting features much more quickly, possibly too quickly in some cases, exposing biases and gaps in the research data.

Because of the position of MDN, Google, ... this can have a negative impact.
Instead of viewing Baseline as having shortcomings some will place the "blame" on Samsung Internet or on end users.

A shortcoming in Baseline can too easily escalate to "Don't use Samsung Internet, it is always behind".

Switching to usage numbers for the definition of Baseline is one way of resolving this, but it is a very complex subject, worthy of it's own track. :)

[jhvanderschee]

@romainmenke yes, sorry... off topic. Thank you for preventing a derailing of the discussion.

I agree that Samsung Internet is relevant even though developers think it is not. When you look at https://caniuse.com/?search=dialog and click 'Usage relative' you can see that it is the third mobile browser. Defenitely worth taking into account for all the reasons you just mentioned.

[ddbeck]

@jhvanderschee Ah, I see—I responded to an earlier version without noticing the edits. I edited my comment to reflect that. Thanks!

More broadly, I think the case has been made here about Samsung Internet and Kadir has provided the research that led to this choice in the first place. A number of other issues are already discussing more expansive areas of the Baseline definition (duration of support, usage metrics, etc.), so I won't rehash them here. But the continuing actions on the issue of Samsung Internet specifically are along the lines of:

• For the next revision of the Baseline definition, reconsider the impact on browsers excluded from the set (i.e., are we exposing developers to surprising negative results for browsers not named in the definition?).
• Review and expand our communications around the definition. Some browsers are certain to be left out (IE!) but I suspect we can do more to help developers recognize that those browsers still exist and when their use case requires additional reading.

There's also a project documentation task, which is to document some basic assumptions and practices of the project (e.g., why we use browsers and not browser engines, intended audience, connection to the WebDX research workstream, and so on.). That needs its own issue which I'll write up when I return from vacation.

[romainmenke]

Sounds good!

[atopal]

Thank you Daniel, agreed on next steps. I'll follow up with another issue to collect the discussion results across the open issues.

[romainmenke]

I believe this issue has been resolved by the current definition?

The 30 months + mobile&desktop versions conditions are proxy metrics.
By the time these become true other vendors will have had time to release updates and those will have sufficient adoption.

---

@atopal Do you prefer I close issues I consider to be resolved or do you prefer that I keep them open until there is wider consensus that it can be closed?

[ddbeck]

@romainmenke Kadir is out until the new year, so I'll step in here. I think you're right that the issue here is resolved. I'll go ahead and close it. Thank you!