Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the "Blacklist / Whitelist" nomenclature #1483

Closed
9 of 10 tasks
tlfeng opened this issue Nov 1, 2021 · 19 comments
Closed
9 of 10 tasks

Change the "Blacklist / Whitelist" nomenclature #1483

tlfeng opened this issue Nov 1, 2021 · 19 comments
Labels
enhancement Enhancement or improvement to existing feature or request Meta Meta issue, not directly linked to a PR

Comments

@tlfeng
Copy link
Collaborator

tlfeng commented Nov 1, 2021

Goal of the enhancement

Coming from #2589

Replace non-inclusive terminologies "blacklist" and "whitelist" throughout this repository with inclusive ones.

Context

What is “inclusive language”?

Inclusive language is designed to avoid excluding people on the basis of gender, sexual preference, age, race, ability, etc. By using language that avoids expressions which imply or express ideas that have prejudice to any particular group of people, we aim to create a more equitable community.

Why using “inclusive language” is important?

Using inclusive language helps build an inclusive environment, which accepts diversity and ensures any individuals and groups feel welcome, respected, and safe. Diversity does contribute to create a fair and strong organization, and also leads to diversity of ideas, which change into creativity and innovation.

How to apply “inclusive language” in a software program?

Replace “objectionable language in code bases and documentation. It involves assessing existing code bases and documentation, identifying potentially problematic language, then replacing terms with more preferable language.

Solution

Overview

To solve the issue, breaking changes will be introduced to OpenSearch, which requires backwards compatibility. The exclusionary terms exist in APIs and configurations, which will impact the compatibility with previous versions of OpenSearch and its third-party clients and tools.
We will support both and new APIs or configurations for a period of time, to let users get used to the breaking changes gradually. Considering OpenSearch follows Semantic Versioning, we are going to include the new API and configuration in current minor versions, while continuing to provide support for the previous APIs and configurations for a period of time, then remove support for the old ones in a next major version.

Substitute for the exclusionary terminology

Our proposed preferable terminologies:

  1. Use allowlist to replace whitelist
  2. Use denylist to replace blacklist

Location for the terminologies:

  • code comments
  • internal variable, method and class names
  • Gradle build environment configuration
  • field, method, class and package names that are exposed as API in Java libraries (renaming will impact clients and plugins that depend on OpenSearch code base)
  • setting names and values (including definition, yml configuration files)
  • tests

Versions to introduce the change

  • Add the new usages with inclusive terminology and deprecation notice for the old usages in minor versions of 1.x.
  • Remove the support for the old usages (APIs and settings) in a major version, it could be either 2.0 or 3.0.

Explanation:

  1. The ideal case is to apply inclusive naming to replace the exclusionary terminology as soon as possible, but the backwards compatibility is a key aspect to be considered, so with new usages added soon, the old usages will be kept for a while before removal to keep the compatibility.
  2. Considering OpenSearch follows Semantic Versioning, deprecation notice MUST come at a minor version, and the breaking change, such as API removal MUST come at a major version.
  3. Update in March 2022: In practice, new usages will be added in version 2.0, and old usages will be removed in at least version 3.0.

Procedure for the replacement

1. Replace the exclusionary words that won’t impact the compatibility.

(source of the idea)
Replace all the exclusionary words in the following location:

  • code comments
  • Internal variable, method and class names
  • Gradle build environment configuration

2. Deprecate old usages that having exclusionary words and impact the compatibility, then create alternative usages with the inclusive terms.

(source of the idea)
This step includes the following effort:

  1. Deprecate the setting which contains exclusionary words.
  2. Add proper form of deprecation notice for every setting changes, including in code (such as adding @Deprecated annotation), console output and logs.
  3. Create alternative settings where having name change, so that calls to the new names can have the same effect with the old names.
  4. For settings, adding fallback logic to fallback to the existing setting with old name, when the corresponding new setting is not configured.

The location that will impact the compatibility when changed:

  • Configuration, including setting name.

3. Add tests to check both old and new usages - in Settings.

(source of the idea)
Adding new unit tests to check the both old and new names have got the same functionality.

4. Replace the exclusionary words in log messages and update documentation.

This step includes the following effort:

  1. Replace the exclusionary words in logs.
  2. Update the user and developer documents to introduce the new usages, and notice the deprecation of old usages.

5. Replace the exclusionary words that will impact software programs depend on OpenSearch Java libraries - in Java APIs.

Replace the exclusionary words in all Java APIs, including field, method, class, and package names.
The "Java APIs" refers to those packaged in Java libraries and are published to Maven (https://search.maven.org/search?q=g:org.opensearch https://mvnrepository.com/artifact/org.opensearch)
Impact of this step:
All plugins, clients and tools that use OpenSearch Java APIs from OpenSearch Java libraries which contain non-inclusive terminologies have to make corresponding changes to call new APIs, if they want to upgrade the dependency to a future major version of OpenSearch.

6. Remove the deprecated settings in a future major version.

Sub-issues

Appendix

Location that contains the exclusionary terms and will impact the compatibility when changed

Settings
1 reindex.remote.whitelist (static)
Code: https://github.com/opensearch-project/OpenSearch/blob/1.0.0/modules/reindex/src/main/java/org/opensearch/index/reindex/TransportReindexAction.java#L60

Location that contains the exclusionary terms and will impact software programs depend on OpenSearch Java libraries

API of Java library
1 Painless SPI - multiple classes with name “Whitelist”
Code: https://github.com/opensearch-project/OpenSearch/tree/1.0.0/modules/lang-painless/spi/src/main/java/org/opensearch/painless/spi
Javadoc: https://opensearch.org/javadocs/1.0.0/OpenSearch/modules/lang-painless/spi/build/docs/javadoc/org/opensearch/painless/spi/package-summary.html

Related issues and PRs

Existing PR before the issue created: #744
Related issue: #472

Issues for other repositories in the organization:

@tlfeng tlfeng added enhancement Enhancement or improvement to existing feature or request >breaking Identifies a breaking change. labels Nov 1, 2021
@tlfeng tlfeng changed the title Change the "Blacklist" / "Whitelist" nomenclature Change the "Blacklist / Whitelist" nomenclature Nov 1, 2021
@tlfeng tlfeng added the Meta Meta issue, not directly linked to a PR label Nov 3, 2021
@stockholmux
Copy link
Member

Hey - just noticed that this was on the roadmap for 1.3 yet it's marked "breaking." The body of this issue does describe what's happening but if you just look at the roadmap it's alarming to see "breaking" on a minor release.

Can we remove that tag?

@dblock dblock added >breaking Identifies a breaking change. v1.3.0 and removed >breaking Identifies a breaking change. labels Dec 10, 2021
@dblock
Copy link
Member

dblock commented Dec 10, 2021

@stockholmux I think we want the non-breaking changes in 1.3, so for now I removed breaking instead.

@morsik
Copy link

morsik commented Jan 8, 2022

Waiting for official english dictionary to remove "black" and "white" colors from it due to "excluding people on the basis of gender, sexual preference, age, race, ability"…

How the hell "whitelist" and "blacklist" can even be excluding to anyone?! Don't you know the physics thing called "shadows"? It makes things invisible… dark… (shouldn't even mentiond "black" because someone will kill me in a second because of that!).

I just really want to understand the reasoning for this. And reasoning should be reasonable! Not some imaginary problem (you obviously have here).

@dblock
Copy link
Member

dblock commented Jan 10, 2022

@morsik #472 (comment)

On behalf of the maintainers of the project I agree that the term 'master' may be offensive to some people. This project is committed to being a community where everyone can join and contribute. This means using inclusive and welcoming language. The repo defaults to main for the branch and, in a similar vein, supporting inclusive language should extend to the API and the rest of the software as long as it can be made in a backward compatible way.

@morsik
Copy link

morsik commented Jan 10, 2022

@dblock "may be offensive to some people" is not explanation. You can cover anything behind this sentence just for sake of not answering this question. I'm asking about real explanation, who and how is offended by this?

@dblock
Copy link
Member

dblock commented Jan 11, 2022

@dblock "may be offensive to some people" is not explanation. You can cover anything behind this sentence just for sake of not answering this question. I'm asking about real explanation, who and how is offended by this?

I have offended Engineers on my teams over the years by using non-inclusive language in technical conversations. I've referenced race, gender and sexual orientation, physical and mental ability. I am not going to ask these individuals to explain here how they were offended, or make a list of who they are. But I've been asked directly, 1:1, to change the words I use, therefore I support this change. Other people's experiences matter to me.

@morsik
Copy link

morsik commented Jan 11, 2022

I am not going to ask these individuals to explain here how they were offended

This seems kinda weird for software engineer… every change I do in my software or infrastructure is made because of some reason which I have to understand why something happens and make clear decision based on circumstances. Based on what you said… you don't even know why you want to make this change.

I've referenced race, gender and sexual orientation, physical and mental ability.

Well, that's your problem that you were a bad person to others, not mine and not other people in community. I don't see why your personal problems should reflect to software. Not to mention that "blacklist" and "whitelist" don't refer to "race, gender and sexual orientation, physical and mental ability" at all.

@jcgraybill
Copy link

Hi @morsik, thanks for expressing your concern here. We definitely don't make breaking changes lightly, and I hear your feedback that every change has a cost.

We're committed to avoiding noninclusive language throughout the project. I recognize some members of the community may be encountering this perspective on "blacklist" and "whitelist" for the first time. Sometimes the realization that a familiar term is actually not the best can be surprising, and curiosity or skepticism about that is totally rational.

"Blacklist" and "whitelist" are recognized as noninclusive terms pretty broadly: this isn't a concept that originated from this project, and our intention is to recognize and respect this rather than to re-debate it.

In this case, I think this change has the benefit of making this terminology more accurate and understandable: as the lists are of things that are allowed and denied, not lists of things that are colors.

@dlvenable
Copy link
Member

This issue came up in the community meeting yesterday. It sounds like the plan is to remove the terms "blacklist" and "whitelist" in 2.0. The reindex.remote.whitelist setting has been around since at least Elasticsearch 6, and I think a longer deprecation path is more appropriate.

It's good that the new fields will come soon in 1.3 so that developers can start making changes soon. But, removing the field from APIs in 2.0 will make migration harder.

@dblock
Copy link
Member

dblock commented Feb 1, 2022

This issue came up in the community meeting yesterday. It sounds like the plan is to remove the terms "blacklist" and "whitelist" in 2.0. The reindex.remote.whitelist setting has been around since at least Elasticsearch 6, and I think a longer deprecation path is more appropriate.

It's good that the new fields will come soon in 1.3 so that developers can start making changes soon. But, removing the field from APIs in 2.0 will make migration harder.

Let's start by implementing a backwards-compatible change and we can decide the deprecation -> removal separately.

@qreshi
Copy link
Contributor

qreshi commented Mar 29, 2022

@dblock Regarding these changes for the plugins, do we have an ETA on when OpenSearch and OpenSearch-Dashboards will rename their references?

Alerting, for example, is referencing Painless and NodeListeners which have things like WhitelistLoader and localNodeMaster, etc. So we can't fully remove our references until core does.

@dblock
Copy link
Member

dblock commented Mar 29, 2022

@tlfeng ^

@tlfeng
Copy link
Collaborator Author

tlfeng commented Mar 29, 2022

Regarding these changes for the plugins, do we have an ETA on when OpenSearch and OpenSearch-Dashboards will rename their references?

Hi @qreshi 😄 Such class/method names that in OpenSearch Java libraries will be renamed in a future major version. At least in version 3.0, but depends on the public acceptance. It is tracked in issue #1683. I will keep updated.
There was a discussion on renaming usages in the public Java APIs (#2453 (comment)). Although deprecating old usage and create new usage is the normal way, there are too many public class/method names that having the non-inclusive name, I'm likely to rename them in-place in a major version instead.

@qreshi
Copy link
Contributor

qreshi commented Mar 29, 2022

@tlfeng Got it, so it's safe to say that these issues being tracked for the plugins don't need to be closed out by 2.0 preview or GA? Since this issue is currently tagged as 2.0.0 and on the roadmap should we disambiguate that by detailing a high-level summary of what we expect to be done by 2.0 and what is targeted for 3.0?

@tlfeng
Copy link
Collaborator Author

tlfeng commented Mar 29, 2022

Since this issue is currently tagged as 2.0.0 and on the roadmap should we disambiguate that by detailing a high-level summary of what we expect to be done by 2.0 and what is targeted for 3.0?

@qreshi That's a reasonable suggestion! I will add more detail to disambiguate the current process. Thank you! 😁

@dblock dblock changed the title Change the "Blacklist / Whitelist" nomenclature Deprecte the "Blacklist / Whitelist" nomenclature Apr 18, 2022
@dblock dblock changed the title Deprecte the "Blacklist / Whitelist" nomenclature Deprecate the "Blacklist / Whitelist" nomenclature Apr 18, 2022
@kartg
Copy link
Member

kartg commented Apr 28, 2022

This issues spans several changes, some of which are not in scope for 2.0.0. I'll go ahead and drop the v2.0.0 tag

@kartg kartg removed the v2.0.0 Version 2.0.0 label Apr 28, 2022
@flavienbwk
Copy link

Our society has a serious problem with intellectual honesty: biased by ultra-minority political opinions from people who just yell loudly.

It's even worse here: OpenSearch engineers, only witnessing their close surrounding is bothered by this "black/white terminology", are totally disrupting the roadmap with a change that will take them months to make.

Have you conducted an scientific survey on this alleged problem ?

@tlfeng tlfeng changed the title Deprecate the "Blacklist / Whitelist" nomenclature Change the "Blacklist / Whitelist" nomenclature Aug 3, 2022
@anasalkouz
Copy link
Member

With 2.4 release, we have deprecated all non-inclusive usages (i.e. master, blacklist and whitelist) and we are planning to remove those deprecated usages on 3.0 release.

@anasalkouz
Copy link
Member

Closing the issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Meta Meta issue, not directly linked to a PR
Projects
None yet
Development

No branches or pull requests

10 participants