Filter and blur sensitive results by term matching

[zackkrida]

Start Date	Project Lead	Actual Ship Date
2023-02-06	@sarayourfriend	TBD

Description

This project matches Openverse images against a list of sensitive terms. All sensitive single image results on the frontend are blurred and viewing them is opt-in for all users.

Development work is ongoing and tracked in a milestone: https://github.com/WordPress/openverse/milestone/12

Documents

• Trust and Safety in the Openverse: Preliminary User Stories and Exploration - A general doc relevant to all content safety projects
• Project Proposal
• Implementation Plans (link to requested plans)
  • Implementation Plan: Managing A Sensitive Terms List #911
    • This wasn't noted as a requested IP in the project proposal as it was already being worked on by Zack, but it is part of this project.
  • Implementation Plan: API sensitive terms filtering and designation #937
    • Milestone here
  • Implementation Plan: Blurring sensitive results #938
• Design Proposal

Issues

• Add non-production feature flag for marking half of results as sensitive #854
• Setup sensitive terms list for the blurring project #1169
• API Milestone: https://github.com/WordPress/openverse/milestone/5
• Frontend Milestone: https://github.com/WordPress/openverse/milestone/12

Prior Art

• Create pre-filtered secondary indexes and add ability to automatically filter sensitive terms at query time openverse-api#1108 | A potential full-fledged implementation for the API
• [Feature] Use metadata keywords to help detect if something is NSFW (original #482) #750
• Consider hiding Flickr behind the 'mature' flag #721

[zackkrida]

Update 2023-03-07

Frontend design work is taking place here. This work assumes that the MVP of this functionality will allow the user to show sensitive results, and to elect to blur sensitive results in the search results view (with blurring as the default value).

We also assume the MVP of this functionality does not display or allow the user to enable specific sensitive content subcategories. However, this active design work will be easily adaptable and compatible with such functionality if we decide to include it.

Done

• Francisco created a design proposal (Safe content browsing flow #791) and we have been soliciting feedback.

Next

• Add non-production feature flag for marking half of results as sensitive #854 @dhruvkb is going to implement a solution which enables developers to mark arbitrary search results as sensitive for the purposes of testing blurring without requiring developers to look at actual sensitive content.
• @panchovm will share an update to the design proposal for additional commentary and approval

Blockers

I have been delayed in publishing my recommendations for a sensitive content terms list for Openverse, but will do so by the end of the week.

[sarayourfriend]

@zackkrida This project currently has the "In RFC" status, but it looks like that means we're skipping the project planning step of this. Is that intentional? I want to clarify before I get to deeply into writing the implementation plan.

[sarayourfriend]

This work assumes that the MVP of this functionality will allow the user to show sensitive results, and to elect to blur sensitive results in the search results view (with blurring as the default value).

Further clarification: are these separate features? Is this re-wording of the quoted summary accurate? "We will enhance our data for what a "sensitive" result might be, thereby filtering them out of queries that do not have mature=True. Additionally, when mature=True and we are including results with sensitive terms, then we will by default blur those results until the user designates that they do not want the blurred results."

The part I want to clarify here is whether we expect results with sensitive terms to appear in queries where mature is not True (i.e., the default query) or if blurred results would only appear for when mature=True.

[zackkrida]

Hi @sarayourfriend! I've fixed the project status to 'In Kickoff'. I erroneously moved it there when the design proposal was created, as I forgot this project included the full scope of the term matching and not just the frontend blurring implementation.

I was of the opinion that we'd want a proposal and implementation plan here, but would like your opinion.

To clarify the update points about the design work:

Users opt-in to seeing sensitive search results. After opting in, blurred sensitive results will be displayed in their search results. If desired, they can further opt-in to unblur these results.

Users will not see sensitive results (those marked as mature internally) in the default query.

For single results, how does this sound:

Individual results will be blurred for all users when visited directly by url, regardless of specified user preferences. Sensitive results will only be unblurred when navigated to from a search results view where 'show sensitive results' is enabled and 'blur sensitive results' is disabled.

[zackkrida]

@panchovm and I decided that it might be best to pause design work after soliciting feedback on the latest iteration of the design. This is primarily to prevent us from having to make further assumptions that may contradict the upcoming planning documents. The designs are in a place where some preliminary development can start (or resume, considering #824) behind a feature flag while we continue to plan.

[sarayourfriend]

Thanks for the clarifications on all accounts. They sound great. I've got about half a technical implementation plan written for this, it turns out the most difficult part will probably be designating which results have sensitive terms in their textual content in the search API in a way that does not degrade performance.

I'll write the kickoff post first and get the project planning generally started. Having technical implementation plans is a good idea here. Blurring likely deserves its own consideration unless we decide to just use regular CSS blurring, but it's something we can discuss in the kickoff discussion.

[zackkrida]

One other idea for that sync session:

• Should we show the eye-closed icon on top of the blurred results? I noticed google does this and it does seem clearer:

[fcoveram]

I also like Google's solution, but there are other product-wise considerations pointed out here. But definitely something to keep in mind.

[zackkrida]

I am thinking of the icon like a label, rather than proposing any changes to the flow or behavior.

There are certain contexts where the blur alone may not be a strong enough visual cue, like the following results:

[fcoveram]

Oh, I see. You are right. The interface needs to provide more context. Good point.

[dhruvkb]

I am not sure the global player needs to be blurred because for it to appear, the user must click a result, choose to unblur that result from the content safety wall, see everything in the single result view, play the audio and then go back to all results. After this has happened, the result item itself is unblurred so the global player playing that item need not be blurred.

As for the eye icon on the blurred images, I would be in favour of a visual context but I am not sure what the best way to indicate this is. I played a bit with the idea and have three thoughts.

• The icon will need to be colored in a way that ensures visibility irrespective of the background. The easiest way to do this is to put the icon in a box with a fixed color (like we do for the licenses).
  
  

• The icon looks a bit different and is placed separately from other iconography associated with the image.
  

• The audio results do not have a corresponding icon. Is that because we can safely believe that text being blurred is unambiguous? What can be done about consistency between different media types?

@fcoveram your design input about these points will be very helpful, thanks.

[fcoveram]

Global player

I am not sure the global player needs to be blurred because for it to appear, the user must click a result, choose to unblur that result from the content safety wall, see everything in the single result view, play the audio and then go back to all results. After this has happened, the result item itself is unblurred so the global player playing that item need not be blurred.

I agree with this. I'm referring to the flow when users play a blurred audio track in the search results page. If the item has not been unblurred, and you click on play, the global player should also show blurred info. In other words, global player should mirror the audio track state.

Here is a quick prototype of this flow.

Global.player.blurred.mp4

Icon indicator

I tried different styles for the icon, and a black icon over a white layer with 60% opacity might work. The ratio contrast required for graphics is lower than for texts. However, the consistency between media types is my main concern.

After testing some ideas, I concluded that the designed consent flow (landing in the search results and enabling the sensitive results) is sufficient for conveying why some content (images and texts) is blurred. The change in the results area is very clear in that new content was added.

[dhruvkb]

The project is very close to completion, with a total of 5 tickets, including 1 blocked ticket (#2550 that needs legal input) and 3 that already have PRs associated with them.

The API ticket for sensitivity in single results #2926 is the biggest blocker to roll-out so getting it assigned and resolved is the next priority item.

[sarayourfriend]

I'm reaching out to the legal team again to ask for clarification on when we can expect a response. The last time I asked they didn't respond 😕

[openverse-bot]

Hi @dhruvkb, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

[dhruvkb]

The latest on this is that we have heard back from legal and these discussions brought up some actionable items:

1. wording changes in the draft document
2. wording changes for "mature" to "sensitive" in our products
3. additional features for the moderation tools project

@sarayourfriend has already implemented points 1 and 2 (thanks! 👏) so I think it should be time to formalise the text into a page on Openverse.org. The third will soon be implemented as a part of that project.

[openverse-bot]

Hi @dhruvkb, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

[dhruvkb]

This has been shipped to production 🎈 🥳 🎉 ! One issue #3081 remains in the milestone blocked on the official translations to be written for the content safety explanation page.

[dhruvkb]

The last issue in the milestone has also been resolved now. The project is shipped!

[AetherUnbound]

I've drafted an announcement post for the feature here: Google Docs Link. @dhruvkb and @WordPress/openverse-maintainers - please take a look and let me know if it sounds okay! I can post it this week and we can move this to "Success" 😄 🚀

[fcoveram]

I will share the document with rmartinezduque asking for feedback.

[rmartinezduque]

Congrats on shipping this new feature! The post looks great to me. I think it clearly communicates the new feature and the benefits it brings. I only suggested a few minor edits, mostly to correct a grammar mistake. If possible, I think it would also be nice to include an image to accompany the announcement (perhaps you already have it in mind). 🙂

[fcoveram]

I can work on a visual for the post ⭐

[AetherUnbound]

Thank you @rmartinezduque and @fcoveram!

[AetherUnbound]

Our announcement P2 has been made: https://make.wordpress.org/openverse/2023/12/11/introducing-enhanced-content-safety-features-on-openverse/

And I've also made an amplification request for the marketing team here: WordPress/Marketing-Team#330

With that, I'm going to move this project to "Success"!