Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sensitive content report moderation workflow #383

Open
12 tasks done
obulat opened this issue Feb 9, 2023 · 44 comments
Open
12 tasks done

Sensitive content report moderation workflow #383

obulat opened this issue Feb 9, 2023 · 44 comments
Assignees
Labels
🧭 project: thread An issue used to track a project and its progress

Comments

@obulat obulat added the 🧭 project: thread An issue used to track a project and its progress label Feb 9, 2023
@obulat obulat moved this to Not Started in Openverse Project Tracker Feb 9, 2023
@krysal
Copy link
Member

krysal commented Feb 17, 2023

Required work to enable the DAG for reported media pending review created at WordPress/openverse-catalog#513. I noticed it hadn't run yet.

@zackkrida zackkrida moved this from Not Started to In Kickoff in Openverse Project Tracker Feb 22, 2023
@zackkrida zackkrida moved this from In Kickoff to Not Started in Openverse Project Tracker Mar 8, 2023
@sarayourfriend sarayourfriend moved this from Not Started to In Kickoff in Openverse Project Tracker Mar 31, 2023
@sarayourfriend sarayourfriend moved this from In Kickoff to Not Started in Openverse Project Tracker Mar 31, 2023
@zackkrida zackkrida moved this from Not Started to In Kickoff in Openverse Project Tracker Apr 12, 2023
@github-project-automation github-project-automation bot moved this to 📋 Backlog in Openverse Backlog Apr 13, 2023
@obulat obulat moved this from In Kickoff to In RFC in Openverse Project Tracker May 3, 2023
@zackkrida zackkrida mentioned this issue May 31, 2023
12 tasks
@sarayourfriend
Copy link
Collaborator

@AetherUnbound has started on the first implementation plan for this project. We probably need to get the next implementation plan also started. The project proposal shows that the API cache management and the initial Django access control implementation plans can be started at any time. If any @WordPress/openverse-maintainers do not currently have an implementation plan they are working on or can take on a new one, please volunteer for one of the ones for this project.

I will start working on #1969 next week.

@openverse-bot
Copy link
Collaborator

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

@sarayourfriend
Copy link
Collaborator

#2126 is pending review from the requested reviewers, @dhruvkb and @krysal. @zackkrida given the delay, do you think it's appropriate for my review of the implementation plan to serve as one of the two approvals for go ahead? @dhruvkb and @krysal, can y'all share an ETA of when you'll be able to review the implementation plan? It's been in the decision round much longer than the expected time span.

I was going to start working on the #1969 IP, but that hasn't happened due to recent incidents and high priority retrospective items taking up lots of time followed by MSR. I hope to be able to start that implementation plan next week.

The recent Redis memory consumption issue has brought to attention that the fine-grained cache IP will need to consider Redis memory usage.

@sarayourfriend
Copy link
Collaborator

After starting to work on the fine-grained cache implementation plan, I opened a discussion about our approach to caching generally: #2453

In that discussion we've decided to explore significantly reducing our cache TTL, potentially low enough such that needing fine-grained cache invalidation on the search endpoint would no longer be necessary. Therefore, I will stall work on the implementation plan to further investigate how changes to the cache will effect performance, especially Elasticsearch and Postgres resource consumption. This is possible now that the thumbnails service is live and we have confirmed that API instability was due almost entirely to long-running thumbnails requests.

Other implementation plans may move forward, such as the Django admin access control and baseline changes, which would unblock several other implementation plans.

@openverse-bot
Copy link
Collaborator

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

@sarayourfriend
Copy link
Collaborator

Apologies for missing an update on this.

@dhruvkb's got a very cool PR for the "locking" of reports, so that moderators know when someone is reviewing a work or not: #4374

I also opened a related issue, that will sit outside the project, for us to consider different ways of logging into Django admin that could make user management and permissions a bit easier: #4342. That is out of scope for this project, and we do not realistically need, nor would benefit from all that complexity. The plan right now does not involved a large number of moderators, nor would they change often, so we don't benefit from making user management overall easier, if the underlying implementation is complicated.

I intended to start working on some issues for this project this week, but got pulled away to things related to an API incident. Hopefully I'll wrap those up early next week and be able to work on some of the issues from this milestone: https://github.com/WordPress/openverse/milestone/22

Krystle's picked up the computer vision labelling implementation plan. Looking forward to seeing that one, especially with the ongoing work in #431 in the works.

@sarayourfriend
Copy link
Collaborator

sarayourfriend commented Jun 5, 2024

To reduce scope and help make more time/room in our schedule for projects important to search, we're cutting the computer vision and metrification aspects of this project. They were always considered "nice to have", and while I'm disappointed @dhruvkb's IP for the metrification part won't be implemented right away, the work is planned and available for us to work on later. It is less critical now that we know who will be doing the moderation. The visibility we'll have with the basic features after we've implemented bulk moderation will be more than sufficient to ensure reports are reviewed in a timely and consistent manner, as monitored by the moderators themselves (following self-accountability, if it needs to be more explicit). @zackkrida do you think we should close the issues in the milestone, or leave them as is?

The computer vision aspect would also be nice to have, and a continuation of #4040, but ultimately not necessary for MVP.

@zackkrida
Copy link
Member

@zackkrida do you think we should close the issues in the milestone, or leave them as is?

I've just peeked at the milestones for the project. I closed the IP milestone, and the two remaining milestones look to contain the correct issues.

Django admin access control and basic functionality changes: WordPress/openverse/milestone/22
Bulk moderation actions: WordPress/openverse/milestone/24

Perhaps you've already made changes, @sarayourfriend? In any case, it all looks good to me.

@sarayourfriend
Copy link
Collaborator

I meant specifically about the metrification issues. There are only three, and they're arguably unnecessary with the current plan of who will do the moderation. I'll close them as won't do for now, and we can revisit our needs in the future once we actually need it.

@zackkrida
Copy link
Member

Ah thanks! I saw that those weren't part of the project's milestones so figured they'd be acceptable to keep as low priority for a rainy day, but you're totally right given our set of moderators.

@openverse-bot
Copy link
Collaborator

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

1 similar comment
@openverse-bot
Copy link
Collaborator

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

@sarayourfriend
Copy link
Collaborator

Oops! I thought I'd left an update here two weeks ago! Apologies.

A single issue remains in the initial milestone, #3640, assigned to @dhruvkb. That one does not block work on the bulk moderation issues, so we could start on those issues as soon as we like.

The backfill had one small issue with foreign keys, fixed in #4512. We ran it in production, and have now dropped the status column in production. The new model for report decision tracking is fully operational!

@openverse-bot
Copy link
Collaborator

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

@zackkrida
Copy link
Member

@dhruvkb do you mind contributing this update while @sarayourfriend is AFK?

@dhruvkb
Copy link
Member

dhruvkb commented Jul 19, 2024

Issue #3640 from milestone https://github.com/WordPress/openverse/milestone/22 has an associated PR #4603, that has been reviewed by @sarayourfriend and the changes that Sara requested have been addressed. Since Sara is AFK, I have requested another reviewer to be picked automatically from the WordPress/openverse-api team.

Also I have started working on issue #3840 from milestone https://github.com/WordPress/openverse/milestone/24 and a PR for that will be up very soon.

@openverse-bot
Copy link
Collaborator

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

@dhruvkb dhruvkb moved this from 🚧 In Progress to 🚢 Shipped in Openverse Project Tracker Aug 3, 2024
@dhruvkb
Copy link
Member

dhruvkb commented Aug 3, 2024

All milestones for this project have now been completed, so the project has been moved to the "🚢 Shipped" column.

@zackkrida zackkrida reopened this Aug 7, 2024
@zackkrida
Copy link
Member

@WordPress/openverse-maintainers this project will be marked a success after the tooling has actually been used in a two-week trial by the sub-group of maintainers who will perform moderation.

Feedback from this trial may inform further improvements or changes.

@sarayourfriend
Copy link
Collaborator

It would be great to split feedback between bugs with the functionality implemented for this project and with iterative improvements and additional features, just so we don't accidentally expand the scope of this project beyond its original goals for the foundational functionality.

@openverse-bot
Copy link
Collaborator

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

@sarayourfriend
Copy link
Collaborator

@zackkrida do you know when the two-week trial will start, and if so, can you share the details here? If not, how can I help get that moving forward?

Nothing to add in the way of an update beyond that, though.

@zackkrida
Copy link
Member

@sarayourfriend given upcoming team AFK and travel, I think the trial should take place from October 14-25th.

Additionally, I think we should stop sending reminder pings for shipped projects:
#4912

@openverse-bot

This comment was marked as outdated.

@sarayourfriend
Copy link
Collaborator

See #383 (comment)

Also, I opened a PR to stop these pings on shipped projects as suggested by Zack: #4993.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🧭 project: thread An issue used to track a project and its progress
Projects
Status: 🚢 Shipped
Development

No branches or pull requests

7 participants