Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAP-1781 Cleanup Access Logs more than 3 months old #2408

Merged
merged 6 commits into from
Nov 15, 2024
Merged

MAP-1781 Cleanup Access Logs more than 3 months old #2408

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8064004
Cleanup Access Logs more than 3 months old
tobyprivett Nov 15, 2024
c593e00
Fix rubocop issues
tobyprivett Nov 15, 2024
24212cb
Prefer a cutoff date of 6 months
tobyprivett Nov 15, 2024
815c129
Add a spec to test the access_logs:cleanup task
tobyprivett Nov 15, 2024
199d640
Prefer cutoff date of 1 year
tobyprivett Nov 15, 2024
5cd6df9
`reenable` lets us run a rake task multiple times in a spec
tobyprivett Nov 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions helm_deploy/values-preprod.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ generic-service:
REDIS_URL: url

cronJobs:
- name: access-log-cleanup
schedule: "0 2 * * *"
command: ["bundle", "exec", "rake", "access_logs:cleanup"]
- name: token-cleanup
schedule: "0 1 * * 0"
command: ["bundle", "exec", "rake", "doorkeeper:db:cleanup"]
Expand Down
3 changes: 3 additions & 0 deletions helm_deploy/values-production.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,9 @@ sidekiq:
GOVUK_NOTIFY_API_KEY: govuk_notify_api_key

cronJobs:
- name: access-log-cleanup
schedule: "0 2 * * *"
command: ["bundle", "exec", "rake", "access_logs:cleanup"]
- name: token-cleanup
schedule: "0 1 * * 0"
command: ["bundle", "exec", "rake", "doorkeeper:db:cleanup"]
Expand Down
3 changes: 3 additions & 0 deletions helm_deploy/values-staging.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ sidekiq:
GOVUK_NOTIFY_API_KEY: govuk_notify_api_key

cronJobs:
- name: access-log-cleanup
schedule: "0 2 * * *"
command: ["bundle", "exec", "rake", "access_logs:cleanup"]
- name: token-cleanup
schedule: "0 1 * * 0"
command: ["bundle", "exec", "rake", "doorkeeper:db:cleanup"]
Expand Down
3 changes: 3 additions & 0 deletions helm_deploy/values-uat.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,9 @@ sidekiq:
GOVUK_NOTIFY_API_KEY: govuk_notify_api_key

cronJobs:
- name: access-log-cleanup
schedule: "0 2 * * *"
command: ["bundle", "exec", "rake", "access_logs:cleanup"]
- name: token-cleanup
schedule: "0 1 * * 0"
command: ["bundle", "exec", "rake", "doorkeeper:db:cleanup"]
Expand Down
14 changes: 14 additions & 0 deletions lib/tasks/access_logs.rake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
namespace :access_logs do
desc 'Cleanup access_logs older than 6 months'
task cleanup: [:environment] do
cutoff_date = 6.months.ago
access_logs_count = AccessLog.where('timestamp < ?', cutoff_date).count
number_of_iterations = (access_logs_count.to_f / 1000).ceil

puts "Cleaning up #{access_logs_count} access_logs in #{number_of_iterations} iterations"

1.upto(number_of_iterations).each do
AccessLog.where('timestamp < ?', cutoff_date).limit(1000).delete_all
end
end
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest possibly adding a test for this task but it's up to you

Loading