404 errors - Successful Grants collection #2504

seanluyk · 2021-09-08T15:46:20Z

Describe the bug
Items in the Successful Grants collection are all returning 404 errors.

To Reproduce
Steps to reproduce the behavior:

Navigate to collection
Click on some items
See error

Additional context
Items in this collection were recently moved as part of a consolidation project. I will check to see if other items in the moved collections are impacted.

seanluyk · 2021-09-08T15:53:00Z

Possibly related to this issue, it doesn't appear to be possible to deposit items to this collection:

###Steps to reproduce###

Describe/deposit item
ERA returns the message: File is infected with a virus.

I've tried this on multiple files that I know do not contain viruses (e.g. brand new text file, attached)

test.txt

pgwillia · 2021-09-08T16:07:31Z

Possibly related to this issue, it doesn't appear to be possible to deposit items to this collection:

###Steps to reproduce###
1. Describe/deposit item

2. ERA returns the message: File is infected with a virus.
I've tried this on multiple files that I know do not contain viruses (e.g. brand new text file, attached)

test.txt

This is the rollbar details of the https://rollbar.com/ualbertalib/jupiter/items/1177/ seems to be a filesystem issue so I've tagged @henryzhang87

pgwillia · 2021-09-08T16:08:40Z

Describe the bug
Items in the Successful Grants collection are all returning 404 errors.

To Reproduce
Steps to reproduce the behavior:
1. Navigate to collection

2. Click on some items

3. See error
Additional context
Items in this collection were recently moved as part of a consolidation project. I will check to see if other items in the moved collections are impacted.

I wasn't able to reproduce this in Firefox/Ubuntu or find this in rollbar. Could you provide more information? Is there a particular item that's failing I could try? Are you logged in?

seanluyk · 2021-09-08T16:12:51Z

@pgwillia I'm logged in (Mac OS 11, latest version of Safari, not on VPN). A user first reported the issue with this collection, and then I noticed that all items contained are also returning 404 errors. Just tried on Chrome and got the same error.

I'm not experiencing this issue with another other collections. I'll email you the file so you can try to replicate.

pgwillia · 2021-09-08T16:13:25Z

Describe the bug
Items in the Successful Grants collection are all returning 404 errors.
To Reproduce
Steps to reproduce the behavior:
1. Navigate to collection

2. Click on some items

3. See error
Additional context
Items in this collection were recently moved as part of a consolidation project. I will check to see if other items in the moved collections are impacted.
I wasn't able to reproduce this in Firefox/Ubuntu or find this in rollbar. Could you provide more information? Is there a particular item that's failing I could try? Are you logged in?

I was able to reproduce as an admin user visiting https://era.library.ualberta.ca/items/15462189-fce1-4a40-a88d-3dceeb8b74ea. Doesn't seem to show up in the rollbar logs.

seanluyk · 2021-09-08T16:17:32Z

@pgwillia FYI a user without admin privileges first reported the "file is infected with a virus" problem.

seanluyk · 2021-09-08T16:32:05Z

@pgwillia @henryzhang87 it appears that the issue is more widespread. We've received reports in the ERA Helpdesk regarding the Research Publications and Certificate in Computer Game Development collections

seanluyk · 2021-09-08T16:39:39Z

@pgwillia @henryzhang87 re the 404 errors issue, is it possible this is ISP related? Leah can access, but myself and another user (Telus) aren't able to

henryzhang87 · 2021-09-08T17:06:06Z

Will have a look

leahvanderjagt · 2021-09-08T17:24:42Z

Hi everyone @pgwillia @henryzhang87 do you think Beau's changes noted in Slack could be behind these problems?

Beau has been changing the underlying software managing some of our services. My understanding is that so far there hasn't been any impact to any of the services since he started last Wednesday. Today he has some servers supporting ERA and DMP on the list to migrate.

What we know:

Users NOT logged in can access any collection and download any item
Logged in users, including but not limited to admin, are encountering an error "file is infected with a virus" and this is suspicious to us because a) I don't know if this is baked into the Jupiter code and b) we are having too many user reports of this problem given the current date which should reflect normal circumstances
Unfortunately, while this is affecting some collections/items, it is not affecting all, so it is item/collection intermittent (not time-intermittent)

Consequence:

ERA Admin cannot deposit, edit or take any action on a number of collections or items
Logged in users are encountering many 404 errors
Logged in users are encountering frequent "file is infected with a virus" messages, causing them to panic and waste time running their files through file corruption detection software

henryzhang87 · 2021-09-08T17:31:13Z

Thank you Leah and Sean. I knew the changes that Beau recently made. I am not sure if it is related, and I am investigating. May I suggest testing on the staging server https://era-app-stg-1.library.ualberta.ca/ to see if the workflow is broken or not there? If it works on staging, then it increases the likelihood of a possible consequence of the recent changes. If it fails on staging, then we need to delve into the problem deeper.

…

On Wed, Sep 8, 2021 at 11:24 AM leahvanderjagt ***@***.***> wrote: Hi everyone @pgwillia <https://github.com/pgwillia> @henryzhang87 <https://github.com/henryzhang87> do you think Beau's changes noted in Slack could be behind these problems? Beau has been changing the underlying software managing some of our services. My understanding is that so far there hasn't been any impact to any of the services since he started last Wednesday. Today he has some servers supporting ERA and DMP on the list to migrate. What we know: - Users NOT logged in can access any collection and download any item - Logged in users, including but not limited to admin, are encountering an error "file is infected with a virus" and this is suspicious to us because a) I don't know if this is baked into the Jupiter code and b) we are having too many user reports of this problem given the current date which should reflect normal circumstances - Unfortunately, while this is affecting some collections/items, it is not affecting all, so it is item/collection intermittent (not time-intermittent) Consequence: - ERA Admin cannot deposit, edit or take any action on a number of collections or items - Logged in users are encountering many 404 errors - Logged in users are encountering frequent "file is infected with a virus" messages, causing them to panic and waste time running their files through file corruption detection software — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2504 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYUCLFKLNNFFVQCBZGF4OLUA6L6NANCNFSM5DVEV2IQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- Thanks Henry Zhang Senior Digital Preservation Specialist

leahvanderjagt · 2021-09-08T17:35:41Z

Hi @henryzhang87 and @pgwillia I just had a chat with Weiwei and she will be in touch with you right away about the changes Beau made.
Let me check what else is happening in my email to make sure i don't have to cancel a mtg to test, one sec pls

pgwillia · 2021-09-08T23:47:04Z

I think that the problem is that a former community/collection doesn't exist any longer and is throwing a ActiveRecord::RecordNotFound when we try to find it in our edit history

irb(main):002:0> item = Item.find('ed2e7a53-b522-47b8-a636-9ec757aa3352')
irb(main):003:0> item = item.decorate
irb(main):004:0> item.history
  PaperTrail::Version Load (2.3ms)  SELECT "versions".* FROM "versions" WHERE "versions"."item_id" = $1 AND "versions"."item_type" = $2 ORDER BY "versions"."created_at" ASC, "versions"."id" ASC  [["item_id", "ed2e7a53-b522-47b8-a636-9ec757aa3352"], ["item_type", "Item"]]
  Item Load (1.3ms)  SELECT "items".* FROM "items" WHERE "items"."id" = $1 LIMIT $2  [["id", "ed2e7a53-b522-47b8-a636-9ec757aa3352"], ["LIMIT", 1]]
  Community Load (1.2ms)  SELECT "communities".* FROM "communities" WHERE "communities"."id" = $1 LIMIT $2  [["id", "d1ddec9f-951a-48c2-8853-09bbc2d38422"], ["LIMIT", 1]]
  Collection Load (1.2ms)  SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2  [["id", "872324e8-7614-4a5b-b09f-a89d57c4ed33"], ["LIMIT", 1]]
Traceback (most recent call last):
       14: from (irb):4
       13: from app/decorators/item_decorator.rb:6:in `history'
       12: from app/decorators/item_decorator.rb:6:in `map'
       11: from app/decorators/item_decorator.rb:7:in `block in history'
       10: from app/decorators/item_decorator.rb:7:in `new'
        9: from app/decorators/humanized_change_set.rb:13:in `initialize'
        8: from app/decorators/humanized_change_set.rb:35:in `generate_html_diffs'
        7: from app/decorators/humanized_change_set.rb:35:in `each_key'
        6: from app/decorators/humanized_change_set.rb:40:in `block in generate_html_diffs'
        5: from app/decorators/humanized_change_set.rb:173:in `member_of_paths'
        4: from app/decorators/humanized_change_set.rb:173:in `map'
        3: from app/decorators/humanized_change_set.rb:176:in `block in member_of_paths'
        2: from app/decorators/humanized_change_set.rb:176:in `map'
        1: from app/decorators/humanized_change_set.rb:179:in `block (2 levels) in member_of_paths'
ActiveRecord::RecordNotFound (Couldn't find Collection with 'id'=872324e8-7614-4a5b-b09f-a89d57c4ed33)

jupiter/app/decorators/humanized_change_set.rb

Lines 172 to 185 in 389fcad

    
           def member_of_paths 
        
             humanized_member_of_paths = @changeset[:member_of_paths].map do |change_item| 
        
               next [] if change_item.nil? 
        
               change_item.map do |path| 
        
                 community_id, collection_id = path.split('/') 
        
                 community_title = Community.find(community_id).title 
        
                 collection_title = Collection.find(collection_id).title 
        
                 "#{community_title}/#{collection_title}" 
        
               end 
        
             end 
        
             humanized_member_of_paths.map { |change_item| change_item.join(', ') } 
        
           end

pgwillia · 2021-09-08T23:56:17Z

This has likely been a problem since August 20th when the items was last edited or when the "Successful Grants Library (SSHRC)" collection was deleted.

seanluyk · 2021-09-09T00:16:10Z

Very interesting, Tricia. Here is the process that was followed in case it helps: -new collections were created -items were assigned to the new collections -where possible the old collections were deleted (*problem objects in a few collections are preventing deletion) Sean Luyk

…

On Wed., Sep. 8, 2021, 17:56 Tricia Jenkins, ***@***.***> wrote: This has likely been an issue since August 20th. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2504 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADRQ3TRYBSZRIKOSBH6SMKDUA7Z2XANCNFSM5DVEV2IQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

pgwillia · 2021-09-09T16:47:05Z

This is an option for display

given the id, Papertrail (the gem we're using for the edit history) can help us bring things back to life, if we want more information

item= Item.find('d74e6ce9-0858-43be-84bd-7c5a28829d1a')
item.destroy
# Time passes....
item= Item.new(id: 'd74e6ce9-0858-43be-84bd-7c5a28829d1a')    # creating a new object with the same id, re-establishes the link
versions = item.versions    # versions ordered by versions.created_at, ascending
item= versions.last.reify  # the item as it was before destruction
item.save                   # the item lives!

Maybe we give admin users the option to restore from this point by clicking an icon
(https://fontawesome.com/v6.0/icons?d=gallery&p=1&q=undo&s=solid)

seanluyk · 2021-09-09T22:24:36Z

@pgwillia I'm in support of the approach you propose to displaying deleted collections in the admin view. I'm interested in learning more about how restoring collections might work, and what the impacts would be too (perhaps best discussed in our meeting)

pgwillia · 2021-09-15T21:07:06Z

@seanluyk how does this look?

seanluyk · 2021-09-15T21:20:43Z

@pgwillia this looks great and makes lots of sense. Thanks!

This was referenced Sep 15, 2021

Edit History of Deleted Collection #2517

Merged

Review Changes with Stakeholders before 2.1 deploy #2524

Closed

pgwillia closed this as completed Nov 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

404 errors - Successful Grants collection #2504

404 errors - Successful Grants collection #2504

seanluyk commented Sep 8, 2021

seanluyk commented Sep 8, 2021

pgwillia commented Sep 8, 2021

pgwillia commented Sep 8, 2021

seanluyk commented Sep 8, 2021

pgwillia commented Sep 8, 2021 •

edited

Loading

seanluyk commented Sep 8, 2021

seanluyk commented Sep 8, 2021 •

edited

Loading

seanluyk commented Sep 8, 2021

henryzhang87 commented Sep 8, 2021

leahvanderjagt commented Sep 8, 2021

henryzhang87 commented Sep 8, 2021 via email

leahvanderjagt commented Sep 8, 2021

pgwillia commented Sep 8, 2021 •

edited

Loading

pgwillia commented Sep 8, 2021 •

edited

Loading

seanluyk commented Sep 9, 2021 via email

pgwillia commented Sep 9, 2021

seanluyk commented Sep 9, 2021

pgwillia commented Sep 15, 2021 •

edited

Loading

seanluyk commented Sep 15, 2021

404 errors - Successful Grants collection #2504

404 errors - Successful Grants collection #2504

Comments

seanluyk commented Sep 8, 2021

seanluyk commented Sep 8, 2021

pgwillia commented Sep 8, 2021

pgwillia commented Sep 8, 2021

seanluyk commented Sep 8, 2021

pgwillia commented Sep 8, 2021 • edited Loading

seanluyk commented Sep 8, 2021

seanluyk commented Sep 8, 2021 • edited Loading

seanluyk commented Sep 8, 2021

henryzhang87 commented Sep 8, 2021

leahvanderjagt commented Sep 8, 2021

henryzhang87 commented Sep 8, 2021 via email

leahvanderjagt commented Sep 8, 2021

pgwillia commented Sep 8, 2021 • edited Loading

pgwillia commented Sep 8, 2021 • edited Loading

seanluyk commented Sep 9, 2021 via email

pgwillia commented Sep 9, 2021

seanluyk commented Sep 9, 2021

pgwillia commented Sep 15, 2021 • edited Loading

seanluyk commented Sep 15, 2021

pgwillia commented Sep 8, 2021 •

edited

Loading

seanluyk commented Sep 8, 2021 •

edited

Loading

pgwillia commented Sep 8, 2021 •

edited

Loading

pgwillia commented Sep 8, 2021 •

edited

Loading

pgwillia commented Sep 15, 2021 •

edited

Loading