feat(website): support excel metadata files in data upload form

[fhennig]

resolves #3334

preview URL: https://excel-upload.loculus.org

Summary

• throws an error if the sheet is empty
• shows a warning if there are multiple sheets in the excel file
• the cell type is encoded in the sheet, so all dates are passed correctly regardless of format.
• the original file handle is kept around and if the original file changes, the file is unselected.

Screenshot

PR Checklist

• All necessary documentation has been adapted.
• The implemented feature is covered by an appropriate test.
  • testfiles are added and parsed to check if dates are correct etc.

[fhennig]

TODO: don't apply the processing for every UploadComponent (currently it also does it for the fasta upload)

[fhennig]

TODO: Testing

• Test multiple excel versions
• Test number formats (localized?)
• test date formats

[chaoran-chen]

Thanks Felix, this looks great to me! I tested it with for WNV using data that I just uploaded to #3469. Below are a few minor comments.

[theosanderson]

The preview seems messed up at the moment - I can't log in (I'm sure that's not to do with the changes you've made just some weird state thing, or pre-existing bug)

[fhennig]

I made an empty commit to try and trigger a redeployment of the preview, I also see the error you're seeing. I can' imagine that the changes I have made are somehow responsible.

[theosanderson]

When you rebase it should be fixed :)

[fhennig]

I rebased, it works!

I'm getting started on the compression handling, I found a lib, looks like it should be easy to implement

[fhennig]

handling compression is in fact not easy, it was a pain. but I think it should work now, the tests passed locally.

[fhennig]

the lzma lib needs to be swapped out as well.

[theosanderson]

Sorry for the pain - yes in fairness I'd forgotten how many different methods we support :(

[fhennig]

no worries 😄

I thought "surely there's a a just unpack it lib" - I have used commandline tools that do that before. But alas, I couldn't find one.

And then some of the libs are quite old, I had to try out a few different ones before finding ones that work well and have type support etc. But now it looks like it's working! (EDIT: Shit, test failed again)

[fhennig]

Ok, so this library that I used for xz (xz-decompress) also has some issues, because vite thinks it's commonJS and it's wonky to import.

Now I also found this: https://www.npmjs.com/package/libarchive.js but it requires a custom init call and putting a wasm module somewhere.

I have to continue this tomorrow. Why are all the xz/lzma libs so weird to use? 😢

---

Alternatively, we just defer this (who would upload xz compressed excel files?!) and just add a popup that that file combination doesn't work yet and that's that. I don't feel like it's really worth spending a lot of time on this.

[theosanderson]

Alternatively, we just defer this (who would upload xz compressed excel files?!) and just add a popup that that file combination doesn't work yet and that's that. I don't feel like it's really worth spending a lot of time on this.

👍

[chaoran-chen]

Alternatively, we just defer this (who would upload xz compressed excel files?!) and just add a popup that that file combination doesn't work yet and that's that. I don't feel like it's really worth spending a lot of time on this.

Agree, this sounds good to me!

[chaoran-chen]

Nice, works for me!