unstructured: fix metadata order mixed up

[lambda-science]

Fix #331

Because unstructured integration code use python sets, file orders are not preserved leading to metadata attribution to wrong filepaths. Using lists fix this as list are ordered.

Also C/C from the issue, I added:

And actually I'm not sure why we need a set logic here to make filepath unique. I feel like it's up to the user to provide unique paths ? For example what happens if a user provide 10 path and 10 metadata but then some filepath are duplicated so then we have 8 filepath and 10 metadata ? It will raise error from normalize_metadata I guess

What I think is that:

We can support directories as path BUT then metadata should max be of a length of 1 (same metadata for all files in directory). Because I'm not sure it's clear how path.glob() orders files (leading to metadata attribution confusion)
If direct paths to files are provided: don't make them unique with sets.

[lambda-science]

I'm sorry for the long commit list it's weird. It took all my previous history also from #242
Maybe I should have created a new branch :/

Also: tests are failing because no hard drive space :p

[anakin87]

Hey... Thanks for the fix!

I generally agree with your approach.

Let's also update the docstrings about meta to reflect the changes...

[lambda-science]

Sounds good to me ! :)

[anakin87]

🚀