Vacuum isuses for ADLS

[Blajda]

Environment

Delta-rs version: git 649fdce

---

Bug

What happened:
Calling table.vacuum with a ADLS backed table results in a panic. Working around the panic results in no files being picked up.

thread 'vacuum::test_adls_vacuum' panicked at 'byte index 80 is out of bounds of adls2://[email protected]/mystorage/mytable', library/core/src/str/mod.rs:107:9

(Ignore the Index number. Modified the paths of my accounts)

Adding a / to the end of the table URI does not work since some normalization is performed during table load but would not matter due to the below issues.

More details:

I removed the + 1 on this line to check if it was the main issue but encounter other issues.

list_obj for ADLS may have two issues:

1. When ObjectMeta is created path from the callee of list_obj is used and not the actual paths from the API.
2. Currently recursive is set to false. Vacuum only calls list_obj for the table's URI hence only _delta_log and partition directories are being returned. This might be an issue with how vacuum is implemented.

See: https://github.com/delta-io/delta-rs/blob/main/rust/src/storage/azure/mod.rs#L369-L375

[Blajda]

Hi @roeap, looking for you analysis on this. If (2.) should be handled from the vacuum side I can take that work on.

[roeap]

@Blajda - thanks for the report! will take a look and see what needs to be done ...

[Blajda]

Closing since #683 is merged and the vacuum tests passed.