Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MongoDB: Error while using translate ValueError: max() iterable argument is empty #134

Open
amotl opened this issue Apr 8, 2024 · 5 comments
Assignees

Comments

@amotl
Copy link
Member

amotl commented Apr 8, 2024

@delusion8399 added a report at crate/mongodb-cratedb-migration-tool#25. Because we are archiving that repository, we are adding the report here instead.

Hi, getting this error while using translate

    type = max(types, key=lambda item: types[item]["count"])
ValueError: max() arg is an empty sequence

image

@amotl
Copy link
Member Author

amotl commented Apr 16, 2024

Dear @delusion8399,

in order to approach that problem, maybe you can share your collection schema with us, and a few samples worth of data? We would love to add corresponding improvements to make the program more robust.

With kind regards,
Andreas.

@amotl
Copy link
Member Author

amotl commented Apr 16, 2024

Silly Reproduction

I have been able to reproduce the problem on a narrow scope:

$ python3.10
>>> types = {}
>>> max(types, key=lambda item: types[item]["count"])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: max() arg is an empty sequence

Thoughts

I guess your collection may use a MongoDB data type the program does not know about, trips on it, and fails to convey that flaw in a good way. If you could tell us about all data types used within the collection at hand, it could already give us more hints to investigate more closely. Thanks!

@amotl amotl self-assigned this Apr 20, 2024
@amotl amotl changed the title MongoDB I/O adapter: Error while using translate [I/O] MongoDB adapter: Error while using translate May 28, 2024
@amotl
Copy link
Member Author

amotl commented Sep 2, 2024

Dear @delusion8399,

that patch improved quite a few edge case situations in the MongoDB table loader.

Maybe you would like to give it another try after upgrading to the most recent version of cratedb-toolkit?

With kind regards,
Andreas.

@amotl
Copy link
Member Author

amotl commented Sep 2, 2024

I've just been able to spot this shape of schema information on real-world data, which seems to trip that very error. So, I may assume a fix is neigh.

"items": {
    "count": 1,
    "types": {
        "ARRAY": {
            "count": 1,
            "types": {}
        }
    }
}

Most probably, it is about empty arrays/lists, where naturally an inner type can't be derived from.

@amotl amotl changed the title [I/O] MongoDB adapter: Error while using translate MongoDB: Error while using translate ValueError: max() iterable argument is empty Sep 2, 2024
@delusion8399
Copy link

Thank you so much for taking this so seriously.
I was able to do that manually using CSV dump.

Anyways If In future I got such case I would definitely use and contribute to this tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants