Efficiently Loading Relationship Data from a Python Pandas DataFrame with Missing Nodes

[guidataiku]

Hello community!

I'm trying to load relationship data from a Python DataFrame, but some rows reference nodes that may not exist. I've found two workarounds so far:

• Processing the DataFrame to remove rows pointing to non-existent nodes (which can be computationally expensive).
• Converting the DataFrame to a CSV and using the IGNORE_ERRORS flag (introduced in version 0.7.0).

I'm wondering if there's a more efficient approach that avoids writing to a file and offers better performance.

Any suggestions or best practices would be greatly appreciated!

Thank you!

[ray6080]

Hi @guidataiku , I believe the best way is to COPY data frame directly with IGNORE_ERRORS. We have an issue on supporting this #4534. Will try to prioritize this.