Null value in column "expanded_url" of relation "tweet_urls_1135540987445600258" violates not-null constraint

[0xbrayo]

Fails with this error:
Error processing Twitter archive: Error inserting tweet_urls_1135540987445600258: null value in column "expanded_url" of relation "tweet_urls_1135540987445600258" violates not-null constraint

[DefenderOfBasic]

thank you for the report @0xbrayo ! are you able to share your archive with me so we can try and reproduce this? Maybe upload it to drive & share a link here or email [email protected] ?

[0xbrayo]

Uploaded to google drive. Emailed you the link.

[0xbrayo]

@DefenderOfBasic Any updates?

[DefenderOfBasic]

@ri72miieop is looking into it now.

[ri72miieop]

hi @0xbrayo there was an issue with the tweet 1155140493451440129 where it didn't have a field that is required from our side. I think this was some problem with the Twitter export. I removed that tweet and uploaded your archive: https://www.community-archive.org/user/1135540987445600258. Welcome. :)

[0xbrayo]

Thanks😊

[DefenderOfBasic]

Thanks @ri72miieop !!

[DefenderOfBasic]

And thank you @0xbrayo for reporting this!!

[0xbrayo]

@ri72miieop @DefenderOfBasic can a fix be pushed to the code, subsequent uploads from me or other people might cause the same error. Warn users that some tweets have been dropped due to malformed schema etc. Also I'm assuming we can do updates by requesting archives in the future and uploading those.

[ri72miieop]

yeah, you're right. I think this is pretty rare and I'd like to understand when it happens better, but I could also check later the db to see which tweet_urls are set to empty and this way people no longer get errors moving forward so it's a win-win.

yeah you can update it by re-uploading a more recent archive