Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Number of characters is different between the Copy action CSV and dataframe #121

Open
TimvdHeijden opened this issue Dec 8, 2022 · 0 comments

Comments

@TimvdHeijden
Copy link
Contributor

TimvdHeijden commented Dec 8, 2022

When writing a dataframe to a SQL database it gives an error in ADF because of max number of characters exceeded (Operation on target Copy all_boards to SQL failed: Failure happened on 'Sink' side. ErrorCode=SqlBulkCopyInvalidColumnLength,.... This was the case with a column with a string version of a list of dicts.
The max characters in Python were 1870, so the schema definition in the database became varchar(1870), in the CSV the max length of values was 1870 as wel after checking as a txt file and after loading it again in Python.

However in ADF it is causing the error during the copy action. It does not give the error if we manually enter the text_length parameter in df_to_azure() as 1873

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant