Skip to content
This repository has been archived by the owner on Oct 17, 2024. It is now read-only.

column names and table names create with underscores..? #4

Closed
mkoenings opened this issue Nov 30, 2022 · 3 comments
Closed

column names and table names create with underscores..? #4

mkoenings opened this issue Nov 30, 2022 · 3 comments

Comments

@mkoenings
Copy link

Hello,
thanks for this great plugin , we are definitely planning to use this in a project.
I wanted to try the functionality in an easy example: read from CSV ( with header, comma sep., "" ) , write into a SQL server table.
The data is kind of garbled, it creates columns with underscores like c_c_u_s_t_k_e_y instead of c_custkey, same for schema and table name. Is there an easy fix ?
Thanks!

2022-11-30T17:09:59.245174Z [info ] 2022-11-30 18:09:59,244 Inserting with SQL: INSERT INTO #customer cmd_type=elb consumer=True name=target-mssql producer=False stdio=stderr string_id=target-mssql
2022-11-30T17:09:59.247173Z [info ] (c_c_u_s_t_k_e_y, c_n_a_m_e, c_a_d_d_r_e_s_s, c_n_a_t_i_o_n_k_e_y, c_p_h_o_n_e, c_a_c_c_t_b_a_l, c_m_k_t_s_e_g_m_e_n_t, c_c_o_m_m_e_n_t) cmd_type=elb consumer=True name=target-mssql producer=False stdio=stderr string_id=target-mssql
2022-11-30T17:09:59.250183Z [info ] VALUES (:c_c_u_s_t_k_e_y, :c_n_a_m_e, :c_a_d_d_r_e_s_s, :c_n_a_t_i_o_n_k_e_y, :c_p_h_o_n_e, :c_a_c_c_t_b_a_l, :c_m_k_t_s_e_g_m_e_n_t, :c_c_o_m_m_e_n_t) cmd_type=elb consumer=True name=target-mssql producer=False stdio=stderr string_id=target-mssql

extractors:
`- name: tap-csv
variant: meltanolabs
pip_url: git+https://github.com/MeltanoLabs/tap-csv.git
settings:
- name: files
kind: array
value:
- entity: customer
path: customer.csv
keys:
- C_CUSTKEY
loaders:

@mkoenings
Copy link
Author

I found out that the function conform_name in singer_sdk/sinks/sql.py is the culprit.
It tries to snake_case everything, which leads to funny results, when the names are all upper case...
I read data from Snowflake mainly, where everything is uppercase. I can't believe that this is so unusual that the singer sdk cant handle it ?

@visch
Copy link

visch commented Dec 5, 2022

This is a bug in the Singer SDK. See meltano/sdk#1205 I bypassed the conforming stuff for now as it has some bugs in it. I personally don't think we should tweak the names unless the target requires us to.

@hholgersen
Copy link
Contributor

@mkoenings I have created a fix for the column casing, but before I merge I created a v0 release because this is a breaking change.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants