Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Provide better observability for data ingestion scenarios #94194

Open
kevin-v-ngo opened this issue Dec 22, 2022 · 0 comments
Open

Provide better observability for data ingestion scenarios #94194

kevin-v-ngo opened this issue Dec 22, 2022 · 0 comments
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-observability

Comments

@kevin-v-ngo
Copy link

kevin-v-ngo commented Dec 22, 2022 •
edited by cockroach-jira-scripts
Loading

This issue tracks provide better observability into data ingestion for workloads onboarding and using the COPY and IMPORT INTO commands.

Jira issue: CRDB-22732
@kevin-v-ngo kevin-v-ngo added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Dec 22, 2022
@rafiss rafiss mentioned this issue Feb 28, 2023
Merged
craig bot pushed a commit that referenced this issue Mar 1, 2023
@rafiss
Merge #97808
a683fc7 
97808: sql: make COPY TO/FROM respect timeouts and cancellation, and appear in SHOW QUERIES r=otan a=rafiss

informs #94194

### sql: use conn executor state machine for COPY FROM 

Using the conn executor state machine allows a lot of bespoke logic to
be removed. (If we were to remove the copy_from_atomic_enabled setting
and make it always true, we could remove even more custom logic.)

This then makes it easy to hook up to cancellation and introspection.

Release note (bug fix): The COPY FROM command now respects the
statement_timeout and transaction_timeout settings.

Release note (bug fix): The COPY FROM command now appears in the results
of SHOW QUERIES.

---

### sql: make stmt/txn timeout apply to COPY TO

This commit simplifies the conn executor state transitions a bit, and
adds in timers for stmt/txn timeouts analogously to how they were for
COPY FROM.

No release note since COPY TO is new.

Co-authored-by: Rafi Shamim <[email protected]>
@rafiss rafiss removed their assignment Jul 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-observability
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rafiss @kevin-v-ngo