Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Very slow container data loading #131877

Closed
safaci2000 opened this issue Oct 4, 2024 · 3 comments
Closed

Very slow container data loading #131877

safaci2000 opened this issue Oct 4, 2024 · 3 comments
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-community Originated from the community X-blathers-untriaged blathers was unable to find an owner

Comments

@safaci2000
Copy link

safaci2000 commented Oct 4, 2024

Describe the problem

I'm curious to know why the data set I'm loading is so much slower. I'm trying to initialize cockroachdb using very similar dataset and letting the docker container load the .SQLs.

I'm not sure if this is cockroachdb issues or some bad choices made in how the docker setup works.

The load time for postgres is: around 12 seconds. The load time for cockroach DB is around 1 min and 20 seconds. Give or take.

I'm trying to understand if this is expected, or I'm doing something silly.

To Reproduce

Initialize a docker-compose or docker stack as you like with the data provided here: https://github.com/go-jet/jet-test-data/tree/master/init/cockroach

container parameters: "start-single-node", "--accept-sql-without-tls"
tested against the latest release: cockroachdb/cockroach:v23.1.27

Expected behavior

Load time should be relatively comparable to what postgres or mysql is for a similar data set.

load time for Mysql/Maria is around 7-8 seconds, Postgres 12-13 seconds, Cockroachdb DB is well at the minute mark. Sqlite is well, local disk IO, so ignoring that.

Excluding all of my own code, and only looking at the docker container data loading, it's still at the minute mark just to load data.

Environment:

  • CockroachDB version used: v23.1.27
  • Browser name and version: NA
  • Operating system and version: Mac OS X M2 chipset

Jira issue: CRDB-42732

@safaci2000 safaci2000 added the C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. label Oct 4, 2024
Copy link

blathers-crl bot commented Oct 4, 2024

Hi @safaci2000, please add branch-* labels to identify which branch(es) this C-bug affects.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

Copy link

blathers-crl bot commented Oct 4, 2024

Hello, I am Blathers. I am here to help you get the issue triaged.

Hoot - a bug! Though bugs are the bane of my existence, rest assured the wretched thing will get the best of care here.

I was unable to automatically find someone to ping.

If we have not gotten back to your issue within a few business days, you can try the following:

  • Join our community slack channel and ask on #cockroachdb.
  • Try find someone from here if you know they worked closely on the area and CC them.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@blathers-crl blathers-crl bot added O-community Originated from the community X-blathers-untriaged blathers was unable to find an owner labels Oct 4, 2024
@alyshanjahani-crl
Copy link
Collaborator

This issue was discussed in community slack: https://cockroachdb.slack.com/archives/CP4D9LD5F/p1728053066398149

TLDR: Use of individual insert into statements on a single node can lead to poor performance, prefer using IMPORT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-community Originated from the community X-blathers-untriaged blathers was unable to find an owner
Projects
None yet
Development

No branches or pull requests

2 participants