Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Long Strings processing performance degradation in CH 18.12+ #3499

Closed
den-crane opened this issue Oct 30, 2018 · 3 comments
Closed

Long Strings processing performance degradation in CH 18.12+ #3499

den-crane opened this issue Oct 30, 2018 · 3 comments

Comments

@den-crane
Copy link
Contributor

den-crane commented Oct 30, 2018

Connected to ClickHouse server version 18.14.10 revision 54409.

"Macros" CREATE_INSERT:
create table xxxx (row String) engine=Memory;
insert into xxxx(row) select arrayStringConcat( arrayMap(x -> 'x'||toString(x)||':"'||toString(9999999)||'"',range(3000)), ',' )||',"test":"111",' from numbers(100000);

  1. set max_block_size=65536;

  2. CREATE_INSERT

  3. select 1 from xxxx where row like '%test:0%'
    0 rows in set. Elapsed: 4.383 sec
    Processed 100.00 thousand rows, 4.69 GB (22.81 thousand rows/s., 1.07 GB/s.)

  4. set max_block_size=5000;
    select 1 from xxxx where row like '%test:0%'
    0 rows in set. Elapsed: 4.546 sec

  5. drop table xxxx;

  6. exit / enter CH client
    set max_block_size=5000;
    CREATE_INSERT;

  7. select 1 from xxxx where row like '%test:0%'
    0 rows in set. Elapsed: 0.963 sec.

  8. set max_block_size=65536;
    select 1 from xxxx where row like '%test:0%';
    0 rows in set. Elapsed: 1.061 sec.

@den-crane
Copy link
Contributor Author

den-crane commented Oct 30, 2018

The same server, downgrade to ClickHouse version 18.10.3.
select 1 from xxxx where row like '%test:0%'
0 rows in set. Elapsed: 0.686 sec. Processed 100.00 thousand rows, 4.69 GB (145.68 thousand rows/s., 6.83 GB/s.)

@KochetovNicolai is it somehow connected to LowCardinality ?

@den-crane
Copy link
Contributor Author

fixed in 18.14.13 ?

Connected to ClickHouse server version 18.14.13 revision 54409.

SELECT 1 FROM xxxx WHERE row LIKE '%test:0%'

0 rows in set. Elapsed: 0.697 sec. Processed 100.00 thousand rows, 4.69 GB (143.42 thousand rows/s., 6.73 GB/s.)

@alexey-milovidov
Copy link
Member

alexey-milovidov commented Nov 8, 2018

Yes. #3530

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants