Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix DateTime vs DateTime('UTC') inconsistency on data read. #2400

Merged
merged 1 commit into from
May 25, 2018

Conversation

bocharov
Copy link
Contributor

@bocharov bocharov commented May 22, 2018

Original problem was described in PR #1650 and partially fixed by
b5af4c9, however one place was forgotten and it was causing query
exceptions when trying to read DateTime column across parts with
different DateTime and DateTime('UTC') data type.

DB::Exception: Type mismatch for column timeslot. Column has type DateTime, got type DateTime('UTC')

cc @alexey-milovidov @bobrik @vavrusa @dqminh

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Original problem was described in PR ClickHouse#1650 and partially fixed by
b5af4c9, however one place was forgotten and it was causing query
exceptions when trying to read DateTime column across parts with
different DateTime and DateTime('UTC') data type.
@dqminh
Copy link
Contributor

dqminh commented May 23, 2018

The bug seems to only trigger for table created implicitly by materialized view which didnt specify the column type. For example we see the bug happen when querying timeslot field.

CREATE MATERIALIZED VIEW IF NOT EXISTS r0.affected_mv
ENGINE = ReplicatedAggregatingMergeTree('zookeeper', '{replica}', date, (date, id, timeslot), 1024) AS
SELECT
    date,
    id,
    toStartOfHour(r.timeslot) AS timeslot
FROM r0.another_table AS r;

So what we think happened that trigger this bug is:

  1. We have an existing set of shards that replicate from each others A, B, C i.e ( inserts to A goes to B and C, inserts to B go to A and C, etc. )
  2. The servers were upgraded from old version to 1.1.54342, existing timeslot fields are in DateTime type.
  3. We need to replace A with another server, calling it A1.
  4. A1's tables were bootstraped from scratch with current table definition, at which point its accepting replication traffic from B and C, as well as replicating its inserts to B and C
  5. Table created implicitly by materialized view in A1 having its timeslot type defined as DateTime('UTC') instead of DateTime
  6. Replicated data from A1 landed in B and C, so when we query those data in B and C, we started seeing this bug because B and C has mixed DateTime and DateTime('UTC') data
  7. Shutdown A1, detach and reattach all affected partitions in B and C reset the data type for timeslot in affected parts to DateTime, mitigating this bug temporarily.

@alexey-milovidov alexey-milovidov merged commit dcebf08 into ClickHouse:master May 25, 2018
@alexey-milovidov
Copy link
Member

Thank you!
The fix is totally correct.

@HunterSherms
Copy link

I'm still seeing this error on 1.1.54390 when writing to a table with a materialized view that aggregates the max date into another table.

CREATE MATERIALIZED VIEW IF NOT EXISTS
${database}._${view} TO ${database}._${max_table}
    SELECT
        group_id,
        date,
        maxState(id) AS max_id,
        maxState(inserted_at) AS max_inserted_at
    FROM ${database}._${table}
    GROUP BY (group_id, date)

Where inserted_at is a DateTime in the source and target tables.
On attempting an insert into the source table:

Column has type AggregateFunction(max, DateTime), got type AggregateFunction(max, DateTime('UTC')): while pushing to view default._view_name, e.what() = DB::Exception

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants