unexpected TableVersionMismatched error in the execution of copy into statement

[dantengsky]

    @dantengsky  I often see  errors like this when I  test `copy into ontime200 .. ` manually , even on a new table  just created (with table name never used before)

 2022-09-26T12:53:10.803063Z ERROR common_meta_api::schema_api_impl: error=TableVersionMismatched: 160 expect `== 163` but `168`  while `update_table_meta

I deleted local ./databend and testbucket/admin/1 on minio

Originally posted by @youngsofun in #7903 (comment)

[dantengsky]

For non-concurrent insertion of a table like the above scenario, no TableVersionMismatched is expected.

[Xuanwo]

FuseTable::commit_to_meta_server(
    ctx.as_ref(),
    &self.table_info,
    &self.meta_location_generator,
    new_snapshot,
)
.await

In try_commit, we will use fuse table's table_info to build UpdateTableMetaReq:

let req = UpdateTableMetaReq {
    table_id,
    seq: MatchSeq::Exact(table_version),
    new_table_meta,
};

So, if we commit twice, will we meet TableVersionMismatched, get the latest table meta seq and try to commit again?

[dantengsky]

turns out, before table commit, CopyInterpreterV2::upsert_copied_files_info will be called

pls note the self.copy_files_to_table just build the pipeline, not executed it yet, but the table IS resolved.

https://github.com/datafuselabs/databend/blob/4ea72440ad7018a76d97b2e2e27b179e8aa953bd/src/query/service/src/interpreters/interpreter_copy_v2.rs#L397-L408

and the execution of upsert_copied_files_info will call SchemaApi::upsert_table_copied_file_info ultimately, which will increase the version of table (meta).

@lichuang please correct me I'm mistaken here

https://github.com/datafuselabs/databend/blob/ae60a183b326ed43403e429a9048cb69a84c3d8d/src/meta/api/src/schema_api_impl.rs#L1829-L1831

thus, later when the pipeline is executed, the version of the table that commits working on, is staled.

Not sure how to fix it yet, @youngsofun @lichuang any suggestions?

[BohuTANG]

Any progress/suggestions? cc @lichuang

[Xuanwo]

reopen this issue for #8307 has been reverted.