The "Snapshot Predecode Duration" metric doesn't take syncSchema time in to account

[CalvinNeo]

Bug Report

1. Minimal reproduce step (Required)

We have a scenario that involves heavy syncSchema(up to 7min) during PrehandleSnapshot.

However, there is no easy way for us to achieve this solution from metrics. The "Snapshot Predecode Duration" does not include syncSchema time(such as AtomicGetStorageSchema).

2. What did you expect to see? (Required)

There is a easy way to find out a heavy syncSchema or PrehandleSnapshot job, through metrics.

3. What did you see instead (Required)

There is no easy way.

4. What is your TiFlash version? (Required)

nightly

[JaySon-Huang]

What make syncSchema take 7 min?

[CalvinNeo]

I deleted 3000 tables

[JaySon-Huang]

3000.0/(7 * 60) = 7.14. So it means dropping one table cost about 7 seconds, still too slow.
Are you dropping 3000 empty tables? However, dropping one table even containing a large amount of data takes 7 seconds still too slow. Do we have logs or can we reproduce this scenario to explore why?

[CalvinNeo]

"Drop 3000 tables cost 7min" is a rough description, I have a doc about this.