ansible: add & update metric panels for syncer and misc

[lance6716]

What problem does this PR solve?

add grafana panels for #772

What is changed and how it works?

modify grafana json.

1. multi-selection on source and instance
   

but seems histogram_quantile, rate can't work on multi-source

2. add panels for worker opErrCounter
   

3. change some metrics for worker

• replace filter $instance to $source_id:
  dm_worker_task_state

4. metrics for syncer

• replace filter $instance to $source_id:
  dm_syncer_remaining_time, dm_syncer_binlog_file, dm_syncer_binlog_transform_cost_count, dm_syncer_skip_binlog_duration_count, dm_syncer_read_binlog_duration_bucket, dm_syncer_binlog_transform_cost_bucket, dm_syncer_dispatch_binlog_duration_bucket, dm_syncer_binlog_event_size_bucket, dm_syncer_queue_size, dm_syncer_added_jobs_total, dm_syncer_finished_jobs_total, dm_syncer_add_job_duration_bucket, dm_syncer_conflict_detect_duration_bucket, dm_syncer_skip_binlog_duration_bucket, dm_syncer_unsynced_table_number, dm_syncer_shard_lock_resolving
• keep $instance:
  • dm_syncer_replication_lag: report by heartbeat, no source_id currently
  • dm_relay_binlog_file{instance="$instance", node="relay"} - ON(instance, job) dm_syncer_binlog_file{instance="$instance", task="$task", node="syncer"}: relay doesn't have source_id currently
  • dm_syncer_txn_duration_time_bucket, dm_syncer_stmt_duration_time_bucket: used by DBConn in many place (checkpoint, onlineddl, ...), seems it's more appropriate to represent a instance's state

Check List

Tests

• Manual test (see panels in broswer)

Code changes

Side effects

Related changes

[codecov]

Codecov Report

Merging #803 into master will increase coverage by 0.3925%.
The diff coverage is 39.3162%.

@@               Coverage Diff                @@
##             master       #803        +/-   ##
================================================
+ Coverage   57.0981%   57.4907%   +0.3925%     
================================================
  Files           205        211         +6     
  Lines         21104      23489      +2385     
================================================
+ Hits          12050      13504      +1454     
- Misses         7890       8731       +841     
- Partials       1164       1254        +90     

[GMHDBJD]

LGTM

[csuzhangxc]

add panels for worker opErrCounter

How about re-order panels for opErrCounter? like:

1. before any operate error
2. source bound error
3. start error
4. pause error
5. resume error
6. auto resume error
7. update error
8. stop error

[lance6716]

add panels for worker opErrCounter

How about re-order panels for opErrCounter? like:

1. before any operate error
2. source bound error
3. start error
4. pause error
5. resume error
6. auto resume error
7. update error
8. stop error

visual result updated in description of this PR #803 (comment) , some error showed in panel are generated by repeatly call resume-task and pause-task

[csuzhangxc]

LGTM

[lance6716]

/run-all-tests

[lance6716]

/run-all-tests