Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[#22843] YSQL: Add YbNumCatalogCacheIdMisses to display PG catalog ca…
…che misses Summary: YSQL provides a gflag `--ysql_catalog_preload_additional_table_list` that customers can use to preload a list of catalog tables to build a set of catalog caches. This can help to speed up the first query as it will see fewer cache misses. However it requires some manual work in order to figure out what to put in the list of catalog tables to preload. One way is to make a connection and execute `SET yb_debug_log_docdb_requests = true` then parse the postgres logs to see what catalog tables are being read in the read requests. We already have a counter `NumCatalogCacheMisses` that is exported to the PG webserver 13000 port. It is the number of accumulated cache misses that is recorded in the PG shared memory. Each time `SearchCatCacheMiss` is invoked, `NumCatalogCacheMisses` is incremented. At the end of the top-most query boundary, we compute the delta since last time and add it to the counter stored in shared memory. This counter can be accessed via the 13000 port. This diff expands the counter `NumCatalogCacheMisses` with `YbNumCatalogCacheIdMisses` which is an array of counters, one for each catalog cache, indexed by its id. In this way from the 13000 port we can see how many cache misses have happened for each catalog cache. If a catalog cache shows a large number of cache misses, it is an indication that its corresponding catalog table should be added to the list of catalog tables to preload. I also did some variable/function renames according to YB naming convention. Jira: DB-11741 Test Plan: (1) ./yb_build.sh --cxx-test pgwrapper_pg_libpq-test --gtest_filter PgLibPqTest.CatalogCacheIdMissMetricsTest (2) manual test ``` ./bin/yb-ctl create --rf 1 ./bin/ysqlsh ysqlsh (11.2-YB-2.23.0.0-b0) Type "help" for help. yugabyte=# create table foo(id int); CREATE TABLE yugabyte=# insert into foo values (10); INSERT 0 1 yugabyte=# \q wget 'http://127.0.0.1:13000/prometheus-metrics?reset_histograms=false&show_help=false' -O - 2>/dev/null | grep CatalogCacheMisses | grep count handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 12 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_0_pg_aggregate_fnoid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_1_pg_am_name_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_2_pg_am_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_3_pg_amop_opr_fam_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_4_pg_amop_fam_strat_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_5_pg_amproc_fam_proc_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_6_pg_attribute_relid_attnam_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_7_pg_attribute_relid_attnum_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_8_pg_auth_members_member_role_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_9_pg_auth_members_role_member_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_10_pg_authid_rolname_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 1 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_11_pg_authid_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 1 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_12_pg_cast_source_target_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_13_pg_opclass_am_name_nsp_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_14_pg_opclass_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_15_pg_collation_name_enc_nsp_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_16_pg_collation_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_17_pg_conversion_default_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_18_pg_conversion_name_nsp_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_19_pg_constraint_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_20_pg_conversion_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_21_pg_database_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_22_pg_default_acl_role_nsp_obj_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 2 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_23_pg_enum_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_24_pg_enum_typid_label_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_25_pg_event_trigger_evtname_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_26_pg_event_trigger_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_27_pg_foreign_data_wrapper_name_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_28_pg_foreign_data_wrapper_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_29_pg_foreign_server_name_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_30_pg_foreign_server_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_31_pg_foreign_table_relid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_32_pg_index_indexrelid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_33_pg_language_name_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_34_pg_language_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_35_pg_namespace_nspname_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 1 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_36_pg_namespace_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_37_pg_operator_oprname_l_r_n_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_38_pg_operator_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_39_pg_opfamily_am_name_nsp_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_40_pg_opfamily_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_41_pg_partitioned_table_partrelid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_42_pg_proc_proname_args_nsp_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_43_pg_proc_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_44_pg_publication_pubname_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_45_pg_publication_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_46_pg_publication_rel_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_47_pg_publication_rel_prrelid_prpubid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_48_pg_range_rngtypid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_49_pg_class_relname_nsp_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 4 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_50_pg_class_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 1 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_51_pg_replication_origin_roiident_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_52_pg_replication_origin_roname_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_53_pg_rewrite_rel_rulename_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_54_pg_sequence_seqrelid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_55_pg_statistic_ext_name_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_56_pg_statistic_ext_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_57_pg_statistic_relid_att_inh_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_58_pg_subscription_subname_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_59_pg_subscription_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_60_pg_subscription_rel_srrelid_srsubid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_61_pg_tablespace_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_62_pg_transform_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_63_pg_transform_type_lang_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_64_pg_ts_config_map_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_65_pg_ts_config_cfgname_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_66_pg_ts_config_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_67_pg_ts_dict_dictname_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_68_pg_ts_dict_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_69_pg_ts_parser_prsname_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_70_pg_ts_parser_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_71_pg_ts_template_tmplname_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_72_pg_ts_template_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_73_pg_type_typname_nsp_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 2 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_74_pg_type_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_75_pg_user_mapping_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_76_pg_user_mapping_user_server_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheMisses_77_pg_yb_tablegroup_oid_index_count{metric_id="yb.ysqlserver",metric_type="server",exported_instance="dev-server-myang:9000"} 0 1719605588722 ``` Reviewers: kfranz, smishra, mihnea Reviewed By: kfranz Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D35792
- Loading branch information