Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend QueryManager query type #9874

Merged
merged 3 commits into from
Aug 13, 2021
Merged

Extend QueryManager query type #9874

merged 3 commits into from
Aug 13, 2021

Conversation

mx-psi
Copy link
Member

@mx-psi mx-psi commented Aug 11, 2021

What does this PR do?

Extends QueryManager query type to allow mappings.

Motivation

Needed for IBM i integration, see #9720.

Additional Notes

As of now it should be a no-op.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • PR title must be written as a CHANGELOG entry (see why)
  • Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
  • PR must have changelog/ and integration/ labels attached

@codecov
Copy link

codecov bot commented Aug 11, 2021

Codecov Report

Merging #9874 (6a88158) into master (9950fe0) will increase coverage by 0.03%.
The diff coverage is 100.00%.

Flag Coverage Δ
active_directory 100.00% <ø> (ø)
activemq_xml 82.31% <ø> (ø)
aerospike 86.97% <ø> (+0.36%) ⬆️
airflow 89.94% <ø> (ø)
amazon_msk 87.94% <ø> (ø)
ambari 86.98% <ø> (ø)
apache 95.43% <ø> (ø)
aspdotnet 93.87% <ø> (ø)
avi_vantage 92.07% <ø> (ø)
azure_iot_edge 82.01% <ø> (ø)
btrfs 82.91% <ø> (ø)
cacti 83.95% <ø> (ø)
cassandra_nodetool 94.19% <ø> (ø)
ceph 91.02% <ø> (ø)
cilium 85.84% <ø> (+1.88%) ⬆️
cisco_aci 95.88% <ø> (ø)
clickhouse 96.55% <ø> (ø)
cloud_foundry_api 95.98% <ø> (+0.12%) ⬆️
cockroachdb 97.18% <ø> (ø)
consul 91.74% <ø> (ø)
coredns 96.36% <ø> (ø)
couch 95.19% <ø> (+0.24%) ⬆️
couchbase 81.45% <ø> (ø)
crio 100.00% <ø> (ø)
datadog_checks_base 89.62% <100.00%> (+0.44%) ⬆️
datadog_checks_dev 79.81% <ø> (-0.04%) ⬇️
datadog_checks_downloader 80.64% <ø> (ø)
datadog_cluster_agent 97.50% <ø> (ø)
directory 94.70% <ø> (ø)
disk 91.15% <ø> (-0.49%) ⬇️
dns_check 94.00% <ø> (ø)
dotnetclr 100.00% <ø> (ø)
druid 97.70% <ø> (ø)
ecs_fargate 77.65% <ø> (ø)
eks_fargate 94.05% <ø> (ø)
elastic 88.54% <ø> (ø)
envoy 93.77% <ø> (+0.25%) ⬆️
etcd 93.09% <ø> (ø)
exchange_server 100.00% <ø> (ø)
external_dns 100.00% <ø> (ø)
fluentd 94.77% <ø> (ø)
gearmand 77.27% <ø> (+1.29%) ⬆️
gitlab 89.94% <ø> (ø)
gitlab_runner 90.32% <ø> (ø)
glusterfs 80.09% <ø> (+0.92%) ⬆️
go_expvar 91.95% <ø> (ø)
gunicorn 93.53% <ø> (ø)
haproxy 95.22% <ø> (+0.17%) ⬆️
harbor 91.58% <ø> (ø)
hazelcast 92.39% <ø> (ø)
hdfs_datanode 90.00% <ø> (ø)
hdfs_namenode 87.94% <ø> (ø)
http_check 89.96% <ø> (+1.82%) ⬆️
ibm_db2 93.65% <ø> (ø)
ibm_mq 90.00% <ø> (ø)
ibm_was 97.44% <ø> (ø)
iis 93.01% <ø> (ø)
istio 78.91% <ø> (+0.63%) ⬆️
kafka_consumer 80.95% <ø> (ø)
kong 92.21% <ø> (ø)
kube_apiserver_metrics 97.35% <ø> (ø)
kube_controller_manager 97.14% <ø> (ø)
kube_dns 98.85% <ø> (ø)
kube_metrics_server 100.00% <ø> (ø)
kube_proxy 100.00% <ø> (ø)
kube_scheduler 96.41% <ø> (ø)
kubelet 89.47% <ø> (ø)
kubernetes_state 89.69% <ø> (ø)
kyototycoon 85.96% <ø> (ø)
lighttpd 83.64% <ø> (ø)
linkerd 85.14% <ø> (+1.14%) ⬆️
linux_proc_extras 96.22% <ø> (ø)
mapr 82.62% <ø> (ø)
mapreduce 82.27% <ø> (ø)
marathon 83.12% <ø> (ø)
marklogic 95.32% <ø> (ø)
mcache 93.39% <ø> (ø)
mesos_master 89.68% <ø> (ø)
mesos_slave 93.63% <ø> (ø)
mongo 94.74% <ø> (+0.28%) ⬆️
mysql 85.28% <ø> (+0.14%) ⬆️
nagios 89.53% <ø> (ø)
network 77.76% <ø> (+1.00%) ⬆️
nfsstat 95.20% <ø> (ø)
nginx 95.11% <ø> (+0.93%) ⬆️
nginx_ingress_controller 98.30% <ø> (ø)
openldap 96.33% <ø> (ø)
openmetrics 97.14% <ø> (ø)
openstack 51.30% <ø> (ø)
openstack_controller 90.51% <ø> (ø)
oracle 92.44% <ø> (+0.60%) ⬆️
pdh_check 97.77% <ø> (ø)
pgbouncer 91.50% <ø> (ø)
php_fpm 90.04% <ø> (+0.43%) ⬆️
postfix 88.04% <ø> (ø)
postgres 92.11% <ø> (+0.24%) ⬆️
powerdns_recursor 95.93% <ø> (ø)
process 85.12% <ø> (ø)
prometheus 94.17% <ø> (ø)
proxysql 99.62% <ø> (ø)
rabbitmq 93.73% <ø> (ø)
redisdb 87.26% <ø> (ø)
rethinkdb 97.93% <ø> (ø)
riak 99.22% <ø> (ø)
riakcs 93.61% <ø> (ø)
sap_hana 93.04% <ø> (ø)
scylla 100.00% <ø> (ø)
snmp 91.20% <ø> (+0.04%) ⬆️
snowflake 94.18% <ø> (+0.58%) ⬆️
sonarqube 95.69% <ø> (ø)
spark 93.64% <ø> (ø)
sqlserver 82.04% <ø> (ø)
squid 100.00% <ø> (ø)
ssh_check 91.08% <ø> (ø)
statsd 87.36% <ø> (+1.05%) ⬆️
supervisord 92.30% <ø> (ø)
system_core 91.04% <ø> (ø)
system_swap 98.30% <ø> (ø)
tcp_check 88.83% <ø> (ø)
teamcity 80.00% <ø> (ø)
tls 97.04% <ø> (+0.87%) ⬆️
tokumx 58.40% <ø> (?)
twemproxy 78.33% <ø> (ø)
twistlock 80.74% <ø> (ø)
varnish 84.57% <ø> (+0.24%) ⬆️
vault 94.91% <ø> (+0.53%) ⬆️
vertica 92.30% <ø> (ø)
voltdb 96.81% <ø> (ø)
vsphere 89.74% <ø> (+0.05%) ⬆️
win32_event_log 86.03% <ø> (+0.28%) ⬆️
windows_service 95.83% <ø> (ø)
wmi_check 92.91% <ø> (ø)
yarn 90.30% <ø> (ø)
zk 85.17% <ø> (+0.25%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

djmitche
djmitche previously approved these changes Aug 11, 2021
Copy link
Contributor

@djmitche djmitche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a newcomer here, but .. it seems odd to me to add this possibility to the validation checks, without a corresponding change to handle the case of query being a mapping. Has support for that case already been merged?

datadog_checks_base/datadog_checks/base/utils/db/query.py Outdated Show resolved Hide resolved
Co-authored-by: Dustin J. Mitchell <[email protected]>
@mx-psi
Copy link
Member Author

mx-psi commented Aug 12, 2021

it seems odd to me to add this possibility to the validation checks, without a corresponding change to handle the case of query being a mapping. Has support for that case already been merged?

@djmitche The QueryManager initializer takes an executor parameter

that runs queries. The idea here is that future users of QueryManager may want to pass queries with additional context to the executor; in particular we want to use this for a future check (IBM i) where the executor also gets a timeout for each query. Existing users of QueryManager would keep getting just queries (strings) and would not need to support this.

Instead of doing this, we could pass the executor argument type as a new query_type argument to QueryManager, defaulting to str (or a boolean for str vs dict). That way existing users of QueryManager would have the same behavior today and there is no room for errors because of someone passing a mapping. Does that sound better to you than the existing patch?

@djmitche
Copy link
Contributor

djmitche commented Aug 12, 2021

I think this is OK as-is. As you've said, it will allow for future support for mappings, and in the interim only increases the set of valid configs. Thanks for the explanation!

Co-authored-by: Ofek Lev <[email protected]>
@mx-psi mx-psi force-pushed the mx-psi/backport-querymanager branch from eb15705 to 6a88158 Compare August 12, 2021 14:29
@mx-psi mx-psi requested a review from ofek August 12, 2021 15:02
@mx-psi mx-psi merged commit 36dfad0 into master Aug 13, 2021
@mx-psi mx-psi deleted the mx-psi/backport-querymanager branch August 13, 2021 07:41
github-actions bot pushed a commit that referenced this pull request Aug 13, 2021
* Add 'timeout' field to QueryManager queries

(cherry picked from commit 7c33f22)

* Apply suggestions from code review

Co-authored-by: Dustin J. Mitchell <[email protected]>

* Address review comment

Co-authored-by: Ofek Lev <[email protected]>

Co-authored-by: Dustin J. Mitchell <[email protected]>
Co-authored-by: Ofek Lev <[email protected]> 36dfad0
mx-psi added a commit that referenced this pull request Aug 13, 2021
Revert "Add 'timeout' field to QueryManager queries"

This reverts commit 7c33f22.

Extend `QueryManager` query type (#9874)

* Add 'timeout' field to QueryManager queries

(cherry picked from commit 7c33f22)

* Apply suggestions from code review

Co-authored-by: Dustin J. Mitchell <[email protected]>

* Address review comment

Co-authored-by: Ofek Lev <[email protected]>

Co-authored-by: Dustin J. Mitchell <[email protected]>
Co-authored-by: Ofek Lev <[email protected]>
(cherry picked from commit 36dfad0)
mx-psi added a commit that referenced this pull request Aug 16, 2021
Revert "Add 'timeout' field to QueryManager queries"

This reverts commit 7c33f22.

Extend `QueryManager` query type (#9874)

* Add 'timeout' field to QueryManager queries

(cherry picked from commit 7c33f22)

* Apply suggestions from code review

Co-authored-by: Dustin J. Mitchell <[email protected]>

* Address review comment

Co-authored-by: Ofek Lev <[email protected]>

Co-authored-by: Dustin J. Mitchell <[email protected]>
Co-authored-by: Ofek Lev <[email protected]>
(cherry picked from commit 36dfad0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants