Provide support for scraping Azure Database for PostgreSQL Flexible server & Hyperscale #1870
Comments
tomkerkhove
changed the title
tomkerkhove
changed the title
|
Adding this to the next shipping cycle but you can currently already achieve this with our generic scraper - https://docs.promitor.io/configuration/v2.x/metrics/generic-azure-resource What you have as an example should actually work (without discovery though): |
|
Let's introduce a server type as per https://docs.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-compare-single-server-flexible-server This should be something like: name: azure_postgre_sql_cpu_percent
description: "The CPU percentage on the server"
resourceType: PostgreSql
scraping:
schedule: "0 */2 * ? * *"
azureMetricConfiguration:
metricName: cpu_percent
aggregation:
type: Average
interval: 00:01:00
resources: # Optional, required when no resource discovery is configured
- serverName: Promitor-1
type: Flexible # Options: Single (default), Flexible |
tomkerkhove
added
agents:scraper
|
Can i use regex here ?
|
|
Hi [11:27:07 INF] Booting up Promitor v2.5.0 - Thank you for using Promitor! |
|
This is not implemented yet and you can only use the Generic scraper for now. See https://docs.promitor.io/configuration/v2.x/metrics/generic-azure-resource |
Where do you want to use regex? We don't support that today. |
|
regex in server name. |
|
No, not today. Resource discovery will be your best bet once this feature is implemented. |
|
Is it possible to add azure tags into scraper?
```
flexible_server_cpu_average{.....instance_name="Microsoft.DBForPostgreSql/flexibleServers/master-smds-dblab2-dev-99", owner="szalek"}
```
this owner is the tag: owner on flex server
I did
```yaml
metrics:
- name: flexible_server_cpu_average
description: "flexible_server_cpu_average"
resourceType: Generic
azureTags:
include:
- rolegroup
- owner
azureMetricConfiguration:
metricName: cpu_percent
aggregation:
type: Average
resources:
- resourceUri: Microsoft.DBForPostgreSql/flexibleServers/master-smds-dblab2-dev-99
resourceGroupName: RG-weu-vnet-tomtom-default
```
seems no success
|
|
You can find full YAML docs on https://docs.promitor.io/configuration/v2.x/metrics/#metrics but no, Azure tags are not supported but is tracked in #599 |
|
Any plans to make tags available?
This ticket is still open so I assume is under development, right?
|
|
It is being tracked and will eventually come. |
|
So I have a WIP PR open which gives the current result: This is based on the current configuration approach: metrics:
- name: promitor_demo_postgresql_simple
description: "Availability (%) of promitor.io measured in Azure Application Insights"
resourceType: PostgreSql
azureMetricConfiguration:
metricName: cpu_percent
aggregation:
type: Average
resources:
# Application Insights with data in the service itself (classic, deprecated)
- serverName: fezfze3
- name: promitor_demo_postgresql_flex
description: "Availability (%) of promitor.io measured in Azure Application Insights"
resourceType: PostgreSql
azureMetricConfiguration:
metricName: cpu_percent
aggregation:
type: Average
resources:
# Application Insights with data in the service itself (classic, deprecated)
- serverName: fezfze2
type: Flexible
- name: promitor_demo_postgresql_hyperscale
description: "Availability (%) of promitor.io measured in Azure Application Insights"
resourceType: PostgreSql
azureMetricConfiguration:
metricName: cpu_percent
aggregation:
type: Average
resources:
# Application Insights with data in the service itself (classic, deprecated)
- serverName: fezfze
type: Hyperscale
- name: promitor_demo_postgresql_discovered
description: "Availability (%) of promitor.io measured in Azure Application Insights"
resourceType: PostgreSql
azureMetricConfiguration:
metricName: cpu_percent
aggregation:
type: Average
resourceDiscoveryGroups:
- name: postgres-databasesA few things to note:
Would this fit your needs? One thing that I have doubts about is using the same scraper which keeps it simple, but if you want to do resource discovery you cannot filter on the type which can be a pain given the metric surface is not identical across all resources. That said, we can add resource type exclusion later on as we're introducing that for resource discovery in the future. Thoughts? |
|
Hi Tom
Please tell me if for Flexes code
- name: promitor_demo_postgresql_flex
description: "Availability (%) of promitor.io measured in Azure Application Insights"
resourceType: PostgreSql
azureMetricConfiguration:
metricName: cpu_percent
aggregation:
type: Average
resources:
# Application Insights with data in the service itself (classic, deprecated)
- serverName: fezfze2
type: Flexible
searches for server named fezfze2 only or it will find as well servers like fezfze2-3, my-fezfze2-98, etc?(so any name contained fezfze2 string)
With discovery
- name: promitor_demo_postgresql_discovered
description: "Availability (%) of promitor.io measured in Azure Application Insights"
resourceType: PostgreSql
azureMetricConfiguration:
metricName: cpu_percent
aggregation:
type: Average
resourceDiscoveryGroups:
- name: postgres-databases
It will search all databases of Postgres in subscription, right?
Anything else should be enabled to use it?
Best regards
…Łukasz Szałek / Expert Software Engineer / CPP CMR / DBLAB-SOJUZ
|
It will only scrape
Yes, all flavors of PostgreSQL |
|
If only scrapes specific name, not regex it is not my case and will not help me
The only way will be to use discovery?
|
You can use resource discovery to automatically discover resources. Everything that is declared in YAML is static, because it's using metrics as code. You can learn about the ways to discover on https://docs.promitor.io/configuration/v2.x/resource-discovery but basically it supports the following for now:
Name filters (ie regex) is another one that can be considered but is not supported today. Azure Tags are typically the best approach. But your current question is beyond the scope of this issue as this is purely for scraping for that service, not how to find resources for it so suggest to open a discussion. |
|
I need guidance
resourceDiscoveryGroups:
- name: ??
type: ??
criteria:
include:
resourceGroups:
- myRG
regions:
- westeurope
What I need in name/type for flexes?
|
|
I did
version: v1
azureMetadata:
tenantId: *****
subscriptionId: ****
resourceGroupName: promitor
cloud: Global
metricDefaults:
azureTags:
include:
- rolegroup
- owner
aggregation:
interval: 00:05:00
scraping:
schedule: "0 * * ? * *"
resourceDiscoveryGroups:
- name: flexes_rdg
type: SqlManagedInstance
criteria:
include:
resourceGroups:
- RG-weu-vnet-tomtom-default
regions:
- westeurope
metrics:
- name: flexible_server_cpu_average
description: "flexible_server_cpu_average"
resourceType: Generic
azureTags:
include:
- rolegroup
- owner
azureMetricConfiguration:
metricName: cpu_percent
aggregation:
type: Average
resourceDiscoveryGroups:
- name: flexes_rdg
Results
Failed to scrape resource collection flexes_rdg: Name does not resolve
|
|
I think it's best to go through the documentation as you seem to be mixing the configuration for our scraper & resource discovery agent. https://promitor.io/concepts/ This feature is not completed yet (hence why the issue is still open) and thus you won't be able to run this yourself already. |
|
Hi
I am trying to setup Promitor discovery
And I see
Azure Landscape │ Success │ Everything is well-configured. ┃
┃ Resource Discovery Groups │ Success │ Everything is well-configured. ┃
┃ Azure Authentication │ Failed │ Validation failed: ┃
┃ │ │ Azure authentication is not configured ┃
┃ │ │ correctly - No identity secret was ┃
┃ │ │ configured for service principle ┃
┃ │ │ authentication
Config
***@***.***
What is the “key” to setup the secret
Tried this style
azureAuthentication:
appId: ******
appKey: ***
no success
this neither
azureAuthentication:
appId: "" # [Deprecated] Prefer identity.id
appKey: "" # [Deprecated] Prefer identity.key
mode: "ServicePrincipal"
identity:
id: ""
key: ""
binding: ""
Thanks for suggestions
|
|
Can you please open a Q&A discussion as this is unrelated to this issue? |
tomkerkhove
changed the title
Proposal
I would like to be able to select multiple Flex servers for monitoring either by using regex in URI or in resourceDiscoveryGroup
Example
resources:
resourceUri: Microsoft.DBForPostgreSql/flexibleServers/{.myproject.}
Component
Resource Discovery, Scraper
Contact Details
[email protected]
The text was updated successfully, but these errors were encountered: