New resources for Log analytics datasources #8589

PriyankaRanganath · 2020-09-23T07:50:08Z

Add support for Log analytics workspace data source resources:

azurerm_log_analytics_datasource_iis_logs
azurerm_log_analytics_datasource_linux_performance_collection
azurerm_log_analytics_datasource_linux_performance_object
azurerm_log_analytics_datasource_linux_syslog_collection
azurerm_log_analytics_datasource_linux_syslog

…rces to log analytics

…ce documentation

PriyankaRanganath · 2020-09-25T06:28:21Z

Is there any way to review and merge this PR? I understand that there is no official documentation for log analytics data sources properties from Microsoft. However, we have taken reference from the equivalent PowerShell cmdlets for each of these data sources and implemented accordingly. The community can use this meanwhile until official documentation and supporting go SDK is available. There is already implementation for windows event and windows performance counter data sources. I dont see why this PR would be an issue if everything is implemented properly.

WodansSon

@PriyankaRanganath, thank you so much for opening this PR, this mostly LGTM, however I left a few comments to address. Once those are taken care of we can get this merged! 🚀

azurerm/internal/services/loganalytics/log_analytics_datasource_iis_logs_resource.go

...rnal/services/loganalytics/log_analytics_datasource_linux_performance_collection_resource.go

.../internal/services/loganalytics/log_analytics_datasource_linux_syslog_collection_resource.go

azurerm/internal/services/loganalytics/log_analytics_datasource_linux_syslog_resource.go

website/docs/r/log_analytics_datasource_iis_logs.html.markdown

website/docs/r/log_analytics_datasource_linux_performance_collection.html.markdown

website/docs/r/log_analytics_datasource_linux_syslog_collection.html.markdown

PriyankaRanganath · 2020-10-08T12:52:45Z

Hi @WodansSon ,

Thank you for reviewing the PR! Resolved comments as requested

katbyte

Thanks for the PR @PriyankaRanganath, while overall this looks great it appears the SDK/swagger is incomplete and needs to be updated and unfortunately we need the SDK to define these types else they can change on Azure's side without correctly handling semantic versioning - as such these types need to be present in the Azure SDK/Swagger. There is an open issue here and until this is fixed i'm not sure we can merge this

katbyte · 2020-10-08T16:49:58Z

azurerm/internal/services/loganalytics/log_analytics_datasource_iis_logs_resource.go

+type dataSourceIISLogsProperty struct {
+	State string `json:"state"`
+}


this looks like an SDK/swagger bug? could we get an issue opened and this fixed there?

katbyte · 2020-10-08T16:51:41Z

azurerm/internal/services/loganalytics/log_analytics_datasource_iis_logs_resource.go

+
+	if v, ok := d.GetOk("on_premise_enabled"); ok {
+		if onPremiseEnabled := v.(bool); onPremiseEnabled {
+			onPremiseState = "OnPremiseEnabled"


This should be exposed by the SDK as a constant? which probably ties in to the swagger bug for the missing dataSourceIISLogsProperty struct?

azurerm/internal/services/loganalytics/log_analytics_datasource_iis_logs_resource.go

katbyte · 2020-10-08T16:53:12Z

...rnal/services/loganalytics/log_analytics_datasource_linux_performance_collection_resource.go

+type dataSourceLinuxPerformanceCollectionProperty struct {
+	State string `json:"state"`
+}


same comment above: could we get this fixed in the swagger and SDK?

katbyte · 2020-10-08T16:53:50Z

...rnal/services/loganalytics/log_analytics_datasource_linux_performance_collection_resource.go

+	perfCollectionState := "Disabled"
+
+	if v, ok := d.GetOk("enabled"); ok {
+		if enabled := v.(bool); enabled {
+			perfCollectionState = "Enabled"
+		}
+	}


Hi @katbyte , there is no support for SDK or even official documentation from Microsoft yet. Is it ok to still merge this PR ?
I am trying to get hold of Microsoft prod team but as mentioned there is no ETA on it yet

Please see the linked issue below for more details, but we typically refrain from merging anything that is not currently supported by the SDK as it can become problematic down the line.

Thanks @katbyte ! will wait for SDK and official documentation release

katbyte · 2020-10-08T18:57:07Z

...internal/services/loganalytics/log_analytics_datasource_linux_performance_object_resource.go

+type dataSourceLinuxPerformanceObjectProperty struct {
+	ObjectName          string                                        `json:"objectName"`
+	InstanceName        string                                        `json:"instanceName"`
+	IntervalSeconds     int                                           `json:"intervalSeconds"`
+	PerformanceCounters []dataSourceLinuxPerformanceObjectCounterName `json:"performanceCounters"`
+}
+
+type dataSourceLinuxPerformanceObjectCounterName struct {
+	CounterName string `json:"counterName"`
+}


this should also be in the swagger & sdk

katbyte · 2020-10-08T18:57:32Z

.../internal/services/loganalytics/log_analytics_datasource_linux_syslog_collection_resource.go

+type dataSourceLinuxSysLogCollectionProperty struct {
+	State string `json:"state"`
+}


azurerm/internal/services/loganalytics/log_analytics_datasource_linux_syslog_resource.go

katbyte · 2020-10-08T18:58:48Z

azurerm/internal/services/loganalytics/log_analytics_datasource_linux_syslog_resource.go

+type dataSourceLinuxSysLogProperty struct {
+	SysLogName       string                          `json:"syslogName"`
+	SysLogSeverities []dataSourceLinuxSysLogSeverity `json:"syslogSeverities"`
+}
+
+type dataSourceLinuxSysLogSeverity struct {
+	Severity string `json:"severity"`
+}


this should also be added to the SDK/swagger

katbyte · 2020-10-08T19:22:43Z

linked issue: Azure/azure-rest-api-specs#9072

katbyte · 2020-10-22T17:39:15Z

@PriyankaRanganath - given this is blocked on the swagger issue, i am going to close it. Once the service team has addressed the missing swagger & SDK supports these properties we can reopen this PR and move it forward.

ghost · 2020-11-22T14:51:37Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!

PriyankaRanganath and others added 8 commits September 14, 2020 15:28

Initial commits for linux syslog and linux syslog collection data sou…

4172565

…rces to log analytics

added make formatting

6d596cf

added log analytics linux syslog and linux syslog collection datasour…

a39026e

…ce documentation

added linux performance collection datasource

af57c46

added iis logs data source

4fa025b

added source for linux perfromance object

3a69f0a

Merge branch 'master' into feature/log_analytics_linux_syslog_datasource

50ccae9

added formatting

c8535cd

ghost added size/XXL documentation labels Sep 23, 2020

PriyankaRanganath marked this pull request as draft September 23, 2020 07:50

Priyanka Ranganath added 3 commits September 23, 2020 14:41

Added formatting for log analytics datasource test files

fa1844c

added formatting errors

42b0b6d

added formatting

c3aef07

PriyankaRanganath marked this pull request as ready for review September 23, 2020 12:40

PriyankaRanganath changed the title ~~Added Log analytics datasource resource features~~ New resources for Log analytics datasource resources Sep 23, 2020

PriyankaRanganath changed the title ~~New resources for Log analytics datasource resources~~ New resources for Log analytics datasources Sep 23, 2020

fixed documentation for log analytics

1d6f0c9

PriyankaRanganath marked this pull request as draft September 24, 2020 10:56

Merge branch 'master' into feature/log_analytics_linux_syslog_datasource

81ff73c

PriyankaRanganath marked this pull request as ready for review September 25, 2020 05:59

PriyankaRanganath mentioned this pull request Sep 25, 2020

Configure Data Sources in a Log Analytics Workspace #3182

Open

WodansSon added this to the v2.31.0 milestone Oct 7, 2020

WodansSon requested changes Oct 7, 2020

View reviewed changes

WodansSon added the service/log-analytics label Oct 7, 2020

Priyanka Ranganath added 4 commits October 8, 2020 12:48

resolved PR comments

aac220b

added more facility types to linux syslog and make formatted code

9c98edf

corrected test case condition check

3eb6572

corrected test case condition check

f04923f

corrected test case condition check

c0926ca

jackofallops modified the milestones: v2.31.0, v2.32.0 Oct 8, 2020

katbyte requested changes Oct 8, 2020

View reviewed changes

katbyte added the waiting-response label Oct 8, 2020

more comments resolved

b6306f8

jackofallops modified the milestones: v2.32.0, v2.33.0 Oct 15, 2020

tombuildsstuff modified the milestones: v2.33.0, Blocked Oct 21, 2020

katbyte closed this Oct 22, 2020

ghost locked as resolved and limited conversation to collaborators Nov 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New resources for Log analytics datasources #8589

New resources for Log analytics datasources #8589

PriyankaRanganath commented Sep 23, 2020 •

edited by WodansSon

Loading

PriyankaRanganath commented Sep 25, 2020 •

edited

Loading

WodansSon left a comment

PriyankaRanganath commented Oct 8, 2020 •

edited

Loading

katbyte left a comment

katbyte Oct 8, 2020

katbyte Oct 8, 2020

katbyte Oct 8, 2020

katbyte Oct 8, 2020

PriyankaRanganath Oct 13, 2020

katbyte Oct 15, 2020

PriyankaRanganath Oct 19, 2020

katbyte Oct 8, 2020

katbyte Oct 8, 2020

katbyte Oct 8, 2020

katbyte commented Oct 8, 2020

katbyte commented Oct 22, 2020

ghost commented Nov 22, 2020

New resources for Log analytics datasources #8589

New resources for Log analytics datasources #8589

Conversation

PriyankaRanganath commented Sep 23, 2020 • edited by WodansSon Loading

PriyankaRanganath commented Sep 25, 2020 • edited Loading

WodansSon left a comment

Choose a reason for hiding this comment

PriyankaRanganath commented Oct 8, 2020 • edited Loading

katbyte left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

katbyte commented Oct 8, 2020

katbyte commented Oct 22, 2020

ghost commented Nov 22, 2020

PriyankaRanganath commented Sep 23, 2020 •

edited by WodansSon

Loading

PriyankaRanganath commented Sep 25, 2020 •

edited

Loading

PriyankaRanganath commented Oct 8, 2020 •

edited

Loading