x-pack/filebeat/input/entityanalytics/provider/activedirectory/internal/activedirectory: new package #37854

efd6 · 2024-02-05T02:27:59Z

Proposed commit message

The activedirectory package provides a query interface to Active Directory user and group entities. Queries can be either a complete pull of the directory, or a subset of entries since a checkpoint time.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

Testing requires setting up an active directory test server (or pointing the test at an existing server). There are notes in the test file explaining how pointing the tests at the server is handled. I used instructions here for setting up a local VM test server.

Partial test output from the directory of the test server.

=== RUN   Test/full
    activedirectory_test.go:81: user: [
        	{
        		"id": "CN=Administrator,CN=Users,DC=testserver,DC=local",
        		"user": {
        			"accountExpires": "2185-07-21T23:34:33.709551516Z",
        			"adminCount": "1",
        			"badPasswordTime": "133517595269561536",
        			"badPwdCount": "0",
        			"cn": "Administrator",
        			"codePage": "0",
        			"countryCode": "0",
        			"dSCorePropagationData": [
        				"2024-01-22T06:52:50Z",
        				"2024-01-22T06:52:50Z",
        				"2024-01-22T06:37:40Z",
        				"1601-01-01T18:12:16Z"
        			],
        			"description": "Built-in account for administering the computer/domain",
        			"distinguishedName": "CN=Administrator,CN=Users,DC=testserver,DC=local",
        			"instanceType": "4",
        			"isCriticalSystemObject": true,
        			"lastLogoff": "0",
        			"lastLogon": "2024-02-08T06:51:02.1812823Z",
        			"lastLogonTimestamp": "2024-02-05T16:15:38.1137138Z",
        			"logonCount": "8",
        			"memberOf": [
        				"CN=Group Policy Creator Owners,CN=Users,DC=testserver,DC=local",
        				"CN=Domain Admins,CN=Users,DC=testserver,DC=local",
        				"CN=Enterprise Admins,CN=Users,DC=testserver,DC=local",
        				"CN=Schema Admins,CN=Users,DC=testserver,DC=local",
        				"CN=Administrators,CN=Builtin,DC=testserver,DC=local"
        			],
        			"name": "Administrator",
        			"objectCategory": "CN=Person,CN=Schema,CN=Configuration,DC=testserver,DC=local",
        			"objectClass": [
        				"top",
        				"person",
        				"organizationalPerson",
        				"user"
        			],
        			"objectGUID": "kUXoCTwYv0iZNc6UadAk1w==",
        			"objectSid": "AQUAAAAAAAUVAAAA0JU2Fq1k30YZ7UPx9AEAAA==",
        			"primaryGroupID": "513",
        			"pwdLastSet": "2024-01-22T06:15:39.8703568Z",
        			"sAMAccountName": "Administrator",
        			"sAMAccountType": "805306368",
        			"uSNChanged": "20494",
        			"uSNCreated": "8196",
        			"userAccountControl": "66048",
        			"whenChanged": "2024-02-05T16:15:38Z",
        			"whenCreated": "2024-01-22T06:36:59Z"
        		},
        		"groups": [
        			{
        				"cn": "Group Policy Creator Owners",
        				"dSCorePropagationData": [
        					"2024-01-22T06:37:40Z",
        					"1601-01-01T00:00:01Z"
        				],
        				"description": "Members in this group can modify group policy for the domain",
        				"distinguishedName": "CN=Group Policy Creator Owners,CN=Users,DC=testserver,DC=local",
        				"groupType": "-2147483646",
        				"instanceType": "4",
        				"isCriticalSystemObject": true,
        				"member": "CN=Administrator,CN=Users,DC=testserver,DC=local",
        				"memberOf": "CN=Denied RODC Password Replication Group,CN=Users,DC=testserver,DC=local",
        				"name": "Group Policy Creator Owners",
        				"objectCategory": "CN=Group,CN=Schema,CN=Configuration,DC=testserver,DC=local",
        				"objectClass": [
        					"top",
        					"group"
        				],
        				"objectGUID": "q72u7gQzkkGd1X2IFbK6Hw==",
        				"objectSid": "AQUAAAAAAAUVAAAA0JU2Fq1k30YZ7UPxCAIAAA==",
        				"sAMAccountName": "Group Policy Creator Owners",
        				"sAMAccountType": "268435456",
        				"uSNChanged": "12391",
        				"uSNCreated": "12354",
        				"whenChanged": "2024-01-22T06:37:40Z",
        				"whenCreated": "2024-01-22T06:37:40Z"
        			},
        			{
        				"adminCount": "1",
        				"cn": "Domain Admins",
        				"dSCorePropagationData": [
        					"2024-01-22T06:52:50Z",
        					"2024-01-22T06:37:40Z",
        					"1601-01-01T00:04:16Z"
        				],
        				"description": "Designated administrators of the domain",
        				"distinguishedName": "CN=Domain Admins,CN=Users,DC=testserver,DC=local",
        				"groupType": "-2147483646",
        				"instanceType": "4",
        				"isCriticalSystemObject": true,
        				"member": "CN=Administrator,CN=Users,DC=testserver,DC=local",
        				"memberOf": [
        					"CN=Denied RODC Password Replication Group,CN=Users,DC=testserver,DC=local",
        					"CN=Administrators,CN=Builtin,DC=testserver,DC=local"
        				],
        				"name": "Domain Admins",
        				"objectCategory": "CN=Group,CN=Schema,CN=Configuration,DC=testserver,DC=local",
        				"objectClass": [
        					"top",
        					"group"
        				],
        				"objectGUID": "dcLZNxKP90+za+sRLhh8kA==",
        				"objectSid": "AQUAAAAAAAUVAAAA0JU2Fq1k30YZ7UPxAAIAAA==",
        				"sAMAccountName": "Domain Admins",
        				"sAMAccountType": "268435456",
        				"uSNChanged": "12770",
        				"uSNCreated": "12345",
        				"whenChanged": "2024-01-22T06:52:50Z",
        				"whenCreated": "2024-01-22T06:37:40Z"
        			},
        			{
        				"adminCount": "1",
        				"cn": "Enterprise Admins",
        				"dSCorePropagationData": [
        					"2024-01-22T06:52:50Z",
        					"2024-01-22T06:37:40Z",
        					"1601-01-01T00:04:16Z"
        				],
        				"description": "Designated administrators of the enterprise",
        				"distinguishedName": "CN=Enterprise Admins,CN=Users,DC=testserver,DC=local",
        				"groupType": "-2147483640",
        				"instanceType": "4",
        				"isCriticalSystemObject": true,
        				"member": "CN=Administrator,CN=Users,DC=testserver,DC=local",
        				"memberOf": [
        					"CN=Denied RODC Password Replication Group,CN=Users,DC=testserver,DC=local",
        					"CN=Administrators,CN=Builtin,DC=testserver,DC=local"
        				],
        				"name": "Enterprise Admins",
        				"objectCategory": "CN=Group,CN=Schema,CN=Configuration,DC=testserver,DC=local",
        				"objectClass": [
        					"top",
        					"group"
        				],
        				"objectGUID": "8NElU6E6TU61DNFLIe7VHQ==",
        				"objectSid": "AQUAAAAAAAUVAAAA0JU2Fq1k30YZ7UPxBwIAAA==",
        				"sAMAccountName": "Enterprise Admins",
        				"sAMAccountType": "268435456",
        				"uSNChanged": "12773",
        				"uSNCreated": "12339",
        				"whenChanged": "2024-01-22T06:52:50Z",
        				"whenCreated": "2024-01-22T06:37:40Z"
        			},
        			{
        				"adminCount": "1",
        				"cn": "Schema Admins",
        				"dSCorePropagationData": [
        					"2024-01-22T06:52:50Z",
        					"2024-01-22T06:37:40Z",
        					"1601-01-01T00:04:16Z"
        				],
        				"description": "Designated administrators of the schema",
        				"distinguishedName": "CN=Schema Admins,CN=Users,DC=testserver,DC=local",
        				"groupType": "-2147483640",
        				"instanceType": "4",
        				"isCriticalSystemObject": true,
        				"member": "CN=Administrator,CN=Users,DC=testserver,DC=local",
        				"memberOf": "CN=Denied RODC Password Replication Group,CN=Users,DC=testserver,DC=local",
        				"name": "Schema Admins",
        				"objectCategory": "CN=Group,CN=Schema,CN=Configuration,DC=testserver,DC=local",
        				"objectClass": [
        					"top",
        					"group"
        				],
        				"objectGUID": "Ti+p/M4gtECARFBF3cNSpw==",
        				"objectSid": "AQUAAAAAAAUVAAAA0JU2Fq1k30YZ7UPxBgIAAA==",
        				"sAMAccountName": "Schema Admins",
        				"sAMAccountType": "268435456",
        				"uSNChanged": "12769",
        				"uSNCreated": "12336",
        				"whenChanged": "2024-01-22T06:52:50Z",
        				"whenCreated": "2024-01-22T06:37:40Z"
        			}
        		],
        		"whenChanged": "2024-02-05T16:15:38Z"
        	},
        	{
        		"id": "CN=Guest,CN=Users,DC=testserver,DC=local",
        		"user": {
        			"accountExpires": "2185-07-21T23:34:33.709551516Z",
        			"badPasswordTime": "0",
        			"badPwdCount": "0",
        			"cn": "Guest",
        			"codePage": "0",
        			"countryCode": "0",
        			"dSCorePropagationData": [
        				"2024-01-22T06:37:40Z",
        				"1601-01-01T00:00:01Z"
        			],
        			"description": "Built-in account for guest access to the computer/domain",
        			"distinguishedName": "CN=Guest,CN=Users,DC=testserver,DC=local",
        			"instanceType": "4",
        			"isCriticalSystemObject": true,
        			"lastLogoff": "0",
        			"lastLogon": "2185-07-21T23:34:33.709551616Z",
        			"logonCount": "0",
        			"memberOf": "CN=Guests,CN=Builtin,DC=testserver,DC=local",
        			"name": "Guest",
        			"objectCategory": "CN=Person,CN=Schema,CN=Configuration,DC=testserver,DC=local",
        			"objectClass": [
        				"top",
        				"person",
        				"organizationalPerson",
        				"user"
        			],
        			"objectGUID": "hSt/40XJQU6cf+J2XoYMHw==",
        			"objectSid": "AQUAAAAAAAUVAAAA0JU2Fq1k30YZ7UPx9QEAAA==",
        			"primaryGroupID": "514",
        			"pwdLastSet": "2185-07-21T23:34:33.709551616Z",
        			"sAMAccountName": "Guest",
        			"sAMAccountType": "805306368",
        			"uSNChanged": "8197",
        			"uSNCreated": "8197",
        			"userAccountControl": "66082",
        			"whenChanged": "2024-01-22T06:36:59Z",
        			"whenCreated": "2024-01-22T06:36:59Z"
        		},
        		"whenChanged": "2024-01-22T06:36:59Z"
        	},
        	{
        		"id": "CN=krbtgt,CN=Users,DC=testserver,DC=local",
        		"user": {
        			"accountExpires": "2185-07-21T23:34:33.709551516Z",
        			"adminCount": "1",
        			"badPasswordTime": "0",
        			"badPwdCount": "0",
        			"cn": "krbtgt",
        			"codePage": "0",
        			"countryCode": "0",
        			"dSCorePropagationData": [
        				"2024-01-22T06:52:50Z",
        				"2024-01-22T06:37:40Z",
        				"1601-01-01T00:04:16Z"
        			],
        			"description": "Key Distribution Center Service Account",
        			"distinguishedName": "CN=krbtgt,CN=Users,DC=testserver,DC=local",
        			"instanceType": "4",
        			"isCriticalSystemObject": true,
        			"lastLogoff": "0",
        			"lastLogon": "2185-07-21T23:34:33.709551616Z",
        			"logonCount": "0",
        			"memberOf": "CN=Denied RODC Password Replication Group,CN=Users,DC=testserver,DC=local",
        			"msDS-SupportedEncryptionTypes": "0",
        			"name": "krbtgt",
        			"objectCategory": "CN=Person,CN=Schema,CN=Configuration,DC=testserver,DC=local",
        			"objectClass": [
        				"top",
        				"person",
        				"organizationalPerson",
        				"user"
        			],
        			"objectGUID": "rdk2F/qu4Eud52Q5bTXc7g==",
        			"objectSid": "AQUAAAAAAAUVAAAA0JU2Fq1k30YZ7UPx9gEAAA==",
        			"primaryGroupID": "513",
        			"pwdLastSet": "2024-01-22T06:37:40.4305135Z",
        			"sAMAccountName": "krbtgt",
        			"sAMAccountType": "805306368",
        			"servicePrincipalName": "kadmin/changepw",
        			"showInAdvancedViewOnly": true,
        			"uSNChanged": "12785",
        			"uSNCreated": "12324",
        			"userAccountControl": "514",
        			"whenChanged": "2024-01-22T06:52:50Z",
        			"whenCreated": "2024-01-22T06:37:40Z"
        		},
        		"groups": [
        			{
        				"cn": "Denied RODC Password Replication Group",
        				"dSCorePropagationData": [
        					"2024-01-22T06:37:40Z",
        					"1601-01-01T00:00:01Z"
        				],
        				"description": "Members in this group cannot have their passwords replicated to any read-only domain controllers in the domain",
        				"distinguishedName": "CN=Denied RODC Password Replication Group,CN=Users,DC=testserver,DC=local",
        				"groupType": "-2147483644",
        				"instanceType": "4",
        				"isCriticalSystemObject": true,
        				"member": [
        					"CN=Read-only Domain Controllers,CN=Users,DC=testserver,DC=local",
        					"CN=Group Policy Creator Owners,CN=Users,DC=testserver,DC=local",
        					"CN=Domain Admins,CN=Users,DC=testserver,DC=local",
        					"CN=Cert Publishers,CN=Users,DC=testserver,DC=local",
        					"CN=Enterprise Admins,CN=Users,DC=testserver,DC=local",
        					"CN=Schema Admins,CN=Users,DC=testserver,DC=local",
        					"CN=Domain Controllers,CN=Users,DC=testserver,DC=local",
        					"CN=krbtgt,CN=Users,DC=testserver,DC=local"
        				],
        				"name": "Denied RODC Password Replication Group",
        				"objectCategory": "CN=Group,CN=Schema,CN=Configuration,DC=testserver,DC=local",
        				"objectClass": [
        					"top",
        					"group"
        				],
        				"objectGUID": "Ij75/i03bkSTx4bqdQlZ3w==",
        				"objectSid": "AQUAAAAAAAUVAAAA0JU2Fq1k30YZ7UPxPAIAAA==",
        				"sAMAccountName": "Denied RODC Password Replication Group",
        				"sAMAccountType": "536870912",
        				"uSNChanged": "12433",
        				"uSNCreated": "12405",
        				"whenChanged": "2024-01-22T06:37:40Z",
        				"whenCreated": "2024-01-22T06:37:40Z"
        			}
        		],
        		"whenChanged": "2024-01-22T06:52:50Z"
        	}
        ]

How to test this PR locally

Related issues

For [Entity Analytics] Active Directory integrations#8559

Use cases

Screenshots

Logs

efd6 · 2024-02-05T04:06:07Z

/test

efd6 · 2024-02-05T05:30:58Z

/test

elasticmachine · 2024-02-05T08:47:37Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2024-02-09T03:56:57.296+0000
Duration: 181 min 48 sec

Test stats 🧪

Test	Results
Failed	0
Passed	28853
Skipped	2020
Total	30873

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-02-05T09:03:33Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

…al/activedirectory: new package The activedirectory package provides a query interface to Active Directory user and group entities. Queries can be either a complete pull of the directory, or a subset of entries since a checkpoint time.

elasticmachine · 2024-02-09T04:55:16Z

💚 Build Succeeded

Buildkite Build
Commit: f49a8e9

History

💚 Build #2727 succeeded ddcb364
💔 Build #2722 failed f1d7fd3
💔 Build #2721 failed 55d6702
💔 Build #2425 failed 3a51725
💔 Build #2424 failed 2e5e5bd

cc @efd6

efd6 added enhancement Filebeat Filebeat backport-skip Skip notification from the automated backport with mergify Team:Security-Service Integrations Security Service Integrations Team 8.13-candidate labels Feb 5, 2024

efd6 self-assigned this Feb 5, 2024

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Feb 5, 2024

efd6 force-pushed the i8559-activedirectory branch from 5869af5 to 2e5e5bd Compare February 5, 2024 02:30

efd6 force-pushed the i8559-activedirectory branch from 2e5e5bd to 3a51725 Compare February 5, 2024 05:35

efd6 marked this pull request as ready for review February 5, 2024 09:03

efd6 requested review from a team as code owners February 5, 2024 09:03

efd6 force-pushed the i8559-activedirectory branch 3 times, most recently from f1d7fd3 to ddcb364 Compare February 8, 2024 04:51

efd6 mentioned this pull request Feb 8, 2024

x-pack/filebeat/input/entityanalytics/provider/activedirectory: new package #37919

Merged

7 tasks

efd6 force-pushed the i8559-activedirectory branch from ddcb364 to f49a8e9 Compare February 9, 2024 03:56

andrewkroh approved these changes Feb 12, 2024

View reviewed changes

efd6 merged commit e9d299b into elastic:main Feb 12, 2024
145 of 147 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

x-pack/filebeat/input/entityanalytics/provider/activedirectory/internal/activedirectory: new package #37854

x-pack/filebeat/input/entityanalytics/provider/activedirectory/internal/activedirectory: new package #37854

efd6 commented Feb 5, 2024 •

edited

Loading

efd6 commented Feb 5, 2024

efd6 commented Feb 5, 2024

elasticmachine commented Feb 5, 2024 •

edited

Loading

Build stats

Test stats 🧪

elasticmachine commented Feb 5, 2024

elasticmachine commented Feb 9, 2024

x-pack/filebeat/input/entityanalytics/provider/activedirectory/internal/activedirectory: new package #37854

x-pack/filebeat/input/entityanalytics/provider/activedirectory/internal/activedirectory: new package #37854

Conversation

efd6 commented Feb 5, 2024 • edited Loading

Proposed commit message

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

efd6 commented Feb 5, 2024

efd6 commented Feb 5, 2024

elasticmachine commented Feb 5, 2024 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

elasticmachine commented Feb 5, 2024

elasticmachine commented Feb 9, 2024

💚 Build Succeeded

History

efd6 commented Feb 5, 2024 •

edited

Loading

elasticmachine commented Feb 5, 2024 •

edited

Loading