-
Notifications
You must be signed in to change notification settings - Fork 507
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update field mapping documentation for Security Analytics #2422
Changes from 8 commits
8919145
d1fb758
ea27388
322ad47
be82052
80f0d1a
c4ac64a
c9c0dec
6ae1125
86910c5
5abffb6
33a4c8a
3e5e39a
6a3838e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -28,21 +28,46 @@ Defining a new detector involves naming the detector, selecting a data source an | |||||
* Use the **Log type**, **Rule severity**, and **Source** dropdown menus to filter the rules you want to select from. | ||||||
* Use the **Search** bar to search for specific rules. | ||||||
|
||||||
To quickly select one or more known rules and dismiss others, first deselect all rules by moving the **rule name** toggle to the left, then search for your target rule names and select each individually by moving its toggle to the right. | ||||||
To quickly select one or more known rules and dismiss others, first deselect all rules by moving the **Rule name** toggle to the left, then search for your target rule names and select each individually by moving its toggle to the right. | ||||||
{: .tip } | ||||||
|
||||||
1. In the **Detector schedule** section, set how often the detector will run. Specify a unit of time and a corresponding number to set the interval. | ||||||
1. Select the **Next** button in the lower-right corner of the screen to continue. The Configure field mapping page appears. | ||||||
|
||||||
## Step 2. Making field mappings | ||||||
## Step 2. Create field mappings | ||||||
|
||||||
Field mapping matches field names for the rule with field names from the log being used to provide data. The mappings are automatically applied once the detector is defined in previous steps. This page offers the user the option to map log-specific field names to the internal rule field names. | ||||||
The field mapping step matches field names from the rule with field names from the log index being used to provide data. Creating field mappings allows the system to accurately convey event data from the log to the detector and then use the data for triggering alerts. | ||||||
|
||||||
For example, if you prefer to have the log field name UserID rather than EventID correspond to the event_uid rule field name, you can use the **Log field name** dropdown menu to select **UserID**. | ||||||
The data source (log index), log type, and detection rules specified in the first step determine which fields are available for mapping. For example, when "Windows logs" is selected as the log type, this parameter along with the specific detection rules determine the list of rule field names available for the mapping. Similarly, the selected data source (log index) determines the list of log field names that are available for the mapping. | ||||||
cwillum marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
<img src="{{site.url}}{{site.baseurl}}/images/Security/field_map.png" alt="Rule and log field mapping example"> | ||||||
#### A note on field names | ||||||
|
||||||
To make any changes to the automatically populated mappings, use the dropdown arrows across from the rule field names to specify a preferred log field name for the mapping. After completing the mappings, select the **Next** button in the lower-right corner of the screen. The Set up alerts page appears and displays settings for an alert trigger. | ||||||
The field mapping process requires that you are familiar with the field names in the log index and have an understanding of the data contained in those fields. If you have an understanding of the log fields in the index, the mapping is typically a simple and straightforward process. | ||||||
cwillum marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
Security Analytics takes advantage of pre-packaged Sigma rules for security event detection. Therefore, the rule field names are derived from a Sigma rule field standard. To make them easier to identify, however, we create aliases for the Sigma rule fields based on the open source Elastic Common Schema (ECS) specification. These alias rule field names are the field names used in these steps. | ||||||
cwillum marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
Although the ECS rule field names are largely self explanatory, you can find pre-defined mappings of the Sigma rule field names with ECS rule field names for all supported log types in the GitHub Security Analytics repository. First navigate to the [OSMappings](https://github.com/opensearch-project/security-analytics/tree/main/src/main/resources/OSMapping) folder, select the folder named for the log type, and open the `fieldmappings.yml` file. For example, to see the Sigma rule fields that correspond to ECS rule fields for the Windows log type, open the [fieldmappings.yml file](https://github.com/opensearch-project/security-analytics/blob/main/src/main/resources/OSMapping/windows/fieldmappings.yml) in the **windows** folder. | ||||||
cwillum marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
### Pending field mappings | ||||||
|
||||||
Once you navigate to the Configure field mapping page, the system attempts to automatically map fields between the two sources. Those field names that are not automatically mapped appear in the **Pending field mapping** table. In this table you can manually map rule fields to log fields. | ||||||
cwillum marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
<br><img src="{{site.url}}{{site.baseurl}}/images/Security/pending-mappings.png" alt="Field mapping example for pending mappings" width="900"> | ||||||
|
||||||
* The Rule field name column lists field names based on all of the pre-packaged rules associated with the selected log type. | ||||||
cwillum marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The bulleted list should ideally be introduced by a brief sentence ending in a colon. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added ... |
||||||
* The log field name column includes a dropdown list for each of the rule fields. Each dropdown list contains field names extracted from the log index. | ||||||
* To map a rule field name to a log field name, use the dropdown arrow to open the list of log fields and select the log field name from the list. To search for names in the log field list, enter text in the **Select a mapping field** box. | ||||||
<br><img src="{{site.url}}{{site.baseurl}}/images/Security/log-field.png" alt="Field mapping example for pending mappings" width="600"> | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Reference the image in the text that precedes it. ", as shown in the following image." usually works. |
||||||
* Once the log field name is selected and mapped to the rule field name, the icon in the Status column to the right changes to a green check mark. | ||||||
* Make as many matches between field names as possible to complete an accurate mapping for rule and log fields. | ||||||
|
||||||
### Default field mappings | ||||||
|
||||||
The **Default mapped fields** table contains mappings that the system made automatically after defining the detector. As shown in the image that follows, when the field names are similar to one another the system can successfully match the two. | ||||||
cwillum marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
<br><img src="{{site.url}}{{site.baseurl}}/images/Security/default-mappings.png" alt="Field mapping example for pending mappings" width="900"> | ||||||
|
||||||
Nevertheless, it's a good idea to review the mappings and verify that they are correct and matched as expected. If you find a mapping that doesn't appear to be accurate, you can use the dropdown list as described in the [Pending field mappings](#pending-field-mappings) section above to correct the field mapping. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we use something simpler than "Nevertheless"? It may not be understood by Global English speakers. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I added this ... |
||||||
|
||||||
After completing the mappings, select the **Next** button in the lower-right corner of the screen. The **Set up alerts** page appears and displays settings for an alert trigger. | ||||||
|
||||||
## Step 3. Setting up alerts | ||||||
|
||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.