It’s a technical release to conclude the migration of UCC UI into this repository and we also happen to release 1 feature.
+apiVersion from globalConfig.json and bump schemaVersion (6c22704)additional_packaging.py file¶To extend the build process, you can create additional_packaging.py file in the same file level where you have your globalConfig file.
This file should have additional_packaging function which accepts 1 argument: add-on name.
Example of how to utilize it:
+ucc-gen finishes all its necessary steps.ucc-gen feature which was not implemented.We can use this feature to map each field with meaningful value to display in the table. For example, The category field contains 1, 2, and 4 values, but when those values are displayed as it is then the user might get confused as those values do not signify the meaning of their mapping. To avoid this confusion user can map each field with meaningful value as shown in the example below.
+{
+ "name": "account",
+ "title": "Account",
+ "table": {
+ "header": [
+ {
+ "field": "name",
+ "label": "Name"
+ },
+ {
+ "field": "key_id",
+ "label": "Key ID"
+ },
+ {
+ "field": "category",
+ "label": "Region Category",
+ "mapping": {
+ "1": "Global",
+ "2": "US Gov",
+ "4": "China"
+ }
+ }
+ ],
+ "actions": [
+ "edit",
+ "delete"
+ ]
+ },
+ "entity": [
+ {
+ "field": "name",
+ "label": "Name",
+ "type": "text",
+ "required": true,
+ },
+ {
+ "field": "key_id",
+ "label": "Key ID",
+ "type": "text",
+ },
+ {
+ "field": "category",
+ "label": "Region Category",
+ "type": "singleSelect",
+ "required": true,
+ "defaultValue": 1,
+ "options": {
+ "disableSearch": true,
+ "autoCompleteFields": [
+ {
+ "label": "Global",
+ "value": 1
+ },
+ {
+ "label": "GovCloud",
+ "value": 2
+ },
+ {
+ "label": "China",
+ "value": 4
+ }
+ ]
+ }
+ }
+ ]
+}
+This is how it looks like in the UI:
+
It is possible to extend the default behaviour of the UCC-generated REST handlers.
+For example, if your add-on requires an API key to operate and you want to validate this API key during its creation, you may want to use the custom REST handlers to achieve that.
+++Note that
+ucc-genwill not override the REST handler code if you use correct file name underbinfolder.
Below there an example of how it can be done. It contains a REST handler for creating an organization with organization_id and organization_api_key fields, not custom one, just generated by ucc-gen.
But below that there is a custom handler CustomRestHandler class which has additional steps for configuration creation and edit operations, specifically handleEdit and handleCreate methods.
import import_declare_test
+
+import logging
+
+from splunktaucclib.rest_handler import admin_external, util
+from splunktaucclib.rest_handler.admin_external import AdminExternalHandler
+from splunktaucclib.rest_handler.endpoint import (
+ RestModel,
+ SingleModel,
+ field,
+ validator,
+)
+
+util.remove_http_proxy_env_vars()
+
+
+fields = [
+ field.RestField(
+ "organization_id",
+ required=True,
+ encrypted=False,
+ default=None,
+ validator=validator.AllOf(
+ validator.String(
+ max_len=50,
+ min_len=1,
+ ),
+ validator.Pattern(
+ regex=r"""^\d+$""",
+ ),
+ ),
+ ),
+ field.RestField(
+ "organization_api_key",
+ required=True,
+ encrypted=True,
+ default=None,
+ validator=validator.AllOf(
+ validator.String(
+ max_len=50,
+ min_len=1,
+ ),
+ validator.Pattern(
+ regex=r"""^[a-z0-9]+$""",
+ ),
+ ),
+ ),
+]
+model = RestModel(fields, name=None)
+
+
+endpoint = SingleModel(
+ "addon_name_organization", model, config_name="organization"
+)
+
+
+def _validate_organization(organization_id, organization_api_key):
+ # Some code to validate the API key.
+ # Should return nothing if the configuration is valid.
+ # Should raise an exception splunktaucclib.rest_handler.error.RestError if the configuration is not valid.
+ ...
+
+
+class CustomRestHandler(AdminExternalHandler):
+ def __init__(self, *args, **kwargs):
+ AdminExternalHandler.__init__(self, *args, **kwargs)
+
+ def handleList(self, confInfo):
+ AdminExternalHandler.handleList(self, confInfo)
+
+ def handleEdit(self, confInfo):
+ _validate_organization(
+ self.payload.get("organization_id"),
+ self.payload.get("organization_api_key"),
+ )
+ AdminExternalHandler.handleEdit(self, confInfo)
+
+ def handleCreate(self, confInfo):
+ _validate_organization(
+ self.payload.get("organization_id"),
+ self.payload.get("organization_api_key"),
+ )
+ AdminExternalHandler.handleCreate(self, confInfo)
+
+ def handleRemove(self, confInfo):
+ AdminExternalHandler.handleRemove(self, confInfo)
+
+
+if __name__ == "__main__":
+ logging.getLogger().addHandler(logging.NullHandler())
+ admin_external.handle(
+ endpoint,
+ handler=CustomRestHandler,
+ )
+++UCC 5.18.0 natively supports custom REST handlers for the modular inputs.
+
One of the common scenarios is to delete a checkpoint after you are deleting an +input in the Inputs page. Otherwise, users may face the wierd consequences if +they create an input with the same name as the input that was deleted and this +newly created input will be reusing the old checkpoint because the names of +the inputs are the same. We would like to avoid this situation in the add-on.
+This can be done without a need to modify the REST handler code generated
+automatically by running ucc-gen.
Below is the automatically generated REST handler code for a modular input REST +handler.
+import import_declare_test
+
+from splunktaucclib.rest_handler.endpoint import (
+ field,
+ validator,
+ RestModel,
+ DataInputModel,
+)
+from splunktaucclib.rest_handler import admin_external, util
+from splunktaucclib.rest_handler.admin_external import AdminExternalHandler
+import logging
+
+util.remove_http_proxy_env_vars()
+
+
+fields = [
+ field.RestField(
+ 'interval',
+ required=True,
+ encrypted=False,
+ default=None,
+ validator=validator.Pattern(
+ regex=r"""^\-[1-9]\d*$|^\d*$""",
+ )
+ ),
+
+ field.RestField(
+ 'disabled',
+ required=False,
+ validator=None
+ )
+
+]
+model = RestModel(fields, name=None)
+
+
+
+endpoint = DataInputModel(
+ 'example_input_one',
+ model,
+)
+
+
+if __name__ == '__main__':
+ logging.getLogger().addHandler(logging.NullHandler())
+ admin_external.handle(
+ endpoint,
+ handler=AdminExternalHandler,
+ )
+New file needs to be created in the bin folder of the add-on. Let’s call it
+splunk_ta_uccexample_delete_checkpoint_rh.py (name can be different).
And put the following content into the file.
+import import_declare_test
+
+from splunktaucclib.rest_handler.admin_external import AdminExternalHandler
+
+
+class CustomRestHandlerDeleteCheckpoint(AdminExternalHandler):
+ def __init__(self, *args, **kwargs):
+ AdminExternalHandler.__init__(self, *args, **kwargs)
+
+ def handleList(self, confInfo):
+ AdminExternalHandler.handleList(self, confInfo)
+
+ def handleEdit(self, confInfo):
+ AdminExternalHandler.handleEdit(self, confInfo)
+
+ def handleCreate(self, confInfo):
+ AdminExternalHandler.handleCreate(self, confInfo)
+
+ def handleRemove(self, confInfo):
+ # Add your code here to delete the checkpoint!
+ AdminExternalHandler.handleRemove(self, confInfo)
+Then, in globalConfig file you need to change the behaviour of the UCC to reuse +the REST handler that was just created.
+{
+ "name": "example_input_one",
+ "restHandlerModule": "splunk_ta_uccexample_delete_checkpoint_rh", <----- new field
+ "restHandlerClass": "CustomRestHandlerDeleteCheckpoint", <----- new field
+ "entity": [
+ "..."
+ ],
+ "title": "Example Input One"
+}
+After ucc-gen command is executed again, the generated REST handler for this
+input will be changed to the following.
import import_declare_test
+
+from splunktaucclib.rest_handler.endpoint import (
+ field,
+ validator,
+ RestModel,
+ DataInputModel,
+)
+from splunktaucclib.rest_handler import admin_external, util
+from splunk_ta_uccexample_delete_checkpoint_rh import CustomRestHandlerDeleteCheckpoint # <----- changed
+import logging
+
+util.remove_http_proxy_env_vars()
+
+
+fields = [
+ field.RestField(
+ 'interval',
+ required=True,
+ encrypted=False,
+ default=None,
+ validator=validator.Pattern(
+ regex=r"""^\-[1-9]\d*$|^\d*$""",
+ )
+ ),
+
+ field.RestField(
+ 'disabled',
+ required=False,
+ validator=None
+ )
+
+]
+model = RestModel(fields, name=None)
+
+
+
+endpoint = DataInputModel(
+ 'example_input_one',
+ model,
+)
+
+
+if __name__ == '__main__':
+ logging.getLogger().addHandler(logging.NullHandler())
+ admin_external.handle(
+ endpoint,
+ handler=CustomRestHandlerDeleteCheckpoint, # <----- changed
+ )
+This feature allows dynamic loading options for singleSelect and multipleSelect fields when options for that field depend on other fields’ values. It loads options via API call to the endpoint mentioned in endpointUrl under options when any dependencies field is updated and all required dependencies fields are non-null.
All non-required dependencies fields can be of any type, but all required dependencies fields should only be of single-select type.
+All dependencies fields’ values are added to the endpoint URL as query parameters.
+{
+ "field": "bucket_name",
+ "label": "S3 Bucket",
+ "type": "singleSelect",
+ "required": true,
+ "options": {
+ "disableonEdit": true,
+ "dependencies": [
+ "aws_account",
+ "aws_iam_role",
+ "aws_s3_region",
+ "private_endpoint_enabled",
+ "sts_private_endpoint_url",
+ "s3_private_endpoint_url"
+ ],
+ "endpointUrl": "splunk_ta_aws/splunk_ta_aws_s3buckets"
+ }
+}
+++ + + + + + +Note: When using the text type field, add debounce using the custom hook to reduce the number of API calls.
+
Using this functionality, the Inputs page form can be divided into distinct sections, each comprising relevant fields. If the isExpandable property is set to true in the global config file, the group will be of the collapsible panel type.
The groups will be displayed at the bottom of the form.
+label to display the title of a specific group.fields to specify the list of fields in a group. All fields must be present in the entity.options:isExpandable can be used to hide/show fields of the group. The default value is false.expand to show all fields of the group while opening the form. The default value is false.{
+ "name": "aws_config",
+ "title": "Config",
+ "groups": [
+ {
+ "label": "AWS Input Configuration",
+ "options": {
+ "isExpandable": false
+ },
+ "fields": [
+ "name",
+ "aws_account"
+ ]
+ },
+ {
+ "label": "Splunk-related Configuration",
+ "options": {
+ "isExpandable": false
+ },
+ "fields": [
+ "sourcetype",
+ "index"
+ ]
+ },
+ {
+ "label": "Advanced Settings",
+ "options": {
+ "expand": false,
+ "isExpandable": true
+ },
+ "fields": [
+ "polling_interval"
+ ]
+ }
+ ],
+ "entity": []
+},
+This is how it looks like in the UI:
+
UCC allows you to add Auth support in the configuration page. In UCC, OAuth2.0 of the Authorization Code Flow grant type is being used. It only supports the standard parameters specified in RFCP749 for obtaining an authorization code.
Auth can be used inside the entity tag. Use type: "oauth" in the entity list and also need to specify the options next to the type: "oauth".
type field value must be oauth.options:
auth_type must be present. It can have either [“basic”, “oauth”] (If we want basic and oauth both support) or [“oauth”] (If we want oauth support only).basic This must be present only if auth_type: [“basic”]oauth This will have list of fields you want to add in oauth authentication flow. In the given example, it is client_id, client_secret, redirect_url, scope and endpoint.client_id this is client id for the your app for which you want authclient_secret this is client secret for the your app for which you want authredirect_url this will show redirect url which needs to be put in app’s redirect url.endpoint this will be endpoint for which we want to build oauth support. For example for salesforce that will be either “login.salesforce.com” or “test.salesforce.com” or any other custom endpoint.auth_code_endpoint this must be present and its value should be endpoint value for getting the auth_code using the app. If the url to get auth_code is https://login.salesforce.com/services/oauth2/authorize then this will have value /services/oauth2/authorizeaccess_token_endpoint this must be present and its value should be endpoint value for getting access_token using the auth_code received. If the url to get access token is https://login.salesforce.com/services/oauth2/token then this will have value /services/oauth2/tokenauth_label this allow user to have custom label for Auth Type dropdownoauth_popup_width width in pixels of the popup window that will open for oauth authentication (Optional, defaults to 600)oauth_popup_height height in pixels of the popup window that will open for oauth authentication (Optional, defaults to 600)oauth_timeout timeout in seconds for oauth authentication (Optional, defaults to 180 seconds)oauth_state_enabled to include state for oauth authentication (default value is false)auth_endpoint_token_access_type optional parameter that is mapped into value of token_access_type query param in authorisation url
Fields allowed in basic and oauth fields as of now:
+oauth_field: This should be kept as it is and without any change.label: This can be changed if the user wants to change the label of the field in UI.field: For now this user must keep it as it is for mandatory fields as mentioned above.help : This can be changed if user wants to change the help text displayed below field.encrypted : This should be true if user wants that particular field encrypted else no need to have this parameter.required: To specify whether the field is required or not. The default value is true.defaultValue: The initial input value. (string, number or boolean)options:placeholder: The placeholder for the field.disableonEdit: When the form is in edit mode, the field becomes uneditable. Default value: falseenable: The enable property sets whether a field is enabled or not. Default value: true++[!WARNING] +Placeholder attribute is deprecated and will be removed in one of the following versions. Instead, we recommend using “help” attribute.
+
"configuration": {
+ "title": "Configurations",
+ "description": "Configure your servers and templates.",
+ "tabs": [
+ {
+ "name": "account",
+ "title": "Account",
+ "entity": [
+ {
+ "field": "name",
+ "label": "Name",
+ "type": "text",
+ "required": true,
+ "help": "Enter a unique name for each Crowdstrike falcon host account.",
+ },
+ {
+ "type": "oauth",
+ "field": "oauth",
+ "label": "Not used",
+ "options": {
+ "auth_type": [
+ "basic",
+ "oauth"
+ ],
+ "basic": [
+ {
+ "oauth_field": "username",
+ "label": "User Name",
+ "field": "username",
+ "help": "Enter Account name."
+ },
+ {
+ "oauth_field": "password",
+ "label": "Password",
+ "field": "password",
+ "encrypted": true,
+ "help": "Enter Password."
+ },
+ {
+ "oauth_field": "security_token",
+ "label": "Securtiy Token",
+ "field": "security_token",
+ "encrypted": true,
+ "help": "Enter Security Token."
+ }
+ ],
+ "oauth": [
+ {
+ "oauth_field": "client_id",
+ "label": "Client Id",
+ "field": "client_id",
+ "help": "Enter Client Id."
+ },
+ {
+ "oauth_field": "client_secret",
+ "label": "Client Secret",
+ "field": "client_secret",
+ "encrypted": true,
+ "help": "Enter Client Secret."
+ },
+ {
+ "oauth_field": "redirect_url",
+ "label": "Redirect url",
+ "field": "redirect_url",
+ "help": "Please add this redirect URL in your app."
+ },
+ {
+ "oauth_field": "scope",
+ "label": "Scope",
+ "field": "scope",
+ "help": "Enter the scope for the authorization code with ',' separating each scope.",
+ "required": false
+ },
+ {
+ "oauth_field": "endpoint",
+ "label": "Endpoint",
+ "field": "endpoint",
+ "help": "Enter Endpoint"
+ }
+ ],
+ "auth_label": "Auth Type",
+ "oauth_popup_width": 600,
+ "oauth_popup_height": 600,
+ "oauth_timeout": 180,
+ "auth_code_endpoint": "/services/oauth2/authorize",
+ "access_token_endpoint": "/services/oauth2/token",
+ "auth_endpoint_token_access_type": "offline"
+ }
+ }
+ ],
+ }
+ ]
+}
+This is how the Add Account modal looks after adding the above code to the globalConfig.json file:
+This is how Add Account looks when auth_type is basic:
+
This is how Add Account looks when auth_type is oauth:
+
This feature allows us to download and unpack libraries with appropriate binaries for the indicated operating system during the build process. +To do this, you need to expand the meta section in global config with the os-dependentLibraries field. This field takes the following attributes:
+| Property | +Type | +Description | +default value | +
|---|---|---|---|
| name* | +string | +Name of the library we want to download. | +- | +
| version* | +string | +Specific version of given library. | +- | +
| dependencies | +boolean | +(Optional) Parameter which determines whether the --no-deps flag will be used when installing package from pip. When the value is set to true the library will be installed along with all its dependencies. When the value is set to false (default) {name}={version} must be present in packages requirements.txt. |
+false | +
| platform* | +string | +The platform for which we want to download the specified library. The value depends on the available wheels for a given library e.g. for this wheel cryptography-41.0.5-cp37-abi3-manylinux_2_28_x86_64.whl platform is manylinux_2_28_x86_64. | +- | +
| python_version* | +string | +Python version compatible with the library. | +- | +
| target* | +string | +Path where the selected library will be unpacked. | +- | +
| os* | +string | +The name of the operating system for which the library is intended. Using this parameter, an appropriate insert into sys.path will be created. It takes 3 values windows, linux and darwin. | +- | +
Generally, the wheel name convention is
{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl.
+For example for this particular library:
grpcio-1.54.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
+your pip parameters are:
and your pip command should look like this:
+pip install --no-deps --platform manylinux_2_17_x86_64 --python-version 37 --target your/path/to/target --only-binary=:all: grpcio==1.54.2
A dot in the platform part indicates that a given distribution supports several platforms. +In this case “.” in manylinux_2_17_x86_64.manylinux2014_x86_64 means this distribution supports both manylinux_2_17_x86_64 and manylinux2014_x86_64.
+for more informations, we recommend watching .whl and manylinux platform
+ ...
+ "meta": {
+ "name": "<TA name>",
+ "restRoot": "<restRoot>",
+ "version": "<TA version>",
+ "displayName": "<TA display name>",
+ "schemaVersion": "<schema version>",
+ "os-dependentLibraries": [
+ {
+ "name": "cryptography",
+ "version": "41.0.5",
+ "platform": "manylinux2014_x86_64",
+ "python_version": "37",
+ "os": "linux",
+ "target": "3rdparty/linux"
+ },
+ {
+ "name": "cryptography",
+ "version": "41.0.5",
+ "platform": "win_amd64",
+ "python_version": "37",
+ "os": "windows",
+ "target": "3rdparty/windows"
+ },
+ {
+ "name": "cryptography",
+ "version": "41.0.5",
+ "dependencies": true,
+ "platform": "manylinux2014_x86_64",
+ "python_version": "37",
+ "os": "linux",
+ "target": "3rdparty/linux_with_deps"
+ },
+ {
+ "name": "cffi",
+ "version": "1.15.1",
+ "platform": "win_amd64",
+ "python_version": "37",
+ "os": "windows",
+ "target": "3rdparty/windows"
+ }
+ ]
+ }
+Running the build for the above configuration will result in the creation of the following structure:
+output
+ └──<TA>
+ ├── bin
+ ...
+ └── lib
+ └── 3rdparty
+ ├── linux
+ │ ├── cryptography
+ │ └── cryptography-41.0.5.dist-info
+ ├── linux_with_deps
+ │ ├── _cffi_backend.cpython-37m-x86_64-linux-gnu.so
+ │ ├── cffi
+ │ ├── cffi-1.15.1.dist-info
+ │ ├── cryptography
+ │ ├── cryptography-41.0.5.dist-info
+ │ ├── pycparser
+ │ └── pycparser-2.21.dist-info
+ └── windows
+ ├── _cffi_backend.cp37-win_amd64.pyd
+ ├── cffi
+ ├── cffi-1.15.1.dist-info
+ ├── cryptography
+ └── cryptography-41.0.5.dist-info
+During the build process, a python script “import_declare_test.py” will be created in output/ta_name/bin to manipulate system paths.
+In each input using the specified libraries, this script must be imported.
+Currently, three operating systems are supported: Windows, Linux and Darwin.
+If, for development purposes, there is a need to create other custom manipulations on sys.path,
+create your own script called “import_declare_test.py” and place it in the package/bin folder.
+This way, when building the TA, the default script will be replaced with the one created by the developer.
+The default script for the above configuration will look like this:
import os
+import sys
+import re
+from os.path import dirname
+
+ta_name = 'demo_addon_for_splunk'
+pattern = re.compile(r'[\\/]etc[\\/]apps[\\/][^\\/]+[\\/]bin[\\/]?$')
+new_paths = [path for path in sys.path if not pattern.search(path) or ta_name in path]
+new_paths.insert(0, os.path.join(dirname(dirname(__file__)), "lib"))
+new_paths.insert(0, os.path.sep.join([os.path.dirname(__file__), ta_name]))
+sys.path = new_paths
+
+bindir = os.path.dirname(os.path.realpath(os.path.dirname(__file__)))
+libdir = os.path.join(bindir, "lib")
+platform = sys.platform
+if platform.startswith("linux"):
+ sys.path.insert(0, os.path.join(libdir, "3rdparty/linux_with_deps"))
+ sys.path.insert(0, os.path.join(libdir, "3rdparty/linux"))
+if platform.startswith("win"):
+ sys.path.insert(0, os.path.join(libdir, "3rdparty/windows"))
+This feature allows us to pass a Javascript function as a string to apply customized validation to form data.
+By using this approach, developers can write custom JavaScript code where they can write their business logic and by validating they may return error messages which will be displayed at the top of the form.
+This custom javascript function have a parameter (for ex. dataDict) which contains the form data object.
+This function will be called after all validators have validated the data form.
+{
+ "name": "example_input_one",
+ "title": "Example Input One",
+ "entity": [],
+ "options": {
+ "saveValidator": "function start_data_validator(dataDict) { const provided_datetime = new Date(dataDict['start_date']).getTime(); const current_datetime = new Date().getTime(); if (provided_datetime > current_datetime) { return 'Start date should not be in future'; }}"
+ }
+}
+This feature allows us to pass broarder description on Input and Configuration page displayed under main description.
+| Property | +Type | +Description | +
|---|---|---|
| text* | +string | +Text used for that description, you can put \n to add a breakline | +
| links | +object | +To enable including links inside description | +
| Property | +Type | +Description | +
|---|---|---|
| slug* | +string | +Used to identify place for link to appear, put inside text, surrounded by 2 squared brackets | +
| link* | +string | +Link to be used | +
| linkText* | +string | +Text to be inserted instead of slug | +
{
+ "name": "example_input_one",
+ "title": "Example Input One",
+ "entity": [],
+ "subDescription": {
+ "text": "Ingesting data from to Splunk Cloud?\nRead our [[blogPost]] to learn more about Data Manager and it's availability on your Splunk Cloud instance.",
+ "links": [
+ {
+ "slug": "blogPost",
+ "link": "https://splk.it/31oy2b2",
+ "linkText": "blog post"
+ }
+ ]
+ }
+}
+