MC-380 - PaloAlto Wildfire - Connection Test

[dmandyna-r7]

Proposed Changes

Description

Describe the proposed changes:

• Added connection test
• Removed unsupported key words
• Added supported_version

Jira Link

https://issues.corp.rapid7.com/browse/MC-380

PR Requirements

Developers, verify you have completed the following items by checking them off:

Testing

Unit Tests

Review our documentation on generating and writing plugin unit tests

• Unit tests written for any new or updated code

In-Product Tests

If you are an InsightConnect customer or have access to an InsightConnect instance, the following in-product tests should be done:

• Screenshot of job output with the plugin changes
• Screenshot of the changed connection, actions, or triggers input within the InsightConnect workflow builder

Style

Review the style guide

• For dependencies, pin OS package and Python package versions
• For security, set least privileged account with USER nobody in the Dockerfile when possible
• For size, use the slim SDK images when possible: komand/python-3-37-slim-plugin and komand/python-3-37-plugin
• For error handling, use of PluginException and ConnectionTestException
• For logging, use self.logger
• For docs, use changelog style
• For docs, validate markdown with make validate which calls mdl to lint help.md

Functional Checklist

• Work fully completed
• Functional
  • Any new actions/triggers include JSON test files in the tests/ directory created with icon-plugin run -c sample $action > tests/$action.json
  • Tests should all pass unless it's a negative test. Negative tests have a naming convention of tests/$action_bad.json
  • Unsuccessful tests should fail by raising an exception causing the plugin to die and an object should be returned on successful test
  • Add functioning test results to PR, sanitize any output if necessary
    • Single action/trigger icon-plugin run -T tests/example.json --debug --jq
    • All actions/triggers shortcut icon-plugin run -T all --debug --jq (use PR format at end)
  • Add functioning run results to PR, sanitize any output if necessary
    • Single action/trigger icon-plugin run -R tests/example.json --debug --jq
    • All actions/triggers shortcut icon-plugin run -R all --debug --jq (use PR format at end)

Assessment

Run

{
  "body": {
    "log": "Connect: Connecting..\nrapid7/Palo Alto https://example.com Step name: get_report\n",
    "meta": {},
    "output": {
      "success": true
    },
    "status": "ok"
  },
  "type": "action_event",
  "version": "v1"
}

docker run --rm -i rapid7/paloalto_wildfire:1.2.0 --debug test < tests/get_report.json

Autogenerate with:

{
  "body": {
    "error": "Connection test failed!\n\nInvalid API key provided. Verify your API key configured in your connection is correct. Response was: 'Invalid apikey or expired apikey'",
    "log": "Connect: Connecting..\nrapid7/Palo Alto https://example.com Step name: get_report\nConnection test failed!\n\nInvalid API key provided. Verify your API key configured in your connection is correct. Response was: 'Invalid apikey or expired apikey'\nTraceback (most recent call last):\n  File \"/usr/local/lib/https://example.com\", line 29, in test\n    https://example.com\"https://example.com\")\n  File \"/usr/local/lib/https://example.com\", line 302, in submit_urls\n    response = https://example.com, data=data, files=data)\n  File \"/usr/local/lib/https://example.com\", line 581, in post\n    return https://example.com'POST', url, data=data, json=json, **kwargs)\n  File \"/usr/local/lib/https://example.com\", line 533, in request\n    resp = https://example.com, **send_kwargs)\n  File \"/usr/local/lib/https://example.com\", line 653, in send\n    r = dispatch_hook('response', hooks, r, **kwargs)\n  File \"/usr/local/lib/https://example.com\", line 31, in dispatch_hook\n    _hook_data = hook(hook_data, **kwargs)\n  File \"/usr/local/lib/https://example.com\", line 80, in _raise_errors\n    raise WildFireException(results[\"error\"][\"error-message\"])\https://example.com 'Invalid apikey or expired apikey'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/https://example.com\", line 311, in handle_step\n    output = https://example.com['body'], 'action', logger, log_stream, is_test, is_debug)\n  File \"/usr/local/lib/https://example.com\", line 421, in start_step\n    output = func()\n  File \"/usr/local/lib/https://example.com\", line 31, in test\n    raise ConnectionTestException(https://example.com, data=e)\https://example.com Connection test failed!\n\nInvalid API key provided. Verify your API key configured in your connection is correct. Response was: 'Invalid apikey or expired apikey'\n",
    "meta": {},
    "status": "error"
  },
  "type": "action_event",
  "version": "v1"
}

docker run --rm -i rapid7/paloalto_wildfire:1.2.0 --debug test < tests/get_report_bad.json

Plugin Validation

[*] Validating plugin at .

[*] Running Integration Validators...
[*] Executing validator HelpValidator
[*] Executing validator ChangelogValidator
[*] Executing validator CloudReadyConnectionCredentialTokenValidator
[*] Executing validator RequiredKeysValidator
[*] Executing validator UseCaseValidator
[*] Executing validator SpecPropertiesValidator
[*] Executing validator SpecVersionValidator
[*] Executing validator FilesValidator
[*] Executing validator TagValidator
[*] Executing validator DescriptionValidator
[*] Executing validator TitleValidator
[*] Executing validator VendorValidator
[*] Executing validator DefaultValueValidator
[*] Executing validator IconValidator
[*] Executing validator RequiredValidator
[*] Executing validator VersionValidator
[*] Executing validator DockerfileParentValidator
[*] Executing validator ProfanityValidator
[*] Executing validator AcronymValidator
[*] Executing validator JSONValidator
[*] Executing validator OutputValidator
[*] Executing validator RegenerationValidator
[*] Executing validator HelpInputOutputValidator
[*] Executing validator SupportValidator
[*] Executing validator RuntimeValidator
[*] Executing validator VersionPinValidator
[*] Executing validator EncodingValidator
[*] Executing validator ExampleInputValidator
[*] Executing validator CloudReadyValidator
[*] Executing validator SupportedVersionValidator
[*] Executing validator UnapprovedKeywordsValidator
[*] Executing validator HelpExampleValidator
[*] Plugin failed validation! The following validation errors occurred:

Validator "VersionPinValidator" failed! 
        Cause: All Python dependencies must be version pinned. Please update all modules in requirements.txt with a specific version pin e.g. lxml==3.7.1

Validator "EncodingValidator" failed! 
        Cause: A forbidden character(s) was found in the 'description' field of the spec.yaml file: {'‐'}
                A forbidden character(s) was found in the 'description' field of the spec.yaml file: {'‐'}
                A forbidden character(s) was found in the 'description' field of the spec.yaml file: {'‐'}
                A forbidden character(s) was found in the 'enum' field of the spec.yaml file: {'‐'}
                A forbidden character(s) was found in the 'description' field of the spec.yaml file: {'‐'}


----
[*] Total time elapsed: 1612.274ms

icon-validate --all .

In-product Testing

[dmandyna-r7]

I was unable to perform in-product testing, kept getting below error when trying to import the plugin:

[mberezin-r7]

Please regenerate the plugin with icon-plugin so that it may add the supported version to this help.md section.

[mrinehart-r7]

I was unable to perform in-product testing, kept getting below error when trying to import the plugin:

I ran into this myself - I tried your branch as well as what's in master and neither imported properly. Let's work to figure out what is causing this to happen.

[dmandyna-r7]

Hey @mrinehart-r7,
I've managed to figure it out, there was a bunch of invalid characters in the plugin-spec.yml, I've fixed them and managed to add the plugin and do the in-product testing. I've attached the screenshots to the PR overview. Let me know if you have any questions :)
Thanks!

[mrinehart-r7]

Hey @mrinehart-r7, I've managed to figure it out, there was a bunch of invalid characters in the plugin-spec.yml, I've fixed them and managed to add the plugin and do the in-product testing. I've attached the screenshots to the PR overview. Let me know if you have any questions :) Thanks!

@dmandyna-r7 nice work! I'm not seeing the in-product screenshots though - can you double-check you attached them?

Edit: I fixed the version pin validator failure - can you double-check those are the right versions (I used PyPi.org to check)? Also, the unit test is failing because it can't find the JSON file, can you update the test to include mocked data?

[dmandyna-r7]

[dmandyna-r7]

Hey @mrinehart-r7,

I re-wrote the unit test case, as the one that already existed was failing because it was expecting valid creds in the ./tests/submit_file.json file. I've updated the unit test to use a mocked response from Wildfire, to ensure we are handling unsupported formats correctly for that action.

Additionally, I've applied black formatting to an action in Jira plugin (last commit) - the validators were failing due to it not being formatted correctly.

Let me know if you have any comments 🙂

Thank you!

[mrinehart-r7]

Hey @mrinehart-r7,

I re-wrote the unit test case, as the one that already existed was failing because it was expecting valid creds in the ./tests/submit_file.json file. I've updated the unit test to use a mocked response from Wildfire, to ensure we are handling unsupported formats correctly for that action.

Additionally, I've applied black formatting to an action in Jira plugin (last commit) - the validators were failing due to it not being formatted correctly.

Let me know if you have any comments 🙂

Thank you!

Perfect, thank you @dmandyna-r7 !