[SOAR-18543] Palo Alto Cortex XDR

[ablakley-r7]

Proposed Changes

Description

Describe the proposed changes:

• Update query start and end time logic to utilise end time of query in non pagination runs. This is to prevent duplicate events being continually processed and raised as new events.
• Update error handling to return error data in response
• Update unit tests for new pagination logic
• Update custom config to read similarly to other plugins

PR Requirements

Developers, verify you have completed the following items by checking them off:

Testing

Unit Tests

Review our documentation on generating and writing plugin unit tests

• Unit tests written for any new or updated code

In-Product Tests

If you are an InsightConnect customer or have access to an InsightConnect instance, the following in-product tests should be done:

• Screenshot of job output with the plugin changes
• Screenshot of the changed connection, actions, or triggers input within the InsightConnect workflow builder

Style

Review the style guide

• For dependencies, pin OS package and Python package versions
• For security, set least privileged account with USER nobody in the Dockerfile when possible
• For size, use the slim SDK images when possible: rapid7/insightconnect-python-3-38-slim-plugin:{sdk-version-num} and rapid7/insightconnect-python-3-38-plugin:{sdk-version-num}
• For error handling, use of PluginException and ConnectionTestException
• For logging, use self.logger
• For docs, use changelog style
• For docs, validate markdown with insight-plugin validate which calls icon_validate to lint help.md

Functional Checklist

• Work fully completed
• Functional
  • Any new actions/triggers include JSON test files in the tests/ directory created with insight-plugin samples
  • Tests should all pass unless it's a negative test. Negative tests have a naming convention of tests/$action_bad.json
  • Unsuccessful tests should fail by raising an exception causing the plugin to die and an object should be returned on successful test
  • Add functioning test results to PR, sanitize any output if necessary
    • Single action/trigger insight-plugin run -T tests/example.json --debug --jq
    • All actions/triggers shortcut insight-plugin run -T all --debug --jq (use PR format at end)
  • Add functioning run results to PR, sanitize any output if necessary
    • Single action/trigger insight-plugin run -R tests/example.json --debug --jq
    • All actions/triggers shortcut insight-plugin run --debug --jq (use PR format at end)

Assessment

You must validate your work to reviewers:

1. Run insight-plugin validate and make sure everything passes
2. Run the assessment tool: insight-plugin run -A. For single action validation: insight-plugin run tests/{file}.json -A
3. Copy (insight-plugin ... | pbcopy) and paste the output in a new post on this PR
4. Add required screenshots from the In-Product Tests section

[joneill-r7]

if we've passed an alert limit into the custom config this could break this logic? is it better to use the lookback type key we've used in other cases?

[ablakley-r7]

You're correct, we should enter this if there is an alert_limit present in the custom config but we're not currently doing a backfill.

[joneill-r7]

It's such a big update so sorry if some of the questions aren't needed! just want to make sure I'm following everything and we're not using resources/time when we don't need to be

[joneill-r7]

should we pass this into get_query_times, we call this as soon as get_query_times gets entered.

[joneill-r7]

I think we can just do - if any(lookback_values) and it iterates for us?

[joneill-r7]

we seem to be doing a lot of converting back and forward. would it be easier to follow keeping this as a datetime obj up until we then decide what start_time we want to keep? and then one conversion of datetime_obj -> unix?

[joneill-r7]

if we're changing the state time at all, should we not reset the search_from and search_to?

[joneill-r7]

do we have an upper limit on this LAST_SEARCH_TO ? want to make sure we don't somehow page so high that we hit an offset limit on their API

[joneill-r7]

in this loop, should we only add the new hash if it's not deduped? saves the amount of values being saved to the state and also how many are being checked against on the next run?

[joneill-r7]

I know this isn't changing the logic, but do we think we need to dedupe when we're paging? if we're using offsets we could maybe speed up the task execution time by skipping the dedupe and hashing until we're on the first and last page?