Zendesk connector: add ability to skip tickets that include tags defined in env var, exclude deleted tickets server side

[jwbrown77]

Description

@yuhongsun96 - apologies if I am tagging the wrong person, but you seem to be the most active committer to the Zendesk connector.

I have a use case where I would like to ingest Zendesk tickets, however I need to be able to filter out certain tickets based on if they have a configured tag present.

There is a similar functionality with Zendesk articles that allows users to filter out articles based on certain labels.

This PR:

• Adds a new optional config: ZENDESK_CONNECTOR_SKIP_TICKET_TAGS - a comma separated list of tags that when any are present, the ingestion should skip.
• Add an if statement to the connector that performs this check, then continues to the next ticket iteration loop if the condition is met
• Unrelated, but according to Zendesk's Incremental Ticket API docs (https://developer.zendesk.com/api-reference/ticketing/ticket-management/incremental_exports/), an optional parameter called exclude_deleted can be set to true to have their API not return deleted tickets. This should be more efficient than performing the check client side.

How Has This Been Tested?

• I tested with no change. It worked, but as designed, according to "Explorer", it pulled all tickets from the configured start date.
• I deleted what it ingested.
• I applied my patch and tested again, and the result was a lower number of tickets. I spot checked several tickets and none of them had the tags I had configured in the environment variable.
• In addition, I manually tested the if statement in a simple script and verified it was behaving as desired:

ignore = ["ignore_this", "ignore_that"]

tags = ["test", "ignore_that"]

if any(tag in ignore for tag in tags):
    print("should ignore")
else:
    print("should not ignore")

Accepted Risk (provide if relevant)

N/A

Related Issue(s) (provide if relevant)

N/A

Mental Checklist:

• All of the automated tests pass
• All PR comments are addressed and marked resolved
• If there are migrations, they have been rebased to latest main
• If there are new dependencies, they are added to the requirements
• If there are new environment variables, they are added to all of the deployment methods
• If there are new APIs that don't require auth, they are added to PUBLIC_ENDPOINT_SPECS
• Docker images build and basic functionalities work
• Author has done a final read through of the PR right before merge

Backporting (check the box to trigger backport action)

Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.

• This PR should be backported (make sure to check that the backport attempt succeeds)

[vercel]

@jwbrown77 is attempting to deploy a commit to the Danswer Team on Vercel.

A member of the Team first needs to authorize it.