Skip to content
This repository has been archived by the owner on Nov 30, 2022. It is now read-only.

1154 email upon privacy request receipt #1303

Merged
merged 13 commits into from
Sep 14, 2022
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ The types of changes are:
* Adds ability to send email notification upon privacy request completion [#1282](https://github.com/ethyca/fidesops/pull/1282)
* Enable new manual webhooks in privacy request execution [#1285](https://github.com/ethyca/fidesops/pull/1285)
* Added human readable label to ConnectionType endpoint [#1297](https://github.com/ethyca/fidesops/pull/1297)
* Adds ability to send email notification upon privacy request receipt [#1303](https://github.com/ethyca/fidesops/pull/1303)
* Add table for consent (#1301)[https://github.com/ethyca/fidesops/pull/1301]

### Docs
Expand Down
1 change: 1 addition & 0 deletions data/config/fidesops.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,4 @@ enabled = true

[notifications]
send_request_completion_notification = true
send_request_receipt_notification = true
pattisdr marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 2 additions & 0 deletions docs/fidesops/docs/guides/configuration_reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ The `fidesops.toml` file should specify the following variables:
|`enabled` | `FIDESOPS__ADMIN_UI__ENABLED` | bool | False | True | Toggle whether the Admin UI is served from `/`
| Fidesops Notification Variables|---|---|---|---|---|
|`send_request_completion_notification` | `FIDESOPS__NOTIFICATIONS__SEND_REQUEST_COMPLETION_NOTIFICATION` | bool | True | True | Whether a notification will be sent to data subjects upon privacy request completion
|`send_request_receipt_notification` | `FIDESOPS__NOTIFICATIONS__SEND_REQUEST_RECEIPT_NOTIFICATION` | bool | True | True | Whether a notification will be sent to data subjects upon privacy request receipt


### An example `fidesops.toml` configuration file
Expand Down Expand Up @@ -118,6 +119,7 @@ enabled = true

[notifications]
send_request_completion_notification = true
send_request_receipt_notification = true
```

Note: The configuration is case-sensitive, so the variables must be specified in `lowercase`.
Expand Down
1 change: 1 addition & 0 deletions docs/fidesops/docs/guides/email_communications.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ Supported modes of use:

- Subject Identity Verification - sends a verification code to the user's email address prior to processing a subject request. For more information on identity verification, see the [Privacy Requests](privacy_requests.md#subject-identity-verification) guide.
- Erasure Request Email Fulfillment - sends an email to configured third parties to process erasures for a given data subject. See [creating email Connectors](#email-third-party-services) for more information.
- Privacy Request Receipt Notification - sends an email to user's email address with privacy request receipt notification.
- Privacy Request Completion Notification - sends an email to user's email address with privacy request completion notification, including a download link to data package, for access requests. For more information on request completion notification, see the [Privacy Requests](privacy_requests.md#request-completion-notification) guide.

## Prerequisites
Expand Down
18 changes: 15 additions & 3 deletions docs/fidesops/docs/guides/privacy_requests.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,24 @@ to continue privacy request execution. Until the Privacy Request identity is ve
```


## Request Completion Notification
## Request Notifications

By default, a request completion email will be sent to users, along with a link to download their data, if applicable. To change this behavior, set the `send_request_completion_notification`
variable in your `fidesops.toml`. You must also set up an [EmailConfig](./email_communications.md) that lets fidesops send automated emails
By default, emails will be sent to users at various points in the request lifecycle.

To change this default behavior for any email type, set the variables under the `notifications` category in your `fidesops.toml`.

You must also set up an [EmailConfig](./email_communications.md) that lets fidesops send automated emails
to your users. If using a custom privacy center, ensure that you intake an email identity, which is required for email notifications throughout fidesops.

### Request Receipt

An email will be sent to users to notify them that their privacy request has been received.

### Request Completion

Upon access request completion, an email will be sent to users to notify them of request completion, along with a link to download their data, if applicable.


!!! Note
For security purposes, the data package download link is a one-time link and expires in 24 hrs by default. To change TTL, update the `subject_request_download_link_ttl_seconds`
variable in your `fidesops.toml`.
Expand Down
1 change: 1 addition & 0 deletions fidesops.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,4 @@ enabled = true

[notifications]
send_request_completion_notification = false
send_request_receipt_notification = false
61 changes: 57 additions & 4 deletions src/fidesops/ops/api/v1/endpoints/privacy_request_endpoints.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# pylint: disable=too-many-branches,too-many-locals,too-many-lines
# pylint: disable=too-many-branches,too-many-locals,too-many-lines, too-many-statements

import csv
import io
Expand Down Expand Up @@ -60,8 +60,10 @@
from fidesops.ops.common_exceptions import (
EmailDispatchException,
FunctionalityNotConfigured,
IdentityNotFoundException,
IdentityVerificationException,
NoCachedManualWebhookEntry,
PolicyNotFoundException,
TraversalError,
ValidationError,
)
Expand All @@ -73,7 +75,7 @@
from fidesops.ops.models.datasetconfig import DatasetConfig
from fidesops.ops.models.email import EmailConfig
from fidesops.ops.models.manual_webhook import AccessManualWebhook
from fidesops.ops.models.policy import CurrentStep, Policy, PolicyPreWebhook
from fidesops.ops.models.policy import ActionType, CurrentStep, Policy, PolicyPreWebhook
from fidesops.ops.models.privacy_request import (
ExecutionLog,
PrivacyRequest,
Expand All @@ -87,6 +89,7 @@
from fidesops.ops.schemas.email.email import (
EmailActionType,
FidesopsEmail,
RequestReceiptBodyParams,
SubjectIdentityVerificationBodyParams,
)
from fidesops.ops.schemas.external_https import PrivacyRequestResumeFormat
Expand All @@ -103,7 +106,10 @@
RowCountRequest,
VerificationCode,
)
from fidesops.ops.service.email.email_dispatch_service import dispatch_email_task
from fidesops.ops.service.email.email_dispatch_service import (
dispatch_email,
dispatch_email_task,
)
from fidesops.ops.service.privacy_request.request_runner_service import (
generate_id_verification_code,
queue_privacy_request,
Expand Down Expand Up @@ -232,7 +238,10 @@ async def create_privacy_request(
)
created.append(privacy_request)
continue # Skip further processing for this privacy request

if config.notifications.send_request_receipt_notification:
_send_privacy_request_receipt_email_to_user(
db, policy, privacy_request_data.identity.email
)
pattisdr marked this conversation as resolved.
Show resolved Hide resolved
if not config.execution.require_manual_request_approval:
AuditLog.create(
db=db,
Expand Down Expand Up @@ -299,6 +308,40 @@ def _send_verification_code_to_user(
)


def _send_privacy_request_receipt_email_to_user(
db: Session, policy: Optional[Policy], email: Optional[str]
) -> None:
"""Helper function to send request receipt email to the user"""
if not email:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this function will error if no email is provided, should the function be optional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to keep email optional here so that this helper function can encapsulate handling err cases for params needed to send an email. Is that what you meant?

logger.error(
IdentityNotFoundException(
"Identity email was not found, so request receipt email could not be sent."
)
)
return
if not policy:
logger.error(
PolicyNotFoundException(
"Policy was not found, so request receipt email could not be sent."
)
)
return
request_types: Set[str] = set()
for action_type in ActionType:
if policy.get_rules_for_action(action_type=ActionType(action_type)):
request_types.add(action_type)
try:
dispatch_email(
db=db,
action_type=EmailActionType.PRIVACY_REQUEST_RECEIPT,
to_email=email,
email_body_params=RequestReceiptBodyParams(request_types=request_types),
)
except EmailDispatchException as exc:
# catch early since this failure isn't fatal to privacy request, unlike the subject id verification email
logger.info("Email dispatch failed with exception %s", exc)


def privacy_request_csv_download(
db: Session, privacy_request_query: Query
) -> StreamingResponse:
Expand Down Expand Up @@ -1095,8 +1138,18 @@ async def verify_identification_code(
)
try:
privacy_request.verify_identity(db, provided_code.code)
policy: Optional[Policy] = Policy.get(
db=db, object_id=privacy_request.policy_id
)
if config.notifications.send_request_receipt_notification:
_send_privacy_request_receipt_email_to_user(
db, policy, privacy_request.get_persisted_identity().email
)
pattisdr marked this conversation as resolved.
Show resolved Hide resolved
except IdentityVerificationException as exc:
raise HTTPException(status_code=HTTP_400_BAD_REQUEST, detail=exc.message)
except EmailDispatchException as exc:
# not fatal to request lifecycle, do not raise error, continue with request
logger.info("Email dispatch failed with exception %s", exc)
except PermissionError as exc:
logger.info("Invalid verification code provided for %s.", privacy_request.id)
raise HTTPException(status_code=HTTP_403_FORBIDDEN, detail=exc.args[0])
Expand Down
1 change: 1 addition & 0 deletions src/fidesops/ops/core/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,7 @@ class FidesopsNotificationSettings(FidesSettings):
"""Configuration settings for data subject and/or data processor notifications"""

send_request_completion_notification: Optional[bool] = True
send_request_receipt_notification: Optional[bool] = True

class Config:
env_prefix = "FIDESOPS__NOTIFICATIONS__"
Expand Down
3 changes: 3 additions & 0 deletions src/fidesops/ops/email_templates/get_email_template.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
EMAIL_ERASURE_REQUEST_FULFILLMENT,
PRIVACY_REQUEST_COMPLETE_ACCESS_TEMPLATE,
PRIVACY_REQUEST_COMPLETE_DELETION_TEMPLATE,
PRIVACY_REQUEST_RECEIPT_TEMPLATE,
SUBJECT_IDENTITY_VERIFICATION_TEMPLATE,
)
from fidesops.ops.schemas.email.email import EmailActionType
Expand All @@ -27,6 +28,8 @@ def get_email_template(action_type: EmailActionType) -> Template:
return template_env.get_template(SUBJECT_IDENTITY_VERIFICATION_TEMPLATE)
if action_type == EmailActionType.EMAIL_ERASURE_REQUEST_FULFILLMENT:
return template_env.get_template(EMAIL_ERASURE_REQUEST_FULFILLMENT)
if action_type == EmailActionType.PRIVACY_REQUEST_RECEIPT:
return template_env.get_template(PRIVACY_REQUEST_RECEIPT_TEMPLATE)
if action_type == EmailActionType.PRIVACY_REQUEST_COMPLETE_ACCESS:
return template_env.get_template(PRIVACY_REQUEST_COMPLETE_ACCESS_TEMPLATE)
if action_type == EmailActionType.PRIVACY_REQUEST_COMPLETE_DELETION:
Expand Down
1 change: 1 addition & 0 deletions src/fidesops/ops/email_templates/template_names.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
SUBJECT_IDENTITY_VERIFICATION_TEMPLATE = "subject_identity_verification.html"
EMAIL_ERASURE_REQUEST_FULFILLMENT = "erasure_request_email_fulfillment.html"
PRIVACY_REQUEST_RECEIPT_TEMPLATE = "privacy_request_receipt.html"
PRIVACY_REQUEST_COMPLETE_DELETION_TEMPLATE = "privacy_request_complete_deletion.html"
PRIVACY_REQUEST_COMPLETE_ACCESS_TEMPLATE = "privacy_request_complete_access.html"
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Privacy Request Received</title>
</head>
<body>
<main>
{% if request_types|length > 1 %}
<p>
The following requests have been received:
</p>
<ul>
{% for type in request_types %}
<li> {{type}} </li>
{% endfor %}
</ul>
{% else %}
<p>
Your {{request_types[0]}} request has been received
</p>
{% endif %}
</main>
</body>
</html>
7 changes: 7 additions & 0 deletions src/fidesops/ops/schemas/email/email.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ class EmailActionType(str, Enum):
# verify email upon acct creation
SUBJECT_IDENTITY_VERIFICATION = "subject_identity_verification"
EMAIL_ERASURE_REQUEST_FULFILLMENT = "email_erasure_fulfillment"
PRIVACY_REQUEST_RECEIPT = "privacy_request_receipt"
PRIVACY_REQUEST_COMPLETE_ACCESS = "privacy_request_complete_access"
PRIVACY_REQUEST_COMPLETE_DELETION = "privacy_request_complete_deletion"

Expand Down Expand Up @@ -58,6 +59,12 @@ class FidesopsEmail(
]


class RequestReceiptBodyParams(BaseModel):
"""Body params required for privacy request receipt email template"""

request_types: List[str]


class AccessRequestCompleteBodyParams(BaseModel):
"""Body params required for privacy request completion access email template"""

Expand Down
3 changes: 1 addition & 2 deletions src/fidesops/ops/service/connectors/email_connector.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@
from fidesops.ops.service.connectors.query_config import ManualQueryConfig
from fidesops.ops.service.email.email_dispatch_service import dispatch_email
from fidesops.ops.util.collection_util import Row, append
from fidesops.ops.util.logger import Pii

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -77,7 +76,7 @@ def test_connection(self) -> Optional[ConnectionTestStatus]:
],
)
except EmailDispatchException as exc:
logger.info("Email connector test failed with exception %s", Pii(exc))
logger.info("Email connector test failed with exception %s", exc)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated this, too, to let the email dispatch service handle the PII masking for these exceptions

return ConnectionTestStatus.failed
return ConnectionTestStatus.succeeded

Expand Down
10 changes: 9 additions & 1 deletion src/fidesops/ops/service/email/email_dispatch_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
EmailServiceSecrets,
EmailServiceType,
FidesopsEmail,
RequestReceiptBodyParams,
SubjectIdentityVerificationBodyParams,
)
from fidesops.ops.tasks import DatabaseTask, celery_app
Expand Down Expand Up @@ -51,6 +52,7 @@ def dispatch_email(
Union[
AccessRequestCompleteBodyParams,
SubjectIdentityVerificationBodyParams,
RequestReceiptBodyParams,
List[CheckpointActionRequired],
]
] = None,
Expand Down Expand Up @@ -106,6 +108,12 @@ def _build_email(
{"dataset_collection_action_required": body_params}
),
)
if action_type == EmailActionType.PRIVACY_REQUEST_RECEIPT:
base_template = get_email_template(action_type)
return EmailForActionType(
subject="Your request has been received",
body=base_template.render({"request_types": body_params.request_types}),
)
if action_type == EmailActionType.PRIVACY_REQUEST_COMPLETE_ACCESS:
base_template = get_email_template(action_type)
return EmailForActionType(
Expand Down Expand Up @@ -167,4 +175,4 @@ def _mailgun_dispatcher(
)
except Exception as e:
logger.error("Email failed to send: %s", Pii(str(e)))
raise EmailDispatchException(f"Email failed to send due to: {e}")
raise EmailDispatchException(f"Email failed to send due to: {Pii(e)}")
Loading