Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove user-provided details from various logs #1100

Merged
merged 16 commits into from
Jun 10, 2024

Conversation

davidmrdavid
Copy link
Collaborator

@davidmrdavid davidmrdavid commented May 23, 2024

In various log events, our telemetry captures user-provided details that may contain sensitive information, particularly in the "Details" column. For safety, this PR does two things:
(1) We stop logging the reason for client operations like terminate, suspend, and resume. These are user-provided strings, and they have minimal use during investigations.

(2) When it comes to logging exceptions, we avoid logging the exception message, which may contain sensitive data. To do this, the orchestratorCompleted action now always carries a FailureDetails object, which can be inspected to obtain only the error trace and error type of an exception. This doesn't mean we do not honor the user's choice of error propagation mode, it just means that the FailureDetails property is always populated. Obviously, this may have a memory perf impact, but I found all alternatives to have tricky tradeoffs. Given that FailureDetails is positioned to be the preferred error serialization mode of future SDKs, I think it should be 'ok' to leverage it here.

@davidmrdavid davidmrdavid marked this pull request as ready for review May 23, 2024 22:19
@davidmrdavid davidmrdavid requested a review from cgillum May 23, 2024 22:19
Copy link
Member

@cgillum cgillum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers for me, but I recommend doing a bit of due diligence on the two points raised so far in this PR to ensure we're not regressing anything major.

We'll definitely want to call out this change clearly in the release notes.

@davidmrdavid davidmrdavid requested a review from cgillum June 7, 2024 22:14
Copy link
Member

@cgillum cgillum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach looks sound to me.

}

private Exception exception;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this moved (and no longer readonly)? In C#, the normal convention is to put fields at the top of the class definition.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was moved without realizing (I tried undoing a bunch of changes and failed to move this to the original position). Will restore the readonly as well. Thanks for the convention note.

}

private Exception? exception { get; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: better to use a readonly field instead of a property (and move it to the top of the class definition). ALso, properties don't start with a lowercase letter.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I'll be following the format as the other exception variable in TaskActivityFailure, where this is readonly but not a property. Will move it to the top of the class as well.

src/DurableTask.Core/Logging/LogEvents.cs Outdated Show resolved Hide resolved
@@ -134,6 +134,14 @@ public OrchestrationRuntimeState(IList<HistoryEvent>? events)
/// </remarks>
public FailureDetails? FailureDetails => ExecutionCompletedEvent?.FailureDetails;

/// <summary>
/// Failure that caused an orchestrator to complete, if any.
/// <remarks>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: <remarks> is a sibling to <summary>, not a nested child. See the FailureDetails property above for an example of how to correctly use <remarks>.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out. I wish Visual Studio had flagged this somehow. Im sure there's a way to set that up, but I'll leave that to another day due to time :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be fixed in next commit

@davidmrdavid
Copy link
Collaborator Author

Incorporated feedback, merging as such. Thanks for the prompt reviews!

@davidmrdavid davidmrdavid merged commit 180d14d into main Jun 10, 2024
2 checks passed
@davidmrdavid davidmrdavid deleted the dajusto/remove-details-from-telemetry branch June 10, 2024 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants