Update Google Cloud Log Parser to include protoPayload.status

[tmoney-coder]

Describe the problem:

The current parser for Google Cloud logs is missing a critical log section, the protoPayload.status. This contains valuable information when present, including why a operation failed.

"status": {
            "code": 7,
            "message": "Permission 'iam.serviceAccounts.create' denied on resource (or it may not exist).",
            "details": [
                {
                    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
                    "reason": "IAM_PERMISSION_DENIED",
                    "domain": "iam.googleapis.com",
                    "metadata": {
                        "permission": "iam.serviceAccounts.create"
                    }
                }
            ]
        }

A great example of this is when a compromised account attempts an operation they do not have permission for. The severity will show "ERROR" in the current parser but it doesn't give any reason why. The original log from GC would have a protoPayload.status.message field that would contain the actual reason, along with a protoPayload.status.details[0].reason field. Both of these fields should be added to parsers/jsonl_plugins/gcp_log.py

To Reproduce:

Create an account in Google Cloud and then perform an action that the account doesn't have permission to perform, such as creating a service account.

Then download the logs with dftimewolf or in the raw JSON format.

Pass the json file to log2timeline.py

Review the json data from the plaso file, the status data is missing

Below is a sample of a message that can be used to test the parsing

{"logName": "projects/THEPROJECTHERE/logs/cloudaudit.googleapis.com%2Factivity", "resource": {"type": "service_account", "labels": {"unique_id": "", "email_id": "", "project_id": "THEPROJECTHERE"}}, "insertId": "12awewewe", "severity": "ERROR", "timestamp": "2024-10-16T20:05:20.755382Z", "protoPayload": {"@type": "type.googleapis.com/google.cloud.audit.AuditLog", "status": {"code": 7, "message": "Permission 'iam.serviceAccounts.create' denied on resource (or it may not exist).", "details": [{"@type": "type.googleapis.com/google.rpc.ErrorInfo", "reason": "IAM_PERMISSION_DENIED", "domain": "iam.googleapis.com", "metadata": {"permission": "iam.serviceAccounts.create"}}]}, "authenticationInfo": {"principalEmail": "[email protected]", "serviceAccountDelegationInfo": [{"firstPartyPrincipal": {"principalEmail": "[email protected]"}}], "principalSubject": "serviceAccount:[email protected]"}, "requestMetadata": {"callerIp": "34.125.178.195", "callerSuppliedUserAgent": "(gzip),gzip(gfe)", "requestAttributes": {"time": "2024-10-16T20:05:20.819805391Z", "auth": {}}, "destinationAttributes": {}}, "serviceName": "iam.googleapis.com", "methodName": "google.iam.admin.v1.CreateServiceAccount", "authorizationInfo": [{"resource": "projects/THEPROJECTHERE", "permission": "iam.serviceAccounts.create", "resourceAttributes": {"type": "iam.googleapis.com/ServiceAccount"}, "permissionType": "ADMIN_WRITE"}], "resourceName": "projects/THEPROJECTHERE", "request": {"@type": "type.googleapis.com/google.iam.admin.v1.CreateServiceAccountRequest", "account_id": "theattacker", "name": "projects/THEPROJECTHERE", "service_account": {"display_name": "This is the attacker account"}}, "response": {"@type": "type.googleapis.com/google.iam.admin.v1.ServiceAccount"}}}

Here is example code to add to the parser

status = self._GetJSONValue(proto_payload, 'status')
    if status:
        event_data.status_message = self._GetJSONValue(status,'message')

I added this to line 133 and tested. The following is the output of the test. Notice that the status_message is populated now.

{
    "__container_type__": "event",
    "__type__": "AttributeContainer",
    "data_type": "gcp:log:entry",
    "date_time": {
        "__class_name__": "TimeElementsInMicroseconds",
        "__type__": "DateTimeValues",
        "time_elements_tuple": [
            2024,
            10,
            16,
            20,
            5,
            20,
            755382
        ],
        "time_zone_offset": 0
    },
    "display_name": "OS:/data/input/tmpj60bncyu.jsonl",
    "event_subtype": "google.iam.admin.v1.CreateServiceAccount",
    "filename": "/data/input/tmpj60bncyu.jsonl",
    "inode": "-",
    "log_name": "projects/THEPROJECTHERE/logs/cloudaudit.googleapis.com%2Factivity",
    "message": "User [email protected] performed google.iam.admin.v1.CreateServiceAccount on projects/THEPROJECTHERE",
    "parser": "jsonl/gcp_log",
    "pathspec": {
        "__type__": "PathSpec",
        "location": "/data/input/tmpj60bncyu.jsonl",
        "type_indicator": "OS"
    },
    "request_account_identifier": "theattacker",
    "request_metadata": [
        "callerIp: 34.125.178.195",
        "callerSuppliedUserAgent: (gzip),gzip(gfe)",
        "requestAttributes: {'time': '2024-10-16T20:05:20.819805391Z', 'auth': {}}",
        "destinationAttributes: {}"
    ],
    "request_name": "projects/THEPROJECTHERE",
    "resource_labels": [
        "unique_id: ",
        "email_id: ",
        "project_id: THEPROJECTHERE"
    ],
    "resource_name": "projects/THEPROJECTHERE",
    "service_account_display_name": "This is the attacker account",
    "service_name": "iam.googleapis.com",
    "severity": "ERROR",
    "sha256_hash": "61a28cb38efecc36c68de5c8c1413b832e5f10a8627b5652a2c5c161891f7675",
    "status_message": "Permission 'iam.serviceAccounts.create' denied on resource (or it may not exist).",
    "timestamp": 1729109120755382,
    "timestamp_desc": "Recorded Time",
    "user": "[email protected]"
}