DPU hours for individual glue-job runs with aws-cli/1.18.69

[jusatdeloitte]

Hi, I'd like to get the data that is displayed under the "Job run monitoring" section in the console. Specifically I need the column 'DPU hours' from the table "Job runs". I wasn't able to find an API for this. Thanks for your help.

[RyanFitzSimmonsAK]

Hi @jusatdeloitte, thanks for reaching out. You can get DPU hours by using get-job-run on any job with auto scaling enabled. It'll be returned as DPUSeconds in the response. Hope this helps, and let me know if you have any follow-up questions!

[jusatdeloitte]

Thanks for your concise response @RyanFitzSimmonsAK . Below is an example where I try this approach. However, I didn't discover DPUSeconds. Can you see what I'm doing wrong here?

$ aws glue get-job-run --job-name my_job_name --run-id jr_123456780bb552f0fb324157807a17ac3ac281655d7248df2e69743cbaa31281
{
    "JobRun": {
        "Id": "jr_123457852f0fb324157807a17ac3ac281655d7248df2e69743cbaa31281",
        "Attempt": 0,
        "JobName": "my_job_name",
        "StartedOn": 1687793708.493,
        "LastModifiedOn": 1687793745.846,
        "CompletedOn": 1687793745.846,
        "JobRunState": "STOPPED",
        "Arguments": {
            "--enable-auto-scaling": "true"
        },
        "PredecessorRuns": [],
        "AllocatedCapacity": 32,
        "ExecutionTime": 30,
        "Timeout": 300,
        "MaxCapacity": 32.0,
        "WorkerType": "G.1X",
        "NumberOfWorkers": 32,
        "LogGroupName": "/aws-glue/jobs",
        "GlueVersion": "4.0"
    }
}

[RyanFitzSimmonsAK]

You probably need to update your CLI version, since you're on a heavily outdated version. I'd recommend going up to CLI v2 if possible.

Migration guide - https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration.html
Updating guide - https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

Let me know how that works for you.

[jusatdeloitte]

Nice, it was a version issue. I didn't need to go all the way to v2 to make it work though. Thanks for your help!