This API is deprecated and will eventually be removed. We recommend you use ListClusters, DescribeCluster, ListSteps, ListInstanceGroups and ListBootstrapActions instead.
DescribeJobFlows returns a list of job flows that match all of the supplied parameters. The parameters can include a list of job flow IDs, job flow states, and restrictions on job flow creation date and time.
Regardless of supplied parameters, only job flows created within the last two months are returned.
If no parameters are supplied, then job flows matching either of the following criteria are returned:
Job flows created and completed in the last two weeks
Job flows created within the last two months that are in one of the following states: RUNNING, WAITING, SHUTTING_DOWN, STARTING
Amazon EMR can return a maximum of 512 job flow descriptions.
", "deprecated":true }, + "DescribeNotebookExecution":{ + "name":"DescribeNotebookExecution", + "http":{ + "method":"POST", + "requestUri":"/" + }, + "input":{"shape":"DescribeNotebookExecutionInput"}, + "output":{"shape":"DescribeNotebookExecutionOutput"}, + "errors":[ + {"shape":"InternalServerError"}, + {"shape":"InvalidRequestException"} + ], + "documentation":"Provides details of a notebook execution.
" + }, "DescribeSecurityConfiguration":{ "name":"DescribeSecurityConfiguration", "http":{ @@ -259,6 +273,20 @@ ], "documentation":"Provides information for all active EC2 instances and EC2 instances terminated in the last 30 days, up to a maximum of 2,000. EC2 instances in any of the following states are considered active: AWAITING_FULFILLMENT, PROVISIONING, BOOTSTRAPPING, RUNNING.
" }, + "ListNotebookExecutions":{ + "name":"ListNotebookExecutions", + "http":{ + "method":"POST", + "requestUri":"/" + }, + "input":{"shape":"ListNotebookExecutionsInput"}, + "output":{"shape":"ListNotebookExecutionsOutput"}, + "errors":[ + {"shape":"InternalServerError"}, + {"shape":"InvalidRequestException"} + ], + "documentation":"Provides summaries of all notebook executions. You can filter the list based on multiple criteria such as status, time range, and editor id. Returns a maximum of 50 notebook executions and a marker to track the paging of a longer notebook execution list across multiple ListNotebookExecution calls.
Sets the Cluster$VisibleToAllUsers value, which determines whether the cluster is visible to all IAM users of the AWS account associated with the cluster. Only the IAM user who created the cluster or the AWS account root user can call this action. The default value, true, indicates that all IAM users in the AWS account can perform cluster actions if they have the proper IAM policy permissions. If set to false, only the IAM user that created the cluster can perform actions. This action works on running clusters. You can override the default true setting when you create a cluster by using the VisibleToAllUsers parameter with RunJobFlow.
Starts a notebook execution.
" + }, + "StopNotebookExecution":{ + "name":"StopNotebookExecution", + "http":{ + "method":"POST", + "requestUri":"/" + }, + "input":{"shape":"StopNotebookExecutionInput"}, + "errors":[ + {"shape":"InternalServerError"}, + {"shape":"InvalidRequestException"} + ], + "documentation":"Stops a notebook execution.
" + }, "TerminateJobFlows":{ "name":"TerminateJobFlows", "http":{ @@ -1309,6 +1364,25 @@ }, "documentation":"The output for the DescribeJobFlows operation.
" }, + "DescribeNotebookExecutionInput":{ + "type":"structure", + "required":["NotebookExecutionId"], + "members":{ + "NotebookExecutionId":{ + "shape":"XmlStringMaxLen256", + "documentation":"The unique identifier of the notebook execution.
" + } + } + }, + "DescribeNotebookExecutionOutput":{ + "type":"structure", + "members":{ + "NotebookExecution":{ + "shape":"NotebookExecution", + "documentation":"Properties of the notebook execution.
" + } + } + }, "DescribeSecurityConfigurationInput":{ "type":"structure", "required":["Name"], @@ -1497,6 +1571,29 @@ "min":1 }, "ErrorMessage":{"type":"string"}, + "ExecutionEngineConfig":{ + "type":"structure", + "required":["Id"], + "members":{ + "Id":{ + "shape":"XmlStringMaxLen256", + "documentation":"The unique identifier of the execution engine. For an EMR cluster, this is the cluster ID.
" + }, + "Type":{ + "shape":"ExecutionEngineType", + "documentation":"The type of execution engine. A value of EMR specifies an EMR cluster.
An optional unique ID of an EC2 security group to associate with the master instance of the EMR cluster for this notebook execution. For more information see Specifying EC2 Security Groups for EMR Notebooks in the EMR Management Guide.
" + } + }, + "documentation":"Specifies the execution engine (cluster) to run the notebook and perform the notebook execution, for example, an EMR cluster.
" + }, + "ExecutionEngineType":{ + "type":"string", + "enum":["EMR"] + }, "FailureDetails":{ "type":"structure", "members":{ @@ -2889,6 +2986,44 @@ }, "documentation":"This output contains the list of instances.
" }, + "ListNotebookExecutionsInput":{ + "type":"structure", + "members":{ + "EditorId":{ + "shape":"XmlStringMaxLen256", + "documentation":"The unique ID of the editor associated with the notebook execution.
" + }, + "Status":{ + "shape":"NotebookExecutionStatus", + "documentation":"The status filter for listing notebook executions.
START_PENDING indicates that the cluster has received the execution request but execution has not begun.
STARTING indicates that the execution is starting on the cluster.
RUNNING indicates that the execution is being processed by the cluster.
FINISHING indicates that execution processing is in the final stages.
FINISHED indicates that the execution has completed without error.
FAILING indicates that the execution is failing and will not finish successfully.
FAILED indicates that the execution failed.
STOP_PENDING indicates that the cluster has received a StopNotebookExecution request and the stop is pending.
STOPPING indicates that the cluster is in the process of stopping the execution as a result of a StopNotebookExecution request.
STOPPED indicates that the execution stopped because of a StopNotebookExecution request.
The beginning of time range filter for listing notebook executions. The default is the timestamp of 30 days ago.
" + }, + "To":{ + "shape":"Date", + "documentation":"The end of time range filter for listing notebook executions. The default is the current timestamp.
" + }, + "Marker":{ + "shape":"Marker", + "documentation":"The pagination token, returned by a previous ListNotebookExecutions call, that indicates the start of the list for this ListNotebookExecutions call.
A list of notebook executions.
" + }, + "Marker":{ + "shape":"Marker", + "documentation":"A pagination token that a subsequent ListNotebookExecutions can use to determine the next set of results to retrieve.
The EC2 unit limits for a managed scaling policy. The managed scaling activity of a cluster is not allowed to go above or below these limits. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.
" + "documentation":"The EC2 unit limits for a managed scaling policy. The managed scaling activity of a cluster is not allowed to go above or below these limits. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.
" } }, "documentation":"Managed scaling policy for an Amazon EMR cluster. The policy specifies the limits for resources that can be added or terminated from a cluster. The policy only applies to the core and task nodes. The master node cannot be scaled after initial configuration.
" @@ -3047,6 +3182,113 @@ "type":"double", "min":0.0 }, + "NotebookExecution":{ + "type":"structure", + "members":{ + "NotebookExecutionId":{ + "shape":"XmlStringMaxLen256", + "documentation":"The unique identifier of a notebook execution.
" + }, + "EditorId":{ + "shape":"XmlStringMaxLen256", + "documentation":"The unique identifier of the EMR Notebook that is used for the notebook execution.
" + }, + "ExecutionEngine":{ + "shape":"ExecutionEngineConfig", + "documentation":"The execution engine, such as an EMR cluster, used to run the EMR notebook and perform the notebook execution.
" + }, + "NotebookExecutionName":{ + "shape":"XmlStringMaxLen256", + "documentation":"A name for the notebook execution.
" + }, + "NotebookParams":{ + "shape":"XmlString", + "documentation":"Input parameters in JSON format passed to the EMR Notebook at runtime for execution.
" + }, + "Status":{ + "shape":"NotebookExecutionStatus", + "documentation":"The status of the notebook execution.
START_PENDING indicates that the cluster has received the execution request but execution has not begun.
STARTING indicates that the execution is starting on the cluster.
RUNNING indicates that the execution is being processed by the cluster.
FINISHING indicates that execution processing is in the final stages.
FINISHED indicates that the execution has completed without error.
FAILING indicates that the execution is failing and will not finish successfully.
FAILED indicates that the execution failed.
STOP_PENDING indicates that the cluster has received a StopNotebookExecution request and the stop is pending.
STOPPING indicates that the cluster is in the process of stopping the execution as a result of a StopNotebookExecution request.
STOPPED indicates that the execution stopped because of a StopNotebookExecution request.
The timestamp when notebook execution started.
" + }, + "EndTime":{ + "shape":"Date", + "documentation":"The timestamp when notebook execution ended.
" + }, + "Arn":{ + "shape":"XmlStringMaxLen256", + "documentation":"The Amazon Resource Name (ARN) of the notebook execution.
" + }, + "OutputNotebookURI":{ + "shape":"XmlString", + "documentation":"The location of the notebook execution's output file in Amazon S3.
" + }, + "LastStateChangeReason":{ + "shape":"XmlString", + "documentation":"The reason for the latest status change of the notebook execution.
" + }, + "NotebookInstanceSecurityGroupId":{ + "shape":"XmlStringMaxLen256", + "documentation":"The unique identifier of the EC2 security group associated with the EMR Notebook instance. For more information see Specifying EC2 Security Groups for EMR Notebooks in the EMR Management Guide.
" + }, + "Tags":{ + "shape":"TagList", + "documentation":"A list of tags associated with a notebook execution. Tags are user-defined key value pairs that consist of a required key string with a maximum of 128 characters and an optional value string with a maximum of 256 characters.
" + } + }, + "documentation":"A notebook execution. An execution is a specific instance that an EMR Notebook is run using the StartNotebookExecution action.
The unique identifier of the notebook execution.
" + }, + "EditorId":{ + "shape":"XmlStringMaxLen256", + "documentation":"The unique identifier of the editor associated with the notebook execution.
" + }, + "NotebookExecutionName":{ + "shape":"XmlStringMaxLen256", + "documentation":"The name of the notebook execution.
" + }, + "Status":{ + "shape":"NotebookExecutionStatus", + "documentation":"The status of the notebook execution.
START_PENDING indicates that the cluster has received the execution request but execution has not begun.
STARTING indicates that the execution is starting on the cluster.
RUNNING indicates that the execution is being processed by the cluster.
FINISHING indicates that execution processing is in the final stages.
FINISHED indicates that the execution has completed without error.
FAILING indicates that the execution is failing and will not finish successfully.
FAILED indicates that the execution failed.
STOP_PENDING indicates that the cluster has received a StopNotebookExecution request and the stop is pending.
STOPPING indicates that the cluster is in the process of stopping the execution as a result of a StopNotebookExecution request.
STOPPED indicates that the execution stopped because of a StopNotebookExecution request.
The timestamp when notebook execution started.
" + }, + "EndTime":{ + "shape":"Date", + "documentation":"The timestamp when notebook execution started.
" + } + }, + "documentation":"" + }, + "NotebookExecutionSummaryList":{ + "type":"list", + "member":{"shape":"NotebookExecutionSummary"} + }, "OnDemandProvisioningAllocationStrategy":{ "type":"string", "enum":["lowest-price"] @@ -3084,7 +3326,7 @@ "Port":{ "type":"integer", "max":65535, - "min":0 + "min":-1 }, "PortRange":{ "type":"structure", @@ -3607,6 +3849,58 @@ "TERMINATE_CLUSTER" ] }, + "StartNotebookExecutionInput":{ + "type":"structure", + "required":[ + "EditorId", + "RelativePath", + "ExecutionEngine", + "ServiceRole" + ], + "members":{ + "EditorId":{ + "shape":"XmlStringMaxLen256", + "documentation":"The unique identifier of the EMR Notebook to use for notebook execution.
" + }, + "RelativePath":{ + "shape":"XmlString", + "documentation":"The path and file name of the notebook file for this execution, relative to the path specified for the EMR Notebook. For example, if you specify a path of s3://MyBucket/MyNotebooks when you create an EMR Notebook for a notebook with an ID of e-ABCDEFGHIJK1234567890ABCD (the EditorID of this request), and you specify a RelativePath of my_notebook_executions/notebook_execution.ipynb, the location of the file for the notebook execution is s3://MyBucket/MyNotebooks/e-ABCDEFGHIJK1234567890ABCD/my_notebook_executions/notebook_execution.ipynb.
An optional name for the notebook execution.
" + }, + "NotebookParams":{ + "shape":"XmlString", + "documentation":"Input parameters in JSON format passed to the EMR Notebook at runtime for execution.
" + }, + "ExecutionEngine":{ + "shape":"ExecutionEngineConfig", + "documentation":"Specifies the execution engine (cluster) that runs the notebook execution.
" + }, + "ServiceRole":{ + "shape":"XmlString", + "documentation":"The name or ARN of the IAM role that is used as the service role for Amazon EMR (the EMR role) for the notebook execution.
" + }, + "NotebookInstanceSecurityGroupId":{ + "shape":"XmlStringMaxLen256", + "documentation":"The unique identifier of the Amazon EC2 security group to associate with the EMR Notebook for this notebook execution.
" + }, + "Tags":{ + "shape":"TagList", + "documentation":"A list of tags associated with a notebook execution. Tags are user-defined key value pairs that consist of a required key string with a maximum of 128 characters and an optional value string with a maximum of 256 characters.
" + } + } + }, + "StartNotebookExecutionOutput":{ + "type":"structure", + "members":{ + "NotebookExecutionId":{ + "shape":"XmlStringMaxLen256", + "documentation":"The unique identifier of the notebook execution.
" + } + } + }, "Statistic":{ "type":"string", "enum":[ @@ -3849,6 +4143,16 @@ }, "documentation":"The timeline of the cluster step lifecycle.
" }, + "StopNotebookExecutionInput":{ + "type":"structure", + "required":["NotebookExecutionId"], + "members":{ + "NotebookExecutionId":{ + "shape":"XmlStringMaxLen256", + "documentation":"The unique identifier of the notebook execution.
" + } + } + }, "String":{"type":"string"}, "StringList":{ "type":"list",