SparkContext causes failed status in YARN cluster mode

[im281]

17/09/28 13:59:40 INFO Client:
client token: N/A
diagnostics: Shutdown hook called before final status was reported.
ApplicationMaster host: 172.31.1.110
ApplicationMaster RPC port: 0
queue: default
start time: 1506605178944
final status: FAILED

The application runs and prints out the results

[im281]

This was run on an AWS cluster BTW

[skaarthik]

What version of Mobius are you using?

[im281]

2.02 on the AWS cluster

…

________________________________ From: Kaarthik Sivashanmugam <[email protected]> Sent: Sunday, October 1, 2017 9:18:54 PM To: Microsoft/Mobius Cc: im281; Author Subject: Re: [Microsoft/Mobius] SparkContext causes failed status in YARN cluster mode (#669) What version of Mobius are you using? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#669 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AJnUcEXPS_vuy4KnMpLx7rnbBz_f89k2ks5soGQugaJpZM4PnUU9>.

[im281]

sorry we are using Spark version 2.02 that works with the latest Mobius

[skaarthik]

Do you get this error in both client and cluster mode deployments? (--deploy-mode client or --deploy-mode cluster in sparkclr-submit)

[im281]

Yes in both client and cluster mode

…

On Thu, Oct 12, 2017 at 10:15 AM Kaarthik Sivashanmugam < ***@***.***> wrote: Do you get this error in both client and cluster mode deployments? (--deploy-mode client or --deploy-mode cluster in sparkclr-submit) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#669 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJnUcOumFwJKGrGaSxtcuagGTwCfJBBbks5srkkbgaJpZM4PnUU9> .

[im281]

Mobius only works on Spark 2.02? On Thu, Oct 12, 2017 at 10:56 AM Iman Mohtashemi <[email protected]> wrote:

…

Yes in both client and cluster mode On Thu, Oct 12, 2017 at 10:15 AM Kaarthik Sivashanmugam < ***@***.***> wrote: > Do you get this error in both client and cluster mode deployments? (--deploy-mode > client or --deploy-mode cluster in sparkclr-submit) > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#669 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AJnUcOumFwJKGrGaSxtcuagGTwCfJBBbks5srkkbgaJpZM4PnUU9> > . >