Add Kerberos Support

[pkasinathan]

Need to add kerberos support for spark magic to contact kerberized Livy Server.

Jupyter SparkMagic - Kerberos Authentication Error:

[pkasinathan]

Hi @aggFTW,

We've added kerberos support and implemented on our environment. My team is going to send a PR for this change.

Please let me know if there is any code level agreement that we need to sign or we can just contribute.

Thanks
Prabhu

[aggFTW]

Hi @prabhu1984.
Nice to see this! There is nothing to sign, as long as your team understands the license sparkmagic uses.

Keep in mind that the Spark IPython magics is only one way of using the library. Kerberos support should also be added for the kernels via configurations and for the server extension.

This is great to see. Looking forward to the PR!

@jasongrout It'd be great to have Ian and team look at the upcoming PR!

[brockn]

Just saw this, I also implemented Kerberos support over in #314

[pranayhasan]

The #314 together with #282 enables Kerberos support for sparkmagic which we are currently using at Linkedin. Merging these PRs made it possible to connect to Livy. It along with ipywidget's PR for Password makes it work. Is there a reason there is no progress on this?

[pranayhasan]

Also can this be added to v0.3 milestone?

[aggFTW]

@prabhu1984 and @brockn, you there? 😄

@pranayhasan, no reason, they just didn't reply to the threads. Should we close these PRs by Wednesday if there's no more activity from them, and then you could submit one yourself?

[brockn]

My user went a different direction but please go ahead!

[aggFTW]

Thanks @brockn. Closed #314

[praveen-kanamarlapudi]

@aggFTW Are we planning to merge #314 ?

Sorry we missed the updates. Are you waiting for anything from us?

[aggFTW]

It seems @brockn won't work on #314 anymore, so no.

@praveenkanamarlapudi are you with @prabhu1984? If so, feel free to send a PR with your changes. Otherwise, I believe @pranayhasan will submit a PR with their changes.

[praveen-kanamarlapudi]

@aggFTW yah I work with @prabhu1984. Let me rebase the code and make PR in couple of hours.

[praveen-kanamarlapudi]

@aggFTW, from the conversation in #314, I think we are more inclined to keep kerberos ticket management out of sparkmagic.

We implemented #284 keeping kerberos ticket management as part of sparkmagic.

If you want to keep ticket management out of sparkmagic, I will make changes accordingly and send PR. Please let me know and I will make a PR.

[aggFTW]

I think that's a good simplification to start, yes. Looking forward to the PR.

@pranayhasan it would be great if you can help code review/submit further PRs so that you make sure your scenario is covered.

Thank you both!

[pranayhasan]

Sure, thanks @aggFTW. I consolidated #314 with #284 which I pushed to my fork. Here is the link: Kerberos Authentication Consolidated

Looking forward for the PR from @praveenkanamarlapudi

[aggFTW]

Any news @praveenkanamarlapudi?

@pranayhasan, I don't know how pressed for time you are here, but feel free to send a PR tomorrow in case we don't hear anything from @praveenkanamarlapudi or @prabhu1984.

[praveen-kanamarlapudi]

@aggFTW Sorry for the delay. We are working on a PR for livy. So missed to send PR for this task. Let me make it by tomorrow.

[praveen-kanamarlapudi]

Made PR #355

[pranayhasan]

Thanks @praveenkanamarlapudi. I shall review it tonight!

[pranayhasan]

@aggFTW Thanks a lot for the follow up!