Add HTML repr to scheduler_info and incorporate into client and cluster reprs

[jacobtomlinson]

Scheduler info

This PR builds upon #4853 and adds an HTML repr to Client.scheduler_info() and cluster.scheduler_info.

Everything is drawn with basic HTML and some inline CSS styling. Most of the heavy lifting is being done by the Jupyter CSS.

I've made heavy use of f-strings throughout but perhaps switching to jinja for HTML reprs may be a better choice in the long run as it will hugely simplify code and duplication. Plus allow for partial templating and reuse. But that would add jinja2 as a dependency so may be better to have a separate discussion about that.

Before

After

Client

I've then expanded the Client HTML repr to add a little info about the client and then reuse the scheduler_info repr.

The client info shows how the client was connected, either with a cluster object, a scheduler file or directly with an address. It also shows more information about the cluster object and the path to the scheduler file if applicable.

Before

After

Cluster

Lastly I've updated the cluster HTML repr and widget. The repr got the same treatment as Client and adds a little cluster specific info and then reuses scheduler_info.

The Cluster class adds minimal cluster info but adds a kwarg to _repr_html_ so that classes lower down can add their own info. I imagine for KubeCluster we may add some Kubernetes specific info, for EC2Cluster some AWS specific info, etc.

For the ipywidget I've added a tabs view where the first tab reuses the HTML repr and the second tab contains the scaling controls which existed before. This should bring some consistency to folks with/without ipywidgets installed.

Before (without ipywidgets)

After (without ipywidgets)

Before (with ipywidgets)

After (with ipywidgets)

[fjetter]

That's amazing

[jacobtomlinson]

It should all handle dark mode too.

[mrocklin]

I think that this is great. Thank you @jacobtomlinson . Two questions:

1. Does this break anything downstream? Presumably things that used to be strings are now more complex objects.

   We probably can't know this for sure, and I think that it's worth the breakage, but I just wanted this concern voiced before we smash the merge button.

2. What else can we do here? While we have you on a roll with HTML reprs are there other things we should consider augmenting?

[jacobtomlinson]

@mrocklin

1. The scheduler_info dict objects are now derived dict objects with an HTML repr. So they still behave as dicts. All the other changes are to HTML reprs. My hope here is that not too much gets broken downstream.
2. I've found this a pretty satisfying exercise. It's not particularly difficult but has high visibility and high impact. I'm happy to keep building things like this for other objects from time to time. If folks have suggestions I'm very happy to hear them.

[jacobtomlinson]

My biggest worry here is putting HTML inside strings inside Python. It's not particularly easy to parse with your eyes due to a lack of syntax highlighting.

String manipulation gets messy quickly.

I think a next step might be to refactor all of this out into some jinja2 templates. That would make building more things in the future easier and most importantly more maintainable.

[mrocklin]

Sounds good to me :)

…

On Thu, May 27, 2021 at 9:02 AM Jacob Tomlinson ***@***.***> wrote: My biggest worry here is putting HTML inside strings inside Python. It's not particularly easy to parse with your eyes due to a lack of syntax highlighting. String manipulation gets messy quickly. I think a next step might be to refactor all of this out into some jinja2 templates. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4857 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTFOT6BG7UGAKKZS3BDTPZGIJANCNFSM45T62VTA> .

[jrbourbeau]

This looks great, thanks for all your work here @jacobtomlinson : )

My biggest worry here is putting HTML inside strings inside Python. It's not particularly easy to parse with your eyes due to a lack of syntax highlighting.

Agreed that HTML strings in Python are difficult to parse. Incorporating some kind of templating would be a welcomed improvement, though can probably be handled in a follow-up PR

[jacobtomlinson]

I've nudged things here to support async for the Client and Cluster objects. There was a small issue if you tried to view the repr before awaiting them.

I expect there may be a small conflict when #4865 is merged. But once resolved this should be good to go.

[GenevieveBuckley]

Hey @jacobtomlinson I notice that the light grey text colour fails the contrast accessibility check here on a white background. Can we update that so it's readable? (Potentially we'll need to be different shades of grey for light vs dark mode backgrounds, not super sure about that.)

[mrocklin]

@jacobtomlinson checking in here. I'd love to see this in. It looks like the who_has/has_what stuff was merged. Is it the right time to gently nudge you on this one? :)

[jacobtomlinson]

Yeah I'll take a look at this today.

[jacobtomlinson]

Failed tests are now passing for me locally so 🤞🏻 this is ready for review.

[jrbourbeau]

Thanks @jacobtomlinson! This looks great : )

[jacobtomlinson]

Thanks @jrbourbeau for the test fixes and the review. Provided CI passes this should be good to merge.

[jacobtomlinson]

Test failure seems flaky, rerunning.

[jrbourbeau]

Thanks @jacobtomlinson!

(test failures are unrelated)