[SIP-30] Remove Tabs in SQL Lab

[suddjian]

[SIP-30] Remove Tabs in SQL Lab

Motivation

SQL Lab is devoting screen space to a custom tab UI that is less effective than that offered natively by the browser. Other upcoming features such as more powerful edit history/undo, and better navigation/discovery make tabs less important. The new design directions coming from the superset design group specify a tabless UI. Browser-based tabs would be a better solution for nearly every SQL Lab use case.

Proposed Change

• Remove the tab bar in SQL Lab
• Include the id of the current queryEditor in the SQL Lab url: myapp.com/superset/sqllab/42
• Links throughout Superset referencing SQL Lab queries should use urls with a queryEditor id
• Implement a user-level flag to switch between the tabful and tabless UI
• Stabilize backend-managed SQL Lab state, and make it the default option.

This change can actually be made without significant changes to state management. SQL Lab state includes some number of queryEditors ("tabs"). Instead of each queryEditor referencing an in-page tab, the SQL Lab url will contain the id of the queryEditor being used.

This will depend on the backend implementation of state management for SQL Lab which is currently behind a feature flag. Without backend SQL Lab state, your local state would accumulate until there is too much to manage, without a practical way to clean it besides clearing your localStorage or editing it laboriously.

One change that will be tricky is offering users a quality migration experience to the tabless UI. More details on that down the page.

A note on preventing lost work in a tabless UI

Nobody wants to write a query, then close their browser and lose it forever. Mitigating that in this new design will be important. The full UI/UX redesigns call for a robust history feature, but there are some things we can do to protect people's work without the history feature.

When you navigate to SQL Lab without specifying an id in the URL, an id will be generated for you just as if you had clicked the "+ tab" button in the current UI. This will allow users to re-open a closed tab from the browser history, even if they didn't save the query. If a user wrote a query but didn't save it, and closes the browser tab, SQLLab should also warn and offer to save the query before the tab closes. If you run or save your query, those will also be ways to recover your work.

Just for reference, here's the design we're working towards. This SIP is not about implementing everything on this screen, only replacing the in-page tabs with browser tabs.

New or Changed Public Interfaces

• "Run query in a new tab" button in the query history will open a new browser tab.
• "Share query" should use the new url format, and should save the query if it hasn't been saved.
• Links in the saved queries screen and query search should use the new url format.
• Remove the ctrl-t keyboard shortcut?

Migration Plan and Compatibility

We should allow users to migrate to the new system at their own pace. Tabs are a heavily-used feature, and should not be ripped out without a proper transition phase to educate and re-orient users.

A per-user feature flag will be added indicating whether they are using the tabful or tabless version of SQLLab. SQLLab will appear unchanged as long as a user's flag is off, with the exception of a small banner in the corner notifying the user of the new tabless experience, and allowing them to flip the flag on. Once they flip the flag SQLLab will change to the tabless UI. There should be an explanation of the change, with useful info on how to navigate this brave new world.

After some time, we can turn this flag on by default for new users. Eventually when the tabless UI is deemed stable, people are happy with the change, and we've made sure that there isn't a negative impact to existing user needs, we can switch everyone over.

Rejected Alternatives

• Keeping in-page tabs - too cumbersome to use, not as accessible, adds unnecessary state complexity
• Switching over immediately to a tabless UI without a feature flag - too jarring to users, want to give them the opportunity to change to the new system when it's convenient for them.
• Developing the new SQL Lab designs in a separate screen alongside the original SQL Lab - too much code complexity and chance of breaking changes, not enough opportunity to reuse existing components.
• Re-organizing SQL Lab frontend state around a tabless UI - too much work for now, and breaks compatibility between the new and old version of the page. It will likely make sense to re-organize state in a future release once all users have migrated to the tabless UI.

[issue-label-bot]

Issue-Label Bot is automatically applying the label #enhancement to this issue, with a confidence of 0.81. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[dpgaspar]

Can you rename this to SIP-30 ?

[suddjian]

Oops! Thanks

[toop]

This good ideal ！！！We are looking forward to your revised ！Thank you very much. @suddjian

[cgivre]

Personally, I do not like this as I like to have multiple queries open for reference. This would get a -1 from me.

[suddjian]

Well the idea here is to better support native browser tab workflows so that people can still have multiple queries open with a less complicated UI. Do you think that could be a good option for your use case? Is there specific value you see in page-based tabs that can't be done with browser tabs?

[mistercrunch]

About change management, we did something similar for both "explore v2" and "dashboard v2", where in the case of "dashboard v2", the dashboards had to be validated (and in some cases tweaked) manually individually.

Things can get intricate as components, endpoints and objects may become shared in intricate ways. For example, the work of deprecating "explore v1" became fairly intricate as it was sharing things with v2. Clearly there are pros and cons here, and if we decide to have 2 versions overlap, we should be deliberate on whether we want to go with a "not DRY" (where code is duplicated to evolve independently) or a tangled approach (where both versions share components and objects).

One thing to mention is that the design is not very clear around state management, and making sure people don't loose work. Personally I'd push towards a "Google Docs"-type pattern, forcing people to title the session quickly, and autosaving as the query is executed. We want to make sure that whatever happens to the browser tab, no work will be lost. We should have clear answers to "what happens if I have 6 tabs open, close the browser, and want to get back to some of the tabs that were opened before? what if I had not explicitly saved? is a tab really a query session? can I restore the session if I close the tab, or just the latest state?

[suddjian]

I've edited the SIP with the latest updates to this proposal. I think this addresses the concerns that have been raised.

[robdiciuccio]

The primary goal of state management in SQL Lab seems to be to ensure queries are not lost, though I'd love to see more feedback on this point. This could be solved for by a robust query history mechanism, as currently exists, or like the one proposed in the screenshot above from SIP-34. Creating a unique ID for a SQL Lab browser tab could be useful for grouping queries by session, or to retain some additional state upon page reload, though using UUIDs for this would be preferable. We'll also need to differentiate between the SQL Lab session ID and a saved query ID, as these represent separate objects.

This open PR is also relevant to the query sharing conversation.

[michellethomas]

To clarify, this SIP is to enable the start of removing tabs (feature flag), but we are not voting on the end result of what sqllab would look like without tabs?

As part of this SIP it would be useful to describe what the process will be to get feedback from users and align on the end result for sqllab designs and when that will happen. I know some of the use cases were already discussed in the cartel meetings, but since this will be a big change, it would be nice to get feedback from current superset users to make sure the designs cover their use cases.

Can you also add details about what the migration strategy should be for existing superset deployments that have tabs? Once we switch over all of the existing queries in tabs will be lost right?, so we'll need to do some work to encourage users to save their queries because tabs will be going away. Are their ways to have a smoother transition, maybe an option to have warnings in the app before people switch over?

[suddjian]

@michellethomas

The redesign of sqllab is being treated as an ever-evolving north star to align everyone on while developing Superset. This SIP is just about how we can go about achieving a tabless UI.

I agree that feedback from users will be highly useful in implementing the new designs, but I think that's out of scope for this SIP. There are other ongoing efforts to set up user feedback channels.

Migration is an important part of this work, since this has such a large impact on users. My proposal as outlined in the SIP is to give each user a button to switch to a tabless UI. Until the tabless UI is deemed "stable", there will be an option to switch back to tabs, with your tab state kept intact. Before you decide to go tabless, we can have a message describing the change to the user. We will need to explain that saving queries is the best way to avoid lost work, but you will also be able to use the sqllab history feature or your browser history to find past queries.

@robdiciuccio

I think you're correct regarding state management. The way I'm thinking of this change is to use the existing tab ID in the same way that you're describing a unique session ID. I think migrating to UUIDs or something similar is a good idea. We can differentiate between ID types with different URL paths, perhaps /sqllab/saved/:id and /sqllab/session/:id.

[sylvia-tomiyama]

+1 to all of the questions above. As the folks on this thread are aware, Tabs are a very popular feature in our deployment so we want to be careful about changes and want to make sure we know what we're voting on. (I also recognize that folks have iterated on the proposal based on feedback, and I appreciate it :))

A few additional questions I have:
(1) Will users have the option to flip back to the tabbed version if they don't like the tabless version?
(2) If most users choose not to migrate, and/or if they flip back, what will we do then? I assume our goal with this transition period is to ensure that we're making the experience better for users, so I'd like a clear process and commitment to address major points of feedback if we find that most people are opting out of this.
(3) What's the value proposition for the user? As context, tabs are a heavily-used feature in our deployment, though people also have frustrations with it. Ideally we'll be able to say why/how this is a net improvement for users. Right now, to me it seems like there's an idea of a much better experience in the long-term but this is an intermediate step that doesn't add clear value for users that creates work/requires users to deal with change. In general, I think we should ensure that when we require people to migrate, the benefits should clearly outweigh the costs and for me right now this isn't super clear. So I'd love to have this spelled out in terms that users can understand (e.g. -- does this solve the problem of losing in-progress work when local storage is cleared? Does it make it easier to access the in-progress work?)
(4) If the added value isn't clear, then what are things we could add/modify to this proposal so that it does? For example -- making it easier to find saved queries and/or your query history; automatically adding each new tab you create to your saved queries; etc.
(5) Can you help me understand whether the proposed will have any regressions relative to the current experience / make sure that the proposal will meet the use cases people have for tabs today? Mainly, people use tabs as a way to (1) organize their work and (2) save in-progress work. A common scenario is that someone may have two or more different subject areas they are running ad hoc analysis on, and that they are working on over a period of days or even weeks/months. I'm not 100% certain after reading your proposal (and its modifications) a few times, so I'd appreciate your help.
(6) If this is a necessary step for the long-term plan, and that's our primary/only reason taking this step (e.g. if this middle step isn't a clearly better experience for users but we're doing it because it's our only way to transition to the ultimate experience), then I'd like clarity/to first vote on what the long-term plan is.

Also, apologies that I didn't send this sooner -- I had a lengthy comment half-written that I lost :(

[michellethomas]

Thanks for the clarification on the migration process, that makes sense.

[suddjian]

Thanks for all the feedback @sylvia-tomiyama! I think I can address all of these, but let me know if I missed anything.

1. In short, yes. This is addressed in the migration section and in my other comment above.

2. A goal of allowing users to switch back is for them to be comfortable with the change. You're right that users could potentially choose not to switch, but I don't think anyone can offer an answer to that at this stage. We'll need to see what happens, gather feedback, and understand why users have made their choice before we can respond effectively. It may just be an issue of messaging, or a problem with the implementation, a missing feature people need, or something else entirely.

3. I admit that "remove tabs" doesn't sound like it has value for users, but nestled within that proposal is "support web browser navigation patterns", which offers a lot of value. Web browsers have more effective methods of organization and navigation than are available in SQL Lab. Many users we've spoken with have a knee-jerk reaction against removing tabs initially, but conclude that it is a good change after further reflection. I agree that the benefits should be demonstrated to outweigh the costs before users would be required to switch, which is the motivation for the migration strategy. The language around that could use some clarification though, so I'll work on that.

4. I think removing tabs has sufficient value on its own. Saving your query automatically may be warranted, but it may also clutter users' saved query lists with work that wasn't meant to be saved. Maybe a list view of your SQL Lab "sessions" could serve that purpose? An advantage of the migration strategy is that if such changes are necessary, we can build them while continuing to user-test the tabless SQL Lab in parallel.

5. I recommend saving your query if it's something you'll be working on for an extended period of time, even if we don't remove tabs. An issue with tabs is they appear to be persistent but they are actually quite brittle and unstable. As for organization, native browser tabs and windows will offer better features than the current UI does. A unique URL for each session also allows using other means of organization such as browser bookmarks, spreadsheets or other documents.

6. While this is a necessary step for the long-term plan, I also think that SQL Lab would function better without tabs than with tabs, today. Discussions regarding the long-term plan should happen in SIP-34.

[sylvia-tomiyama]

Thanks for clarifying! Makes a lot of sense.

(1)/(2) My goal was to confirm that we'd use the feedback/experience during this period to make any changes, and I think that your answer shows we will, so you have my +1 on the migration process. Thanks :)

(3)/(4)/(5) I talked to a couple users at our company and I think I understand why switching from SQL lab tabs to browser tabs could help with organization, but I'm not totally convinced it's enough value to counteract the loss of the current ease of preserving and re-accessing in-progress work. I really like your suggestion to show a list of SQL lab "sessions" and think this would would tip this into the "obvious net positive for users" category.

Here's my thought process on this: I want to reiterate that saving in-progress work is really important. We ran some in-depth research on Superset with users at our company, and this was one of the key painpoints (losing work) that came up that contributed to a lot of mistrust in Superset. In a world where people are used to working with things like google docs (and honestly even pre-cloud word autosaved), I feel like the baseline expectation is that all in-progress work is saved automatically.

That's why re: #5 I understand this brittleness with tabs and have been advocating for addressing it for a while now. Even though we provide the same suggestion you do to our users, every time the local storage gets cleared for some reason or another, we get frustrated and surprised users complaining that their work has been lost.

So I really like your suggestion of providing a list view of your SQL lab sessions (assuming that your sessions get auto-saved, which I think it would since we generate a unique session id each time?) -- it seems like it would address both the permanence issue AND also make it easier to manage the clutter of your work. (I think that searching through browsing history is a stretch and unlikely to be discoverable). If we have this, it seems like a complete solution and one that seems more clearly like an improvement.

(6) OK. Especially with the careful consideration and process we have for migration, I feel comfortable with moving that conversation outside of this SIP.

Thanks for taking the time to comment and iterate! I shared this proposal with a couple of users here and they appreciated the thought that was put into it.

[suddjian]

I want to respond to questions raised in the Superset Meetup, and clarify the scope and purpose of this proposal.

This proposal is specifically about improving user experience by moving the tab implementation from the UI to the browser, and not about changing state management - though it does depend on the state management improvements from SIP-23. This should allow the same user productivity, while being a more modern and practical solution.

Browser tabs will allow for better support of upcoming features such as more powerful edit history/undo. Supporting unique URLs for each SQL Lab session will also unlock new use cases that are currently impossible. For example, if you close a tab in SQL Lab you can never get it back, but in this design, if you close a tab you simply have to get back to the same URL again.

This pattern was identified during the redesign process as a better UX pattern than the current design. Participants were generally in favor of the change due to the improved usability, stability, and cleaner UI of browser tabs, as well as some positive user feedback.

I’d like to get some more votes on this SIP in the email thread. If you have not voted yet, your vote will be much appreciated.

[benceorlai]

The goal here was to clarify the scope and functionality (the "What"), understanding that some implementation details (the "How") will be further addressed in the subsequent PR. But I will be glad to discuss further - even in a small group format or chat - if anyone thinks the intended functionality needs to be clarified further. Please see the attached PDF with wireframes of the current/new interaction.
Download PDF / Open from Drive

[jiegzhan]

Development started? Any timelines roughly? Thanks.

[graceguo-supercat]

Hi @benceorlai I think understanding some implementation details (the "How") part will help me make decision. In the current behavior, I can easily see all my old queries in a list of sql lab tabs (and within 1 browser tab). Do we still have this benefit (see all the old queries in a central place) in the single sql lab tab solution?
I can find all my queries from query history....but every time i need to switch browser window to do it, which is far less convenient than a single click.

[jiegzhan]

I saw sip-30 has been approved. Do we have any timelines roughly? Thanks.

[suddjian]

@jiegzhan No timelines currently to my knowledge, but I'll let you know if that changes.

[jiegzhan]

Is anyone working on this? Thanks.

[rusackas]

Voted through, so closing! Thanks!

[yhp353134]

请问你这个是什么版本，界面比较好看

[rusackas]

Does anyone on this thread have any intention to follow through on this? If not, I think we should consider it closed/denied.