Implement a drag & drop model for building layouts

[Bojhan]

A big part of Drupal's UX flaws is its inability to provide a great experience for some of the core site building concepts. I have raised this concern many times, and we even outlined it as a strategy for Drupal 8.

Given that Backdrop seems to focus on this sitebuilder type of audience, I'd love for this to happen. Clearly its a big effort, but in terms of UX it will set it apart. From my point of view, improving UX by removing a bunch of stuff - does very little if you leave the core concepts intact, that is what most people will care about.

The actual model could be rather simple, you need to decide between either real live or abstracted view of "a display". This display can then be configured, by pulling items (blocks, fields, menu's, etc.) on top of it. The tricky part is making this a cohesive experience across data models.

I wouldn't mind helping with the interaction design and research, but the biggest problem is and has been resources.

[saltednut]

I am interested in looking at this too. Are we looking at porting any existing contrib or creating a new system when its decided how plugins should work in backdrop? I think it might be a mistake to fork panels due to complexity. It would be nice to have a Panels IPE-like experience without the extra baggage provided by the full panels system. Distributions like Panopoly have shown us that setting something like this up in Drupal 7 required a lot of different modules. In addition, it involved greatly changing the default UX provided by ctools (re: panopoly_magic, and *_widgets)

[quicksketch]

Hi @Bojhan, thanks for opening this issue! I've said it elsewhere already, but the number 1 thing I'd like to see in Backdrop 1.0 that's not in D8 is a layout tool. I saw an earlier post you made ("Drupal component library" or something like that? I can't find it at the moment), that lined up pretty close with what I was thinking of also.

I think for the first pass we should not attempt to do an in-place editor or front-end manipulation. It has too many challenges around describing which pages you're editing: whether it's a single page or multiple. Not to mention rendering and moving around content with actual things inside of them is both heavy (web browsers can have a hard time moving huge chunks of the page) and difficult to implement.

I think we can learn a lot and borrow heavily from Panels, but for an out-of-box tool Panels demands too much knowledge to operate.

The basics of what I'm thinking of consist of the following:

• Layouts are moved to a dedicated module (say layout.module) and are provided via "plugins" (which may just be a hook and a class). Core ships with 3-6 layouts out of the box.
• The responsibilities of layouts are removed entirely from themes, and in fact because the layout module would handle the content of the page, pretty much all of the "page.tpl.php" file would no longer exist. It would be logical to rename html.tpl.php back to page.tpl.php since we wouldn't have a need to separate them any more.
• This means that layouts would be useable between themes. Regardless of the theme enabled, the layouts would work work regardless. A well-written theme would style the out-of-box layouts provided by core if they desired (i.e. adding backgrounds and styling).
• The layout module would allow overriding of any existing paths, or creation of new paths. So you could have layout-only pages (such as the homepage), or take an existing path and wrap a layout around it, like the "Page manager existing pages" module. You could then drop in the "existing content" block or (maybe longer-term), pull out individual pieces of the original content and put them into different regions.
• Each layout for a path would not have the concept of "variants" like Panels does now. Instead you'd set up a selection filter for each individual layout. So for example if you wanted to lay out the node/%node page differently for "blog" and "photo" content types, you'd make two different layouts at the same URL. Layout module would loop through all layout configurations at a current path and use the first one that matches. If none match, it would fallback to a site-wide default layout.
• Editing a layout would be similar to editing a panel today, as an admin-only task with placeholders in a drag and drop UI. We'd try to consolidate as many settings as possible (and eliminate many of them) onto the main page for editing a layout, or perhaps have an advanced collection of settings that open in dialogs (similar to Views). This is probably where things are most unclear: deciding what to cut and keep from Panels and how to build out this UI.

The concept of "context" is also important to address, but honestly even D8 didn't figure this out and currently blocks of no concept of context whatsoever. Starting with building a foundation for stand-alone layouts that can replace the terrible one-layout-for-everything would be an excellent start.

[dalejung]

So this is from my experience with D6 and Panels, it's a few years old, I hope it applies.

I think that the drag-and-drop layout should be decoupled from panels or whatever the new system is called. In practice, people rarely change the layout and bigger editorial teams will have set templates.

The reason we used panels was for semantic groupings of plugins. This was useful for tiered caching, as many panels would invalidate as infrequently as their components. In most cases we were able to get a cache hit on the entire panel and not just individual blocks.

Another big reason was the ability of panels to pass in args to their plugins. Having the ability to decouple each plugin from what was going on outside it made re-use incredibly simple. Sometimes the nid came from the url, sometimes from a node references, the plugin didn't have to care.

When it came to the actual layout, the panels rendered each plugin and then threw them to a tpl. In this way, we didn't necessarily care about the ordering of the plugins. This was especially true for more complicated layouts that didn't fit any type of grid. The Panel was just there to define an api for plugins and group them into tpls. Obviously we did modifications to remove all the automatic html wrappers and make the tpl override names easy.

This was by far the most robust system I could come up with. Themers were happy because the DOM made sense to them and they could accomodate any design.

My worry is that a pre-occupation with layouts would make it harder to decouple all the benefits listed above from the drag-and-drop gui.

Also, this my be reflective of my experience with magazine publishers, but we could never get the "whole page is a panel" to make sense. If you suppose that every page is really 3 regions (left, content, right) and you have 3 variations of those regions, then that requires you build 27 panels for each permutation. What we did is have each variation be a special panel that was tied to a region called a panel_template. These were outputted via a panel_block that selected the proper variant via a cascade of default->context->node. I suppose one could put a panel_block into a page wide panel, but I assume people are talking about a flatter hierarchy than that.

[dalejung]

Oh, the other added benefit of plugins that define their arguments is that you can automate invalidating their caches. Generally we had still make a pass and make the invalidation rule more granular, but it's still better than timed/not cache.

[Bojhan]

@quicksketch Interesting, I think our original ideas do encompass much of this experience. Frankly I could hash out the key parts in under 15 minutes. But the challenge is not in the solution, but in the scope. I'd love to get the basics of a layout placement system with drag and drop before even thinking about page creation and selection filters.

The whole reason most of the SCOTCH work stranded, is because we were too ambitious. Given that we are dealing with far fewer resources here, I'd aim for getting the basic block drag and drop in regions, layouts that represent the actual groups, a nice way to select the right block - first. Then expand the scope to creating page, and finally mapping different layouts to one URL.

The whole problem with Panels et al, is that it focuses on doing it all - Backdrop should by its nature start with getting the key patterns right.

[matt2000]

May I suggest:

Layouts:define regions available and their placement in the page
Contexts: define what goes in regions, in what order
Themes: define the markup/style of regions and things in them; also how the 'order' defined in context is interpreted.

[jenlampton]

I re-titled this issue "Implement a drag & drop model for building layouts" because blocks and fields should probably be two separate issues, but the bigger picture here is layouts.

[duckzland]

so far the best layouting system via GUI that I've encountered is WordPress Headway theme framework. I believe that kind of system can be integrated with Drupal theme system renderable arrays. Basically things like context, display suite, fields, blocks, regions is mashed up together in a single Layout array.

each page has its own layout array stored, the layout array can be reused for other similar pages, such as page.layout will serve all pages, page--blog.layout will serve page with blog node type etc..

the structure of the array can be something like :
-- Area Header
-- Region Logo
-- Block Logo
-- Region Navigation
-- Block Menu 1
-- Areas (or wrapper) Content
--- Regions Content
--- Block 1
--- Field 1 displaying title
--- Field 2 displaying content
--- Region Sidebar
--- block C
--- Block D
--- Field C
--- Area Footer
-- Region Footer left
-- Region Footer Right

I've implemented such system in http://drupal.org/project/sigmaone but isn't expanding it further due to shifting focus to WP.

Hope this helps in creating something like headway for backdrops.

[quicksketch]

Hi guys, sorry this issue is starting to look like one brain-dump after another, and I'm about to add another one. :(

@jenlampton and I went through the ideas of the basic plan above and refined it to particular paths. Calculating how to deal with the options we're going to be having (i.e. contexts, arguments, etc.) It's not a fully baked plan but it's a start, and more specific than the general outline above.

---

List of Layouts page: admin/structure/layouts

• Lists all layouts, grouped by path (if multiple "variants" in Panels terms)
• Each layout has a drop button of operations (Edit, Clone, Delete, Disable/Enable)
• If a layout path has multiple "variants", they are collapsed within the row, the drop button contains only the options Rearrange, Delete, Disable.
• Multiple variants under a singe path can be expanded in the list, showing their different selection critria in place of the path.
• Button at the top of the page for "Add new layout"

Add new Layout form: admin/structure/layouts/add

Form includes options for:

• Human and Machine name for each layout (even if there's another layout for a different variant at the same path).
• Path to override or create.
• Contexts: If paths contain '%' signs, AJAX-create a drop-down for each '%' in the path, requesting a "context" for each argument.
  • For known paths (i.e. 'node/%') as defined by hook_context_paths() (or similarly named), lock the select list to the known available context.
  • For unknown paths (i.e. 'node/%/webform-results/submissions/view/%webform_submission'), display a drop down list for each unknown argument, while maintaining "locked" select lists for each known argument ("node" in this example).
• Selection criteria: For each context, display a "Narrow criteria" next to each context, which opens a dialog of available selection criteria for that context.
  • After selecting a criteria, edit settings in the dialog.
  • After saving the dialog, the new criteria can be edited or deleted from a list below each context.
• Layout selection: This may not be grouped as Panels is today. Instead just list all layouts directly on one page.

Save the layout setup. On to the next page: the layout content.

Layout Edit page: admin/structure/layouts/manage/[layout_name]

• This is the main layout page for placing blocks within regions.
• Each region has settings for display style, accessible via a gear/dropdown (like Views) for editing. Includes options: Add, Configure
• Adding a new block to a region will display modal with options for editing that block's settings.
• After the block is added to a region, blocks may be moved via drag and drop to other regions. Blocks do not contain a preview, just the name of the block.
• Each individual block may be configured via a dropbutton (or individual buttons) that contains the options for Configure and Delete.
• Save button at the bottom of the page.

When editing a layout, the "Settings" form for the layout containing the original options may be access via a tab: admin/structure/layouts/manage/[layout_name]/settings

• This form contains all the original options present on the add new layout form. Machine names may not be changed.

[quicksketch]

@Bojhan pointed me at his video of a mockup at http://bojhan.nl/blocks-layout-prototype, which is remarkably close to what's being suggested here. The idea for setting up paths, arguments, and contexts on the first page is almost identical. Some of the further ideas (additional data sources) might be out of our initial scope, but overall it's a great visualization of approximately where we'd like to take this.

[quicksketch]

I'm starting to make headway on this issue. Although the overall amount of work is a little intimidating, I think by leveraging the existing code from Panels, this should be a surmountable task. Instead of a "ground-up" implementation, I'm working at this task from the perspective of porting the architecture of Panels, with the UI described above and in Bojhan's video. So essentially this becomes a porting and UI job instead of trying to reinvent the wheel. I'll post to my sandbox when I get something worth sharing.

[quicksketch]

My work on Layout module still isn't close to pull-request status, but I've been making steady progress for the last month. The in-progress branch is located at https://github.com/quicksketch/backdrop/tree/86/layout_module

Despite my intentions to mirror @Bojhan's mockups, we're ending up with something a lot closer to the Panels UI than I had anticipated. Because we're basically "porting" Panels' concepts to Backdrop, the UI ends up following suit in a lot of ways.

So far, the only things working are creation of new layout paths, creation of new menu items, the very basics of automatically and manually identifying contexts, and saving/loading/cloning/etc. Some screenshots:

Layout listing:

There's still a ton of work to do, but at least this shows some progress.

[quicksketch]

I've still been pushing forward on layouts. Recently we got to a point where you can actually add and move blocks around on the page, which is a great milestone in the building out of this module. While the module is somewhat stable, I made a branch for anyone who wants to try it out at https://github.com/quicksketch/backdrop/tree/86/layout_module_20140716_demo.

I'll be putting together a video to demonstrate the current status of things shortly.

[quicksketch]

Here's the video of current status: http://youtu.be/pMKcjEetKAg

It's coming along nicely and actually has some usefulness now. The largest remaining tasks are:

• Integrating fully with the bootstrap/menu to take over the full content of the page.
• Limiting listed conditionals/blocks based on available contexts (right now they're all just always shown).
• Make it so single plugins can expand into multiple items (e.g. the "Field" plugin should let you place any individual field).

And then there are lots of little things on top of that: form contexts, integrating properly with views, finishing porting the available plugins, styling things... still lots to do for sure, but it's good progress and feels real at this point.

[quicksketch]

It's been a while since an update, but this has been pushing forward steadily. All 3 items from my last update are now complete. Layouts can completely take over an existing page and use (or not) the existing content on the page.

Additionally, we've built out:

• Administrative previews and a list of conditions are shown on the main layout page.
• The upgrade path is complete to migrate Drupal 7 blocks into both a default layout and a node/% layout if necessary.
• Ported fully responsive 1 column, 2 column, and a Bartik-style 3-3-4 column layout (for the upgrade path).
• We made a replacement "header" block that contains the elements of a header, including the logo, site title, slogan, and a primary menu (by default this is the user menu).
• Block module has been stripped down to nothing but "custom blocks". It no longer affects the display of the page in any way (as that's what Layout does now).

I've also removed all remaining references to page manager/ctools/panels, and trimmed down the scope of the first merge to just include replacing the functionality currently in core. We can port the remaining CTools plugins as individual PRs at a later time after we get in the base architecture.

Here's a screenshot of the current interface, now with header and footer regions:

As always, the progress is open and located at https://github.com/quicksketch/backdrop/tree/86/layout_module

I'll be preparing a new video with our current status as soon as possible. The only remaining tasks at this point are:

• Continuing to fix the tests so existing tests are passing (mostly modules that provide blocks, e.g. node, book, menu, etc.).
• Build new tests to cover at least the basic functionality of Layout and replace the tests that we've removed out of block.test, e.g. node-type conditions, path conditions, user-role conditions, block weights, and positioning.

[Bojhan]

Wow :) I will give it a spin

[quicksketch]

That would be excellent @Bojhan! I'd love to hear your thoughts. Ultimately this ended up architecturally very similar to Panels since we started by porting the functionality. UI-wise it is simplified, but doesn't quite follow the workflow your original videos outlined. I remember the biggest take-away from your video was how additional contexts were added was very different than Panels today. For the initial version of Layouts though, we haven't added the ability to support relationships or even additional contexts (besides those from a URL). The infrastructure is there however, so after we get the basics hammered out we can work on the UI to support additional contexts and relationships.

[quicksketch]

Here's the current status of our tests. Mostly functioning but a lot of block-related failures: https://travis-ci.org/quicksketch/backdrop/builds/35674324

[klonos]

This is so exciting!!! Can't wait for the video demo as well as the chance to try a backdrop build that includes the feature.

Thanx Nate for all the hard work.

[quicksketch]

Glad you are excited about it @klonos! I am too. This is going to be a ridiculously huge improvement over the current system.

The new video has been published to our YouTube channel.

You can watch the video directly at https://www.youtube.com/watch?v=xoD6gDZmE3o

[klonos]

Just watched the video. Looks amazing!!!

Can't wait to start building my first Backdrop CMS theme.

Are you planning to publish any guide for themers that come from the D6/D7 world?

[Bojhan]

First off, whooa! I can only imagine the amount of work it took to pull this thogheter. Although it does not have all of the UX ideas that we put forward, I think it does a great job in significantly reducing the complexity compared to something like panels.

While I would love to help figure out some of the interaction design on this, I'd like to know how useful that is at this stage. I want to avoid my Drupal-type contributions, were I mostly just spend time making UI's that never go anywhere.

Long term

Preview / In-line model

The biggest step any of these tools would make is to provide a way to preview settings and/or a model to do it inline with preview. Its often hard to see how settings affect the "real" thing - ideally we pull that closer in every way possible. This will be the true differentiator between an implementation driven UI and a user mental model driven UI.

Smarter context selection and browsing

As we are adding more and more contexts, conditions, blocks etc. the real challenge will be in exposing a useable browser. This would be a separate issue, but perhaps something another contributor could help tackling. We should draw some inspirations how other systems do this, not Panels.

Short term

A lot of my comments here you probably already considered. But these are improvements you can make without major recamps.

Listing

The listing of layouts is a little odd, we don't really support "nesting" in tables very nicely in core. I will need to do a few explorations to see how we can make this look nicer. The new styling we have for tables would make a world of difference.

The default profile should probably come with two different context ones for content types and users? Good defaults really help here.

Layout viewer

There are a number of obvious visual improvements that we could make here. Non of these are critical for release. But mostly open doors, some ideas:

• Removing the buttons for adding and configuring, with nice +'s and cog wheels. That would make the UI easier to scan.
• Fixing the alignment. of blocks
• Having conditions as a link that opens a modal, rather than a fieldset.

Known available contexts

In order to get a "node-type" context available. You have to know it exists. What would be nice if we have a description, or "recommended" contexts that a user can pick from. That would help with that discovery part (even if it is just the ones you support now).

I really like the auto-recognition, that really helps people understand what is going on.

Selecting context per block

A big leap that you could take here is introducing more of a "Browser" function. Similar to what Views does when you are browsing through available filters or fields. Right now its a drill-down which is relatively nice already! But it would be good for exploring purposes to allow you to browse and search for specific contexts.

Thanks!

[quicksketch]

The more I look at it, the less I like the 'Title type' field (as it currently is);

@jenlampton has had a lot of things to say about the Title options as well. We don't want to make it a block, because it isn't moveable (and probably shouldn't be). Her suggestion was to put an edit link next to the title, similar to what we do for blocks. Then the title settings would open in a modal, like a block.

Not sure that I like the ability to edit custom blocks when adding them to a layout...

I agree on this one too. We'll need to think about it. One of our followup tasks is adding "Custom text" blocks that save into config and are not entities at all. These would be per-block and not globally available. 9/10 times, I'd probably use these instead of Block entities anyway.

The order of the block list no longer changes between adding blocks (which is good), but the order still seems random

Yeah, let's leave the order of blocks to the followup, where we'll add the filter box and a dropdown categories filter (similar to views).

It seems a better idea to automatically remove it, and have the error serve to explain why it was removed.

Not sure about this. I don't think it's going to be common to try this in the first place unless you were actually trying to break something (good job on the attempt 😉), but since we don't show the error until they save the page, which one would we remove? The first one or the newly added one?

Probably just a CSS issue, but when the sidebar is empty (either no visible blocks, or no blocks at all), the sidebar region still takes up space

This was actually an intentional change. Collapsible regions can often be a confusing concept and can increase the variability of the page. e.g. users may expect a max-width of 600px for the left column for both images and text, but suddenly when a block is removed, it fills the entire width, which can make readability difficult. We kept the collapsing columns for the Bartik layout (for compatibility), but for the two-column layout and others we'll add in the future, they will probably not collapse if empty. If you wanted a page with a single column and no blocks, it'd be up to you to set a different layout for that particular path.

[quicksketch]

I just updated the PR again. Now the message for "You have unsaved changes" shows up immediately after adding or modifying a block.

[jenlampton]

Some feedback for layouts - first woohoo! happy dance
... and then some more valuable feedback.

1. We need to clean up the edit layout DragonDrop(tm) interface, see Clean up the edit layout interface #373

2. Fix up header block text "Contains elements typically contained..." should be "Contains elements typical of a site header including..."

3. Configure Block "Title type" options should be Default, Custom, None. No need to repeat the word "title".

4. Text on admin listing page. "used on all paths without a specific layout" should be "used on all paths unless otherwise specified"

5. When you navigate to the "Create new layout" page and then Cancel, validation fires and you are required to fill out all the fields.

6. When you have filled out all the fields, and then cancel, the form is cleared but you are left on the same page. You should be returned to the list of layouts.

7. When editing a layout, both "Save layout" button and the "Cancel" button should return the viewer to the list of layouts.

8. When editing the layout "settings" and you hit "Save" you should be returned to the layout page.

9. When creating a new layout, the content from the default layout gets copied into the new layout. But sometimes the same block gets placed more than once (I have header twice and main menu three times, but powerd by backdrop only once). This happens for layouts with and without a context. I can't figure out why this is happening, but it happens consistently.

10. Changes to the Create new layout form.

• "Layout title" should be "Layout name".
• Layout order should be intuitive: 1 then 2 then 3/3/4

11. Conditions are missing when editing the default "Administrative layout". I wanted to change the admin layout to be 2 columns. But after doing so I instantly realized that I had a column on my layout-editing-layout, and wanted to add a condition for "not THIS path" but I can't :(

12. Layout listing page:
    "Path and layout" should be "Layout and Path"

[ghost]

Not sure if this got lost above, but:

• Allow default layouts to be reset/reverted

[quicksketch]

Thanks all.

9. When creating a new layout, the content from the default layout gets copied into the new layout. But sometimes the same block gets placed more than once (I have header twice and main menu three times, but powerd by backdrop only once). This happens for layouts with and without a context. I can't figure out why this is happening, but it happens consistently.

This looks like it's actually because editing a block puts a duplicate UUID in the region for that block. Once the duplicate UUIDs are in the default layout, they show up when cloning as each block is reassigned new UUIDs.

4. Text on admin listing page. "used on all paths without a specific layout" should be "used on all paths unless otherwise specified"

I'm not sure about this new text. It makes it sound like you can "otherwise specify" certain paths that wouldn't use the default layout. But there's no place to specify a list of paths, you just have to make new layouts at those paths.

All the other feedback is great and I'll work on implementing it. Thanks @BWPanda for the last update too, sounds like a good idea to be able to revert the defaults as well.

[quicksketch]

11. Conditions are missing when editing the default "Administrative layout". I wanted to change the admin layout to be 2 columns. But after doing so I instantly realized that I had a column on my layout-editing-layout, and wanted to add a condition for "not THIS path" but I can't :(

You can't put conditions on a default layout, because there would be nothing onto which you could fallback after the default. Instead, if you wanted to put a layout one everything except for one path, you'd duplicate the default, and put it at that path and change as needed. Due to the needs of different paths providing different contexts, you can't use a layout at more than one path.

[quicksketch]

Okay, updated the PR again with all recommendations from @jenlampton (except those noted in my last two comments). It looks like the ability to "Revert" a layout has gone completely missing. Though the code is all there, layouts modified are never marked as having been so. I'll take a look at this some time this weekend.

[quicksketch]

Alright, the last fix we have here is to make it so that default layouts are revertible. All the code was already in place, we just weren't updating the "storage" flag when saving. Once that was added, the reverting functionality reappeared and worked correctly.

[quicksketch]

@jenlampton and I have been reviewing this (and demoing) at Pacific Northwest Drupal Summit and HTML5 Developer Conference. Our PR looks in good shape and we haven't had any serious issues in the past 5 days. Let's get this merged and continue in the followup issue, and add more new issues as necessary (which has already begun).

[quicksketch]

I've merged in backdrop/backdrop#483. Yay!! 6 months of effort and it's a great start. Let's start creating and handling followup issues in #345. Thanks everyone for their amazing contributions and input on this!

[klonos]

Great work everybody. Thanx.

...now lets try to break it. "Kicking the tires" is the expression I believe ;)