Let's get Backdrop ready to run on Pantheon!

[kreynen]

While it is possible to spin up an instance of Backdrop on Pantheon, it requires jumping through a few hoops. Nothing serious at first, but this is more automated with other distributions and could be improved in Backdrop as well.

The first issue users run into is Backdrop can't create the files necessary for the install.

This is because the instance is started in Git mode. This is easy enough to fix for each instance, but it's also possible to ask someone at Pantheon to configure the instances to start in SFTP mode. Until then users simply need to go to the site's dashboard and switch the mode from Git to SFTP there. It can be switched back to Git after the install.

Once in SFTP mode, clicking Try Again will allow the install to continue. The next step requires the user to get the database settings from Pantheon Dashboard. This is something that happens automagically in other distributions. I think this is what @populist and @fluxsauce wanted to address with backdrop/backdrop#60, but for now the user needs to get the database settings from Pantheon Dashboard and use the Advance section of that form to change the Database Host from localhost to dbserver.dev.xxx...xxx.drush.in and update the port.

Success! Now users can start using Backdrop for themselves... until Pantheon updates the database connection information. The site also can't connect to the database after being pushed from dev -> test either. This is because the install process writes static database configuration to settings.php and that information can both change at any point in time for the dev instance and is different on the test and live instances.

To get this to continue working on dev, users need to replace 1 line of the settings.php. Instead of...

$database = 'mysql://pantheon:...

Pantheon requires something more like...

if (!empty($_SERVER['PRESSFLOW_SETTINGS'])) {
  $env = json_decode($_SERVER['PRESSFLOW_SETTINGS'], TRUE);
  if (!empty($env['conf']['pantheon_binding'])) {
    $pantheon_db = $env['databases']['default']['default'];
    $pantheon_conf = $env['conf'];

    //user name and password
    $db_string = $pantheon_db['driver'] . '://' . $pantheon_db['username'] . ':' . $pantheon_db['password'] . '@';
    //host
    $db_string .= 'dbserver.' . $pantheon_conf['pantheon_environment'] . '.' . $pantheon_conf['pantheon_site_uuid'] . '.drush.in' . ':' . $pantheon_db['port'];
    // database
    $db_string .= '/' . $pantheon_db['database'];

    $database = $db_string;
  }   
}

But even with that in place the site won't even load when pushed to test. $config_directories['active'] = 'files/config_' . md5($database) . '/active'; in the settings.php is probably an issue. Because the database connection string is dynamic, this would change unexpectedly in every environment. I hard coded it to the directory created on dev, but that still didn't resolve the issue on live.

Pantheon also reports that their Pantheon Platform API Module is not installed.

Without a backdrop version of that module, I'm not sure how someone would install that.

I don't have time to continue chasing this, but I've achieved my goal of being able to get the new layout tool in front of more people.

[quicksketch]

the next step requires the user to get the database settings from Pantheon Dashboard. This is something that happens automagically in other distributions. I think this is what @populist and @fluxsause wanted to address with backdrop/backdrop#60

That PR needs a bit of work, we now have a dedicated issue for that request at #281

Thanks for opening this issue! It's super helpful just to see what you had to go through to get it set up. We hope to have the "Try Backdrop" link from the homepage of Backdrop CMS load up an instance on Pantheon direclty without any additional configuration. Looks like we have a number of things that need to be addressed, both on our side and at Pantheon. We've got a number of connections at Pantheon who are interested in getting Backdrop running on the platform, so it should be possible to overcome all of these issues. Thanks again for the run down!

[jenlampton]

@kreynen yes, thank you for giving this a new test drive!!! This is really, really helpful. It's been a while since we installed Backdrop on pantheon, and we've changed a bit in backdrop since then ;)

The config directories issue is sort-of a non-issue because those should be explicitly specified anyway (and they will usually be outside the drupal root, and version controlled). I immagine Pantheon is doing something similar for their Drupal 8 instances, and this would be no different for Backdrop - eventually ;)

Thanks for the punch list! :)

[kreynen]

I imagine Pantheon is doing something similar for their Drupal 8 instances, and this would be no different for Backdrop

There are differences between both the standard D7, D8 and even WordPress installs and what is installed on Pantheon. Pantheon instances aren't simply pulling the project from Drupal.org as the upstream repository. These are pulling from "drops" that are Pantheon specific versions of the releases.

https://github.com/pantheon-systems/drops-8/search?q=pantheon

I believe all the public D7 distributions are maintained in a similar way where we package the general use version of the distribution on Drupal.org then merge those changes with the Pantheon Drops-7 changes with each release that is pushed to Pantheon.

https://github.com/phase2/openatrium-drops-7
https://github.com/kreynen/civicrm-starterkit-drops-7
https://github.com/populist/panopoly-drops-7
https://github.com/pantheon-systems/drops-7/network/members

It's possible Pantheon is planning to maintain a similar fork for Backdrop, but currently they are pulling from the main project repository which requires the steps I outlined to install. Without taking one of these approaches, is not going to just install:

• Make Pantheon specific changes in the main branch wrapped in if (isset($_SERVER['PRESSFLOW_SETTINGS'])
• Maintain a Pantheon specific Backdrop with if (isset($_SERVER['PRESSFLOW_SETTINGS'])
• Support any host that injects settings into the $_SERVER variables in the main branch

I think it's important to get Backdrop in front of more people... people who aren't necessarily developers with a local stack where they can get a recent release of Backdrop running in a few minutes.

I also think it's possible to support Pantheon and hosts like it out of the box, but you won't get that by following D8's lead.

[klonos]

Hmm, getting more people to test Backdrop you say? ...then shamelessly promoting #5 over at the .org issue queue ;)

I love simplytest.me for Drupal.

[jenlampton]

I imagine Pantheon is doing something similar for their Drupal 8 instances, and this would be no different for Backdrop

What I meant was that the way they handle the config directory would be the same as they do for D8. That has nothing to do with the upstream repo ;)

We also have an issue in the queue that will allow any host (not just Pantheon) to inject its own database connection info, It needs some tests: backdrop/backdrop#60

Big picture here, the "try a demo" button on our flagship site (https://backdropcms.org) will fire up a Pantheon developer account with Backdrop running. We'll need to sort out all the issues you outlined and make that process as straightforward as possible. We are already working on it - but this issue will go a long way to helping us get that done :)

[populist]

In the short term, I want to propose that we handle the Backdrop CMS in a similar way that we do with Drupal 7 and 8. This basically involves creating a new repository (backdrop-pantheon) which will contain the latest release of Backdrop and some Pantheon specific code (Backdrop versions of Pantheon API/Solr + Pantheon specific settings.php values for DB/CMI).

This way users can have a clean experience spinning up the site on Pantheon (and can easily move off Pantheon) and Backdrop can just update their backdrop-pantheon repository with each new release so user's can see it in the dashboard.

Ideally we can get the DB injection stuff worked out (as per backdrop/backdrop#60), but there will also be some custom code (Pantheon API module for example) which may necessitate this approach for the foreseeable future.

[jenlampton]

An update here, we've created a repo for the pantheon instance: backdrop-pantheon Along with our pantheon-specific changes, Backdrop now installs and runs on Pantheon (thanks @populist!). However, we can't deploy things between environments just yet, so hold yer horses!

We're still working out a sensible workflow for pushing config around, since it ended up being considerably different than Drupal 8.

Here's the current proposition:

• Every time a "database" is moved from one place to another, move the config/active directory along with it (rsyc is fine) This is essential and will probably require some changes to the Pantheon infrastructure.
• Track the config/staging directory under version control, and use vcs to "deploy" config, the same as code. (We will not track, and may even .gitignore the config/active directory)
• Deploying config will go a little something like this:

1. copy all JSON files from config/active to config/staging
2. git add, and git commit the contents of the staging directory
3. push/pull/deploy these changes to the next environment
4. use the GUI on the next environment to review a diff of all the changes
5. use the GUI to import the config changes

• Note: Automating steps 4 and 5 here would be bad news, since without that active directory being tracked in VCS changes that were made directly to this environment can be lost easily, with no way to recover them.

Q: where do we put the config directory?
Q: where do we put the config/active directory?
Q: where do we put the config/staging directory?

I'm recommending we put the config directory in the web root, right "next to" the files directory - but NOT inside. People are going to be syncing files from env to env, and if config went accidentally along with files, they'd be in trouble (far worse trouble than files/db getting out of sync - here we're talking data loss). I'd also like to see both active and staging stay within config, since that makes the most sense for the people using the files.

Q: how do we assure the config directory is writable by the server?

We solved this problem in backdrop core by leaving the config directory inside files. I'm uneasy about leaving it there on Pantheon because people will also be provided with a tool that allows them to shoot themselves in the foot by moving files around (though, I'm now second guessing this decision for core too). Additionally, since people aren't in control of their own file permissions on Pantheon, I expect Pantheon to be smart enough to correctly set the permissions for them. If Pantheon can do this correctly for files perhaps it can also do the same thing for config.

Q: how do we assure the config directory is not readable via the web?

It sounds like Pantheon solved this problem for D8 by making sure yaml files were never accessible via the web. That same trick won't work for JSON - but perhaps if we can do a little permissions setting on the config directory that will do the trick.

I'd love some opinions from the rest of you on how you'd like to handle deploying config. Long term I think we're going to need a new tool on pantheon, since config is neither files, nor, database, nor code, but short term I think we can make it work by treating it like code (and sometimes the database).

[populist]

I think there may be a good case to be made that Backdrop's CMI active configuration should live in the database similar to Drupal 8. Here is the Drupal.org thread on the matter which discusses a few of the reasons - https://www.drupal.org/node/2161591.

If that is possible, then you can use a much simpler model for configuration which is just to have a single /config directory that holds all of the staging configuration for a site. This can be written to by either the webserver or something like Drush and will properly work with VCS.

If that is not possible, then I would use a more complex model for configuration where you have the active configuration managed in the files directory (/files/config/active) and the staging configuration managed outside the files director (/config/staging). README.txt can help point people around.

[jenlampton]

Putting config back in the database is not on the table for Backdrop.

For Backdrop 1.0.0 on Pantheon, perhaps we should not worry about allowing people to deploy config since that is proving difficult. There will be a UI for this within Backdrop itself, so people can export dev / import stage, export stage / import live if they need to move their configs around.

If/When pantheon builds a separate UI for deploying config for Drupal 8 we can revisit and see if they will also build a similar UI for deploying config for Backdrop.

For now, let's just leave the config directory inside files like it is in core, and everyone running Backdrop on Pantheon (@kreynen ) will just need to deploy their files directory along with their database - every time :)

[populist]

How important is it to you to be able to keep the configuration generated from CMI under version control?

There is a pretty clear way to do this already with D8 + Pantheon (https://amsterdam2014.drupal.org/session/drupal-8-cmi-managed-workflow) which can be adopted for Backdrop as per my comments above.

Putting all of the configuration (active and staging) into the files directory requires using SFTP or a Web UI to download/upload configuration files for deployments which may not be the best developer experience.

[jenlampton]

tl;dr: Without the active directory versioned, I don't think versioning config a good idea.

The primary benefit of versioning the config directory is 1) being able to see a diff of what's currently active (what's been changed on live?) and 2) being able to pull that active config (from one place) into staging (in another).

There's already a web UI for deploying config, and the developer will need to use it anyway to get the imports to run (database tables created, etc). Without drush support yet, there's also no way to skip using the UI entirely.

Leaving config in files would require the following deployment workflow:

1. export config site 1 (using backdrop UI, or copy files from disk)
2. import config site 2 (using backdrop UI only)

If we wanted to add versioning config, the currently proposed workflow gets a little bit more cumbersome...

1. export config site 1 (using backdrop UI, or copy files around on disk)
2. add to version control, commit, push site 1
3. pull site 2
4. import config site 2 (using backdrop UI)

The backdrop UI will provide you with a dff between what's currently active and what's about to be imported (in staging) before you run the import. This allows you to catch dangerous things, and I wouldn't recommend anyone proceed with an import blindly. For example, if a field was added on live but not dev, and the config from dev is being imported, running that import will delete the field on live!

Without the active config directory in version control there's no way for a dev to see "what you are about to delete" when importing, without using the Backdrop UI. Diffing "this deployment" vs "last deployment" via vcs is not sufficient.

Having some visibility is worse than having none at all, because it gives people a false sense of safety. A dev checks the diff of config via vcs and thinks things look fine, runs the import, and deletes a ton of stuff that has been added to live since the last deployment.

[populist]

As a followup here, I would certainly defer to you around how the product should work. I would say that it's probably the case - for both Backdrop and Drupal - that there will be lots of learning and experimentation with the best way to use CMI over the next ~18 months. While the underlying technology is good, there are a few unanswered questions about how folks should best use it and what kind of supporting modules / developmental workflows are best to make full use of it.

Considering that you are using active files and that your CLI is still coming, I can see the case for 1.0 on Pantheon to default to just using the files directory (which can be overriden by people of course if they want to try different stuff) and then change it down the line.

However, if you want to work towards something more similar to the D8 workflow (https://amsterdam2014.drupal.org/session/drupal-8-cmi-managed-workflow) happy to brainstorm with you some basic stuff we can do to help make that real for you.

[jenlampton]

I don't think we're interested in moving towards the proposed D8 workflow at this time.

Let's keep it simple and keep it in files until we can see a clear path forward :) I like the idea of giving it some testing in the wild, and developing best practices from what's learned.

[quicksketch]

We've merged in the PR for #281, which lets a server specify configuration overrides (e.g. the database credentials and cache backends). This should enable Pantheon (and other hosts) to automatically configure Backdrop credentials to streamline the installation process.

[kreynen]

I spun up a new instance on Pantheon yesterday and immediately noticed backdrop/backdrop#502 still wasn't fixed in that version. While these changes are getting merged into https://github.com/backdrop/backdrop, Pantheon is using https://github.com/backdrop-ops/backdrop-pantheon as the upstream repo which hasn't been updated since October 23.

I manually updated this install, but what's the process going to be for getting releases to Pantheon? Can we get https://github.com/backdrop-ops/backdrop-pantheon updated to it's easier to get more eyes on the latest updates?

[kreynen]

With the 1.0.0-preview release now available, it would be really nice to be able to preview that version on Pantheon without having to update the code from the upstream repo yourself. What is the process for keeping the Pantheon version in sync with with updates to the primary repo?

[jenlampton]

I think we should set up some github magic behind the scenes to automatically update backdrop-pantheon every time a tag is added to backdrop. But, we don't have anyone coding up our github magic these days. Maybe we can sweet talk @sirkitree into it?

[jenlampton]

I just pushed the code from Backdrop 1.0.0-preview up to the backdrop-pantheon repo (I think, I may have pushed the latest from 1.x) either way, the demo button on backdropcms.org should work if we can get pantheon pulling this latest code. @populist is that automatic?

[populist]

Pantheon will pull the latest code from the backdrop-pantheon repository automatically and will notify existing sites that an update exists in their dashboard.

[populist]

As an update, I also made another pass at the Pantheon settings injection to simplify everything based on new changes in the Backdrop CMS. There are a couple quirks that still need to be resolved (backdrop-ops/backdrop-pantheon#1) but that is the branch for testing.

[kreynen]

Just spun up 1.0 with no errors on the install :) Really amazing work! I also merge the 1.0 updates @jenlampton merged into the Pantheon Drop reop into my existing Backdrop test sites without any issues. So that process (though manual) is working as well.

[quicksketch]

w00t! Thanks @kreynen! Much easier now than it had been, eh?

[populist]

And it has been written up to much excitement - https://www.getpantheon.com/blog/backdrop-cms-10-available-pantheon

[jenlampton]

<3

[jenlampton]

I just wrote up a comparison of our three leading candidates for handling config in backdrop with Git over at #855 (comment) if anyone is curious.