Make Backdrop drush-compatible

[rudiedirkx]

Obviously Drush is the best. Part of Backdrop might be drush compatible, but at least some of it's not.

I suggest keeping a drush sibling compatible at all times. Shouldn't be too hard. Only small API changes for now, right?

[greg-1-anderson]

Right now, Drush does not recognize backdrop at all; it thinks there is no Drupal site at the cwd at all, and will not try to bootstrap.

Moving this forward would clearly require an issue in the Drush queue. I don't know what the other maintainers think, but I for one would rather see Drush support Backdrop than to have a "Brush" fork.

It would be really helpful if Backdrop would add an official file somewhere in its core directory that contains the name "backdrop". Drush could use this file to "fingerprint" Backdrop, just as it is similarly fingerprinting D8 vs. D7 by finder certain well-known files in the D8 core directory.

[msonnabaum]

I'd prefer that drush plan to never support backdrop.

[greg-1-anderson]

Looks like the sentiment is for the Backdrop community to fork Drush (or other tool) to provide CLI support, rather than try to make Drush support everything.

[quicksketch]

I'd love to see Drush support Backdrop. Considering one of the main goals is minimal changes from Drupal 7 (and for the most part, staying that way moving forward), the differences should be drastically less than supporting Drupal 8.

Right now, other than version-checking, drush actually works just fine with Backdrop already. We even use it to install Backdrop in our testing framework: https://travis-ci.org/backdrop/backdrop

However newer versions of drush got strict on version checking (we use 5.7), making it so it ceased to recognize Backdrop.

So I'd love to see drush support Backdrop as well as Drupal, and I'd love to help make that possible, but if it's a non-starter for the Drush maintainers then we'll need to come up with an alternative.

[msonnabaum]

Can't speak for the rest of the maintainers, but it's definitely a non-starter for me.

[danny-englander]

Likewise, I can't fathom developing a "drupal" (Backdrop) site without drush.

[greg-1-anderson]

The thing to do would be to start a discussion in the Drush queue and see what everyone really thinks. If there was a PR that made Drush work with Backdrop with a minimum amount of change to Drush, and if there was a Backdrop maintainer for Drush, then the decision would be more philosophical than technical.

Without support in drush-ops/drush, then the alternative would be:

1. Clone drush-ops/drush to backdrop/drush.
2. Make a backdrop branch of master, and commit backdrop support there.
3. Periodically pull master from upstream, and rebase the backdrop branch.

This would also take a maintainer, of course. Staying on Drush 5.7 forever would be a little sad, but I suppose that's a fallback alternative.

[greg-1-anderson]

If you want Drush to think that Backdrop is D7:

// Drupal 7 root. Additional check for the absence of core/composer.json to
// grant $path is not core/ of a Drupal 8 root.
$candidate = 'includes/common.inc';
if (file_exists($path . '/' . $candidate) && file_exists($path . '/misc/drupal.js') && ((basename($path) != 'core') || !file_exists($path . '/../composer.json'))) {
  return $candidate;
}

In other words, you need includes/common.inc, and misc/drupal.js, and an absence of composer.json.

Of course, all of the bootstrap functions would have to have the same name.

[ergonlogic]

Pull request for basically the changes @greg-1-anderson suggested: backdrop/backdrop#78

[quicksketch]

I don't think we should build in a hack like this (and this is definitely a hack). I'd prefer to see the creation of Backdrop-specific constants and then use those as identifiers (which would require a change to drush of course). Even with Backdrop being similar to D7, there's no way we can trick Drush into working 100% of the time, considering assumptions about Drupal version numbers, URLs, and other things that are probably going to be outside our control.

I think our best chance at this is adjusting Backdrop where needed to make it identifiable from D7/D8, then provide a demonstrable pull request to Drush providing compatibility. If it's truly limited in its scope, it may be acceptable to maintainers, especially if it's done in a generic fashion where Drupal distributions or update servers could benefit from the changes.

[ergonlogic]

So, yes, admittedly it's a hack, but one that cuts to the core of the issue (all those puns are intentional, btw ;)

I think the fact that Backdrop forked after the move to core/ is exactly what will make ongoing Drush support difficult.
There's a desire to drop support for D6 to simplify the task of supporting D8 (drush-ops/drush#186). Whereas adding special-casing for Backdrop is moving in exactly the opposite direction.

Drush's D7 support is essentially a stable API we could target. To the extent possible this keeps the responsibility in Backdrop's hands, where it can be handled responsibly for the best interests of the project. Once Backdrop is bootstrapped, we can alter Drush's behaviour in all kinds of ways in our own drush.inc files, though.

Anyway, I could well be way off base on all of this.

[greg-1-anderson]

While I don't really have time to work on D8 or Backdrop right now, I would be interested in seeing Drush gain the ability to allow commandfiles (plugins) to alter the Drush bootstrap. With this sort of change, it would be possible to distribute backdrop.drush.inc, and have that include the hooks necessary to identify a Backdrop site, and, once identified, include either the core Drush files you need, or the Backdrop replacements as necessary. This capability might also be helpful to Drush core, should D8 support requirements start to differ substantially from Drush's D6 support requirements. I think I will be able to attend the Backdrop talk at Pantheon on Monday; if so, perhaps we can discuss there, afterwards.

[quicksketch]

So, yes, admittedly it's a hack, but one that cuts to the core of the issue (all those puns are intentional, btw ;)

:D
Sounds like a possible approach @greg-1-anderson. That's the sort of thing I'd hope for in getting Backdrop support going.

It's problems like this that really seem to drive home the need for the most compatibility possible. I expect for our initial release cycle, we're going to have limited developer capacity available. Keeping the API changes low makes for less work overall. Over in #81 (YML info files), I'm backpedaling from the change because we can use more-compatible (and faster) alternatives.

[mikemccaffrey]

Any reason not to fork drush, and create a project-specific version called brush?

[quicksketch]

Any reason not to fork drush, and create a project-specific version called brush?

Drupal + Shell = drush
Backdrop + Shell = bash

Clearly that's not going to work. ;)

But seriously, Drush is already intended to work with different APIs: D6/D7/D8. I don't think Backdorp support should be challenging to add by comparison, but the problem we have is interest within the drush project. Thus far, the Drush maintainers have showed mere apathy rather than hostility. Adding support directly in Drush would be the approach requiring the least effort, we just need someone who is trusted and motivated enough to maintain Backdrop support within Drush itself.

[sun]

I don't think Backdorp support

OT: Lovely to see the original typo living on. ❤️ Always wondered how that epic incident happened in the first place, but appears to be more common than I thought 😃

[sirkitree]

I tend to do this all the time as well. But I never really had the
problem in typing Drupal, that was always Durpal.

[greg-1-anderson]

I have a conflict on Monday, and I won't be at BADCamp either. :( I will continue to monitor this issue; perhaps we can discuss at a future SFDUG, if there is no movement sooner than that.

[quicksketch]

I'm working on making this at least possible by giving Drush a way to identify that it's working with Backdrop in #117. However, that issue requires that we initially break existing compatibility with Drush, which means we need a different way of installing Backdrop from the command line when we run our tests, so I made an issue for that as well at #118.

Once we give Drush a way to distinguish between Backdrop and Drupal 7/8, we can work on officially making Drush support Backdrop. Right now it's not possible with Backdrop just being an unidentified version of Drupal somewhere between 7 and 8.

[jenlampton]

I think the next step here is to add a backdrop.drush.inc file into the core directory, so that Drush knows how to work with Backdrop. This will need to come after (or at the same time as) the new discovery mechanism is developef for Drush, over at drush-ops/drush#342.

[jenlampton]

Okay, I found it.

You were right @greg-1-anderson, it turns out drush_drupal_version in drupal.inc wasn't able to identify backdrop as any particular version of Drupal because of the /core directory, and because VERSION is not defined in Backdrop, instead we use BACKDROP_VERSION, but even so that would have returned 1 and not 7.

I added some code to drupal.inc to detect that the site is backdrop in https://github.com/jenlampton/drush/tree/backdrop-drupal-version, but I feel like this isn't the right place for this. Should we instead include a backdrop.inc, or, what is the best approach here to keep the Backdrop out of Drush?

[uberhacker]

After following Jen's steps in #47 (comment) above, I was able to get drush working with the following patch to the latest master version.

diff --git a/includes/drupal.inc b/includes/drupal.inc
index b24dc52..d25c362 100644
--- a/includes/drupal.inc
+++ b/includes/drupal.inc
@@ -49,7 +49,7 @@ function drush_drupal_version($drupal_root = NULL) {
       else {
         // D7 stores VERSION in bootstrap.inc.
         // D6 and below does it in system.module.
-        $version_constant_paths = array('/includes/bootstrap.inc', '/modules/system/system.module');
+        $version_constant_paths = array('/includes/bootstrap.inc', '/modules/system/system.module', '/core/includes/drupal.inc');
         foreach ($version_constant_paths as $path) {
           if (file_exists($drupal_root . $path)) {
             require_once $drupal_root . $path;

[uberhacker]

This is ridiculously hackish, but if you don't want to touch drush, you can also create ./modules/system/system.module in your Backdrop directory with the following contents:

<?php
/**
 * @file
 * Contains constants and function wrappers for Drupal 7.x compatibility.
 */

/**
 * The current system version.
 */
define('VERSION', '7.0-dev');

[greg-1-anderson]

How much of Drush works under that configuration? From earlier comments, it seems like the sql-* commands at least would not work. Do the pm-* commands work? What else?

[uberhacker]

Hey Greg,
I was able to get all database and module operations working. All the sql-* and pm-* commands work along with status, site-install, etc. In fact, I couldn't find a command that didn't work. I did get an error with the Redis contrib module, however. See below:

PHP Fatal error:  Cannot redeclare drupal_path_initialize() (previously declared in ./core/includes/drupal.inc:738) in ./modules/contrib/redis/redis.path.inc on line 40

Do you think it makes sense to move the version stuff into bootstrap.inc or system.module so Backdrop won't require a drush patch?

[uberhacker]

FWIW, I also added the lines from #47 (comment) above in ./includes/bootstrap.inc instead of ./modules/system/system.module and drush continues to work seamlessly.

[greg-1-anderson]

This seems like a reasonable interim approach to me. It's up to the Backdrop folks to decide if they want to add that file, though.

[jenlampton]

@uberhacker omg. I can't believe you got this working!!! 👍 How did you get around the problem of VERSION returning 7, but Backdrop needing the Sql6 class to be loaded? I'm still running into that issue.

Also, I'm conflicted about telling Drush that Backdrop is Drupal 7. At some point we may need to differentiate from what Drupal 7 is doing, and it seems that drush_drupal_version() might be the best/easiest way to do that. If we had B1 in there as @greg-1-anderson suggested, that might make our lives easier in the future.

Also, we should be careful about using things directly from drupal.inc. Yes, that is Backdrop's compatibility layer with Drupal, but we'll eventually want Drush to keep working, even without it.

[uberhacker]

@jenlampton, basically the premise here is fooling drush into thinking Backdrop IS Drupal 7. As long as the Backdrop project keeps Drupal 7 compatibility, this is possible, correct? I'll have to correct myself about drush site-install. Earlier I mentioned this was working. However, it doesn't work since it's assuming settings.php should be somewhere under ./sites. What is wrong with Backdrop continuing to use the current Drupal 7 ./sites infrastructure? I guess that may be a question for @quicksketch. This would eliminate the need of ./core/scripts/install.sh, right? Sorry if I'm oversimplifying things here.

[uberhacker]

It looks like drush dl will download contrib modules in Backdrop root instead of ./modules. So drush dl --destination=modules/contrib is needed.

[jenlampton]

As long as the Backdrop project keeps Drupal 7 compatibility

@uberhacker the backdrop project won't keep complete Drupal 7 compatibility, either now or in the future. The plan is to keep Backdrop 1 close to Drupal 7 to make things easier on contrib developers. Similarly, Backdrop 2 will be kept close to Backdrop 1, and Backdrop 3 close to Backdrop 2. We'll always be moving farther and farther away from Drupal 7 (Backdrop is not D7 long-term-support).

Tying ourselves to Drush for Drupal 7 might be shooting ourselves in the foot.

What is wrong with Backdrop continuing to use the current Drupal 7 ./sites infrastructure?

The biggest reason for moving it is to make it obvious to new users where to enter their database connection info. Burying that file in some drupal-specific file structure is an unnecessary hurdle. Bigger picture: do 80% or more of Drupal sites use a multi-site installation? No. We should not be making 100% of people use the sites directory. In Backdrop it's not used at all - unless it's specifically enabled.

How did you get around the problem of VERSION returning 7, but Backdrop needing the Sql6 class to be loaded? I'm still running into that issue.

I still don't understand how you were able to connect to a database without loading sql6. Unless you maybe were using a D7-style database connection array of arrays?

It looks like drush dl will download contrib modules in Backdrop root instead of ./modules. So drush dl --destination=modules/contrib is needed.

Were you able to get it to download modules from backdropcms.org?

[klonos]

the backdrop project won't keep complete Drupal 7 compatibility, either now or in the future. The plan is to keep Backdrop 1 close to Drupal 7 to make things easier on contrib developers. Similarly, Backdrop 2 will be kept close to Backdrop 1, and Backdrop 3 close to Backdrop 2. We'll always be moving farther and farther away from Drupal 7 (Backdrop is not D7 long-term-support).

Instead of trying the impossible then, why not fork drush into say "brush" and have it be Backdrop-specific only. What is the point of struggling to adapt and use a tool that was tailored specifically for Drupal when the future is to move away from Drupal. Why waste energy trying to "fit a camel through the eye of a needle".

[serundeputy]

Instead of trying the impossible then,

• tis very possible;

why not fork drush

• no;

drush will work it is already sufficiently abstracted our labors will bear fruit (that is to say it is not a waste) 🐉

[uberhacker]

@jenlampton
I didn't do anything special to get the database connection working. I simply created /includes/bootstrap.inc as explained above along with backdrop.drush.inc and the supporting class. Not sure how to download from backdropcms.org. Where would I configure that?

@serundeputy
I can completely understand why @msonnabaum does not want to support Backdrop in drush if the intention is to move away from the current Drupal infrastructure. It is the responsibility of the Backdrop folks to maintain compatibility if they want to continue using the current drush or else a Backdrop specific fork is inevitable. I like the goals of the Backdrop project and will continue to support it's progress but this thing with drush is definitely chasing a rabbit down the hole.

[btopro]

If moving away from the Drupal structure then fork drush as well. I think you'd lose a lot of potential users in doing this but that's a separate issue. One of the most appealing things with drush is the ability to control D6,7,8 and hopefully, Backdrop while utilizing (largely) the same command-set.

[greg-1-anderson]

Let's not worry about who is opposed to plans that are not moving forward anyway. There is plenty of support among the Drush maintainers to abstract Drush to the point where it can (a) be used to bootstrap and support other CMSs, such as Backdrop, and (b) the useful parts of it can be used by other cli tools that need similar functionality. Code re-use is a good thing, and working together is a good thing. Ultimately, not every project is going to be built in exactly the same way, or fill exactly the same use case. The way to manage fragmentation is to do the extra work to ensure that useful code is re-usable in "foreign" environments (applications they were not originally designed for). Separation of concerns is just good design.

[uberhacker]

The #1 principle of the Backdrop philosophy is "Backwards compatibility is important." I generally agree that abstraction is good but changing the infrastructure just for the sake of making it easier for project newcomers is not a compelling enough reason for change. I agree with all the points mentioned here but the question remains - what are we going to do until then?

[greg-1-anderson]

I should let @jenlampton speak for herself, but the impression I get is that your original suggestion is okay in the interim, but it seems that it might be insufficient (module download location, sql access, etc.), so other work (e.g. in progress Backdrop bootstrapping for Drush) is also necessary.

For an interim workaround to the download issue, you could commit a drushrc.php file to BACKDROP_ROOT_/drush and set $command_specific['dl']['destination'] = ... and $command_specific['dl']['source'] = ..., presuming that you are using drupal.org compatible release XML.

[uberhacker]

I can work around the issues but someone with less technical knowledge may have trouble with the download options. Below are the issues I noticed that need work:

1. Modules are downloaded (aka drush dl) to Backdrop root by default instead of the ./modules directory without a destination option.
2. Site install (aka drush site-install) looks in ./sites for a settings.php. Workaround is to use ./core/scripts/install.sh on a *NIX system. Windows batch file doesn't exist?
3. Not necessarily drush related but some Drupal 7 contrib modules are not compatible. The one I noticed is Redis.

Has anyone else been able to get drush working or am I the only one so far? :)

[jenlampton]

the impression I get is that your original suggestion is okay in the interim, but it seems that it might be insufficient (module download location, sql access, etc.), so other work (e.g. in progress Backdrop bootstrapping for Drush) is also necessary.

Yes, that's exactly right @greg-1-anderson. Unfortunately no one else has been able to get drush working with the 'original suggestion' so we'll need confirmation that's actually a viable approach. I'm 99% sure that other work is also necessary.

This is ridiculously hackish, but if you don't want to touch drush, you can also create ./modules/system/system.module in your Backdrop directory

Placing a system.module in the contrib modules directory is not a good idea, as we already have a core module called system.

FWIW, I also added the lines from #47 (comment) above in ./includes/bootstrap.inc instead of ./modules/system/system.module and drush continues to work seamlessly.

We specifically removed the VERSION definition from here and defined a BACKDROP_VERSION instead so that Drupal modules that required a Drupal version wouldn't get 7 unless the Drupal compatibility mode is on. Adding it back here would be a mistake.

If we need VERSION for Drush, a better solution would be to add it into the files that are used for only Drush (but not anything that is used by the rest of Backdrop). Let's find out why we can't add it in the BackdropBoot class instead.

Has anyone else been able to get drush working or am I the only one so far? :)

I wonder if you have been able to get Drush working @uberhacker. The db commands should not be working for you with the setup you describe above, and the fact that you think they are makes me think that something is awry with your set up.

Not necessarily drush related but some Drupal 7 contrib modules are not compatible. The one I noticed is Redis.

You shouldn't be trying to install Drupal modules on Backdrop in the first place. The command drush dl needs to pull Backdrop modules from backdropcms.org (as mentioned above) and this is a non-issue.

For an interim workaround to the download issue, you could commit a drushrc.php file ...

I have added this file into https://github.com/jenlampton/backdrop/tree/47-drush if anyone wants to give it a test.

[serundeputy]

I've followed the steps in comment 47 and put these two files backdrop.drush.inc and backdrop.inc https://gist.github.com/serundeputy/06cc047a8fc285714130 into my ~/.drush directory

and used the line from @uberhacker in drupal.inc

$version_constant_paths = array('/includes/bootstrap.inc', '/modules/system/system.module', '/core/includes/drupal.inc');

using:

• a checkout of drush 7.x branch
• a checkout of backdrop 1.x branch

it does download modules (from d.o) to the root of backdrop, i.e. drush dl redirect puts a redirect into your backdrop root.

drush cc all did remove entries from the cache_page table in my backdrop DB.

drush st was not reporting any DB connection though.

that is what I found for now.

[uberhacker]

I've attached bash shell scripts that get drush working. Albeit, this is hackish, but it does work. Feedback is welcome.

backdrop-install.sh

#!/bin/bash

# Assumptions
# 1. Operating system is an *NIX environment such as MAC/BSD/Linux
# 2. Git is installed
# 3. Drush is installed
# 4. MySQL/MariaDB is installed
# 5. Database named backdrop has been created and granted with user backdrop and password backdrop
# 6. Proper permissions are set where the user running this script can write to /var/www

BACKDROP=/var/www/backdrop
cd $HOME/.drush
git clone https://gist.github.com/06cc047a8fc285714130.git backdrop
drush cc drush
cd /var/www
git clone https://github.com/backdrop/backdrop.git
cd $BACKDROP
php core/scripts/install.sh --account-name=admin --account-pass=admin --db-url=mysql://backdrop:backdrop@localhost/backdrop
mkdir sites/default
ln -s $BACKDROP/settings.php $BACKDROP/sites/default/settings.php
mkdir includes
ln -s $BACKDROP/core/includes/drupal.inc $BACKDROP/includes/bootstrap.inc
drush status
drush sql-connect

backdrop-uninstall.sh

#!/bin/bash
BACKDROP=/var/www/backdrop
cd $BACKDROP
drush sql-drop -y
cd $HOME
rm -rf $BACKDROP
rm -rf $HOME/.drush/backdrop

[quicksketch]

To help avoid needing to place a fake bootstrap.inc file, I filed an issue and PR over in the Drush queue at drush-ops/drush#1724.

The problem we have right now is that drush_drupal_version() includes hard-coded logic. As our compatibility layer wants to expose itself as D7 but this function manually searches very-specific paths, moving the logic to the new Drush "Boot" classes makes it so that Backdrop can control how Drush sees it.

I had a lot of progress making a working Drush w/ Backdrop but I'm worried about the number of commands that won't "just work" with the compatibility layer. Some things like drush dl, drush updb, and drush up require a lot of modification to work with Backdrop and could have terrible consequences if they fail (imagine if drush up drupal replaced Backdrop with Drupal in your installation).

So I think what we'll want to do is clearly enforce what commands are allowed with Backdrop. If the compatibility layer works (drush sql-cli or drush cc all) then great, we'll white-list those. If the command needs rewriting/updating such as drush up, then we'll write our own version of those commands.

[quicksketch]

Yay, drush-ops/drush#1724 was fixed so that we can avoid the problem with drush_drupal_version(). I'll post my drush extension for Backdrop publicly once I get it working reasonably.

[quicksketch]

Here's my initial stab at Drush for Backdrop: https://github.com/quicksketch/backdrop-drush (!)

Things actually work pretty well. Installation instructions may be found in the project README.md file. Be sure you use the latest master branch of Drush for the time being.

Although @greg-1-anderson initially recommended that new BackdropBoot class extend the DrupalBoot7 class, I wasn't really comfortable with Backdrop just pretending to be D7, because some major commands like drush updb and drush dl are never going to be usable with the compatibility layer. We can still make these commands work, but they'll need to be written natively for Backdrop.

So right now BackdropBoot is entirely separate from Drupal and extends the BaseBoot class directly. No hacks and no compatibility layer, just Backdrop-native commands being executed through Drush.

I've filed several follow-ups with Drush (some have already been fixed), in particular drush-ops/drush#1749 seems like it would be helpful when using Drush with Backdrop.

Until that issue is fixed (or something similar), I've marked all Backdrop commands as "hidden" and then they swap themselves in when possible. For example drush cron actually executes drush backdrop-cron. We swap out the command on the fly to run the Backdrop-native one if you're within a Backdrop directory. Some commands like drush sql-cli don't need any modification at all and we run those commands directly.

[sirkitree]

I've reported some errors with this over in backdrop-contrib/backdrop-drush-extension#3

[serundeputy]

Thanks everyone drush is bootstrapping on Backdrop sites and we have some functionality in place:

I'm closing this and any issues or feature requests should now be reported in the backdrop-contrib/drush issue queue: https://github.com/backdrop-contrib/drush/issues

thanks everyone!