Documentation of Configuration Management

[Drupaldise]

Hi, where can I find some docs about the configuration management?
How can i work with the staging dir in files/config/staging - or what is this?
What is the best practice to work with GIT (or other version control) with my exported config files?
In other words how can I deploy my config changes without uploading a file - instead using the filesystem?

Thanks for enlighten me. ;-)

[quicksketch]

Hi @Drupaldise! We have an article coming out in Drupal Watchdog next month that includes some details, but I think generally speaking we don't have documentation on managing config via the file system. We need to figure out even where this documentation would go between the API docs and the Backdrop handbook, but generally I think I can sum it up. Perhaps once we have a place to put it, we can get something official written up.

---

If you're going to be managing configuration outside the user interface (such as through version control or even FTP), you should move the config directories outside of the Backdrop "files" directory. Backdrop by default will put the configuration files in a randomly generated /files/config_[hash] directory, and points settings.php at this path like this:

$config_directories['active'] = 'files/config_b7c6057f5199d7aa921445a3d31d30df/active';
$config_directories['staging'] = 'files/config_b7c6057f5199d7aa921445a3d31d30df/staging';

Because the "files" directory is generally not in version control, you can make managing your config easier (and more secure) if this directory is moved outside the web directory. Generally that means up one level in the file system:

$config_directories['active'] = '../config/active';
$config_directories['staging'] = '../config/staging';

And you'll need to organize your Git repository or site directory to match this structure, so it would be something like:

• .git
• .gitignore
• config
  • active
  • staging
  • versioned (more on this later)
• docroot
  • .htaccess
  • core
  • files
  • layouts
  • modules
  • ... etc. (the Backdrop installation).

I included the .gitignore file in here because you'll probably want to make a few entries in this file. I'd suggest ignoring both the "files" directory and the active and staging config directories individually:

/config/active
/config/staging
/docroot/files

The key to all management of config is the "staging" directory. This directory will be empty all the time unless you're actively moving configuration. We're also introducing our own "versioned" directory that we'll manage in Git.

On your localhost or development machine. Configure your site as you would want to deploy to live. Then to deploy your changes simply copy the "active" files into the "versioned" directory, then check it into Git.

cp config/active/* config/versioned/
git add versioned -A
git commit -m "Updating config files."
git push origin master

Then on your live server, copy the files back out of "versioned" directory into the "staging" (NOT the "active") directory:

ssh [email protected]
git pull origin master
cp config/versioned/* config/staging/

You've now "staged" your configuration for deployment, but it's not yet active. You have to let Backdrop import the configuration to allow it to make necessary database changes and validate the configuration is acceptable (e.g. won't accidentally delete in-use fields).

So visit admin/config/development/configuration in your Backdrop site, where you should now see the staged changes. You can view the differences in each file that was modified, deleted, or added. Click the "Import all" button. If no validation errors occurred, the config will be imported and active. Now all menus, content types, theme settings, etc. that you configured on your localhost will be reflected on the live site.

In the future, we'll probably have a command-line tool (#47) to do this last step without the UI, but it's a good idea to use the UI to view the file diffs anyway.

Note that after you do the import, your "staged" directory will be empty again. Ready for the next time you do a deployment and copy from the "versioned" directory. If you would like to pull down the configuration from live, you can simply do the same process of copying the active directory to versioned, checking it in, pulling down on your local, copying to staging, then importing.

---

There may be other approaches that work as well. We've theorized ideas like managing the "staging" directory all the time, or a combination of symlinks, but so far I think the "just make another directory" for versioned config is the easiest approach we've encountered.

[jenlampton]

@quicksketch That's an interesting approach. I'm going to outline this approach below, along with the two others, in more detail for a better compare / contrast for anyone curious.

The extra directory approach

Configuration of config directory

config/
config/active
config/staging
config/versioned <-- track via Git

Additional set up

1. Move config outside of files (and preferably outside the docroot)
2. Update settings.php to point at the new locations of active & staging

Deploying to prod - moving upstream

1. local: overwrite versioned directory with active (deletions too!)
2. local: add, commit, push via Git
3. production: pull changes
4. production: copy files
5. production: run config importer (via UI)

Sync local from prod - moving downstream

1. production: overwrite versioned directory with active (deletions too!)
2. production: add, commit, push via Git
3. local: pull changes
4. local: copy files
5. local: run config importer (via UI)

Upsides:

1. All files tracked via Git will remain "up to date" after importing staged config

Downsides:

1. Accidental reverts are possible - see (1) below

There are two other approaches we've also been considering:

The versioned staging directory approach

This an idea from @populist and @davidstrauss based on how the config directory is being managed for Drupal 8 on Pantheon.

Configuration of config directory

config/
config/active
config/staging <-- track via Git

Additional set up

1. Move config outside of files (and preferably outside the docroot)
2. Update settings.php to point at the new locations of active & staging

Deploying to prod - moving upstream

1. local: overwrite staging directory with active (deletions too!)
2. local: add, commit, push via Git
3. production: pull changes
4. production: run config importer (via UI)

Sync local from prod - moving downstream

1. production: overwrite staging directory with active (deletions too!)
2. production: add, commit, push via Git
3. local: pull changes
4. local: run config importer (via UI)

Upsides:

1. No extra directories needed
2. One less step (than above) when moving upstream & downstream
3. When deployed, the config changes would "land" in your staging directory on production, ready to be imported.

Downsides:

1. Accidental reverts are possible - see (1) below
2. Git reports staging directory as always "out of date" - see (2) below

And this approach is my personal favorite (and what we'll be recommending in the Drupal Watchdog article, out this May):

The figure-8 approach

This approach was recommended by @heyrocker after creating CMI for Drupal 8. It involves using your settings.php file to define different active and staging directories for each environment.

Configuration of config directory

config/
config/dev-active <-- track via Git
config/prod-active <-- track via Git

Additional set up

1. Move config outside of files (and preferably outside the docroot)
2. Rename config/active to something like config/prod-active
3. rename config/staging to something like config/dev-active.
4. On your local site, update settings.php to point at dev-active for active, and prod-active for staging.
5. On your production site, update settings.php to point at prod-active for active, and dev-active for staging.

Deploying to prod - moving upstream

1. local: add & commit all files in config exactly as they are. push.
2. production: pull changes
3. production: run config importer (via UI)

When you pull upstream, the config files from what was the active directory on local land in what is now the staging directory on production - ready for import.

Sync local from prod - moving downstream

1. production: add & commit all files in config exactly as they are. push.
2. local: pull changes
3. local: run config importer (via UI)

When you pull downstream, the config files from what was the active directory on production land in what is now the staging directory on local - ready for import.

Upsides:

1. Once set up, you never need to think about which directory is which!
2. No extra directories needed
3. One more less step to move both upstream & downstream
4. Commit things as they are, no extra moving files around.

Downsides:

1. Git reports the (staging) directory as always "out of date" - see (2) below
2. Figuring this out may make your head explode

Here is a more detailed explanation of the two downsides mentioned above:

(1) ACCIDENTAL REVERTS ARE POSSIBLE: If there was ever a config change made on production and not actively copied out of the active directory and committed wherever it's supposed to be tracked, the next time code is deployed that production change would be reverted.

(2) GIT SHOWS STATUS: OUT OF DATE. Git is tracking and reporting all the deleted config files in the staging directory. Running a Git status check on any environment will show the deleted files. Developers may get used to seeing deleted files in staging, and train themselves to ignore those warnings. When a config file is actually deleted BY MISTAKE this is very likely to be overlooked!

[Drupaldise]

Hey Guys, thanks for pointing this out. It makes it more clear for me now. In my opinion there must be a topic for that on backdropcms.org Handbook. This feature is so important.

[heyrocker]

Note that my 'figure 8' idea stems from the very early stages of CMI, and I don't think it translates well to the way that the modern system (in Drupal or Backdrop) is architected and the thought-work that has gone into it since then. I hadn't considered the third directory option. It's a solid solution but I wish we had something a little more seamless. The whole thing of git status being out of date never really bothered me.

[jenlampton]

@heyrocker what is it about the figure-8 that doesn't translate well to how Drupal 8 & Backdrop are architected? What is it about the other two options that translates better? I still prefer the figure-8 approach since it removes the need for copying anything anywhere (and needing to carefully think about where to copy it). Also, this set-up lends itself nicely to having more than 2 environments (like on Pantheon or Acquia) just by adding additional directories.

Config	Local	Dev	Test	Live
jen-active	deploy
dev-active		deploy
test-active			deploy
live-active				deploy

[mikemccaffrey]

Move config outside of files (and preferably outside the docroot)

I'm pretty sure this is not possible when you are hosting on Pantheon, since the docroot is also the root of the git repository so you can't put it outside of that, and files is the only area that the web server has permission to update.

So, am I correct in assuming that with Pantheon, you need to keep the config files in the hashed directory in files? Has anyone figured out a way around that?

[quicksketch]

So, am I correct in assuming that with Pantheon, you need to keep the config files in the hashed directory in files? Has anyone figured out a way around that?

You could just add an .htaccess file to prevent it from being accessed directly. We're planning on adding one by default in the future over in #706. However I think the hash will stay either way, for sites that are not running Apache for basic security.

[mikemccaffrey]

FYI, the approach I ended up using on Pantheon was to leave the active config directory in its default location under the files folder where the pantheon web server can make changes, but to put the staging config directory in the site root under version control.

This actually enforces a workflow that I prefer to the default one where the contents of the staging directory are emptied after import. Since backdrop can't delete the files in the staging directory if they are under version control, I can go to the config management page at any time and see all the active config changes that have been made by users since the last code release, just like how with features you can see what has been overridden from the version in code.

The only messy part is how it generates dozens of "Warning: unlink() permission denied" errors after you hit Import All. We should get CMI to play nice with version controlled config directories, perhaps as part of the 2.x plans in #435.

[quicksketch]

The only messy part is how it generates dozens of "Warning: unlink() permission denied" errors after you hit Import All.

We could gracefully handle this one situation. Just suppress errors from unlink with the @ symbol and display a single message instead.

$deleted_all_files = TRUE;
foreach ($config_files as $file) {
  $deleted_all_files = @unlink($file) && $deleted_all_files;
}
if (!$deleted_all_files) {
  backdrop_set_message(t('Failed to deleted some configuration files from the staging directory.'));
}

[heyrocker]

Is there an argument to be made that you shouldn't delete after import at all? I know this argument was made during dev on d8 and i was against it, but i might be warming to it now.

[heyrocker]

I was trying to remember back to how this whole thing came about, and here's a discussion of the problem.

https://www.drupal.org/node/1697256#comment-6641656

It really stems from "How do you stage deletes?" When a config file previously existed and then disappears, the config system sees this as a delete. There are situations where this can get totally foobar'd if your import directory is not cleaned out after every use. Here is an example:

1. You import a bunch of stuff, and then your import is left behind.
2. In your active directory you delete a field.
3. You copy your config from active to import.
4. Your deleted field is still there, because you just copied files from one place to another.
5. When you import your field doesn't get deleted.

Now sure, you can be extremely careful when doing that copy from dir to dir, but we were trying to figure out how to manage that in a more foolproof manner. Note that the management of how to stage deletes was easily the most complicated problem we had to solve in the base system. You can read that entire thread for tons of alternate approaches we tried and finally abandoned.

Just wanted to throw that background out there, for people who might be thinking about trying to work around that directory emptying behavior.

[mikemccaffrey]

We could gracefully handle this one situation. Just suppress errors from unlink with the @ symbol and display a single message instead.

Or we could just check if the system has write permission in the staging directory, and present a single clear notice such as "backdrop cannot automatically remove the staging files".

It really stems from "How do you stage deletes?" When a config file previously existed and then disappears, the config system sees this as a delete. There are situations where this can get totally foobar'd if your import directory is not cleaned out after every use.

This is definitely a problem if the user is manually copying config files around, but should not come up if the config is in version control. We may want to think about how to handle each of these two very different use cases in their own fashion.

[mikemccaffrey]

@heyrocker: It really stems from "How do you stage deletes?" When a config file previously existed and then disappears, the config system sees this as a delete.

Personally, I'm thinking that a better solution for this would be to export a config file with a list of all the elements that have been deleted, which can be checked on import to see if anything needs to be removed. That seems much more foolproof than assuming that a missing file means that configuration and content should be deleted.

What would the drawbacks be to this approach?

[heyrocker]

We actually implemented and later backed out that approach. I highly
recommend you read the linked thread above, it goes into these discussions
in detail, and gives a really good overview of all the pros and cons.

On Mon, Dec 21, 2015 at 1:21 PM, Mike McCaffrey [email protected]
wrote:

@heyrocker https://github.com/heyrocker: It really stems from "How do
you stage deletes?" When a config file previously existed and then
disappears, the config system sees this as a delete.

Personally, I'm thinking that a better solution for this would be to
export a config file with a list of all the elements that have been
deleted, which can be checked on import to see if anything needs to be
removed. That seems much more foolproof than assuming that a missing file
means that configuration and content should be deleted.

What would the drawbacks be to this approach?

—
Reply to this email directly or view it on GitHub
#855 (comment)
.

[thamas]

Hi all,

I just started to collect information about and try Backdrop. And I'd like to tell you that it is really strange that such an important feature like this, which is listed in the second position of the Backdrop homepage(!), has not a dedicated documentation section!

So one have to start search for information and arrives to an issue queue where the actuality of the information available is not known…

[docwilmot]

Hello, I have personally determined to work only on documentation after version 1.4.0. Sorry about this.

[jenlampton]

@docwilmot ++

[thamas]

@docwilmot @jenlampton I know that a new release is coming in a few days and I can imagine how hard you work!
My comment is intended to be more general not specified to these very busy days.
Also adding a page to the documentation with one link to this (or a more relvant issue, blogpost etc.) could be a big step forward I think. :)

[serundeputy]

I've written a drush command bcex that uses the "The versioned staging directory approach".
That can be found at: backdrop-contrib/backdrop-drush-extension#74

And I'd be grateful for testers and reviews of that workflow.
This workflow keeps the git status clean and therefore useful.

This downside:

(1) ACCIDENTAL REVERTS ARE POSSIBLE: If there was ever a config change made on production and not actively copied out of the active directory and committed wherever it's supposed to be tracked, the next time code is deployed that production change would be reverted.

Is one I'm used to living with this in the features for deployment workflow and our team just says don't make config changes on production ever!

I'm for documenting this as well. If we can get consensus that using the versioned staging approach is acceptable or even desirable. I would volunteer to write this documentation.

Now with panttheon.yml file allowing for nested docroot this approach will work well with the Pantheon platform as well. I had/have some issues with testing on Pantheon (permissions), but I've got support requests in for them.

thanks,
~Geoff

[xmacinfo]

Is there any update on Full import/export documentation? The import part does not work.

While moving a Backdrop site to a new host, I add to import all the configurations manually by adding all the files in the staging folder. Something more automatic would have been more helpful. :-)

[Graham-72]

@xmacinfo I share your concern that this does not seem to be documented (if it is, where is it?), but the other day I successfully used the import part. It seems to be critical to do things in the right order.

Some time ago a made a personal copy of the procedure to follow - I think it may have been written somewhere by @jenlampton. This is it:

Suggested procedure for moving a site to new server space:

1. Back-up the existing site by:
   a) Using contributed module ‘Backup and Migrate’ to save site data tables and content as an sql file (only the ‘Quick Backup’ is needed).
   b) Export the current configuration as a gz file by using Configuration Management / Full Export.
   c) Make sure you have an up-to-date copy of the contents of the following directories:
   - files
   - layouts
   - libraries
   - modules
   - themes

2. Upload into the new server space

• Backdrop
• all contributed modules used by the site
• all files (except the config_.... folder)
• layouts
• libraries
• themes

3. Install Backdrop as a new site
4. Activate each module used in the site
5. Import the configuration gz file exported in step 1.
   The Configuration Management function will first load the json files into the staging folder and then check whether they can be loaded as active files. Any discrepancy, such as a missing module, will halt the process with an error message advising, for example, on which module is missing. Either install the missing module and try again, or if perhaps it is a result of a module that is no longer in use, delete the redundant json file from the staging folder before trying again.
6. Import the data tables
   Use Backup and Migrate to restore the site’s data tables saved in step 1.

[herbdool]

I had tried out putting the staging directory into version control, but in the case of a particular site, it can lead to problems. It's probably an edge case, but thought I'd mention it.

This site uses a service to deploy the code. It has the option to deploy via FTP or git. In this case it's deploying via FTP and will only FTP the files which it detects had changed. So what happens is that the staging folder will only contain the files that changed. Backdrop will assume that all the rest of the missing JSON files means the config was deleted. So the sync can't be used to import the new changes; it'll just ruin the site.

So I guess we'll just use a third folder to hold the versioned files and use the export/import tarballs instead, or connect via an ftp app and upload/copy the files from the versioned directory. Though it's bit of a bother in our workflow if we're mostly just using a service to deploy code.

[ghost]

I was bold and created a page about this topic on BackdropCMS.org: https://backdropcms.org/user-guide/configuration-management I don't have access to mark it unpublished, so it's live, but feel free to take it offline if it needs changes first.

I basically just copied @jenlampton's post re. the 3 methods above.

[klonos]

Thanks @BWPanda for doing that 👍 ...I have split that page into 3 separate child pages 😉

[klonos]

@xmacinfo, @Graham-72 is https://backdropcms.org/develop/moving-backdrop-your-local-live-server-using-cmi and https://backdropcms.org/develop/moving-backdrop-site-new-server what you were talking about, or do we need a separate documentation page for import/export specifically?

[klonos]

Our dev docs have the following structure right now (omitting certain unrelated topics):

...

• Moving Backdrop from local to live
• Moving a Backdrop site to a new server
• Configuration Management
  • The extra directory approach
  • The figure-8 approach
  • The versioned staging directory approach

I would like to propose the following restructuring:

• Moving a Backdrop site
  • from local to live
  • to a new server
• Configuration Management
  • The extra directory approach
  • The figure-8 approach
  • The versioned staging directory approach

Reasoning: The initial documentation for Config Management, as written by @BWPanda, was a single-page article, which included a 3 different approaches. Splitting each approach to its own page, allowed for the parent "Configuration Management" page to serve as a landing page. There is nothing substantial there at the moment, but we have a place for things to be added.

I think perhaps it would be helpful to also have a short tutorial that covers the basic principles for the benefit of newcomers and also old-timers like me who forget things.

Moving the 2 "Moving Backdrop ..." documentation pages under a parent page would allow the parent page to serve as a place for that intro documentation you are proposing @Graham-72

[jenlampton]

All these pages are on the API site, right? With our developer documentation?

…

On Sat, Jun 8, 2019, 5:40 PM Gregory Netsas ***@***.***> wrote: ...our dev docs have the following structure right now (omitting certain unrelated topics): ... - Moving Backdrop from local to live - Moving a Backdrop site to a new server - Configuration Management --The extra directory approach --The figure-8 approach --The versioned staging directory approach I would like to propose the following restructuring: - Moving a Backdrop site -- from local to live -- to a new server - Configuration Management --The extra directory approach --The figure-8 approach --The versioned staging directory approach Reasoning: The initial documentation for Config Management, as written by @BWPanda <https://github.com/BWPanda>, was a single-page article, which included a 3 different approaches. Splitting each approach to its own page, allowed for the parent "Configuration Management" page to serve as a landing page. There is nothing substantial there at the moment, but we have a place for things to be added. I think perhaps it would be helpful to also have a short tutorial that covers the basic principles for the benefit of newcomers and also old-timers like me who forget things. Moving the 2 "Moving Backdrop ..." documentation pages under a parent page would allow the parent page to serve as a place for that intro documentation you are proposing @Graham-72 <https://github.com/Graham-72> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#855?email_source=notifications&email_token=AADBER6JEPENB4BWX52C4NDPZQYPNA5CNFSM4A7R3V4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXIAAHQ#issuecomment-500170782>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADBERY5PO6WW5YOKUVOLBTPZQYPNANCNFSM4A7R3V4A> .

[klonos]

Nope; they live under the dev guide in b.org: https://backdropcms.org/develop/configuration-management

[jenlampton]

Let's move them. It's going to get really confusing if we start mixing up our docs.

…

On Sat, Jun 8, 2019, 5:46 PM Gregory Netsas ***@***.***> wrote: Nope; they live under the dev guide in b.org: https://backdropcms.org/develop/configuration-management — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#855?email_source=notifications&email_token=AADBER76LM2PNHKVODIKPZTPZQZE3A5CNFSM4A7R3V4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXIACIY#issuecomment-500171043>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADBER7BFLK37GGH4G44HTLPZQZE3ANCNFSM4A7R3V4A> .

[ghost]

@jenlampton Which of the config management approaches are used on Backdrop's sites:

• B.org: https://github.com/backdrop-ops/backdropcms.org/tree/master/config
• API: https://github.com/backdrop-ops/api.backdropcms.org/tree/master/config
• Forum: https://github.com/backdrop-ops/forum.backdropcms.org/tree/master/config

As developers working on those sites, do we have to use a specific method, or can we use whatever we like?

[jenlampton]

@BWPanda its all documented in the readme on each site:

• https://github.com/backdrop-ops/backdropcms.org/tree/master/config
• https://github.com/backdrop-ops/api.backdropcms.org/tree/master/config
• https://github.com/backdrop-ops/forum.backdropcms.org/tree/master/config

These sites are all using the Versioned staging directory approach.

As developers working on those sites, do we have to use a specific method

You have to use the method that's set up for the site. If you don't like the method we're using we can open a ticket and discuss -- and maybe choose a better one. But everyone needs to use the same method for each site.

[ghost]

These sites are all using the Versioned staging directory approach.

I think I'm confused then... The 'versioned' approach seems to indicate that:

1. Only the 'staging' directory is in git, and
2. "No extra directories needed." But we have to create a dev-active directory in addition to the two from git...

[jenlampton]

Every Backdrop site will always need an active directory. Maybe that bit of the docs needs to be updated? The point might have been that only one directory needs to be in version control. We've also added the live directory in Git on our sites, but only for convenience. It doesn't need to be comitted for this pattern to work.

[ghost]

Discussion re. possible changes to our config management approach are happening in backdrop-ops/backdropcms.org#612, so considering the documentation pages have been created for the 3 approaches in general, should we close this issue now?

[jenlampton]

Sure?

[markabur]

Is there any documentation besides this area?

https://api.backdropcms.org/documentation/working-configuration

That gives three different workflows, but doesn't give any sort of overview of how configuration management works in the first place. For example it says "all configuration is stored in JSON files instead" but it doesn't mention where these files are found.

Typos - "It's" should be "its" in two places:

https://api.backdropcms.org/documentation/working-configuration

Each has it's pros and cons, so please choose the one that best fits your needs.

https://api.backdropcms.org/documentation/versioned-staging-directory

When you git pull on the production server, the staging directory is copied in it's entirety.

[stpaultim]

@markabur - Good points.

This is probably not what you are looking for, but there are some videos that demonstrate how it works in our Youtube channel.

Here is a 30 minute demonstration:
https://youtu.be/9EAXrPYlGt0

[klonos]

Thanks @markabur 👍 ...I've fixed the typos + a grammar mistake. I've also tweaked https://api.backdropcms.org/documentation/working-configuration to mention where the config files are stored by default, as well as where you can look if you want to change that location.

Care to have a look and let us know if that looks a bit better now?

[indigoxela]

Not directly the same, but possibly related: It has also been suggested to make the docblocks of function config_get() and related easier to understand - currently they're pretty misleading.

[markabur]

Thanks @klonos, that's a little better. Really I was looking for some sort of technical explanation of how the system works. E.g. Do the files get read from the filesystem on every page request? Is it safe to edit the configuration files directly? Is there any sort of hierarchy for overriding config files in custom modules or themes, like there is with tpl files? That sort of thing. But I can dig around and figure it out -- I was just wondering of there were more docs that I hadn't seen.

[jenlampton]

but it doesn't mention where these files are found.

The configuration directories are specified in settings.php.

Do the files get read from the filesystem on every page request?

Yes.

Is it safe to edit the configuration files directly?

No.

Is there any sort of hierarchy for overriding config files in custom modules or themes, like there is with tpl files?

No, the only copy that matters is the one in the active directory (but all values can be overridden individually in settings.php)

Thank you for working on this for us!!!

[stpaultim]

Is it safe to edit the configuration files directly?

No.

This surprises me a bit. I need to follow up on this later and understand it better. It's not something I would do often, but I also would have assumed it was possible.

I suppose that "possible" and "safe" are not the same thing. ;-)

[jenlampton]

It's not something I would do often, but I also would have assumed it was possible.

It's possible, it's just not always safe.

Since database schemas are dependent on the state of the config files, changing a file alone (but not also updating the schema) could cause serious problems for your site. You would need to know if there are any ramifications of making the change in order to do it safely.

If there's a UI to change that setting in Backdrop, then the schema changes would handled as part of the submit or update handler on the form, or they could be triggered by using the config-sync UI. (These are the safe ways to manage configuration.)

If there's not a UI to change that setting in Backdrop, it could be because the schema changes that would be necessary haven't been written yet, either because they are too hard, or because that change isn't possible. (Sometimes the only "safe" way to make this kind of change is to delete the old thing and create a new one)

[heyrocker]

When we were developing the config system I always liked to say "90% of the time it will be fine, but the other 10% you'll hose your site so badly it will never recover."

…

On Mon, Nov 30, 2020 at 12:51 PM Jen Lampton ***@***.***> wrote: It's possible, it's just not safe. Since database schemas are dependent on the state of the config files, changing a file alone (but not also updating the schema) could cause serious problems. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#855 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABHY54UPUHRTEKG6HL32JDSSQAWNANCNFSM4A7R3V4A> .

[markabur]

So whenever I synchronize my files folder down from Pantheon, I should also download the latest database -- and vice-versa?

[jenlampton]

So whenever I synchronize my files folder down from Pantheon, I should also download the latest database

Either that, or you should run the config-sync UI on your local site (this is what I do with my Pantheon sites).

and vice-versa?

No, never vice-versa. Always use the config-sync tool when deploying, otherwise you are likely to overwrite anything that's happened in production while you were working locally.

I also recommend using different config directories on your local site than on Pantheon. (see https://www.jenlampton.com/blog/move-your-backdrop-site-to-pantheon for more details)

[markabur]

Thanks, by vice-versa I meant that whenever I copy the database down from Pantheon, I should also copy the config folder -- otherwise there could be schema issues, right? Basically, the idea is that the database snapshot is no good unless the corresponding config files are present.

Is there a way to update configuration programmatically? E.g. could I save a config file in a custom module, and use an update hook to copy it into the staging directory and then sync it? Or is the UI going to be involved no matter what?

[jenlampton]

by vice-versa I meant that whenever I copy the database down from Pantheon, I should also copy the config folder

Ah, that makes more sense :) Yes, this is correct.

Is there a way to update configuration programmatically? E.g. could I save a config file in a custom module, and use an update hook to copy it into the staging directory and then sync it?

Yes, this is possible (though, uncommon). It would be easier to commit the staging directory so you don't need to copy files around (see blog post above for how to do so safely on Pantheon), then use the update hook run the config-sync.

Adding the update hook step would make sense if you had another update that you needed to run anyway -- but if you don't, what's the motivation for adding one? Doing a config sync is simpler than running an update, and there are drush commands for both (for sites not on pantheon) if you wanted it scripted.

[markabur]

Well, on Pantheon the update functions can run automatically when the code is pushed, and there's no need to log into the site to complete the process, so to me it seems simpler and more foolproof to use an update function in that case.

If I'm picturing the workflow correctly -- to me, committing to the staging directory isn't ideal as the commit history wouldn't show which lines changed in the config file, and I probably want the changes stored with a module anyway. E.g. if I have a module that creates a content type or something, and I've exported the config to files stored in that module, then later when I make changes, I'd have to export the updated config into the module (for anyone who installs the module on a fresh site later) and a second copy of it to the staging folder (to update the current site). Yes? So I figure if my module is going to have the authoritative copy of some config anyway, then it would hopefully not be too complicated to deploy that exact file rather than a second copy of it.

TLDR: How do core and contrib deal with config changes? They don't ask people to use the config sync UI, or do they?

[jenlampton]

Well, on Pantheon the update functions can run automatically when the code is pushed

I was afraid you might say that. This isn't the case for Backdrop sites on Pantheon. That "Run updates" checkbox does not actually run the updates.

committing to the staging directory isn't ideal as the commit history wouldn't show which lines changed in the config file

It does for my sites. I commit the entire staging directory from the very beginning of the project. Every time I make a config change, I export the changed files from active to staging, and commit them. (If you never commit removed files from staging then it will track all the changes just like any other file.)

and I probably want the changes stored with a module anyway. E.g. if I have a module that creates a content type

Ah yeah. Using modules to manage content types is a rare occurrence in Backdrop sites now that we have config. Definitely still possible though.

then later when I make changes, I'd have to export the updated config into the module (for anyone who installs the module on a fresh site later) and a second copy of it to the staging folder (to update the current site). Yes?

Yep. But I expect you may still find copying a .json file twice faster than writing an update hook. :)

TLDR: How do core and contrib deal with config changes? They don't ask people to use the config sync UI, or do they?

No, they use update hooks.

The update loads the item that has been changed (a view, for example), modifies it exactly how needed (changes the page title, for example), and saves the item. No config files are ever re-imported, as you might loose any changes a particular site might have made to that item (a new field added to the view, for example).

If you were sure that on your site no other changes had been made to the content type, your update hook could re-import the config if you wanted :)

[herbdool]

When config is synced via the UI or drush it will also trigger hooks such as the one in locale to search for translatable config strings. If config files are directly changed these books don't get triggered.

[ghost]

@herbdool Not entirely true... Drush doesn't do this. See backdrop-contrib/backdrop-drush-extension#75 (comment) and backdrop-contrib/backdrop-drush-extension#162