[4.x]: Plugin license keys using environment variables are frequently overwritten

[MoritzLost]

Description

We're using environment variables for the license keys for Craft and plugin licenses, so we can buy licenses and add them to the production environment directly. This is a much simpler process for sites going live, because it means we can buy the licenses and use them on a site without a developer having to include the new license key in the project config in a development environment and create a pull request. Besides that, it's good practice in general to keep secrets like license keys out of version control.

Our environment looks a little like this:

PLUGIN_LICENSE_KEY_NAVIGATION=XXXX-XXXX-XXXX-XXXX-XXXX-XXXX
PLUGIN_LICENSE_KEY_GOOGLE_MAPS=XXXX-XXXX-XXXX-XXXX-XXXX-XXXX

The license keys are defined using those environment variables in the config/project/project.yaml:

  navigation:
    edition: standard
    enabled: true
    licenseKey: $PLUGIN_LICENSE_KEY_NAVIGATION
    schemaVersion: 2.0.2

While Craft does support this usage, it causes problems when setting up a new project or creating a new development environment. If you don't have a license key yet, Craft will create a new license key and write it to the project config, overwriting the environment variable setting. This happens as soon as you use php craft install/craft, or view the plugin settings in the backend, or do anything at all really. The problem is that there's no way to request a new license key without Craft writing it to the project config – for development purposes, at this point we haven't bought the plugin licenses yet, since the project is still in development. Besides that, we use trial license keys for development and only use the real license keys for the live site.

Anyway, this means the steps to install a project using environment variables for license keys are pretty unintuitive and error-prone:

1. Install the project using php craft install/craft, during the installation (or at the latest once you visit the backend) Craft will create a new license key and write it to the project.yaml.
2. Copy the generated license key to the .env file.
3. Use git restore config/project/project.yaml to restore the previous project.yaml and php craft project-config/apply to force Craft to use the environment variable again.

Steps to reproduce

1. Create an environment variable with the license key for a commercial plugin (PLUGIN_LICENSE_KEY_NAVIGATION) and set it to the current license key used for the plugin.
2. Go to Settings -> Plugins and enter the environment variable with a $-sign in front in the field for the plugin's license key ($PLUGIN_LICENSE_KEY_NAVIGATION).
3. Check that the environment variable was written to config/project/project.yaml, replacing the hardcoded license key.
4. Now set the environment variable to an empty value in your .env and reload the Settings -> Plugins page.
5. At this stage, Craft has created a new license key for the plugin and written it to config/project/project.yaml, forcibly overwriting the environment variable setting.

Expected behavior

There should be a way to create development license keys without having them written to the project.yaml which leads to having to awkwardly reset & apply the file every time. I see two possible solutions:

• Have Craft detect if the license key is defined as an environment variable. Then, whenever it creates a new license key, write it to that variable in the .env file instead of the project.yaml. The commands setup/app-id and setup/security-key already do this, so it's not unprecedented.
• Provide a way to create new development license keys for plugins that doesn't involve Craft writing it to the project.yaml. This could be a Console Command or a function on id.craftcms.com. This would allow us to create license keys for a new dev environment and set it in the .env before the installation.

Actual behavior

There's no way to create new development license keys for plugins without Craft writing them to the project.yaml, which is not great if you're using environment variables to define plugin keys.

Craft CMS version

4.2.2

PHP version

8.1

Operating system and version

No response

Database type and version

No response

Image driver and version

No response

Installed plugins and versions

No response

[brandonkelly]

I was able to reproduce this, but only if the environment variables were defined in .env, and set to an empty value, e.g.:

PLUGIN_LICENSE_KEY_NAVIGATION=

If the environment variables didn’t exist at all, or were set to a dummy key like XXXX-XXXX-XXXX-XXXX-XXXX-XXXX, Craft was leaving them alone in the project config.

Just fixed it for empty values, for the next Craft 3 and 4 releases – Craft will now leave those alone, and mark them as invalid. It won’t generate new license keys for you anymore; you will need to purchase the plugins from plugins.craftcms.com and create/update the license environment variables.

This is a much simpler process for sites going live, because it means we can buy the licenses and use them on a site without a developer having to include the new license key in the project config in a development environment and create a pull request.

Worth mentioning that this isn’t really true since 3.6, which is actually tied to the reason your plugins were getting license keys automatically. Since 3.6, Craft will be assigned license keys for all installed plugins, with a “trial” status. You can commit chose license keys to your project config, and they can be purchased from production later on. The license key statuses will be changed to “active” without affecting the license key. So there’s no need to make any project config changes at that time.

So the only reason to use environment variables is if you want to keep the keys out of your project config (which is still perfectly valid).

[MoritzLost]

@brandonkelly Thank you! 🚀

Yes, having the environment variables set to an empty value is normal in our workflow. We have a starter project that uses environment variables for license keys, so whenever we set up a new project or a new team member sets up their local development environment for an existing one, the environment variables are empty, since they don't have a license yet. This was causing the project.yaml to be overwritten during the installation. Thank you for fixing that!

It won’t generate new license keys for you anymore; you will need to purchase the plugins from plugins.craftcms.com and create/update the license environment variables.

Just to be clear; we usually purchase the licenses when a project goes live, and work with trial licenses during development. If Craft won't generate trial license keys anymore, will we have any problems? Or can we just continue to use our development environments without license keys for the commercial plugins?

Worth mentioning that this isn’t really true since 3.6, which is actually tied to the reason your plugins were getting license keys automatically. Since 3.6, Craft will be assigned license keys for all installed plugins, with a “trial” status. You can commit chose license keys to your project config, and they can be purchased from production later on. The license key statuses will be changed to “active” without affecting the license key. So there’s no need to make any project config changes at that time.

This wasn't working for us before we switched to environment variables, though now I'm not sure what the exact problem was. Might've been related to our starter project which always included the same license key, so that license key was already bound to a different domain. Anyway, I'll test this once the next version with this fix is released!

[brandonkelly]

Yes, having the environment variables set to an empty value is normal in our workflow.

Yeah that’s obvious in hindsight. Took me a while to get there because I thought you were saying you used XXXX-XXXX-XXXX-XXXX-XXXX-XXXX as the starting value.

Just to be clear; we usually purchase the licenses when a project goes live, and work with trial licenses during development. If Craft won't generate trial license keys anymore, will we have any problems? Or can we just continue to use our development environments without license keys for the commercial plugins?

The original idea was to just have Craft treat empty values the same as invalid values (e.g. XXXX-XXXX-XXXX-XXXX-XXXX-XXXX) or when the environment variable was missing altogether – to treat the empty value as an invalid license.

But that would cause Craft to show license nag alerts at the top of every control panel page, which would get pretty annoying. And as I said it would force you to purchase a license at plugins.craftcms.com when you go live, which is at best a lateral move compared to your prior workflow.

On second thought I realized I could approach this a little differently: continue to treat empty environment variable values as if there’s no license key at all, so Craft requests that a trial license key be assigned (like before), but then at the point where we save out the new license key, we can just update the .env file rather than overwrite the environment variable reference. (Guess that’s probably exactly what you were hoping for in the first place 🙃)

Just made that change for the next release (22165ac).

[brandonkelly]

Craft 3.7.53 and 4.2.3 are both out with that change now.

[MoritzLost]

Yeah that’s obvious in hindsight. Took me a while to get there because I thought you were saying you used XXXX-XXXX-XXXX-XXXX-XXXX-XXXX as the starting value.

@brandonkelly Sorry for the confusion! I meant this as a placeholder for a valid license, should've been more explicit.

On second thought I realized I could approach this a little differently: continue to treat empty environment variable values as if there’s no license key at all, so Craft requests that a trial license key be assigned (like before), but then at the point where we save out the new license key, we can just update the .env file rather than overwrite the environment variable reference. (Guess that’s probably exactly what you were hoping for in the first place 🙃)

That's exactly what I was hoping for! Thanks a bunch, this will make our dev setup much simpler! 🚀

[jakobhaerter]

Thanks @brandonkelly and @MoritzLost!