Allow builds on pushing a tag

[maggie44]

Triggering a new push to the Balena cloud each time a pull request is merged is a little too eager for my liking. I would rather have greater control over when a new release is generated, so I can merge many pull requests together in to the repo before deploying a release. I usually do this by building workflows to run when a tag is pushed:

name: Deploy to BCR

on:
  push:
    tags:
      - "*"

jobs:
  balena_cloud_build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Turnstyle
        uses: softprops/turnstyle@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Deploy to Balena
        uses: balena-io/deploy-to-balena-action@master
        id: build
        with:
          balena_token: ${{ secrets.BALENA_TOKEN }}
          fleet: maggie0002/balena-device-ui

      - name: Log release ID built
        run: echo "Built release ID ${{ steps.build.outputs.release_id }}"

In this workflow though, I get the following error:

2022-04-22T19:10:48.0926755Z --------------------------------------------------------------------------------
2022-04-22T19:10:48.0927219Z [Warn] Node.js version "14.19.0" does not satisfy requirement ">=12.8.0 <13.0.0"
2022-04-22T19:10:48.0927533Z [Warn] This may cause unexpected behavior.
2022-04-22T19:10:48.0927970Z --------------------------------------------------------------------------------
2022-04-22T19:10:48.3431456Z  _            _
2022-04-22T19:10:48.3431806Z | |__   __ _ | |  ____  _ __    __ _
2022-04-22T19:10:48.3432336Z | '_ \ / _` || | / __ \| '_ \  / _` |
2022-04-22T19:10:48.3432561Z | |_) | (_) || ||  ___/| | | || (_) |
2022-04-22T19:10:48.3433211Z |_.__/ \__,_||_| \____/|_| |_| \__,_|
2022-04-22T19:10:48.3433335Z 
2022-04-22T19:10:48.3437068Z 
2022-04-22T19:10:48.3437873Z Logging in to balena-cloud.com
2022-04-22T19:10:48.5834856Z Successfully logged in as: ...
2022-04-22T19:10:48.5835083Z 
2022-04-22T19:10:48.5835222Z Find out about the available commands by running:
2022-04-22T19:10:48.5835404Z 
2022-04-22T19:10:48.5837286Z   $ balena help
2022-04-22T19:10:48.5837666Z 
2022-04-22T19:10:48.5838057Z For further help or support, visit:
2022-04-22T19:10:48.5839126Z https://www.balena.io/docs/reference/balena-cli/#support-faq-and-troubleshooting
2022-04-22T19:10:48.5839400Z 
2022-04-22T19:10:49.7307584Z --------------------------------------------------------------------------------
2022-04-22T19:10:49.7309044Z [Warn] Node.js version "14.19.0" does not satisfy requirement ">=12.8.0 <13.0.0"
2022-04-22T19:10:49.7309681Z [Warn] This may cause unexpected behavior.
2022-04-22T19:10:49.7310815Z --------------------------------------------------------------------------------
2022-04-22T19:10:50.4550023Z git version 2.34.2
2022-04-22T19:10:50.5992885Z Initializing SDK for https://api.balena-cloud.com)
2022-04-22T19:10:50.6930003Z ##[error]Push workflow only works with main branch. Event tried pushing to: refs/tags/0.0.6
2022-04-22T19:10:50.8314833Z Post job cleanup.
2022-04-22T19:10:50.9623943Z [command]/usr/bin/git version
2022-04-22T19:10:50.9678941Z git version 2.35.1
2022-04-22T19:10:50.9728604Z Temporarily overriding HOME='/home/runner/work/_temp/033939ec-2ac2-4f83-a790-b79b1f584ce1' before making global git config changes
2022-04-22T19:10:50.9731125Z Adding repository directory to the temporary git global config as a safe directory
2022-04-22T19:10:50.9738050Z [command]/usr/bin/git config --global --add safe.directory /home/runner/work/balena-device-ui/balena-device-ui
2022-04-22T19:10:50.9788024Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2022-04-22T19:10:50.9829507Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :
2022-04-22T19:10:51.0093157Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2022-04-22T19:10:51.0162122Z http.https://github.com/.extraheader
2022-04-22T19:10:51.0172896Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2022-04-22T19:10:51.0216905Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :
2022-04-22T19:10:51.0700148Z Cleaning up orphan processes

[brennanwilkes]

Any word on when this will be implemented?

[wes8ty]

I am also looking for this feature.

[jellyfish-bot]

[nucleardreamer] This has attached https://jel.ly.fish/7d122467-bbf2-4599-bf5a-0cb42a895c86

[pipex]

@Maggie0002 @brennanwilkes @wes8ty could you share more information about your release/deploy workflows? This will enable us to better design a general feature for the more general use case. Thank you!

[20k-ultra]

This sounds like the commit-to-master workflow described in the readme but triggers on tags.

I am a little concerned if we are just going to make the action start to support a lot of workflows. Most of the work on this action was around leveraging draft releases so you can test on a device before merging/finalizing.

There is a PR which will support this workflow but I want to check if we want to continue down this path of adding more workflows. Right now, building a release on tag event seems more useful than push to master so I'd argue for it to be the last new workflow to be added.

[thgreasi]

Hi,
You can try using balena-io/balena-ci@push-tag rather than balena-io/deploy-to-balena-action@master in case you want to test the not-yet-merged proof-of-concept PR that we have on this.
Let us know how it works for you.

Kind regards,
Thodoris

[wes8ty]

Hello,

So the current challenge I have with the example is that it publishes/creates a new container for every pull request. If you have 10+ people working on a project doing each of their pull requests... this is a lot of container versions... Second, a feature sometimes takes multiple pull requests to implement (at least for us) which means things are expected to break until all the pull requests are complete for that epic. You can't really test the pull request because it takes multiple people in parallel to make it come together. We want to do a release often enough to test using our CI / CD but not on each pull request to where we don't give the developers enough time to actually implement the pull request (unit tests + integration tests + feature + docs...ect).

When we create a release (with tag) this means:

• All planned features / pull requests have been completed and are closed. This would be based on a time cadence or just a need to have a release.
• All unit tests for each pull request have passed
• Integrations tests need to be performed to validate the release functions properly
• If all integrations tests pass and other QA processes, the release is upgraded from draft to release just like the commit-to-master workflow but contains multiple pull requests instead of one.

If we need to test something locally first, we will use the live push feature of Balena as a first-level validation that the code committed within the pull requests works with the hardware. Sometimes this becomes difficult because not everyone has the hardware to test so the only option is the commit-to-master workflow. We are experimenting with a VPN connection and using live push for developers that don't have all the hardware needed to locally test.

Hopefully, this helps with the use case at least for us.

[maggie44]

@wes8ty has listed some great points.

I can add a few other scenarios:

• When external services generate pull requests for you. For example, I have been using external services like Weblate and Crowdin that allow teams of translators to translate the various English strings in to other languages. When they do so, the software generates a pull request with the new changes. I usually want those changes to be combined in to the next release rather than generate a release of its own (their are ways around this such as having the action ignore those branches, but the push on tag solution is my usual chosen route as it allow more granular control).
• Depending on users, there are occasions where you try and keep the number of published releases as few as possible. Many users/clients/developers prefer larger updates less often rather than small updates every day/week. The push on tag allows me to combine updates over time and publish them all at once without having to have a bunch of pull requests pointing to other pull requests, or not be able to use separate branches for different features (an alternative being layered branches of main, dev, etc etc., but it gets messy).

[thgreasi]

@Maggie0002 @wes8ty I would love to get some feedback on my PR on this. As far as I can tell it should satisfy your needs, since I literally based it in the workflow provided at the start top of this issue.
Let me point you to the workflow I used in my test repo for this:
https://github.com/thgreasi/action-tester/blob/master/.github/workflows/balena_ci.yml

[maggie44]

@thgreasi @20k-ultra I haven't tested it, but assuming it runs without being a breaking change (i.e. no changes required to the .yml file) then I think it would serve all the various scenarios (as you say, based on the fact it was built around the initial post then safe to say it would serve my scenarios).

I can see how adding another criteria could broaden the scope and there is concern about opening a can of worms, especially as the pull request just adds one additional type:

-       if (context.eventName === 'push' && context.ref === `refs/heads/${target}`) {
+ 	if (
+ 		context.eventName === 'push' &&
+ 		(context.ref === `refs/heads/${target}` ||
+ 			context.ref.startsWith('refs/tags/'))
+ 	) 

That said, I think the change could be seen as finishing off the existing on -> push -> branches feature, rather than be seen as a new feature. Looking at the GitHub docs, the current push on branch that is already included in the action and action docs is grouped with push on tag feature, and this change would complete that grouping of functionality:

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onpushbranchestagsbranches-ignoretags-ignore

Considering there appears to be demand for tags, that GitHub groups those two features (branch and tag), and that pushing on tags is something we see a lot in other repos I'm thinking it's solid to include the change without having to worry about having to support other features in the future just yet?

[maggie44]

Edit: To really be sure that the grouping of functionality is included, it should probably be tested with glob patterns and excludes (!) for both branch and tags too (v1.*, 'releases/**', !releases/**-alpha). I don't see why excludes wouldn't work, but for glob patterns if the passed tag is used directly from the workflow rather than from the repo it could get an invalid format:

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onpushbranchestagsbranches-ignoretags-ignore

[wes8ty]

So I was able to test the balena-io/balena-ci@push-tag in my workflow. I do see the tags created but I cannot get the version to show up in Balena cloud.

- name: Deploy to [balena-cloud.com]
  uses: balena-io/balena-ci@push-tag
  with:
    environment: balena-cloud.com
    balena_token: ${{ secrets.BALENA_TOKEN }}
    fleet: REDACTED

I do have the balena.yml created in the root of the repository (same level as the Dockerfile) with the version specified.

[thgreasi]

but I cannot get the version to show up in Balena cloud.

Can you clarify which version you expected to see and where @wes8ty ?
If it's the 0.0.0+rev2 that you are referring to, this is based on the version field found in the balena.yml file in your repo. Eg:

type: sw.application
version: '1.2.3'

You can update that though using the actions in the UI (or w/ a PATCH the release.semver field, maybe part of your workflow) if the tag names that you use are intended to be used as release versions (note though that it needs to be semver compliant, thus you will need to be dropping the v prefix from the PATCHed value.

Update: just saw that you mentioned that you already have a balena.yml...
Can you share its trimmed down content?
Do we see any clue in the action's logs? (eg: Error: the version field in balena.yml is not a valid semver)
Does is show up if you use the action w/ the push-to-master workflow rather than tags?

[wes8ty]

Trimmed version. I have tried version: 0.1.1 and version: "0.1.1" combinations.

#
# Url: https://www.balena.io/docs/learn/deploy/deploy-with-balena-button/#balenayml-configuration-file
name: Manager
type: sw.application
version: 0.1.1
description: >-
  TEST
assets:
  repository:
    type: blob.asset
    data:
      url: "REDACTED"
data:
  applicationEnvironmentVariables:
    - TEST1: "/TEST"
  defaultDeviceType: imx8mm-var-dart
  supportedDeviceTypes:
    - imx8mm-var-dart

I do not see any errors in the workflow log.

[wes8ty]

Where is the is_final documented for Balena? If I wanted to make a release as is_final: False do I place this in the GitHub workflow?

- name: Deploy to [balena-cloud.com]
  uses: balena-io/balena-ci@push-tag
  with:
    environment: balena-cloud.com
    balena_token: ${{ secrets.BALENA_TOKEN }}
    is_final: False

I was able to fix the version also today (weekend break really helps). I was ignoring yaml files in the docker ignore file so that is why it was not working.

[thgreasi]

Hi @wes8ty,
afaik we do not support is_final as an input parameter for this action.
The action only sets the is_final field of a release to false when the draft release on PR workflow is used:
https://github.com/balena-io/deploy-to-balena-action#draft-release-workflow-recommended

If your use case is to only finalize the release from the dashboard at a later point, then you could probably workaround that limitation by having your fleet track (be pinned to) a specific release, and change it to track the newer release once you are comfortable with it:
https://www.balena.io/docs/learn/deploy/release-strategy/release-policy/#pin-fleet-to-a-release

[20k-ultra]

Hey everyone. I appreciate the discussion and wish we could implement this feature but I don't think we will.

This github action was built to work around leveraging draft releases (new release modelling change that we will get more documentation for). If we start adding workflows that do not support creating draft releases we will be straying further away from the goal of this action. Most if not all the time building this action was figuring out the complexities of making a release that can be referenced again later on another Github action.

I feel that we (Balena) made a mistake by allowing this action to work by pushing to master since that workflow does not use draft releases. I think that workflow should be removed to avoid confusion honestly. I understand how people could see a master workflow is not that much different from tag commit event so that should be supported too, therefore it (master workflow) should be removed.

I have been talking with some people though about how we can support other workflows that use draft releases. For example, on merge to master, a draft release is built and when a commit is tagged or release is made than that draft version is marked as final. That achieves the goal of this issue while leveraging draft releases.

The benefit of draft releases is that what you tested is what runs on your fleet. If you merge some code into master then build a release which your fleets pull, the build might have differences from resources it pulls at build time (what if there's an exploit and the package your Dockfile step downloads is not the same as what you ran in your development).

For those that just want to build a release without dealing with draft releases than I suggest checking out the community actions in the marketplace which just wrap the balena cli and you can simply run push with those.

[20k-ultra]

hey all again. If you give #181 a read I think we can make this action achieve what everyone wants.

[maggie44]

@20k-ultra @klutchell, thanks for taking a look at this.

I will give some thought to #181, I think you are right there is some room to make it more 'composable'.

In the meantime, I have taken on board what you said about this repo having an action with a very specific purpose, and that the community actions may be better suited. I think you are right, this repo had a very specific purpose, and there may be value in going back to something very simple and building up from there.

So I have had a swing at an action that does what I think I need it to do. It simply wraps the CLI and opens all the functions to the user, without being too prescriptive. Because it only wraps the CLI it also opens up other options like preloading, which is something I had also been meaning to work on. Hopefully I can entice some other community members in to contribute, I have looked around and there are lots of us in the community doing the same thing. I will try and pull everyone together on the forums and see if we can't pull our resources.

Going forward, I would like to see how people use it. I will give it some time and then poke around how different users are implementing it, and based on common use cases may look to build some of those features in.

As always, ideas, thoughts, experiments welcome.

https://github.com/maggie0002/balena-cli-action

[20k-ultra]

This feature was merged in v0.12.0 by #178.

Keep in mind we may introduce breaking changes since we are not at v1.0.0 of this action yet. I want the long term implementation for building on tag is done via #181.