Versioning strategy between Package Control and develop

[jwhendy]

I posted in the ST forums to try and help out with #149 . It was more complicated than I thought, but the thread might be of interest to you.

I forked this and tracked my own branch from ST3 to check the behavior. Upon pushing a commit to the tracked branch:

• Package Control -> Upgrade package worked if, I think, enough time passed between the last commit or since pushing. One time I tried and it said there were no updates, but another time it worked fine.
• the time I was told no updates were available, choosing Package Control -> Upgrade/Overwrite all packages did work.
• pushing a commit, waiting a bit, and starting ST3 does not trigger an auto-update of packages (not sure if it's supposed to)

The recommendation from @wbond was to simply have users clone the develop branch into the Packages directory. This does seem pretty straightforward and would bring the git log with it (allowing users to know for sure what commit they are on).

Initially, all I was trying to do was make it easier to tell what release I was really on per your question about whether or not I was up to date on a feature you added! I did solve that; one can do Package Control -> List Packages and you'll see the date of the latest commit. I was initially hoping that the console output of Deviot could be a different version than the release; they both show [ Deviot 2.1.1 ] so it's not obviously clear there's anything different about the release vs. develop package installed.

I looked in the code and you could make the develop branch have a different version in __init__.py... like __version__ = 'dev-branch'. To do make that a dynamic string based on the commit seemed complicated, though. The way Package Control installs is from the zip, so you don't get the .git folder with it and thus couldn't even look up the recent commit hash if you wanted to...

Feel free to do what makes sense for the package and your aims!

[gepd]

I've add something:

now when you have installed a dev version, the status bar will show the deviot version, at this moment it's Deviot 2.2.0.dev1

The right way to upgrade is using Package Control -> Upgrade/Overwrite all packages.

By default Package Control is set to update each 1 hour, so it will not work if you restart ST.

How to know when deviot is updated?

First you run Upgrade/Overwrite all packages option and then

• ST will display a window, asking to restart ST after the package is updated

• After restart, the status bar will show the version installed (it will be displayed only with .ino .cpp files)

I'm also testing the install_prereleases option, I will check what is more convenient to use

[jwhendy]

Nice! I like that. I think for anyone switching between release and dev versions, they get a nice visual cue of what version they are on.

That, combined with Package Control -> List Packages gives the current "version." I'm torn on how PC shows the date string vs. the commit, but date is still a helpful indicator:

[jwhendy]

Quick note, I had to look at the code to find out that you need a board selection before the status bar info is displayed.

Also, this was interesting as I just picked Uno to pick something, but that prompted me to notice that a past project that was open isn't recognized as being for a nanoatmega328, despite it being in platformio.ini already. Would that be something for a separate enhancement issue: pick up the board/env from any project that contains a platformio.ini?

[gepd]

It will recognize all the environments in the platformio.ini file, but it won't choose any from the list, maybe I can improve it when there is only one environment, but when there is more than one, is the user who should decide

At this moment the current environment is global, if you open a new project, the last environment used will be selected

[FichteFoll]

now when you have installed a dev version, the status bar will show the deviot version, at this moment it's Deviot 2.2.0.dev1

Note that to follow proper semver specifications, this should be 2.2.0-dev.1. See http://semver.org.

If you have the notion of "development release numbers" already, why not choose to tag pre-releases in the first place? The process was outlined in the referenced forum thread, but feel free to ask if anything is unclear. (You can even get update notes for pre-releases that way. I used this for PackageDev, for example.)

By default Package Control is set to update each 1 hour, so it will not work if you restart ST.

That's not exactly correct. Package Control checks for update on startup, but only if it hasn't already within the last hour.

[jwhendy]

Note that to follow proper semver specifications, this should be 2.2.0-dev.1. See http://semver.org.

Just perused and see that is the case, indeed. Items 9-11 were really helpful in understanding the guidelines.

[gepd]

Thanks @FichteFoll

Will the pre-releases have the same timing of the repositories option?

Package Control checks for update on startup, but only if it hasn't already within the last hour.

If that is the case Upgrade/Overwrite all packages will continue working?

Offtopic: Is it normal that Package Control keep installing the package as Deviot (Arduino IDE) instead of Deviot (folder instalation)?
will that affect the users who already have installed the plugin in the Deviot folder?

[jwhendy]

Offtopic: Is it normal that Package Control keep installing the package as Deviot (Arduino IDE) instead of Deviot (folder instalation)?

I think that's defined here. It looks like it's programmed to install to Deviot (Arduino IDE).

Per that thread above, @wbond says that pointing to a repo/tree/branch syntax will give the package the name repo, hence the develop branch installs as simply Deviot.

I'm torn on the pre-releases... I know it really doesn't really matter, but this annoys me (auto complete in git):

$ git checkout # [press tab]
develop                            v0.1.4-beta 
FETCH_HEAD                         v1.0.0 
HEAD                               v1.1.0 
messages-update                    v1.1.10 
ORIG_HEAD                          v1.1.2 
origin/develop                     v1.1.3 
origin/feature/wrong_INO_lines     v1.1.4 
origin/HEAD                        v1.1.5 
origin/master                      v1.1.6 
origin/messages-update             v1.1.7 
origin/pio_structure_checkbox      v1.1.8 
origin/versioning                  v1.1.9 
pio_structure_checkbox             v1.2.0 
upstream/develop                   v1.2.1 
upstream/feature/wrong_INO_lines   v1.2.2 
upstream/HEAD                      v1.2.3 
upstream/master                    v1.2.4 
v0.1.0-beta                        v1.2.5 
v0.1.2-beta                        v2.0.0 
v0.1.3-beta                        v2.0.1 
v0.1.4.1.1-beta                    v2.1.0 
v0.1.4.1-beta                      v2.1.1

At least from my perspective, who cares [much] about a release after it's out, and especially after it's a few versions back? With some of your quick responses to my issues, it would seem like an annoyance to create a pre-release every time you push a change so I can test it? It seems cumbersome for true development pace, and that the -dev.x strategy is less hassle.

This is clearly up to you, but I don't know that I see the advantage of a true git tag. Seems like a lot of overhead. Skimming recent versions you have maybe 10-50+ commits between releases. That's a lot of tags to issue for development!

[gepd]

I think that's defined here. It looks like it's programmed to install to Deviot (Arduino IDE).

Per that thread above, @wbond says that pointing to a repo/tree/branch syntax will give the package > the name repo, hence the develop branch installs as simply Deviot.

In that case, master and develop branchs had the same repo name, the new name is only in the json file you pointed, there's my question.

it would seem like an annoyance to create a pre-release every time you push a change so I can test it? It seems cumbersome for true development pace, and that the -dev.x strategy is less hassle.

I was thinking the same, some times there are users testing commit by commit and the pre-release option won't be so comfortable in that case.

[jwhendy]

Not sure I understand the question. All branches here Deviot, but Package Control has been instructed to call it Deviot (Arduino IDE) in d.json.

For user-added repos, there's no mechanism to provide another name, so it just calls it the repo name. More specifically, I think it's called whatever it downloads the snapshot as. I suppose it could see if the repo base matches a foo.json entry, but I didn't get the sense they want to take on more complexity.

I tried an experiment to fiddle with naming. material-theme is defined in Package Control, but I did git clone [url] mat_theme in the Package directory:

Package Control -> List Packages

It calls it whatever the folder is called. I think all foo.json does is tell Package Control what to name the folder when it downloads.

---

I noticed the "previous_names": ["Deviot"] line in d.json, which looks like Package Control's mechanism to handle renaming. This should cover users who have the previous name installed and still trigger the update based on the docs. Guessing you could also swap back if you wanted and do "previous_names": ["Deviot (Arduino IDE)"].

[FichteFoll]

If that is the case Upgrade/Overwrite all packages will continue working?

This will force an update check.

---

All that said, I still don't understand why you don't just git clone the develop branch and the git pull repeatedly to pull updates. If you want to check every commit, shouldn't you be using the software that introduces the concept of a "commit" directly?

Furthermore, you can write packages in a way that they are not affected by the name they are actually installed as. You'd need to determine the package's name when the module is loaded (either with __package__ on ST3 or with __file__ and the current working dir on ST2) and replace all occurences of it in the code with that string. For imports, use relative imports (for ST2 you must use absolute imports, because ST was kinda weird back then).

Another thing you could do is distribute your own repository.json file with the package listed there and add that via Add Repository instead of the git-repo/tree/branch URL. This allows everything that Package Control supports, i.e. overriding the package name and specifying a branch as a release base, which will then think of each commit as a release.
See https://github.com/wbond/package_control/blob/master/example-repository.json.

[gepd]

All that said, I still don't understand why you don't just git clone the develop branch and the git pull repeatedly to pull updates. If you want to check every commit, shouldn't you be using the software that introduces the concept of a "commit" directly?

The 'problem' is, I know there will be users not familiar with this option, or will have problem to install git

Furthermore, you can write packages in a way that they are not affected by the name they are actually installed as. You'd need to determine the package's name when the module is loaded (either with package on ST3 or with file and the current working dir on ST2) and replace all occurences of it in the code with that string. For imports, use relative imports (for ST2 you must use absolute imports, because ST was kinda weird back then).

Luckily, I had only two lines in my code with this problem and I've already solved them.

Another thing you could do is distribute your own repository.json file with the package listed there and add that via Add Repository instead of the git-repo/tree/branch URL. This allows everything that Package Control supports, i.e. overriding the package name and specifying a branch as a release base, which will then think of each commit as a release.

Thanks, maybe this could be a much easier solution. I'm testing it now.

---

@jwhendy I was thinking in the name option, in the repository file, only as a label. I thought it would take the instalation folder name from the repo name, but I was wrong.

The last suggestion from FichteFoll could be the solution.

[jwhendy]

Not sure the best place to file this, as it applies to the current route to tackle versions, namely a repository.json file. Just some observations/results after tinkering.

---

Maybe this is known, but it appears that a local repository definition will overwrite the package information of the matching package name in the default package_control_channel.

Specifically, I pointed ST3 to my forked repository.json to try some things and upon uninstall/reinstall, I only see my own Deviot version. Seems like a pro/con... seems more finicky if someone wanted to switch from release <-> develop, but also potentially less confusing (just one Deviot instance).

---

With the current method, I can confirm that commits are picked up using Package Control -> Upgrade Packages, so that's awesome. Maybe you already knew this... I just wasn't sure since the develop branch took on v2.1.5, so it was hard to tell if it was really develop or looking at master.

---

I'm not sure what your strategy was with these lines in __init__.py:

VERSION = (2, 1, 5)
__version__ = ".".join([str(s) for s in VERSION[:3]])
if(len(VERSION) > 3):
    __version__ +=  + VERSION[3]

It looks like you were anticipating appending a fourth number to the version? Since you were already using +=, I thought the second + might be a typo. I also got errors about needing a str. What do you think of this?

VERSION = (2, 1, 6, 1)
__version__ = ".".join([str(s) for s in VERSION[:3]])
if(len(VERSION) > 3):
    __version__ +=  '.dev.' + str(VERSION[3])

This is working for me in my versioning branch, and I get this in the bar:

[gepd]

Maybe this is known, but it appears that a local repository definition will overwrite the package information of the matching package name in the default package_control_channel.

Specifically, I pointed ST3 to my forked repository.json to try some things and upon uninstall/reinstall, I only see my own Deviot version. Seems like a pro/con... seems more finicky if someone wanted to switch from release <-> develop, but also potentially less confusing (just one Deviot instance).

It's known, and was what FichteFoll explain above. Package Control do not support switch between branches at this moment, the only way to do this, is forking the repo and use GIT. The idea of use repository.json is just that, override the package_control_channel when you want to back to the master branch, just remove the repository entry (Package Control > Remove Repository)

---

Yes, there is a typo there __version__ += + VERSION[3]

Instead of

VERSION = (2, 1, 6, 1)
__version__ = ".".join([str(s) for s in VERSION[:3]])
if(len(VERSION) > 3):
    __version__ +=  '.dev.' + str(VERSION[3])

I use:

VERSION = (2, 1, 6, '-dev.1')
__version__ = ".".join([str(s) for s in VERSION[:3]])
if(len(VERSION) > 3):
    __version__ += VERSION[3]

I think it's simplier that way

[FichteFoll]

Maybe this is known, but it appears that a local repository definition will overwrite the package information of the matching package name in the default package_control_channel.

Yes, this is intended. You'll have to add/remove the repository file from Package Control's list in order to switch between normal and develop releases.

[jwhendy]

Ah. I saw he said that it "allows everything that Package Control supports, i.e. overriding the package name", but took that to mean specifying Deviot (Arduino IDE), not overriding the default entry with the modified package meta data.

I use: ... I think it's simplier that way.

Looks good to me! Would you only add that fourth string when develop begins to take commits above/beyond the previous release on master?

This was primarily about the discussion of versioning in general, with you making the call on strategy. I do happen to like the solution, so I'm going to mark it as closed. Feel free to re-open, and thanks for taking a look and coming up the solution!

[gepd]

Looks good to me! Would you only add that fourth string when develop begins to take commits above/beyond the previous release on master?
Yes, that is my idea :)