Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CircleCI #3921

Closed
wants to merge 10 commits into from
Closed

CircleCI #3921

wants to merge 10 commits into from

Conversation

lukas-w
Copy link
Member

@lukas-w lukas-w commented Oct 27, 2017

This PR adds tests on CircleCI. CircleCI uses Docker containers to run builds in and allows us to specify the Docker image to use. The ones in use in this PR can be found at lukas-w/lmms-ci-docker. I suggest we move that repo to the LMMS organization.

Reasons to use CircleCI:

  • CI builds are a lot faster. The Docker approach means we can prepare our build environment once instead of setting it up every build (install step on Travis). Also, builds on CircleCI start almost instantly, Travis builds however can take a couple minutes to start.
  • We can use any Linux distro for building. Travis is limited to Ubuntu 12.04 or 14.04 at the moment. With CircleCI, we can easily upgrade to a more recent version of Ubuntu. This is something that we need to do so we can use @tobydox's new mingw PPA, his old one (targeting 14.04) will not receive any updates.
  • CircleCI supports scheduled (cron) jobs and allows us to "upload artifacts" to their servers on each build, making it a good candidate for building & hosting nightly builds. This PR uploads an AppImage on every build, but we may want to change that to just one a day.

CircleCI says they are working on supporting macOS builds and builds on different CPU architectures in the future. Configuration is a bit less intuitive than Travis, I had to use some advanced YAML syntax to avoid too much duplication, but I think it's still worth the switch.

I suggest we merge this and test it for a while to see how we like it and to decide if we want to drop (Linux) Travis builds in favour of CircleCI.

Tagging @tresf because I had to make some changes to the AppImage build scripts. Fuse is not supported inside Docker so I changed it to extract all of linuxdeployqt.AppImage contents.

MinGWCrossCompile.cmake needed fixing because for some reason SET(ENV{PKG_CONFIG} wouldn't work. PKG_CONFIG_EXECUTABLE serves the same purpose so the change should be harmless.

@lukas-w lukas-w requested a review from tresf October 28, 2017 09:56
@gi0e5b06
Copy link

Don't switch. Just add.

@lukas-w
Copy link
Member Author

lukas-w commented Oct 30, 2017

@gi0e5b06 No, there's no point in running the same tests twice, so we'll abandon one of the services eventually. Also, these are services generously offered to us at no cost, so I'd like to avoid using more resources than we need.

APPIMAGETOOL="$USERBIN/appimagetool"
LINUXDEPLOYQTDIR="$HOME/bin/linuxdeployqt"
LINUXDEPLOYQTBIN="$LINUXDEPLOYQTDIR/usr/bin"
LINUXDEPLOYQT="$LINUXDEPLOYQTBIN/linuxdeployqt"
Copy link
Member

@tresf tresf Oct 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$HOME/bin/foo is a search path for some of these portable tools and they were chosen that way for a reason. If fuse isn't supported, $HOME/bin/foo/bin/foo doesn't make sense.

Perhaps we just keep a squashfs-root folder to be consistent with upstream documentation. Also, according to the documentation, we should always execute the AppRun executable to setup the environment properly. This may not be an issue now, but we don't want to use the app other than how it was intended, and furthermore the AppRun can itself point to a wrapper around the launcher.

But the more I think about this the more I wonder if we should have the non-fuse logic as some sort of detect-and-fallback so that people that do this locally can continue to have a copy of the binaries in the default $HOME/bin location?

Copy link
Member

@tresf tresf Oct 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another idea (assuming we detect-and-wrap the fuse logic) is that we can also symlink to $HOME/bin/foo and point it to the AppRun, thus simulating what fuse would have done. Circle-CI may not be the only desktop with FUSE turned off. I remember @jasp00 not liking the reliance on FUSE either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's some inspiration around a wrapper, authored by one of the appimagetool maintainers. https://github.com/TheAssassin/redeclipse-appimage/blob/master/build-appimages.sh#L94-L96

DAYSOLD=2
if env -i which linuxdeployqt > /dev/null 2>&1; then
skipped "System already provides this utility"
elif ! find "$LINUXDEPLOYQT" -mtime -$DAYSOLD 2>/dev/null|grep -q "." > /dev/null 2>&1; then
elif ! find "$APPIMAGE" -mtime -$DAYSOLD 2>/dev/null|grep -q "." > /dev/null 2>&1; then
Copy link
Member

@tresf tresf Oct 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a side-note, I wrote-but-passionately-hate having this internal logic. If there's a submodule we can clone offering a curl|wget binary caching method, we could simplify this. (or move it to another script?)

@@ -84,7 +84,7 @@ IF(NOT DEFINED ENV{MINGW_DEBUG_INFO})
MESSAGE("* CMAKE_CXX_COMPILER : ${CMAKE_CXX_COMPILER}")
MESSAGE("* CMAKE_RC_COMPILER : ${CMAKE_RC_COMPILER}")
MESSAGE("* WINDRES : ${WINDRES}")
MESSAGE("* ENV{PKG_CONFIG} : $ENV{PKG_CONFIG}")
MESSAGE("* PKG_CONFIG_EXECUTABLE : ${PKG_CONFIG_EXECUTABLE}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you confirm this was available in CMake 2.8.9 ?

@zapashcanon
Copy link
Contributor

This could be a way to maintain a correct and up to date list of dependencies needed on common linux distro by using the corresponding (official) docker imager, e.g. debian:testing, debian:unstable, fedora:latest etc. I used to do this with gitlab-ci. It's much more easier than maintaining a wiki as we know clearly if the list is correct, and are warned when it's not anymore.

@lukas-w lukas-w mentioned this pull request Dec 5, 2017
7 tasks
@Reflexe
Copy link
Member

Reflexe commented Mar 16, 2018

They currently also support macOS builds: https://circleci.com/build-environments/#apple-platforms

@lukas-w
Copy link
Member Author

lukas-w commented Apr 21, 2018

The use of $HOME/bin appears to have been dropped in stable-1.2 and master, so I reset package_linux.sh.in to the version from master and re-implemented the fix for non-fuse build via 14575a3. @tresf, I also replaced the -mtime -$DAYSOLD logic with wget -N, which uses the HTTP last-modified header field to decide whether the download is needed.

echo " [.......] Downloading ($(uname -p)): ${url}"
wget "$url" -O "$LINUXDEPLOYQT" -q || (rm "$LINUXDEPLOYQT" && false)
wget -N -q "$url" || (rm "$filename" && false)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@lukas-w
Copy link
Member Author

lukas-w commented Apr 22, 2018

Squashed into 138281b, 19bc043, d9ea65a and 418bcce and merged to master.

@lukas-w lukas-w closed this Apr 22, 2018
@lukas-w lukas-w deleted the circleci branch April 22, 2018 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants