Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option 1: Simplify how we find extras package #33

Closed
wants to merge 2 commits into from

Conversation

jayofdoom
Copy link
Contributor

This completely removes the old logic. PR #35 keeps the older logic, hidden behind an attribute.

  • Current regexp breaks on kernels with double-digit versions (i.e.
    3.11.0-13-generic)
  • Can't find a sensible reason why we can't plug in uname -r directly

This is tested working on Ubuntu Precise 12.04 and tested in theory on Ubuntu Saucy 13.10 (although other problems prevent this cookbook from working there yet; see my recently filed issue).

@bflad
Copy link
Contributor

bflad commented Dec 19, 2013

Short of testing every kernel for versions the cookbook already supports, personally I'd much rather not break compatibility for the edge cases I've heard uname not working. Soon enough I'm sure we'll not even require AUFS (as shown by devmapper support on CentOS/Fedora/RHEL family machines), which will remove this need.

In the meantime, could we just limit the platform_version to just 12.04, 12.10, and 13.04? Or maybe just make the regex support two digit numbers via something like [0-9]{1,2}?

Curious what you think. Thanks!

@jayofdoom
Copy link
Contributor Author

I honestly can't think of a single situation where the longer grep would find something and not just return the entire output of uname -r. I can understand the paranoia, and will resubmit with the version gate per your request because I don't use anything older than Precise, but I honestly don't think there are any cases that uname -r wouldn't work for, and even if there were, they could probably be handled in a less complex way.

@jayofdoom
Copy link
Contributor Author

Actually, given in the docs you show only support for 12.04, 12.10, and 13.04, I'm not sure keeping the old way makes sense at all. A cursory apt-cache search on the three supported distros don't show me any kernels that this would break for, I'd suggest leaving it alone.

I'll leave this one (with fixes for the style failures in the build) and submit an alternate PR where we can hide the old kernel-finding logic behind an attribute, so anyone who might be broken could re-enable the old way of finding it, but honestly I think that's overkill.

@jayofdoom
Copy link
Contributor Author

PR #35 makes using the old logic an option. FWIW; I prefer completely removing it instead, but either solution should be amenable for Saucy.

- Current regexp breaks on kernels with double-digit versions (i.e.
  3.11.0-13-generic)
- Can't find a sensible reason why we can't plug in uname -r directly
@jayofdoom
Copy link
Contributor Author

aha; I have found a way this will fail.

This no longer checks to see if the linux-image-extra package exists before trying to install it. I'll add this check

- Only attempt to install package if it exists -- use apt-cache search
  to check for it
- Use ohai's kernel['release'] in lieu of shelling out to uname
@trinitronx
Copy link
Contributor

Tested this PR on saucy (13.10) and found that it fixes the problem I had with the current stable version of this cookbook (0.21.0 on community.opscode.com as of writing):

================================================================================
Error executing action `install` on resource 'package[linux-image-extra-3.11.0-12-generic
linux-image-extra-virtual
linux-image-extra-3.11.0-13-generic
linux-image-extra-3.11.0-14-generic]'
================================================================================


Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '127'
---- Begin output of apt-cache policy linux-image-extra-3.11.0-12-generic
linux-image-extra-virtual
linux-image-extra-3.11.0-13-generic
linux-image-extra-3.11.0-14-generic ----
STDOUT: linux-image-extra-3.11.0-12-generic:
  Installed: 3.11.0-12.19
  Candidate: 3.11.0-12.19
  Version table:
 *** 3.11.0-12.19 0
        500 http://us.archive.ubuntu.com/ubuntu/ saucy/main amd64 Packages
        100 /var/lib/dpkg/status
STDERR: sh: 2: linux-image-extra-virtual: not found
sh: 3: linux-image-extra-3.11.0-13-generic: not found
sh: 4: linux-image-extra-3.11.0-14-generic: not found
---- End output of apt-cache policy linux-image-extra-3.11.0-12-generic
linux-image-extra-virtual
linux-image-extra-3.11.0-13-generic
linux-image-extra-3.11.0-14-generic ----
Ran apt-cache policy linux-image-extra-3.11.0-12-generic
linux-image-extra-virtual
linux-image-extra-3.11.0-13-generic
linux-image-extra-3.11.0-14-generic returned 127


Resource Declaration:
---------------------
# In /tmp/vagrant-chef-1/chef-solo-1/cookbooks/docker/recipes/aufs.rb

 13:     package extra_package do
 14:       not_if 'modprobe -l | grep aufs'
 15:     end
 16:   end



Compiled Resource:
------------------
# Declared in /tmp/vagrant-chef-1/chef-solo-1/cookbooks/docker/recipes/aufs.rb:13:in `from_file'

package("linux-image-extra-3.11.0-12-generic
linux-image-extra-virtual
linux-image-extra-3.11.0-13-generic
linux-image-extra-3.11.0-14-generic") do
  action :install
  retries 0
  retry_delay 2
  package_name "linux-image-extra-3.11.0-12-generic\nlinux-image-extra-virtual\nlinux-image-extra-3.11.0-13-generic\nlinux-image-extra-3.11.0-14-generic"
  cookbook_name :docker
  recipe_name "aufs"
  not_if "modprobe -l | grep aufs"
end

The old and clunky way of searching apt-cache policy for a grep matched uname -r output was returning multiple results and breaking the cookbook. This PR fixed it for me 😸

@trinitronx
Copy link
Contributor

Looks like the naming scheme is quite ugly for the linux kernel image packages across Ubuntu releases. I just tested on 12.04 precise and found that apt-cache search linux-image-extra-3.5.0-23-generic was returning no results.

================================================================================
Recipe Compile Error in /tmp/kitchen/cookbooks/docker/recipes/default.rb
================================================================================


NoMethodError
-------------
undefined method `strip' for nil:NilClass


Cookbook Trace:
---------------
  /tmp/kitchen/cookbooks/docker/recipes/aufs.rb:4:in `from_file'
  /tmp/kitchen/cookbooks/docker/recipes/default.rb:10:in `from_file'


       Relevant File Content:

       ----------------------

       /tmp/kitchen/cookbooks/docker/recipes/aufs.rb:


         1:  case node['platform']
         2:  when 'ubuntu'
         3:    # Verify the package exists before we attempt to install it
         4>>   extra_package = Mixlib::ShellOut.new('apt-cache search linux-image-extra-' + node['kernel']['release']).run_command.stdout.split(' ').first.strip
         5:    unless extra_package.empty?
         6:      package extra_package do
         7:        not_if 'modprobe -l | grep -q aufs'
         8:      end
         9:    end
        10:
        11:    modules 'aufs' do
        12:      action :load
        13:    end



       [2013-12-30T20:01:45+00:00] ERROR: Running exception handlers

[2013-12-30T20:01:45+00:00] ERROR: Exception handlers complete
[2013-12-30T20:01:45+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
Chef Client failed. 7 resources updated
[2013-12-30T20:01:45+00:00] ERROR: undefined method `strip' for nil:NilClass
[2013-12-30T20:01:45+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
>>>>>> Converge failed on instance <default-ubuntu-1204>.
>>>>>> Please see .kitchen/logs/default-ubuntu-1204.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sudo -E chef-solo --config /tmp/kitchen/solo.rb --json-attributes /tmp/kitchen/dna.json  --log_level info]
>>>>>> ----------------------

Turns out that the only similar kernel images in precise (12.04) are 3.2.* and named with -virtual at the end.

vagrant@default-ubuntu-1204:~$ apt-cache search linux-image-extra-3.5.0-23-generic
vagrant@default-ubuntu-1204:~$ apt-cache search linux-image-extra-3.5.0-23
vagrant@default-ubuntu-1204:~$ apt-cache search linux-image-extra-3.5.0
vagrant@default-ubuntu-1204:~$ apt-cache search linux-image-extra-3.
linux-image-extra-3.2.0-23-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-24-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-25-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-26-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-27-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-29-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-30-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-31-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-32-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-33-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-34-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-35-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-36-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-37-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-38-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-39-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-40-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-41-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-43-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-44-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-45-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-48-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-49-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-51-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-52-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-53-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-54-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-55-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-56-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests
linux-image-extra-3.2.0-57-virtual - Linux kernel image for version 3.2.0 on 64 bit x86 Virtual Guests

In the Docker Installation Documentation, for Ubuntu 12.04 they support a 3.8 kernel version with package names: linux-image-generic-lts-raring linux-headers-generic-lts-raring

So this code probably has the best way to decide which one to use in general, but for 12.04 use those packages above.

Haven't tested on any other VMs yet, but if this still doesn't do the trick: maybe try to first split the output of either uname -r or node['kernel']['release'] on a dash '-' char, and then re-join only the first and second parts with a '-' (or use regex captures instead if you wish), then run the apt-cache search on that, and take the closest match.

@trinitronx
Copy link
Contributor

Something to note: Docker's Kernel Requirements. This pretty much limits us to installing a 3.8 kernel anyway. Since the linux-image-extra-* package really gives us extra kernel modules... namely: aufs, we don't necessarily need it after docker version 0.7. However, they still recommend using it if possible. They officially only support 12.04 and 13.04, and it looks like this cookbook has those as well as 12.10 in the test-kitchen config, so we're good on the testing front.

For Ubuntu 12.04 LTS, the packages they recommend already include AUFS support: linux-image-generic-lts-raring linux-headers-generic-lts-raring

For Ubuntu 13.04, the packages they recommend to use the stock 3.8 kernel and install the extra modules the essentially the same way we do in this PR: apt-get -y install linux-image-extra-$(uname -r)

@jayofdoom
Copy link
Contributor Author

It's really crummy that uname -r returns -generic for some of those :-X

I kinda agree that the packages they suggest are better, however, the earlier version of the kernel for Precise does work, and I don't think it's valuable to force users of the cookbook to upgrade kernel if they don't want to separately (not to mention no way to enforce reboot in chef, so it gives kinda inconsitent results if the user isn't expecting this).

I'll rework this to work with your failing case.

@jayofdoom jayofdoom closed this Jan 1, 2014
@jayofdoom
Copy link
Contributor Author

@trinitronx please see PR #35 for a version of this which shouldn't break precise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants