Cannot fetching update on rhel-based os

[mrqwer88]

Hello!

I try run clair on fedora and centos 6(binary builded on fedora 21).
After start it have errors like -

2015-11-16 07:44:08.473519 I | updater/fetchers: fetching Ubuntu vulneratibilities
2015-11-16 07:44:08.473851 I | updater/fetchers: fetching Debian vulneratibilities
2015-11-16 07:44:08.474215 I | updater/fetchers: fetching Red Hat vulneratibilities
2015-11-16 07:46:11.600821 E | updater/fetchers: could not parse bzr branch output to get the revision number
2015-11-16 07:46:11.600855 E | updater: an error occured when fetching update 'Ubuntu': could not download requested ressource.

bzr versions 2.6.0 and 2.1.1.

Can you help fix it?

[mrqwer88]

Debian and Centos vulneratibilities fetched correctly in result.
How I see problem only with get Ubuntu CVE list.

[Quentin-M]

Hello,

Yeah, I think the output from bzr is different on your platform. You're having that error on both Fedora and CentOS, with bzr 2.6.0 and 2.1.1, right ?

I will try to reproduce it asap in order to fix it. Thanks.

[mrqwer88]

Yes - on both.
Thanks!

[lorenz]

Happens on Ubuntu 15.04 too, bzr 2.7.0dev1

[mrqwer88]

On Ubuntu 12.04 with bzr 2.5.1 too.

[Quentin-M]

Unfortunately, I have not been able to reproduce the error yet. I just tried on Ubuntu 12.04 with bzr 2.5.1 and go1.4.2 and Clair has successfully pulled the Ubuntu vulnerabilities (it takes a long time tho as their repo is very, very slow).

What is the output of, let's say: bzr branch lp:ubuntu-cve-tracker /tmp/ubuntu-cve-tracker on these machines ?

[lorenz]

I tracked it down, it's a locale issue. I'm working on a PR.

[mrqwer88]

Output looks like -

bzr branch lp:ubuntu-cve-tracker /tmp/ubuntu-cve-tracker
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data.  See "bzr help launchpad-login".
Branched 10225 revision(s).               

and on another

bzr branch lp:ubuntu-cve-tracker /tmp/ubuntu-cve-tracker
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data.  See "bzr help launchpad-login".
Ответвлены 10225 ревизий.                

[Quentin-M]

Thanks - I totally understand now.
Waiting for @lorenz's patch, otherwise I'll fix it.

[lorenz]

I have a patch ready, but I need to verify that it works as expected.

[lorenz]

Patch is in #9, could the others try it? It works just fine for me.

[Quentin-M]

That would be part of the fix.

However, I think that the regular expressions (https://github.com/coreos/clair/blob/f229083e1e52d2fea46cb7c69be05b5e5f32c680/updater/fetchers/ubuntu.go#L68) are still too specific. mrqwer88's bzr answers with Branched 10225 revision(s). with the s between parenthesis.

Should probably as well make them more flexible?

[lorenz]

My fix should probably be enough as it pretty much guarantees how the strings will look as long as upstream doesn't change them. @mrqwer88 Can you try #9 and tell us if it works?

[Quentin-M]

Yeah, my point is that I am afraid that revisions and revision(s) is not only a locale change. Let's see what @mrqwer88 has.

[mrqwer88]

Unfortunatly with national locale on Fedora -

write to Launchpad or access private data.  See "bzr help launchpad-login".
Ответвлены 10225 ревизий.            

problem reproduces with
git branch --list

• patch-1

If set LC_ALL to POSIX output -

Branched 10227 revisions. 

and work.

On centos 6 problem reproduced ( "Branched 10225 revision(s).")

[lorenz]

It is most likely enough:

$ locale
LANG=de_DE.UTF-8
LANGUAGE=de_DE:en
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=

With my environment patch:

$ LANG=C LANGUAGE=C locale
LANG=C
LANGUAGE=C
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=

This is essentially equivalent to LC_ALL=C, so if that problem is locale-related then that's enough. I tried to test on CentOS 6 (via Docker) but couldn't get bzr to run.

[mrqwer88]

Looks like with path it run like for me

LANG=C bzr branch lp:ubuntu-cve-tracker /tmp/ubuntu-cve-tracker2
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data.  See "bzr help launchpad-login".
Ответвлены 10227 ревизий.       

With LANGUAGE must work -

LANGUAGE=C bzr branch lp:ubuntu-cve-tracker /tmp/ubuntu-cve-tracker1
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data.  See "bzr help launchpad-login".
Branched 10227 revisions.                

[mrqwer88]

On centos 6 problem with "revisions." vs " revision**(s)**."

[lorenz]

@mrqwer88 How did you get bzr running on CentOS 6? For me it crashes with

TypeError: readline() takes exactly 1 argument (2 given)

[mrqwer88]

From epel -

LANGUAGE=C yum info bzr
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: ftp.hosteurope.de
 * epel: mirror.nsc.liu.se
 * epel-debuginfo: mirror.nsc.liu.se
 * epel-source: mirror.nsc.liu.se
 * extras: ftp.hosteurope.de
 * updates: ftp.hosteurope.de
Installed Packages
Name        : bzr
Arch        : x86_64
Version     : 2.1.1
Release     : 2.el6
Size        : 23 M
Repo        : installed
From repo   : base
Summary     : Friendly distributed version control system
URL         : http://www.bazaar-vcs.org/
License     : GPLv2+
Description : Bazaar is a distributed revision control system that is powerful, friendly,
            : and scalable.  It is the successor of Baz-1.x which, in turn, was
            : a user-friendly reimplementation of GNU Arch.

[lorenz]

Up to bzr 2.5 the old string 'revision(s)' is used, from 2.5 on 'revisions' is used. So we need to change the regex.

[mrqwer88]

May be more simple use -

bzr revno /tmp/ubuntu-cve-tracker2
10227

for get revision number?

[Quentin-M]

I am not against that, it is actually used here, we could use that everywhere.

[Quentin-M]

It should fix for everyone, please re-open if it is not the case.

[mrqwer88]

Thanks a lot!

Now it work good for me.

2015-11-17 08:32:58.357867 I | updater: updating vulnerabilities
2015-11-17 08:32:58.357916 I | updater/fetchers: fetching Ubuntu vulneratibilities
2015-11-17 08:32:58.358028 I | updater/fetchers: fetching Debian vulneratibilities
2015-11-17 08:32:58.359670 I | updater/fetchers: fetching Red Hat vulneratibilities
2015-11-17 08:35:59.727436 I | updater: update finished