Add settings to build rpm #112

kinow · 2019-01-09T01:36:34Z

Not a requirement for Cylc, but could be useful. The .gitignore is to ignore the default build directory. This can also be changed via command line, but I thought simpler to leave the default directory and just ignore it.

The other commit is for adding setup.cfg. In this file, the groups of settings represent either commands or special lifecycle stages. So when you run python setup.py bdist_rpm, the bdist_rpm includes any parameters passed to that command.

For cylc I am including --requires to specify what's required for the Cylc RPM. It would be nice if we had something like python-isodatetime, so that we could have all dependencies properly managed when installing via pip or rpm, but that's not a must-have (we can ask users to simply install via pip for example).

The installation directory is being hard-coded, as the default value (at least in my environment) was /usr/local/, but the CentOS image I had access to was using /usr/.

Tested locally (in a comment).

kinow · 2019-01-09T01:50:36Z

Tested using the official Centos Docker image, tag centos7.

$ docker pull centos:centos7
centos7: Pulling from library/centos
Digest: sha256:184e5f35598e333bfa7de10d8fb1cebb5ee4df5bc0f970bf2b1e7c7345136426
Status: Image is up to date for centos:centos7

Then to build the RPM.

$ python setup.py bdist_rpm
running bdist_rpm
running egg_info
writing isodatetime.egg-info/PKG-INFO
writing top-level names to isodatetime.egg-info/top_level.txt
writing dependency_links to isodatetime.egg-info/dependency_links.txt
reading manifest file 'isodatetime.egg-info/SOURCES.txt'
writing manifest file 'isodatetime.egg-info/SOURCES.txt'
writing 'build/bdist.linux-x86_64/rpm/SPECS/isodatetime.spec'
running sdist
running check
creating isodatetime-2018.11.0
creating isodatetime-2018.11.0/isodatetime
...
...
Wrote: /home/kinow/Development/python/workspace/isodatetime/build/bdist.linux-x86_64/rpm/RPMS/noarch/isodatetime-2018.11.0-1.noarch.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.QZ8aVE
+ umask 022
+ cd /home/kinow/Development/python/workspace/isodatetime/build/bdist.linux-x86_64/rpm/BUILD
+ cd isodatetime-2018.11.0
+ rm -rf /home/kinow/Development/python/workspace/isodatetime/build/bdist.linux-x86_64/rpm/BUILDROOT/isodatetime-2018.11.0-1.x86_64
+ exit 0
Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.mlpHIk
+ umask 022
+ cd /home/kinow/Development/python/workspace/isodatetime/build/bdist.linux-x86_64/rpm/BUILD
+ rm -rf isodatetime-2018.11.0
+ exit 0
moving build/bdist.linux-x86_64/rpm/SRPMS/isodatetime-2018.11.0-1.src.rpm -> dist
moving build/bdist.linux-x86_64/rpm/RPMS/noarch/isodatetime-2018.11.0-1.noarch.rpm -> dist
$ mkdir /tmp/rpms
$ mv dist/*.rpm /tmp/rpms

And to test the RPM.

$ docker run -v /tmp/rpms/:/tmp/rpms -t -i centos:centos7 /bin/bash
[root@9ac01e68cd2a /]# python
Python 2.7.5 (default, Oct 30 2018, 23:45:53) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import isodatetime
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named isodatetime
>>> 
[root@9ac01e68cd2a /]# rpm -ivh /tmp/rpms/isodatetime-2018.11.0-1.noarch.rpm 
Preparing...                          ################################# [100%]
Updating / installing...
   1:isodatetime-2018.11.0-1          ################################# [100%]
[root@9ac01e68cd2a /]# python
Python 2.7.5 (default, Oct 30 2018, 23:45:53) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import isodatetime
>>> dir(isodatetime)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__']
>>> isodatetime.__version__
'2018.11.0'
>>> from isodatetime import data
>>> isodatetime.data.Duration(days=10)
<isodatetime.data.Duration object at 0x7f411f6b5808>
>>> duration = isodatetime.data.Duration(days=10)
>>> duration
<isodatetime.data.Duration object at 0x7f411f6b5870>
>>> print(duration
... 
... )
P10D
>>> print(duration)
P10D

So it appears to be working? This RPM could be generated during a release, and included in the GitHub download page (as other Python modules do). Or even published via a repository to centos/fedora/etc.

Cheers
Bruno

kinow · 2019-01-09T01:55:02Z

More about the generated RPM:

$ rpm -qp isodatetime-2018.11.0-1.noarch.rpm --provides
isodatetime = 2018.11.0-1
$ rpm -qp isodatetime-2018.11.0-1.noarch.rpm --requires
python(abi) = 2.7
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
$ less isodatetime-2018.11.0-1.noarch.rpm
isodatetime-2018.11.0-1.noarch.rpm:
Name        : isodatetime
Version     : 2018.11.0
Release     : 1
Architecture: noarch
Install Date: (not installed)
Group       : Development/Libraries
Size        : 572197
License     : LGPLv3
Signature   : (none)
Source RPM  : isodatetime-2018.11.0-1.src.rpm
Build Date  : Wed 09 Jan 2019 14:20:19 NZDT
Build Host  : kinow-VirtualBox
Relocations : /usr 
Packager    : "isodatetime devs"
Vendor      : "British Crown (Met Office) & Contributors"
URL         : https://github.com/metomi/isodatetime
Summary     : Python ISO 8601 date time parser and data model/manipulation utilities
Description :
isodatetime
===========
...
...
*** Contents:
/usr/lib/python2.7/site-packages/isodatetime-2018.11.0-py2.7.egg-info/PKG-INFO
/usr/lib/python2.7/site-packages/isodatetime-2018.11.0-py2.7.egg-info/SOURCES.txt
/usr/lib/python2.7/site-packages/isodatetime-2018.11.0-py2.7.egg-info/dependency_links.txt
/usr/lib/python2.7/site-packages/isodatetime-2018.11.0-py2.7.egg-info/top_level.txt
/usr/lib/python2.7/site-packages/isodatetime/__init__.py
/usr/lib/python2.7/site-packages/isodatetime/__init__.pyc
/usr/lib/python2.7/site-packages/isodatetime/__init__.pyo
/usr/lib/python2.7/site-packages/isodatetime/data.py
/usr/lib/python2.7/site-packages/isodatetime/data.pyc
/usr/lib/python2.7/site-packages/isodatetime/data.pyo
...
...
/usr/lib/python2.7/site-packages/isodatetime/timezone.py
/usr/lib/python2.7/site-packages/isodatetime/timezone.pyc
/usr/lib/python2.7/site-packages/isodatetime/timezone.pyo
/usr/lib/python2.7/site-packages/isodatetime/util.py
/usr/lib/python2.7/site-packages/isodatetime/util.pyc
/usr/lib/python2.7/site-packages/isodatetime/util.pyo

And it has 152K.

kinow · 2019-01-09T02:46:53Z

Travis-CI appears to be unhappy running pip (not related to this change?).

Also, we can't alter the distribution name. So the RPM installs isodatetime. Similarly, for Cylc it would be cylc. Following RHEL/Centos standard, I think it would be better if it were python-isodatetime and python-cylc.

(unless before running the python setup.py bdist_rpm command, someone did a quick shell + awk/sed/etc to replace name="isodatetime" by name="python-isodatetime" in setup.py...)

matthewrmshin · 2019-01-09T09:18:58Z

setup.cfg

+packager="isodatetime devs"
+
+[install]
+install-lib=/usr/lib/python2.7/site-packages


Is Travis CI failing because of this line?

Oh! You might have found the error! This would change the installation location for the pip package too... quite interesting. Will try to fix it tomorrow. Thanks for the great spot!

So yesterday after reading your comment at home, I thought... well, gotta have to maintain a spec file for the RPM... but really don't want to introduce that right now as we have other pending issues (python3, web gui, etc).

This morning, I spent 15 minutes just looking at the code and blinking only the necessary amount 👀 then realized that

if it's just the name parameter that we need to change, we could override it someway programmatically in setup.py (which is common, look at any medium/large project's setup.py). Found a good StackOverflow example that worked like a charm.

the install_lib had to be specified for me, as I am building under Ubuntu. Then, as we got Update to Python 3.7+ #111 merged, and I started using Anaconda, I realized it used /home/kinow/.../anaconda/usr/lib/....... which means that directory depends on the machine building the RPM... So I reckon it's fair to leave that setting as is, without any customization. And let dev/users build the RPM in CentOS, or even use Travis CI hacks to build with Docker+CentOS.

Updated pull request to have the custom name for bdist_rpm. To test, one can simply try:

python setup.py bdist, which will produce the default binary distribution, i.e. a .tar.gz file

python setup.py bdist_rpm, which will produce the RPM file and RPM source file

If anyone would like to test even further, just grab the RPM, and - in a system without isodatetime, try running rpm -ivh python-isodatetime*noarch.rpm, check the output, and the try importing isodatetime (e.g. python -c 'import isodatetime').

kinow · 2019-01-09T21:53:18Z

Pull request rebased after the Python3 pull request merge.

matthewrmshin

It appears to work in my environment.

matthewrmshin · 2019-01-10T11:36:40Z

One review enough for this.

matthewrmshin reviewed Jan 9, 2019

View reviewed changes

matthewrmshin added this to the next-release milestone Jan 9, 2019

matthewrmshin assigned kinow Jan 9, 2019

kinow added 2 commits January 10, 2019 10:07

Ignore distutils build directory (when rpm)

5221828

Change distribution name only when running bdist_rpm

0b7d62e

kinow force-pushed the add-settings-to-build-rpm branch from 15753f5 to 0b7d62e Compare January 9, 2019 21:42

matthewrmshin approved these changes Jan 10, 2019

View reviewed changes

matthewrmshin merged commit 51337e1 into metomi:master Jan 10, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add settings to build rpm #112

Add settings to build rpm #112

kinow commented Jan 9, 2019

kinow commented Jan 9, 2019

kinow commented Jan 9, 2019

kinow commented Jan 9, 2019 •

edited

Loading

matthewrmshin Jan 9, 2019

kinow Jan 9, 2019

kinow Jan 9, 2019

kinow commented Jan 9, 2019

matthewrmshin left a comment

matthewrmshin commented Jan 10, 2019

Add settings to build rpm #112

Add settings to build rpm #112

Conversation

kinow commented Jan 9, 2019

kinow commented Jan 9, 2019

kinow commented Jan 9, 2019

kinow commented Jan 9, 2019 • edited Loading

matthewrmshin Jan 9, 2019

Choose a reason for hiding this comment

kinow Jan 9, 2019

Choose a reason for hiding this comment

kinow Jan 9, 2019

Choose a reason for hiding this comment

kinow commented Jan 9, 2019

matthewrmshin left a comment

Choose a reason for hiding this comment

matthewrmshin commented Jan 10, 2019

kinow commented Jan 9, 2019 •

edited

Loading