yum packaging precompiled kmod

Overview

Packaging templates for yum and dnf based Linux distros to build NVIDIA driver precompiled kernel modules.

For official packages see this table and developer blog post.

The main branch contains this README and a sample build script. The .spec files can be found in the appropriate rhel7, rhel8, and fedora branches.

Table of Contents

• Overview
• Deliverables
• Prerequisites
  • Clone this git repository
  • Download a NVIDIA driver runfile
  • Install build dependencies
• Demo
• Building with script
  • Fetch script from master branch
  • Usage
• Building Manually
  • Generate tarball from runfile
  • X.509 Certificate
  • Compilation and Packaging
  • Sign RPM packages with GPG signing key
• RPM Repository
  • Other NVIDIA driver packages
  • Python script
  • Generate metadata
  • Enable local repo
• Installing packages
• Modularity Profiles
• Presentations
  • GPU Technology Conference
  • Red Hat Summit
• Related
  • NVIDIA plugin
  • NVIDIA driver
• Contributing

Deliverables

This repo contains the .spec file used to build the following RPM packages:

note: XXX is the first . delimited field in the driver version, ex: 440 in 440.33.01

• RHEL8 or Fedora streams: latest and XXX

  kmod-nvidia-${driver}-${kernel}-${driver}-${rel}.${dist}.${arch}.rpm
  > ex: kmod-nvidia-440.33.01-4.18.0-147.5.1-440.33.01-2.el8_1.x86_64.rpm
  > ex: kmod-nvidia-450.51.06-5.6.11-300-450.51.06-4.fc32.x86_64.rpm

  note: requires genmodules.py to generate modules.yaml for modularity streams.

• RHEL7 flavor: latest

  kmod-nvidia-latest-${kernel}.r${driver}.${dist}.${arch}.rpm
  > ex: kmod-nvidia-latest-3.10.0-1062.18.1.r440.33.01.el7.x86_64.rpm

• RHEL7 flavor: branch-XXX

  kmod-nvidia-branch-XXX-${kernel}.r${driver}.${dist}.${arch}.rpm
  > ex: kmod-nvidia-branch-440-3.10.0-1062.18.1.r440.33.01.el7.x86_64.rpm

These packages can be used in place of their equivalent DKMS packages:

• RHEL8 or Fedora streams: latest-dkms and XXX-dkms

  kmod-nvidia-latest-dkms-${driver}-${rel}.${dist}.${arch}.rpm
  > ex: kmod-nvidia-latest-dkms-440.33.01-1.el8.x86_64.rpm

• RHEL7 flavor: latest-dkms

  kmod-nvidia-latest-dkms-${driver}-${rel}.${dist}.${arch}.rpm
  > ex: kmod-nvidia-latest-dkms-440.33.01-1.el7.x86_64.rpm

The latest and latest-dkms streams/flavors always update to the highest versioned driver, while the XXX and XXX-dkms streams/flavors lock driver updates to the specified driver branch.

note: XXX-dkms is not available for RHEL7

Prerequisites

Clone this git repository:

Supported branches: rhel7, rhel8 & fedora

git clone -b ${branch} https://github.com/NVIDIA/yum-packaging-precompiled-kmod
> ex: git clone -b rhel8 https://github.com/NVIDIA/yum-packaging-precompiled-kmod

Download a NVIDIA driver runfile:

• TRD location: http://us.download.nvidia.com/tesla/ (not browsable)

  ex: http://us.download.nvidia.com/tesla/440.33.01/NVIDIA-Linux-x86_64-440.33.01.run

• UDA location: http://download.nvidia.com/XFree86/Linux-x86_64/

  ex: http://download.nvidia.com/XFree86/Linux-x86_64/440.64/NVIDIA-Linux-x86_64-440.64.run

• CUDA runfiles: cuda_${toolkit}_${driver}_linux.run are not compatible.

  However a NVIDIA driver runfile can be extracted intact from a CUDA runfile:

  sh cuda_${toolkit}_${driver}_linux.run --tar mxvf
  > ex: sh cuda_11.1.0_455.23.05_linux.run --tar mxvf
  
  ls builds/NVIDIA-Linux-${arch}-${driver}.run
  > ex: ls builds/NVIDIA-Linux-x86_64-455.23.05.run

Install build dependencies

note: these are only needed for building not installation

# Compilation
yum install gcc

# Kernel headers and source code
yum install kernel-headers-$(uname -r) kernel-devel-$(uname -r)

# Packaging
yum install rpm-build

# Enable EPEL to install DKMS
yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(rpm -E %rhel).noarch.rpm
yum install dkms

Demo

Building with script

Fetch script from main branch

cd yum-packaging-precompiled-kmod
git checkout remotes/origin/main -- build.sh

Usage

note: distro: fedora32, rhel7, rhel8

./build.sh path/to/*.run ${distro}
> ex: time ./build.sh ~/Downloads/NVIDIA-Linux-x86_64-440.33.01.run rhel8

Building Manually

Generate tarball from runfile

mkdir nvidia-kmod-440.33.01-x86_64
sh NVIDIA-Linux-x86_64-440.33.01.run --extract-only --target .
mv kernel nvidia-kmod-440.33.01-x86_64/
tar -cJf nvidia-kmod-440.33.01-x86_64.tar.xz nvidia-kmod-440.33.01-x86_64

X.509 Certificate

Generate X.509 public_key.der and private_key.priv files.

Example x509-configuration.ini. Replace $USER and $EMAIL values.

openssl req -x509 -new -nodes -utf8 -sha256 -days 36500 -batch \
  -config x509-configuration.ini \
  -outform DER -out public_key.der \
  -keyout private_key.priv

Compilation and Packaging

note: Fedora users may need to export IGNORE_CC_MISMATCH=1

mkdir BUILD BUILDROOT RPMS SRPMS SOURCES SPECS
cp public_key.der SOURCES/
cp private_key.priv SOURCES/
cp nvidia-kmod-440.33.01-x86-64.tar.xz SOURCES/
cp kmod-nvidia.spec SPECS/

rpmbuild \
    --define "%_topdir $(pwd)" \
    --define "debug_package %{nil}" \
    --define "kernel $kernel" \
    --define "kernel_release $release" \
    --define "kernel_dist $dist" \
    --define "driver $version" \
    --define "epoch 3" \
    --define "driver_branch $stream" \
    -v -bb SPECS/kmod-nvidia.spec

# Kernel: 4.18.0-147.5.1
# Driver: 440.33.01
# Stream: latest
> ex: rpmbuild \
    --define "%_topdir $(pwd)" \
    --define "debug_package %{nil}" \
    --define "kernel 4.18.0" \
    --define "kernel_release 147.5.1" \
    --define "kernel_dist .el8_1" \
    --define "driver 440.33.01" \
    --define "epoch 3" \
    --define "driver_branch latest" \
    -v -bb SPECS/kmod-nvidia.spec

Sign RPM package(s) with GPG signing key

If one does not already exist, generate a GPG key pair

gpg --generate-key

Set $gpgKey to secret key ID.

gpgArgs="/usr/bin/gpg --force-v3-sigs --digest-algo=sha512 --no-verbose --no-armor --no-secmem-warning"
for package in RPMS/*/kmod-nvidia*.rpm; do
  rpm \
    --define "%_signature gpg" \
    --define "%_gpg_name $gpgKey" \
    --define "%__gpg /usr/bin/gpg" \
    --define "%_gpg_digest_algo sha512" \
    --define "%_binary_filedigest_algorithm 10" \
    --define "%__gpg_sign_cmd %{__gpg} $gpgArgs -u %{_gpg_name} \
      -sbo %{__signature_filename} %{__plaintext_filename}" \
    --addsign "$package";
done

RPM Repository

Other NVIDIA driver packages

RHEL8 or Fedora

Copy relevant packages from the CUDA repository

* dnf-plugin-nvidia*.rpm
* cuda-drivers-${version}*.rpm
* nvidia-driver-${version}*.rpm
* nvidia-driver-cuda-${version}*.rpm
* nvidia-driver-cuda-libs-${version}*.rpm
* nvidia-driver-devel-${version}*.rpm
* nvidia-driver-libs-${version}*.rpm
* nvidia-driver-NvFBCOpenGL-${version}*.rpm
* nvidia-driver-NVML-${version}*.rpm
* nvidia-kmod-common-${version}*.rpm
* nvidia-libXNVCtrl-${version}*.rpm
* nvidia-libXNVCtrl-devel-${version}*.rpm
* nvidia-modprobe-${version}*.rpm
* nvidia-persistenced-${version}*.rpm
* nvidia-settings-${version}*.rpm
* nvidia-xconfig-${version}*.rpm

RHEL7

Copy relevant packages from the CUDA repository

* yum-plugin-nvidia*.rpm
* cuda-drivers-${version}*.rpm
* nvidia-driver-${flavor}-${version}*.rpm
* nvidia-driver-${flavor}-NVML-${version}*.rpm
* nvidia-driver-${flavor}-NvFBCOpenGL-${version}*.rpm
* nvidia-driver-${flavor}-cuda-${version}*.rpm
* nvidia-driver-${flavor}-cuda-libs-${version}*.rpm
* nvidia-driver-${flavor}-devel-${version}*.rpm
* nvidia-driver-${flavor}-libs-${version}*.rpm
* nvidia-libXNVCtrl-${version}*.rpm
* nvidia-libXNVCtrl-devel-${version}*.rpm
* nvidia-modprobe-${flavor}-${version}*.rpm
* nvidia-persistenced-${flavor}-${version}*.rpm
* nvidia-settings-${version}*.rpm
* nvidia-xconfig-${flavor}-${version}*.rpm

Python script

wget https://raw.githubusercontent.com/NVIDIA/cuda-repo-management/main/genmodules.py

Generate metadata

mkdir my-first-repo
# Precompiled kmod package(s)
cp RPMS/*/kmod-nvidia*.rpm my-first-repo/
# Other NVIDIA driver packages
cp ~/Downloads/*.rpm my-first-repo/

RHEL8 or Fedora

createrepo_c -v --database my-first-repo/
python3 ./genmodules.py my-first-repo/ modules.yaml
modifyrepo_c modules.yaml my-first-repo/repodata

RHEL7

createrepo -v --database my-first-repo

Enable local repo

Create custom.repo file

[custom]
name=custom
baseurl=file:///path/to/my-first-repo
enabled=1
gpgcheck=0

Copy to system path for yum/dnf package manager

sudo cp custom.repo /etc/yum.repos.d/

Clean yum/dnf cache

yum clean all

Installing packages

note: XXX is the first . delimited field in the driver version, ex: 440 in 440.33.01

• RHEL8 or Fedora streams: latest, XXX, latest-dkms, XXX-dkms

  dnf module install nvidia-driver:${stream}
  > ex: dnf module install nvidia-driver:latest

  To switch streams, first uninstall and clear the current stream

  dnf remove nvidia-driver
  dnf module reset nvidia-driver
  

• RHEL7 flavors: latest, branch-XXX, latest-dkms

  yum install nvidia-driver-${flavor}
  > ex: yum install nvidia-driver-latest

  Then to install nvidia-settings

  yum install cuda-drivers

Modularity Profiles

• RHEL8 or Fedora profiles: default, ks, fm, src

  dnf module install nvidia-driver:${stream}/${profile}
  > ex: dnf module install nvidia-driver:450/fm

  The default profile (default) installs all of the driver packages for specified stream using transitive closure

  dnf module install nvidia-driver:${stream}/default

  note: do not need to specify default profile

  The kickstart profile (ks) is used for unattended Anaconda installs of CentOS, Fedora, & RHEL Linux OSes via a configuration file. This profile does not install the cuda-drivers metapackage, which otherwise would attempt to uninstall any existing NVIDIA driver runfiles via a %pretrans hook

  %packages
  @^Minimal Install
  @nvidia-driver:${stream}/ks
  %end

  note: any package warning is fatal to a kickstart installation

  The NvSwitch profile (fm) installs all of the driver packages, as well as Fabric Manager and NCSQ

  dnf module install nvidia-driver:${stream}/fm

  note: this is intended for hardware containing NvSwitch such as DGX systems

  The Source profile (src) installs only the contents of /usr/src/nvidia-${version} which provides nv-p2p.h and other header files used for compiling NVIDIA kernel modules such as GDRCopy and nvidia-fs

  note: this profile is only compatible with precompiled streams (latest, XXX); DKMS streams use kmod-nvidia-latest-dkms

  dnf module install nvidia-driver:${stream}/src

  note: this profile should be combined with another profile, i.e default, ks, or fm

  dnf module install nvidia-driver:${stream}/{default,src}

Presentations

GPU Technology Conference

• Precompiled Kernel Modules: Packaging & Deployment on RHEL8 with Modularity Streams
  • Fall 2020
  • video (MP4)
  • slides (PDF)

Red Hat Summit

• Simplifying NVIDIA GPU Driver Deployment on Red Hat Enterprise Linux
  • Spring → Summer 2020
  • video (MP4)
  • slides (PDF)

Related

NVIDIA plugin

• dnf-plugin-nvidia & yum-plugin-nvidia

  • https://github.com/NVIDIA/yum-packaging-nvidia-plugin

• nvidia-driver (and 6 more)

  • https://github.com/NVIDIA/yum-packaging-nvidia-driver

Contributing

See CONTRIBUTING.md