Skip to content

Latest commit

 

History

History
1070 lines (833 loc) · 38.2 KB

BUILD.adoc

File metadata and controls

1070 lines (833 loc) · 38.2 KB
Raw

Vulkan® Specification Build Instructions and Notes

Table of Contents

Introduction

This README describes how to build the Vulkan API specification, reference pages, and\or other related targets.

It documents how to set up your build environment, build steps and targets, and contains some troubleshooting advice.

Building The Spec

First, clone the Khronos Github repository containing the Vulkan specification to your local Linux, Windows, or Mac PC. The repository is located at https://github.com/KhronosGroup/Vulkan-Docs/.

Next, install all the necessary build tools (see Software Dependencies below).

Finally, go to the root directory of your local repository clone, and do

$ make html

which builds an HTML5 specification output for the core Vulkan 1.1 specification, with no extensions included.

$ ./makeAllExts all

builds the spec targets html, pdf, styleguide, registry, manhtmlpages, and allchecks, with all registered extensions included.

ℹ️
Note

  • make all takes a long time to run, and generates outputs that are irrelevant for most users. Usually make html is used to update the HTML target, which is all that’s needed for quick review of changes made.

  • The default make options build a Vulkan 1.1 specification with no optional extensions.

  • The validusage target is not built as part of make all, due to it needing to be built with all extensions enabled. Building this target will fail otherwise.

These targets generate a variety of output documents in the directory specified by the Makefile variable $(OUTDIR) (by default, out/). The checked-in file out/index.html links to all these targets, or they can individually be found as follows:

Vulkan® Specification

  • html — Single-file HTML5 in $(OUTDIR)/html/vkspec.html, and KaTeX dependency in $(OUTDIR)/katex

  • chunked — Chunked HTML5 in $(OUTDIR)/html/chap?.html

  • pdf — PDF in $(OUTDIR)/pdf/vkspec.pdf

“styleguide” (Vulkan® Documentation and Extensions: Procedures and Conventions)

  • styleguide — Single-file HTML5 in $(OUTDIR)/styleguide.html

XML Registry schema document

  • registry — Single-file HTML5 in $(OUTDIR)/registry.html

Diff spec

  • diff_html — Single-file HTML5 in $(OUTDIR)/html/diff.html

Reference pages

  • manhtmlpages — File-per-entry-point HTML in $(OUTDIR)/man/html/*. Must be built with all extensions enabled (using makeAllExts).

Validator output

  • None at present. The allchecks target writes to standard output unless the underlying script is given additional options.

Valid usage database

  • validusage - json database of all valid usage statements in the specification. Must be built with ./makeAllExts (for now). Output in $(OUTDIR)/validation/validusage.json. A validated schema for the output of this is stored in $(CURDIR)/config/vu-to-json/vu_schema.json

Once you have the basic build working, an appropriate parallelization option to make, such as

make -j 8

may significantly speed up the reference page builds.

If you encounter problems refer to the Troubleshooting section.

Building Specifications For Different API Versions

The Makefile defaults to building a Vulkan 1.1 specification. This is controlled by Asciidoctor attributes passed in the Makefile variable $(VERSIONS) To instead build a Vulkan 1.0 specification, pass

VERSIONS="VK_VERSION_1_0"

on the make command line.

Building With Extensions Included

Extensions are defined in the same source as the core Specification, but are only conditionally included in the output. Asciidoctor attributes of the same name as the extension are used to define whether the extension is included or not — defining such an attribute will cause the output to include the text for that extension.

When building the specification, the extensions included are those specified as a space-separated list of extension names (e.g. VK_KHR_surface) in the Makefile variable $(EXTENSIONS), usually set on the make command line. When changing the list of extensions, it is critical to remove all generated files using the clean_generated Makefile target, as the contents of generated files depends on $(EXTENSIONS). There are several helper scripts which clean these files and then build one or more specified targets for specified extensions:

  • makeExt — generate outputs with one or more extensions enabled. Usage is makeExt extension-names target(s), where extension-names is a space-separated list of extension names, such as VK_EXT_debug_report. If more than one extension is specified, extension-names must be quoted on the command line.

  • makeKHR — generate outputs with all Khronos (VK_KHR_*) extensions enabled. Usage is makeKHR target(s).

  • makeAllExts — generate outputs with all Vulkan extensions enabled. Usage is makeAllExts target(s).

The target(s) passed to these scripts are arbitrary make options, and can be used to set Makefile variables and options, as well as specify actual build targets; you can, for example, do:

$ ./makeAllExts -j 8 VERSIONS="VK_VERSION_1_0" html

The Makefile variable $(APITITLE) defines an additional string which is appended to the specification title. When building with extensions enabled, this should be set to something like (with extension VK_extension_name). The makeExt, makeKHR, and makeAllExts scripts already do this.

The reference pages (the manhtmlpages target) must be built using makeAllExts; there are markup and scripting issues which will probably cause any more restricted set of refpages to fail to build.

Building A Highlighted Extension Diff

The diff_html target in the makefile can be used to generate a version of the specification which highlights changes made to the specification by the inclusion of a particular set of extensions.

Extensions in the Makefile variable $(EXTENSIONS) define the base extensions to be enabled by the specification, and these will not be highlighted in the output. Extensions in the Makefile variable $(DIFFEXTENSIONS) define the set of extensions whose changes to the text will be highlighted when they are enabled. Any extensions in both variables will be treated as if they were only included in $(DIFFEXTENSIONS). $(DIFFEXTENSIONS) can be set when using the make* scripts described above.

In the resulting HTML document, content that has been added by one of the extensions will be highlighted with a lime background, and content that was removed will be highlighted with a pink background. Each section has an anchor of #differenceN, with an arrow (⇒) at the end of each section which links to the next difference section. The first diff section is #difference1.

ℹ️
Note

This output is not without errors. It may instead result in visible [.added]##content## and [.removed]##content##, and so also highlights not being rendered. But such visible markup still correctly encapsulates the modified content.

Alternate and Test Builds

If you are just testing Asciidoctor formatting, macros, stylesheets, etc., you may want to edit vkspec.txt to just include your test code. The asciidoctor HTML build is very fast, even for the whole Specification, but PDF builds take several minutes.

Images Used In The Specification

All images used in the specification are in the images/ directory in the SVG format, and were created with Inkscape. We recommend using Inkscape to modify or create new images, as we’ve had problems using SVG files created by some other tools; especially in the PDF builds.

Validation Scripts

The allchecks Makefile target runs a Python script that looks for markup errors, missing interfaces, macro misuse, and inconsistencies in the specification text. This script is necessarily heuristic, since it’s dealing with lots of hand-written material, but it identifies many problems and can suggest solutions. This script is also run as part of the CI tests in the internal Khronos gitlab repository.

Our Asciidoctor Macros

We use a bunch of custom macros in the reference pages and API spec Asciidoctor sources. The validator scripts rely on these macros as part of their sanity checks, and you should use the macros whenever referring to an API command, struct, token, or enum name, so the documents are semantically tagged and more easily verifiable.

The supported macros are defined in the config/spec-macros/extension.rb asciidoctor extension script.

The tags used are described in the style guide (generated from styleguide.txt).

We (may) eventually tool up the spec and ref pages to the point that anywhere there’s a type or token referred to, clicking on (or perhaps hovering over) it in the HTML view will take reader to the definition of that type/token. That will take some more plumbing work to tag the stuff in the autogenerated include files, and do something sensible in the spec (e.g. resolve links to internal references).

Most of these macros deeply need more intuitive names.

Reference Pages

The reference pages are extracted from the API Specification source, which has been tagged to help identify boundaries of language talking about different commands, structures, enumerants, and other types. A set of Python scripts extract and lightly massage the relevant tagged language into corresponding ref page. Pages without corresponding content in the API spec are generated automatically, when possible (e.g. for Vk*FlagBits pages).

If for some reason you want to regenerate the ref page sources from scratch yourself, you can do so by

rm man/apispec.txt
make apispec.txt

The genRef.py script will generate many warnings, but most are just reminders that some pages are automatically generated. If everything is working correctly, all the man/*.txt files will be regenerated, but their contents will not change.

If you add new API features to the Specification in a branch, make sure that the commands have the required tagging and that ref pages are generated for them, and build properly.

Our stylesheets

We use an HTML stylesheet config/khronos.css derived from the Asciidoctor stylesheet factory “colony” theme, with the default Arial font family replaced by the sans-serif Noto font family.

Marking Normative Language

Normative language is marked as bold, and also with the purple role for HTML output. It can be used to mark entire paragraphs or spans of words. In addition, the normative terminology macros, such as must: and may: and cannot:, always use this role.

The formatting of normative language depends on the stylesheet. Currently it just comes out in purple. We may add a way to disable this formatting at build time.

Imbedding Equations

Where possible, equations should be written using straight asciidoc markup with the eq role. This covers many common equations and is faster than the alternatives. A variety of mathematical symbols are defined using attributes in the included config/attribs.txt. These symbols are defined using attribute names the same as the comparable LaTeX macro names, where possible.

For more complex equations, such as multi-case statements, matrices, and complex fractions, equations should be written using the latexmath: inline and block macros. The contents of the latexmath: blocks should be LaTeX math notation. LaTeX math markup delimiters are now inserted by the asciidoctor toolchain.

LaTeX math is passed through unmodified to all HTML output forms, which is subsequently rendered with the KaTeX engine when the HTML is loaded. A local copy of the KaTeX release is kept in katex/ and copied to the HTML output directory during spec generation. Math is processed into SVGs via asciidoctor-mathematical for PDF output.

The following caveats apply:

  • The special characters < , > , and & can currently be used only in [latexmath] block macros, not in latexmath:[] inline macros. Instead use \lt, \leq, \gt, and \geq for <, ⇐, >, and >= respectively. & is an alignment construct for multiline equations, and should only appear in block macros anyway.

  • AMSmath environments (e.g. \begin{equation*}, {align*}, etc.) cannot be used in KaTeX at present, and have been replaced with constructs supported by KaTeX such as {aligned}.

  • Arbitrary LaTeX constructs cannot be used. KaTeX and asciidoctor-mathematical are only equation renderers, not full LaTeX engines. Imbedding LaTeX like \Large or \hbox{\tt\small VK\_FOO} may not work in any of the backends, and should be avoided.

See the “style guide” (Vulkan Documentation and Extensions) document for more details of supported LaTeX math constructs.

Asciidoc Anchors And Xrefs

In the API spec, sections can have anchors (labels) applied with the following syntax. In general the anchor should immediately precede the chapter or section title and should use the form [[chapter-section-label]]. For example,

For example, in chapter synchronization.txt:

[[synchronization-primitives]]
Synchronization Primitives

Cross-references to those anchors can then be generated with, for example,

See the <<synchronization-primitives>> section for discussion of fences,
semaphores, and events.

You can also add anchors on arbitrary paragraphs, using a similar naming scheme.

Anything whose definition comes from one of the autogenerated API include files (.txt files in the directories basetypes, enums, flags, funcpointers, handles, protos, and structs) has a corresponding anchor whose name is the name of the function, struct, etc. being defined. Therefore you can say something like:

Fences are used with the +++<<vkQueueSubmit>>+++ command...

Software Dependencies

This section describes the software components used by the Vulkan spec toolchain.

Before building the Vulkan spec, you must install the following tools. Minimum versions known to be working are shown. Later versions may work as well; however, specific versions are sometimes required and when this is known, it is noted below.

  • GNU make (make version: 4.0.8-1; older versions probably OK)

  • Python 3 (python, version: 3.4.2)

  • Ruby (ruby, version: 2.5.3)

    • The Ruby development package (ruby-dev) may also be required in some environments.

  • Git command-line client (git, version: 2.1.4). The build can progress without a git client, but branch/commit information will be omitted from the build. Any version supporting the following operations should work:

    • git symbolic-ref --short HEAD

    • git log -1 --format="%H"

  • Ghostscript (ghostscript, version: 9.10). This is for the PDF build, and it can still progress without it. Ghostscript is used to optimize the size of the PDF, so it will be order of magnitude smaller if it is included.

  • The following dependencies are required only if building the chunked target (chunked HTML output).

    • Node (nodejs, version: 8.11.1)

      • Node Package Manager (npm, version: 5.8.0), for installing Lunr

    • Lunr (lunr, version: 2.3.6), installed with npm. One of the lunr files, lunr.js, is also incorporated into this repository under config/chunkindex/lunr.js, so it will always be available at spec load time. It is possible that there will be incompatibilities between the installed version of lunr used to build the index, and the copy of lunr.js included with the repository. If so, update the repository copy of the file from your lunr distribution, or install the specific lunr version 2.3.6 corresponding to the repository copy.

      • Ensure that the installed lunr package is found by node at runtime. This may require setting the environment variable NODE_PATH to the path where lunr is installed. This path will probably be /usr/lib/node_modules, if you install nodejs and npm from deb.nodesource.com .

The following Ruby Gems and platform package dependencies must also be installed. This process is described in more detail for individual platforms and environment managers below. Please read the remainder of this document (other than platform-specific parts you don’t use) completely before trying to install.

  • Asciidoctor (asciidoctor, version: 1.5.8). Version 2.0.0 and later will not work at this time due to breaking changes in asciidoctor.

  • Coderay (coderay, version 1.1.2)

  • JSON Schema (json-schema, version 2.8.1)

  • Asciidoctor Diagram (asciidoctor-diagram, version: 1.5.11)

  • Asciidoctor PDF (asciidoctor-pdf, version: 1.5.0.alpha16)

  • Asciidoctor Mathematical (asciidoctor-mathematical, version 0.2.2)

  • Dependencies for asciidoctor-mathematical (There are a lot of these!)

  • KaTeX distribution (version 0.7.0 from https://github.com/Khan/KaTeX. This is cached under katex/, and need not be installed from github.

  • If generating the chunked HTML target:

ℹ️
Note

Older versions of these packages may work, but are not recommended. In particular, the latest versions of asciidoctor-pdf and asciidoctor-mathematical often solve problems we’ve encountered in older versions.

Only the asciidoctor and coderay gems are needed for the HTML make targets. Rest is needed for the PDF builds.

json-schema is only required in order to validate the output of the valid usage extraction scripts to a JSON file. If not installed, validation will be skipped when the JSON is built.

ℹ️
Note

While it’s easier to install just the toolchain components for HTML builds, people submitting MRs with substantial changes to the Specification are responsible for verifying that their branches build both html and pdf targets.

Platform-specific toolchain instructions follow:

Windows (General)

Most of the dependencies on Linux packages are light enough that it’s possible to build the spec natively in Windows, but it means bypassing the makefile and calling functions directly. Considering how easy it is to get an Unix subsystem or VM on Windows, this is not recommended. It is unlikely a direct path will become supported in the future.

Three options for Windows users are described below: Ubuntu / Windows 10 (best, as long as you’re running Windows 10), MinGW, and Cygwin.

Ubuntu / Windows 10

At the time of writing Ubuntu Subsystem is provided in 18.04 LTS and 16.04 LTS versions. These versions are perfectly suitable for building this repo.

You can install Ubuntu Subsystem as described in the official documentation: https://docs.microsoft.com/en-us/windows/wsl/install-win10

The distro image is not kept up-to-date, so it is recommended to run:

sudo apt update
sudo apt full-upgrade

Rest is identical to Linux instructions.

MinGW

MinGW can be obtained here: http://www.mingw.org/

Once the installer has run its initial setup, following the instructions on the website, you should install the mingw-developer-tools, mingw-base and msys-base packages. The msys-base package allows you to use a bash terminal from windows with whatever is normally in your path on Windows, as well as the unix tools installed by MinGW.

In the native Windows environment, you should also install the following native packages:

Once this is setup, and the necessary Ruby Gems are installed, launch the msys bash shell, and navigate to the spec Makefile. From there, you’ll need to set PYTHON= to the location of your python executable for version 3.x before your make command - but otherwise everything other than pdf builds should just work.

ℹ️
 Building the PDF spec via this path has not yet been tested but may be possible - liblasem is the main issue and it looks like there is now a mingw32 build of it available.

Cygwin

When installing Cygwin, you should install the following packages via setup:

// "curl" is only used to download fonts, can be done in another way
autoconf
bison
cmake
curl
flex
gcc-core
gcc-g++
ghostscript
git
libbz2-devel
libcairo-devel
libcairo2
libffi-devel
libgdk_pixbuf2.0-devel
libiconv
libiconv-devel
liblasem0.4-devel
libpango1.0-devel
libpango1.0_0
libxml2
libxml2-devel
make
python3
ruby
ruby-devel
ℹ️
 Native versions of some of these packages are usable, but care should be taken for incompatibilities with various parts of cygwin - e.g. paths. Ruby in particular is unable to resolve Windows paths correctly via the native version. Python and Git for Windows can be used, though for Python you’ll need to set the path to it via the PYTHON environment variable, before calling make.

When it comes to installing the mathematical ruby gem, there are two things that will require tweaking to get it working. Firstly, instead of:

MATHEMATICAL_SKIP_STRDUP=1 gem install asciidoctor-mathematical

You should use

MATHEMATICAL_USE_SYSTEM_LASEM=1 gem install asciidoctor-mathematical

The latter causes it to use the lasem package already installed, rather than trying to build a fresh one.

Recent versions of some gems break the installation process and/or pdf build on some systems. If the above doesn’t work, try:

MATHEMATICAL_USE_SYSTEM_LASEM=1 gem install mathematical -v 1.6.7
gem install ruby-enum -v 0.7.0
gem install asciidoctor-mathematical

The mathematical gem also looks for "liblasem" rather than "liblasem0.4" as installed by the lasem0.4-devel package, so it is necessary to add a symlink to your /lib directory using:

ln -s /lib/liblasem-0.4.dll.a /lib/liblasem.dll.a

Ruby Gems are not installed to a location that is in your path normally. Gems are installed to ~/bin/ - you should add this to your path before calling make:

export PATH=~/bin:$PATH

Finally, you’ll need to manually install fonts for lasem via the following commands:

mkdir /usr/share/fonts/truetype cd /usr/share/fonts/truetype
curl -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/cmex10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/cmmi10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/cmr10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/cmsy10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/esint10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/eufm10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/msam10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/msbm10.ttf

Linux (Debian, Ubuntu, etc.)

System dependencies can be installed via apt:

sudo apt install build-essential python3 git cmake bison flex \
    libffi-dev libxml2-dev libgdk-pixbuf2.0-dev libcairo2-dev \
    libpango1.0-dev fonts-lyx ghostscript libreadline-dev
ℹ️
Note

On Ubuntu versions prior to 18.04 LTS, you will probably need to use the ttf-lyx package instead of fonts-lyx.

These instructions are for the Ubuntu installation and are generally applicable to native Linux environments that use Debian packages, although the exact list of packages to install may differ. Other distributions using different package managers, such as RPM (Fedora) and Yum (SuSE) will have different requirements.

Ruby can also be installed as a system package:

sudo apt install ruby ruby-dev

Ruby packages are often well out of date, so using alternative ruby environments such as rbenv or rvm might be preferable.

Once the Ruby environment is set up, install the required Ruby Gems.

If you will need to generate the chunked HTML target, install the Asciidoctor-chunker dependencies as described below.

Mac OS X

Mac OS X should work in the same way as for Ubuntu by using the Homebrew package manager, with the exception that you can simply install the ruby package via brew rather than using a ruby-specific version manager.

You’ll likely also need to install additional fonts for the PDF build via mathematical, which you can do with:

cd ~/Library/Fonts
curl -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/cmex10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/cmmi10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/cmr10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/cmsy10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/esint10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/eufm10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/msam10.ttf \
     -LO http://mirrors.ctan.org/fonts/cm/ps-type1/bakoma/ttf/msbm10.ttf

Then install the required Ruby Gems.

Ruby Gems

The following ruby gems can be installed directly via the gem install command, once the platform is set up:

gem install --no-rdoc --no-ri asciidoctor -v 1.5.8
gem install --no-rdoc --no-ri coderay json-schema asciidoctor-mathematical asciidoctor-diagram
gem install --no-rdoc --no-ri --pre asciidoctor-pdf

Depending on Ruby environment gem may require sudo.

It may significantly speed up installation if you skip documentation build by passing --no-rdoc --no-ri arguments.

It may be beneficial to use updated packages via:

gem update --no-rdoc --no-ri
gem clean

Asciidoctor-chunker and Index Generation

To generate the chunked HTML target, you must install asciidoctor-chunker and the underlying Roswell compiler and related dependencies. These projects do not seem to support standard software repositories and packaging (e.g. RPM, .deb, etc.), so you will need to follow the How to Install directions for asciidoctor-chunker.

Note that both Roswell and asciidoctor-chunker are installed outside the scope of the Vulkan Specification repository (in system directories, and in your home directory, respectively).

You must also install these Javascript dependencies to generate the index, partly as system packages, and partly with npm. Note that npm is not packaged for Debian 9, thus it’s installed from deb.nodesource.com following https://linuxize.com/post/how-to-install-node-js-on-debian-9/

curl -sL https://deb.nodesource.com/setup_8.x | sudo bash -
# nodejs also installs npm
sudo apt install nodejs
sudo npm install -g [email protected]
setenv NODE_PATH /usr/lib/node_modules

Troubleshooting

This section goes over known problems and solutions for toolchain installation or for build.

If you get arbitrary build errors it can’t hurt to first try resolve it by cleaning the tree:

make clean
git clean -dxf

STEM SVG Errors

If you happen to have _ or other Asciidoctor formating characters in your path, then PDF build using asciidoctor-mathematical may fail with:

asciidoctor: WARNING: image to embed not found or not readable: whatever/<em>stuff/Vulkan-Docs/out/equations_temp/stem-d3355033150173c1d397e342237db405.svg

You simply need to have the repository cloned in a simpler path.

Ghostscript Errors

Ghostscript optimization of the PDF may produce:

   **** Error reading a content stream. The page may be incomplete.
               Output may be incorrect.
   **** Error: File did not complete the page properly and may be damaged.
               Output may be incorrect.

Usually, it is just a problem with the Asciidoc sources (e.g. silent failure to render content that does not fit in the page; such as SVG equations where there is no line break opportunity).

Ruby Gem Versioning Errors

Sometimes, when updating ruby gem packages incompatibilities arise. It is resoleved by identifying the offending packages and downgrading them:

$ gem uninstall package_name
$ gem install package_name --version good_version_number

If you already have the gem dependencies previously installed, if there are new versions, then updating to them instead might help:

$ gem update --no-rdoc --no-ri

ruby-enum

We have seen this PDF build error:

Failed to load AsciiDoc document - wrong constant name default (NameError)

It should not be occurring with updated packages. Make sure you are using ruby-enum 0.7.1 or later, and mathematical 1.6.8 or later. If you are forced to use earlier versions, see gjtorikian/mathematical#69 for a report of a related versioning problem.

prawn

Make sure you are using prawn 2.2.1 or later, and prawn-templates 0.0.5 or later. Incompatibilities between asciidoctor-pdf and earlier versions of these gems affects the PDF build. See KhronosGroup#476

asciidoctor-mathematical gem native extension errors

Installing mathematical gem builds lasem and mtex2MML native binaries. The Dependencies we list should be sufficient for the install to build those native extensions successfully.

If you encounter problems, it is possible to use those binaries from preinstalled locations. See https://github.com/gjtorikian/mathematical#troubleshooting.

Asciidoctor include errors

If you get errors like:

asciidoctor: ERROR: chapters/???.txt: line 189: include file not found: ???/Vulkan-Docs/api/protos/???.txt

you probably forgot to call make clean_generated as stated in the Building With Extensions Included chapter.

Asciidoctor-chunker memory exhaustion

If you get errors like:

ASCIIDOCTOR-CHUNKER: Processing Chap 17 ....
Heap exhausted during garbage collection: 224 bytes available, 288 requested.
...
GC control variables:
   *GC-INHIBIT* = true
   *GC-PENDING* = true
   *STOP-FOR-GC-PENDING* = false
fatal error encountered in SBCL pid 31086(tid 0x7f4816866700):
Heap exhausted, game over.

try specifying a larger dynamic space size, something bigger than 2000:

$ ROSWELLOPTS="dynamic-space-size=2500" ./makeAllExts html chunked

Alternative Ruby environments

The default ruby packages on Linux distro may be out of date. Through the default ruby package, Ubuntu 18.04 provides ruby 2.5, and Ubuntu 16.10 provides ruby 2.3. Those system packages seem to be sufficient to build this repo.

But there are better options; either rvm or rbenv is recommended to install an updated version of Ruby environment.

ℹ️
Note

  • If you are new to Ruby, you should completely remove (through the package manager, e.g. sudo apt purge packagename) all existing Ruby and asciidoctor infrastructure on your machine before trying to use rvm or rbenv for the first time. dpkg -l | egrep 'asciidoctor|ruby|rbenv|rvm' will give you a list of candidate package names to remove.

    • If you already have a favorite Ruby package manager, ignore this advice, and just install the required OS packages and gems.

  • In addition, rvm and rbenv are mutually incompatible. They both rely on inserting shims and $PATH modifications in your bash shell. If you already have one of these installed and are familiar with it, it’s probably best to stay with that one. One of the editors, who is new to Ruby, found rbenv far more comprehensible than rvm. The other editor likes rvm better.

    • Neither rvm nor rbenv work, out of the box, when invoked from non-Bash shells like tcsh. This can be hacked up by setting the right environment variables and PATH additions based on a bash environment.

  • Most of the tools on Bash for Windows are quite happy with Windows line endings (CR LF), but bash scripts expect Unix line endings (LF). The file .gitattributes at the top of the vulkan tree forces such scripts to be checked out with the proper line endings on non-Linux platforms. If you add new scripts whose names don’t end in .sh, they should be included in .gitattributes as well.
Ubuntu/Windows 10 Using Rbenv

Rbenv is a lighter-weight Ruby environment manager with less functionality than rvm. Its primary task is to manage different Ruby versions, while rvm has additional functionality such as managing “gemsets” that is irrelevant to our needs.

A complete installation script for the toolchain on Ubuntu for Windows, developed on an essentially out-of-the-box environment, follows. If you try this, don’t try to execute the entire thing at once. Do each step separately in case of errors we didn’t encounter.

# Install packages needed by `ruby_build` and by toolchain components.
# See https://github.com/rbenv/ruby-build/wiki and
# https://github.com/asciidoctor/asciidoctor-mathematical#dependencies

sudo apt-get install autoconf bison build-essential libssl-dev \
    libyaml-dev libreadline6-dev zlib1g-dev libncurses5-dev \
    libffi-dev libgdbm3 libgdbm-dev cmake libxml2 \
    libxml2-dev flex pkg-config libglib2.0-dev \
    libcairo-dev libpango1.0-dev libgdk-pixbuf2.0-dev \
    libpangocairo-1.0 libreadline-dev

# Install rbenv from https://github.com/rbenv/rbenv
git clone https://github.com/rbenv/rbenv.git ~/.rbenv

# Set path to shim layers in .bashrc
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> .bashrc

~/.rbenv/bin/rbenv init

# Set .rbenv environment variables in .bashrc
echo 'eval "$(rbenv init -)"' >> .bashrc

# Restart your shell (e.g. open a new terminal window). Note that
# you do not need to use the `-l` option, since the modifications
# were made to .bashrc rather than .bash_profile. If successful,
# `type rbenv` should print 'rbenv is a function' followed by code.

# Install `ruby_build` plugin from https://github.com/rbenv/ruby-build

git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build

# Install Ruby 2.5.3 (current as of this writing; earlier may work)
# Setting RUBY_CONFIGURE_OPTS dramatically cuts the install time, see
# https://github.com/rbenv/ruby-build/issues/1054#issuecomment-276934761
RUBY_CONFIGURE_OPTS=--disable-install-doc
export RUBY_CONFIGURE_OPTS
rbenv install 2.5.3

# Configure rbenv globally to always use Ruby 2.5.3.
echo "2.5.3" > ~/.rbenv/version

# Finally, install toolchain components.
# asciidoctor-mathematical also takes in excess of 20 min. to build!
# The same RUBY_CONFIGURE_OPTS advice above may apply here as well.

gem install --no-rdoc --no-ri asciidoctor -v 1.5.8
gem install --no-rdoc --no-ri coderay json-schema asciidoctor-mathematical asciidoctor-diagram
gem install --no-rdoc --no-ri --pre asciidoctor-pdf
Ubuntu/Windows 10 Using RVM

Here are (sparser) instructions for using rvm to setup version 2.3.x:

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable --ruby
source ~/.rvm/scripts/rvm
rvm install ruby-2.3
rvm use ruby-2.3
ℹ️
 Windows 10 Bash will need to be launched with the "-l" option appended, so that it runs a login shell; otherwise RVM won’t function correctly on future launches.

Revision History

  • 2018-12-04 - Update Rbenv and ruby gem installation instructions and package dependencies for Linux and Ubuntu/Windows 10.

  • 2018-10-25 - Update Troubleshooting, and Windows and Linux build. Plus random editing.

  • 2018-03-13 - Rename to BUILD.adoc and update for new directory structure.

  • 2018-03-05 - Update README for Vulkan 1.1 release.

  • 2017-03-20 - Add description of prawn versioning problem and how to fix it.

  • 2017-03-06 - Add description of ruby-enum versioning problem and how to fix it.

  • 2017-02-13 - Move some comments here from ../../../README.md. Tweak asciidoctor markup to more clearly delineate shell command blocks.

  • 2017-02-10 - Add more Ruby installation guidelines and reflow the document in accordance with the style guide.

  • 2017-01-31 - Add rbenv instructions and update the README elsewhere.

  • 2017-01-16 - Modified dependencies for Asciidoctor

  • 2017-01-06 - Replace MathJax with KaTeX.

  • 2016-08-25 - Update for the single-branch model.

  • 2016-07-10 - Update for current state of spec and ref page generation.

  • 2015-11-11 - Add new can: etc. macros and DBLATEXPREFIX variable.

  • 2015-09-21 - Convert document to asciidoc and rename to README.md in the hope the gitlab browser will render it in some fashion.

  • 2015-09-21 - Add descriptions of LaTeX and MathJax math support for all output formats.

  • 2015-09-02 - Added Cygwin package info.

  • 2015-09-02 - Initial version documenting macros, required toolchain components and versions, etc.