Update project docs, add Sphinx docs pipeline (#14)

.github/workflows/docs.yml

@@ -0,0 +1,55 @@
+name: Docs
+on:
+  push:
+    branches: [main]
+  workflow_dispatch:
+env:
+  python-version: 3.11
+  docs-build: docs/build/
+permissions:
+  contents: write
+jobs:
+  docs:
+    runs-on: ubuntu-latest
+    steps:
+      # Fetches remote repository without --progress option.
+      #
+      # The default behavior of @actions/checkout outputs many noisy lines of
+      # status output in the workflow log, which is problematic for log size.
+      #
+      # Refer to the below PR for updates on this issue:
+      # - https://github.com/actions/checkout/pull/1067
+      - name: Checkout latest repository commit
+        # uses: actions/checkout@v3
+        uses: simonbaird/checkout@no-progress-option
+        with:
+          show-progress: false
+          # submodules: recursive
+      # Setup python virtual environment
+      - name: Install poetry
+        run: |
+          bash ci/scripts/lib/install-poetry.sh
+          echo "PATH=${HOME}/.poetry/bin:${PATH}" >> $GITHUB_ENV
+      - name: Setup Python ${{ env.python-version }}
+        uses: actions/setup-python@v3
+        with:
+          python-version: ${{ env.python-version }}
+          cache: poetry
+          cache-dependency-path: poetry.lock
+      # Setup Poetry virtual environment
+      - name: Use matrix python version in Poetry virtualenv
+        run: |
+          poetry env use ${{ env.python-version }}
+      - name: Setup Poetry plugins and project dependencies
+        run: poetry install -n --with docs
+      # Build docs
+      - name: Sphinx build
+        run: |
+          poetry sphinx-build
+      - name: Deploy
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ${{ env.docs-build }}
+          publish_branch: gh-pages
+          force_orphan: true

.gitignore

@@ -1,4 +1,5 @@
 .*
+!*.keep
 .env
 .pypirc
 !/.github
@@ -9,6 +10,7 @@
 !/.gitattributes
 !/.gitignore
 !/.pre-commit-config.yaml
+!/.readthedocs.yaml
 !/**/.pylintrc
 
 # Build artifacts

.pre-commit-config.yaml

@@ -9,12 +9,6 @@ repos:
     args: ["-f", "requirements.txt", "-o", "requirements.txt"]
 - repo: local
   hooks:
-  - id: ci:setup
-    name: 'ci:setup'
-    entry: poetry install -n
-    language: system
-    types: [python]
-    pass_filenames: false
   - id: ci:resolve-modules
     name: 'ci:resolve-modules'
     entry: poetry run poe resolve-modules

.readthedocs.yaml

@@ -0,0 +1,21 @@
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+version: 2
+
+build:
+  os: ubuntu-20.04
+  tools:
+    python: "3.11"
+  jobs:
+    post_install:
+      - pip install poetry==1.5.0
+      # Install docs dependencies into the same readthedocs virtualenv
+      - poetry config virtualenvs.create false
+      # - poetry install --no-root --only docs
+      - poetry install --with docs
+
+# Build documentation under the `docs/build` directory with Sphinx
+sphinx:
+  builder: html
+  configuration: ci/tools/sphinx/conf.py

CONTRIBUTING.md

@@ -15,15 +15,13 @@ This document describes this project's development process. Following these guid
 5. [Improving Documentation](#improving-documentation)
 6. [Development Process](#development-process)
 7. [Contributing Code](#contributing-code)
-    1. [General Guidelines and Philosophy](#general-guidelines-and-philosophy)
-    2. [Getting Started](#getting-started)
-    3. [Setting up your development environment](#setting-up-your-development-environment)
-    4. [Project Structure](#project-structure)
-    5. [Project Scripts](#project-scripts)
-    6. [Coding Style](#coding-style)
-    7. [Building the Project](#building-the-project)
-    8. [Testing](#testing)
-    9. [License](#license)
+    1. [Setting up your development environment](#setting-up-your-development-environment)
+    2. [Project Structure](#project-structure)
+    3. [Project Scripts](#project-scripts)
+    4. [Coding Style](#coding-style)
+    5. [Building the Project](#building-the-project)
+    6. [Testing](#testing)
+    7. [License](#license)
 8. [Pull Request Process](#pull-request-process)
     1. [Review Process](#review-process)
     2. [Addressing Feedback](#addressing-feedback)
@@ -82,23 +80,6 @@ When you have finished working on a topic branch, you can merge your changes bac
 
 ## Contributing Code
 
-### General Guidelines and Philosophy
-
-* Include unit tests when you contribute new features or fix a bug, this:
-  * proves that your code works correctly
-  * guards against breaking changes
-  * lowers the maintenance cost
-
-* Keep compatibility and cohesiveness mind when contributing a change that
-  will impact the public API.
-
-* Create issues for any major changes and enhancements that you wish to make.
-
-* Create a pull request for your changes and coordinate with the project
-  maintainers to get it merged to main.
-
-### Getting Started
-
 To begin contributing, you will need to fork the main repository to work on your changes. Simply navigate to our GitHub page and click the "Fork" button at the top. Once you've forked the repository, you can clone your new repository and start making edits.
 
 In git, it is best to isolate each topic or feature into a “topic branch”. While individual commits allow you control over how small individual changes are made to the code, branches are a great way to group a set of commits all related to one feature together, or to isolate different efforts when you might be working on multiple topics at the same time.
@@ -149,7 +130,25 @@ bash scripts/setup-poetry.sh
 
 ### Project Structure
 
-> TODO
+#### Development Tooling and Environments
+
+The project contains a top-level `ci/` directory containing the CI/CD configuration and environments used in integration, as well as additional configurations for each of the project's development tools when developing locally. Required environments are also automatically configured when using `poetry env use <python version>` and running the install command in a new virtual environment.
+
+This leverages a root-level `pyproject.toml` configuration as the source of truth for all of the project's dependencies and development tools. The `poetry.lock` file is used to lock the project's dependencies to specific versions to ensure that all developers are using the same versions of the project's dependencies. This allows for a consistent development environment across all platforms and Python versions.
+
+Some important caveats and notes about the project's development tools:
+
+* When configuring a new virtual environment, always use `poetry install` to ensure that the project's dependencies are installed and to ensure that the required development tools are installed and configured correctly. This will ensure that the development environment is correctly configured for your virtualenv's Python version.
+
+* When you install the project for the first time, additional required poetry plugins will be installed automatically. These plugins are required for the project's development tools to function correctly, and are automatically added to your poetry configuration with the project's install hook. These plugins are installed globally, and will be available in all of your poetry virtual environments.
+
+* The project is automatically built by poetry each time you install the project (you can find the built project under `ci/tools/poetry/build`). This means that build tools for the CLI or the project's documentation require no pre-requisite setup and only require a single command to build after installation.
+
+#### Project Modules
+
+The main project is split into two main modules: `ocebuild` and `ocebuild_cli`. The `ocebuild` module contains all of the core functionality of the project, while the `ocebuild_cli` module contains the CLI interface for the CLI executable.
+
+The `third_party` module contains all of the third-party code used by the project. This includes vendored code, as well as code that is not available on PyPI. This can include fixes or patches to upstream code, contributed experimental code, or code that is not available on PyPI.
 
 ### Project Scripts
 

README.md

@@ -19,57 +19,153 @@
 
 </div>
 
-OCE-Build is designed to help enable projects to be highly composable and easily maintainable, prioritizing a small and git-friendly footprint.
+OCE-Build is designed to help enable projects to be highly composable and easily
+maintainable, prioritizing a small git-friendly footprint. It is designed to be
+flexible and extensible for existing projects, while also providing a simple
+configuration and interface for new projects to get started quickly.
 
-### Key Features
+## Feature Overview
+
+> **Note**: This project is still in early development and is not yet ready for general use.
+The following is a list of features that are planned for the initial release.
 
 - **Declarative Configuration**
-  - [x] Minimal project configuration
-  - [x] Flexible support for any project structure
-  - [ ] Shared/Template configurations
-    - [x] Support for overriding build pipeline inclusions/exclusions
-    - [ ] Inheritance resolution for extending config.plist configurations.
-  - [ ] Automated config.plist schema conflict resolution
-    - [ ] Diff file creation between two PLIST files
-    - [x] Support for reading declarative config.{yml|yaml} files.
-    - [x] Support for reading and converting existing config.plist files.
+  - Minimal project configuration.
+  - Flexible support for any project structure.
+  - Shared/Template config.plist configurations.
+  - Automated config.plist schema conflict resolution.
 - **Version Management**
-  - [x] Versioning for OpenCore binaries and resources
-    - [x] Versioning for bundled driver and tool binaries
-    - [x] Versioning for supplemental OCBinaryData binaries
-  - [x] Versioning for local & external Kext sources
-    - [x] Fetch external Kext sources by release version or commit
-    - [x] Kext dependency sorting for proper prelink injection
-  - [x] Versioning for local & external SSDT binaries
-    - [x] Versioning for pre-built SSDT files
-    - [x] SSDT dependency sorting for proper ACPI table loading
-  - [x] Lockfile creation for intelligent registry management
+  - Versioning for OpenCore binaries and resources (Kexts, SSDTs, etc).
+  - Automatic version resolution for dependencies.
+  - Lockfile creation for intelligent registry management.
 - **Build Support**
-  - [x] Incremental builds and cache management from lockfile.
-    - [ ] Support for remote cache artifacts
-  - [x] Advanced ACPI language support
-    - [x] Support for building DSL source into compiled AML
-    - [ ] Compiler options with external refs and patchfiles
-  - [ ] CI support for managing and building Kexts with XCode
-
-## Installation
-
-This project is distributed as both a standalone executable and a Python library on PyPI. The standalone executable is intended to be used for simple projects or for CI/CD, while the Python library is intended to be used for supporting library development or complex build pipelines.
-
-See the [install](/docs/install.md) documentation for more information on how to
-install OCE Build as an executable or as a Python library.
-
-## Getting Started
-
->TODO
-
-## Contributing
-
-See the [contributing](/CONTRIBUTING.md) documentation for more information on how to contribute to this project.
-
-## Issues
-
->TODO
+  - Incremental builds and cache management from lockfile.
+  - Advanced ACPI language support, including compiler and patchfile options.
+  - Dependency sorting for proper prelink injection (i.e. SSDTs, Kexts).
+  - CI support for managing and building Kexts with XCode.
+
+## User Documentation
+
+*This section is for users looking to integrate this tool with their own project.
+For information on contributing to this project, refer to the
+[Developer Docs](#developer-documentation)*
+
+### Installation
+
+This project is distributed as both a standalone CLI and a Python library on
+PyPI. The CLI is intended to be used for simple projects or for CI/CD, while the
+library is intended to be used for supporting Python library development or
+complex build pipelines. The library also comes bundled with the CLI and it's
+source code is available under the `ocebuild.cli` module.
+
+The recommended way to install the CLI is from [GitHub releases](https://github.com/Qonfused/OCE-Build/releases). This will provide you with a standalone executable that can be used without installing Python or any other dependencies. The CLI is available for Windows, macOS, and Linux.
+
+To install the CLI / library from PyPI, you will need to have Python 3.8 or
+later installed. You can install Python from [python.org](https://www.python.org/downloads/)
+or from your package manager. For more information on installing Python, refer
+to the [Python documentation](https://docs.python.org/3/using/index.html).
+
+To install the library directly from PyPI using `pip`, simply run the following
+command in your terminal:
+```sh
+pip install ocebuild
+```
+
+Or when using the `poetry` package manager, run:
+```sh
+poetry add ocebuild
+```
+
+Depending on your system, you may need to use `pip3` instead of `pip` when
+installing the library. You may also need to use `sudo` or `pip install --user`
+to install the library globally or locally, respectively.
+
+After installing the library, the CLI will also be available in your PATH, which
+you can invoke by simply running `ocebuild` in your terminal (or `poetry run ocebuild`).
+
+### Getting Started
+
+To get started, you will need to create a `build.yml` file in the root of your
+project. This file will contain all of the information needed to build your EFI.
+You can follow along with the below example or reference one of the [example projects](https://github.com/Qonfused/OCE-Build/tree/main/examples) included in this repository.
+
+The following is a minimal example of a `build.yml` file located under a `src/`
+directory:
+
+```yaml
+# src/build.yml
+---
+build: DEBUG
+version: latest
+---
+ACPI:
+Drivers:
+  - AudioDxe
+  - HfsPlus
+  - OpenRuntime
+  - ResetNvramEntry
+Kexts:
+  Lilu: latest
+  VirtualSMC: latest
+Resources:
+Tools:
+  - OpenShell
+```
+
+Once you have created your `build.yml` file, you can run the build command to
+build your EFI:
+
+```sh
+ocebuild build --cwd src/build.yml
+```
+
+This will create a new `dist/` directory in your project containing your EFI.
+You can also specify a custom output directory by using the `-o` / `--output`
+option. To view all available commands and options, run `ocebuild --help` or run
+`ocebuild <command> --help` for more information on a specific command.
+
+Refer to [docs/configuration.md](https://github.com/Qonfused/OCE-Build/blob/main/docs/configuration.md#build-configuration) for more information on how to setup your build
+configuration.
+
+Note that this does not output a config.plist file. To generate a config.plist
+file, you will need to create a `config.yml` file in the same directory as your
+`build.yml` file. The `config.yml` file contains only the changes you wish to
+make to the default Sample.plist file (located under `dist/Docs/Sample.plist`).
+Refer to [docs/configuration.md](https://github.com/Qonfused/OCE-Build/blob/main/docs/configuration.md#configplist-configuration) for more information.
+
+## Developer Documentation
+
+*This section is for developers looking to contribute to this project. This is
+only meant to be a brief overview of the project structure and development
+guidelines. For more information, refer to the [contributing](/CONTRIBUTING.md)
+documentation.*
+
+### Guidelines and Philosophy
+
+The general philosophy for this project is to keep things as simple as possible.
+This means that the project should be easy to understand, easy to maintain, and
+easy to contribute to.
+
+A summary of the contributing guidelines for this project are as follows:
+
+- Create an [issue](https://github.com/Qonfused/OCE-Build/issues) for any major
+  changes and enhancements that you wish to make.
+  - If you find a security vulnerability, do NOT open an issue. Please refer to our [Security Policy](https://github.com/Qonfused/OCE-Build/security/policy) for reporting security vulnerabilities.
+  - For questions or discussions related to this project, please open a new GitHub
+  [discussion](https://github.com/Qonfused/OCE-Build/discussions) instead of
+  opening an issue.
+- Create a [pull request](https://github.com/Qonfused/OCE-Build/pulls) for your
+  changes and coordinate with the project maintainers to get it merged to main.
+  - Include unit tests when you contribute new features or fix a bug. This:
+    - Proves that your code works correctly and prevents regressions from going
+      unnoticed.
+    - Guards against breaking changes to the public API and ensures that the
+      project can remain backwards compatible.
+    - Lowers the maintenance cost of the project by making it easier to
+      troubleshoot and debug issues.
+  - Keep compatibility and cohesiveness mind when contributing a change that
+    will impact the public API. This helps limit the scope of changes to only
+    what is required to implement the new feature or fix the bug.
 
 ## License