Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python rewrite #4

Merged
merged 249 commits into from
Jul 28, 2023
Merged

Python rewrite #4

merged 249 commits into from
Jul 28, 2023

Conversation

Qonfused
Copy link
Owner

@Qonfused Qonfused commented Jun 20, 2023

Test coverage:

Tasks:

CI (Python)

  • Handle semver versioning schemes
    • Support automatic versioning schemes (e.g. via git tags)
  • Add environment checks for future versions of Python.
  • Ensure namespace and module resolution are not mangled
    • Automatically generate API exports for codegen
  • Sort import order for packages and modules

CLI (Python)

  • Add CLI support for --version flag
  • Add shared context for verbosity and debug flags
  • Add shared context for config and build file paths
  • Commands
    • Build command
    • Lock command
    • Patch command

Library (C++, Python)

  • Errors
    • Traceback frame introspection and suppression
    • Custom error classes (runtime/tests)
    • Runtime validation methods
  • Filesystem
    • Handle extracting remote archives to temporary directories
    • Expose missing POSIX filesystem operations
  • Parsers
    • Tokenizer/AST
      • (Native)
      • Stringified (asserting all values as string)
      • Flat (stringifying all nested keys)
    • Plist - Apple PLIST 1.0
      • Dictionary (root)
      • Array (root)
    • YAML - YAML 1.2
      • (Native)
      • Annotated (see example config.yml)
      • Stringified (asserting all values as string)
      • Flat (stringifying all nested keys)
  • Pipeline
    • Build file/configuration
      • Read and parse build file
      • Normalize entries
      • Validate build file against schema
      • Generate a template build file from an existing EFI structure
    • Config
      • Read and parse config file
      • Parse schema from Sample.plist/config.plist file
        • Validate config.plist against a schema
      • Merge and patch config.plist
        • Handle config inheritance
    • Kexts
      • Extract kexts from archives
      • Read and parse kext manifest
      • Validate kext manifest against build configuration
      • Sort kext load order by OSBundleLibraries field
    • Lockfile
      • Read and parse lockfile
      • Handle parsing build configuration specifiers
      • Handle creating or updating the lockfile
      • Validate lockfile against build configuration
      • Cache package sources and checksums
      • Validate lockfile updates against cached packages
    • OpenCore
      • Read and parse the OpenCore (build) configuration
      • Unpack the OpenCore archive to a temporary directory
      • Extract bundled OpenCore binaries, scripts + documentation
    • SSDTs
      • Extract and wrap iASL binary
      • Handle decompilation of SSDT binaries
      • Handle patching of SSDT source files
      • Handle compilation of SSDT source files
      • Sort SSDTs by resolving symbolic references
  • Sources
    • Handle remote sources
      • GitHub
        • Commits - raw + tarball sources
        • CI/CD - GitHub Actions
        • Releases
      • Dortania build repo
      • Handle arbitrary archives/URLs
    • Handle local sources
      • Filesystem - Raw path and RFC 8089 file: protocol
      • Local cache
  • Versioning

Copy link
Owner Author

@Qonfused Qonfused left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schema docs generation is mostly uncoupled from the main library, though there currently exists a markdown parser in the ocebuild/parsers/schema.py file that may be worth moving separately under the current docsgen pipeline.

A more robust solution worth exploring is to use pandoc (i.e. ran in CI w/ a docker image), which will more reliably convert each schema's LaTeX entry to markdown. There is currently no action needed for the current schema parser which parses and segments these entries.

<dict>
<key>OpenCore-Version</key>
<string>0.9.3</string>
</dict>
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schema docs will need to be updated on a regular cadence following each OpenCore release. Pre-release builds of OpenCore will also need to be smoke-tested against any schema validation in case new schema entries are added that must be specially handled.

Additionally, the usage of the '#Revision' dict mimics the same usage by OCLP in its payload config.plist here. Special care must be taken when introducing metadata keys that may conflict with other projects.

@Qonfused
Copy link
Owner Author

Unfortunately due to a bug in the paths filtering behavior for GitHub actions, there is currently a block preventing progress on the CI for orchestrating integration testing. This will force current progress in this PR to be merged pre-maturely in order to continue.

The core library is mostly stable and tested, however, what is remaining regarding development and test coverage is primarily the pipeline supporting ocebuild_cli:
image

For the CLI, the build and lock commands are mostly finished and are pending the patch command to be implemented, which will handle the remaining config.plist patching for the build pipeline.

@Qonfused Qonfused merged commit 675f425 into main Jul 28, 2023
Qonfused added a commit that referenced this pull request Jul 28, 2023
@Qonfused Qonfused mentioned this pull request Jul 28, 2023
Qonfused added a commit that referenced this pull request Jul 28, 2023
Rewrites the project as a standalone Python library and CLI.

This implementation is not feature complete pending an upstream conflict with GitHub actions; refer to #4 (comment) for details.

For current progress on integration, refer to acidanthera/bugtracker#2311.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature]: Resolve config.plist schema conflicts
1 participant