Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bambamc: add aarch64/arm64 builds #51067

Merged
merged 3 commits into from
Oct 1, 2024
Merged

Conversation

martin-g
Copy link
Contributor

@martin-g martin-g commented Oct 1, 2024

Describe your pull request here


Please read the guidelines for Bioconda recipes before opening a pull request (PR).

General instructions

  • If this PR adds or updates a recipe, use "Add" or "Update" appropriately as the first word in its title.
  • New recipes not directly relevant to the biological sciences need to be submitted to the conda-forge channel instead of Bioconda.
  • PRs require reviews prior to being merged. Once your PR is passing tests and ready to be merged, please issue the @BiocondaBot please add label command.
  • Please post questions on Gitter or ping @bioconda/core in a comment.

Instructions for avoiding API, ABI, and CLI breakage issues

Conda is able to record and lock (a.k.a. pin) dependency versions used at build time of other recipes.
This way, one can avoid that expectations of a downstream recipe with regards to API, ABI, or CLI are violated by later changes in the recipe.
If not already present in the meta.yaml, make sure to specify run_exports (see here for the rationale and comprehensive explanation).
Add a run_exports section like this:

build:
  run_exports:
    - ...

with ... being one of:

Case run_exports statement
semantic versioning {{ pin_subpackage("myrecipe", max_pin="x") }}
semantic versioning (0.x.x) {{ pin_subpackage("myrecipe", max_pin="x.x") }}
known breakage in minor versions {{ pin_subpackage("myrecipe", max_pin="x.x") }} (in such a case, please add a note that shortly mentions your evidence for that)
known breakage in patch versions {{ pin_subpackage("myrecipe", max_pin="x.x.x") }} (in such a case, please add a note that shortly mentions your evidence for that)
calendar versioning {{ pin_subpackage("myrecipe", max_pin=None) }}

while replacing "myrecipe" with either name if a name|lower variable is defined in your recipe or with the lowercase name of the package in quotes.

Bot commands for PR management

Please use the following BiocondaBot commands:

Everyone has access to the following BiocondaBot commands, which can be given in a comment:

@BiocondaBot please update Merge the master branch into a PR.
@BiocondaBot please add label Add the please review & merge label.
@BiocondaBot please fetch artifacts Post links to CI-built packages/containers.
You can use this to test packages locally.

Note that the @BiocondaBot please merge command is now depreciated. Please just squash and merge instead.

Also, the bot watches for comments from non-members that include @bioconda/<team> and will automatically re-post them to notify the addressed <team>.

Summary by CodeRabbit

  • New Features

    • Enhanced build scripts for improved error handling and multi-threaded execution, speeding up compilation.
    • Updated package version to 0.0.50 with support for additional platforms: linux-aarch64 and osx-arm64.
    • Introduced config.guess and config.sub scripts for better system detection and configuration validation.
  • Bug Fixes

    • Improved debugging capabilities in the build process, particularly for MacOS dynamic library loading issues.
  • Documentation

    • Updated meta.yaml to reflect new versioning and platform support.

@martin-g martin-g added aarch64 Related to adding linux-aarch64 support osx-arm64 Related to adding osx-arm64 support labels Oct 1, 2024
Copy link
Contributor

coderabbitai bot commented Oct 1, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

The pull request introduces modifications to the build.sh and meta.yaml files for the bambamc package. Enhancements to build.sh include improved error handling with set -xe and multi-threaded execution of the make command. The meta.yaml file is updated to version 0.0.50, with an incremented build number, new source URL, MD5 checksum, and added support for linux-aarch64 and osx-arm64.

Changes

Files Change Summary
recipes/bambamc/build.sh Enhanced execution behavior by adding set -xe for error handling and updated make to multi-threaded execution with -j ${CPU_COUNT}.
recipes/bambamc/config.guess New script added to determine the canonical system name and handle various system detection tasks.
recipes/bambamc/config.sub New script added to validate and canonicalize configuration types provided as arguments.
recipes/bambamc/meta.yaml Updated to version 0.0.50, incremented build number to 8, specified new source URL and MD5 checksum, and added support for linux-aarch64 and osx-arm64.
recipes/smalt/build.sh Modified to enable debugging and multi-threaded execution in make, with specific adjustments for MacOS testing.

Possibly related PRs

Suggested reviewers

  • bgruening

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (2)
recipes/bambamc/build.sh (1)

7-7: Great optimization for parallel compilation!

The addition of -j ${CPU_COUNT} to the make command enables parallel compilation, which can significantly speed up the build process, especially on multi-core systems.

Consider adding a fallback for CPU_COUNT to ensure the script works even if the variable is not set:

-make -j ${CPU_COUNT}
+make -j ${CPU_COUNT:-1}

This change ensures that if CPU_COUNT is not set, the build will default to using a single core, maintaining compatibility with all environments.

recipes/bambamc/meta.yaml (1)

35-38: LGTM: Additional platform support added correctly.

The 'extra' section has been added to specify support for 'linux-aarch64' and 'osx-arm64' platforms, which aligns with the PR objective. This is the correct approach to extend platform support in Conda recipes.

Consider adding a comment explaining the purpose of these additional platforms for better maintainability. For example:

extra:
  additional-platforms:
    # Support for ARM64 architecture on Linux and macOS
    - linux-aarch64
    - osx-arm64
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 8fc154f and f07ac9b.

📒 Files selected for processing (2)
  • recipes/bambamc/build.sh (1 hunks)
  • recipes/bambamc/meta.yaml (2 hunks)
🔇 Additional comments (4)
recipes/bambamc/build.sh (2)

3-3: Excellent addition for improved error handling and debugging!

The set -xe command enhances the script's robustness:

  • -e ensures the script exits immediately if any command fails, preventing cascading errors.
  • -x enables debug mode, which prints each command before execution, aiding in troubleshooting.

This change aligns with best practices for shell scripting in build environments.


Line range hint 1-9: Clarification needed on aarch64/arm64 support

The PR objectives mention adding aarch64/arm64 builds, but there are no explicit changes in this script related to these architectures. While the current changes improve the build process, they don't directly address the stated objective.

Could you please clarify how the aarch64/arm64 support is being implemented? Is it handled in the recipe's meta.yaml file or through Conda's build matrix?

To verify the aarch64/arm64 support, we can check the meta.yaml file:

This will help us understand how the aarch64/arm64 support is configured in the recipe.

✅ Verification successful

aarch64/arm64 support confirmed in meta.yaml

The meta.yaml includes additional platforms for linux-aarch64 and osx-arm64, ensuring aarch64/arm64 builds are supported as per the PR objectives.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check if meta.yaml exists and search for aarch64/arm64 related entries
if [ -f recipes/bambamc/meta.yaml ]; then
    echo "Contents of meta.yaml:"
    cat recipes/bambamc/meta.yaml
    echo "Searching for aarch64/arm64 related entries:"
    grep -i 'aarch64\|arm64' recipes/bambamc/meta.yaml
else
    echo "meta.yaml file not found in the expected location."
fi

Length of output: 1090

recipes/bambamc/meta.yaml (2)

10-10: LGTM: Build number increment is correct.

The build number has been properly incremented from 7 to 8. This is the correct approach when updating a package without changing its version number.


Line range hint 1-38: Verify build and functionality on new platforms.

The package has been updated to version '0.0.50' with corresponding changes to the source URL and MD5 checksum. While the build and runtime dependencies remain unchanged, it's crucial to ensure that the package builds and functions correctly on the newly added platforms (linux-aarch64 and osx-arm64).

Please run the following script to check if there are any platform-specific build instructions or tests that need to be added:

If the script doesn't find any platform-specific instructions or tests, consider adding them to ensure the package works correctly on the new platforms.

Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Outside diff range and nitpick comments (4)
recipes/bambamc/build.sh (2)

5-6: Great addition for cross-platform compatibility!

Copying newer versions of config.guess and config.sub is crucial for supporting builds on newer architectures, particularly macOS ARM64. This change enhances the package's compatibility across different platforms.

Consider adding a check to ensure the files exist before copying:

if [ -f "${RECIPE_DIR}/config.guess" ] && [ -f "${RECIPE_DIR}/config.sub" ]; then
  cp ${RECIPE_DIR}/config.* .
else
  echo "Warning: config.guess or config.sub not found in RECIPE_DIR"
fi

This will prevent potential errors if the files are missing and provide a helpful warning.


11-11: Excellent optimization for build performance!

The modification to use make -j ${CPU_COUNT} allows the build process to utilize multiple CPU cores, which can significantly reduce compilation time, especially on multi-core systems.

Consider adding an upper limit to the number of jobs to prevent potential resource exhaustion on machines with many cores:

make -j $(( ${CPU_COUNT} < 4 ? ${CPU_COUNT} : 4 ))

This will use up to 4 cores or the number of available cores, whichever is less. Adjust the number 4 as appropriate for your build requirements.

recipes/smalt/build.sh (1)

3-3: Consider the implications of enabling debug mode

The change from set -ef -o pipefail to set -xef -o pipefail enables debug mode (-x), which prints each command and its arguments as they are executed. While this can be helpful for troubleshooting, it may also expose sensitive information in build logs.

Consider whether this level of verbosity is necessary for production builds. If it's intended for debugging purposes only, you might want to make it conditional or remove it before merging.

recipes/bambamc/config.sub (1)

1880-1885: Review the placement of editor configuration directives.

The script includes editor-specific configuration directives at the end:

# Local variables:
# eval: (add-hook 'before-save-hook 'time-stamp)
# time-stamp-start: "timestamp='"
# time-stamp-format: "%:y-%02m-%02d"
# time-stamp-end: "'"
# End:

While these lines are helpful for maintaining the timestamp, they might not be necessary in the committed script or could interfere with other developers' editor settings. Consider removing them or relocating them to a separate file or comment block to avoid potential confusion.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between f07ac9b and 63d8911.

📒 Files selected for processing (5)
  • recipes/bambamc/build.sh (1 hunks)
  • recipes/bambamc/config.guess (1 hunks)
  • recipes/bambamc/config.sub (1 hunks)
  • recipes/bambamc/meta.yaml (2 hunks)
  • recipes/smalt/build.sh (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • recipes/bambamc/meta.yaml
🔇 Additional comments (7)
recipes/bambamc/build.sh (2)

3-3: Excellent addition for improved error handling and debugging!

The set -xe command enhances the script's behavior:

  • -e ensures the script exits immediately if any command fails, preventing cascading errors.
  • -x prints each command before execution, facilitating easier debugging.

These improvements will make the build process more robust and easier to troubleshoot if issues arise.


1-12: Overall script structure looks good!

The script maintains a clear and logical flow:

  1. Setting up the environment
  2. Preparing configuration files
  3. Setting environment variables
  4. Configuring the build
  5. Compiling the code
  6. Installing the package

The new changes integrate seamlessly with this structure, enhancing the script's functionality without disrupting its overall flow.

recipes/smalt/build.sh (3)

9-9: Excellent optimization for build speed

The addition of -j ${CPU_COUNT} to the make command enables parallel compilation, which can significantly speed up the build process on multi-core systems.

This is a great optimization, especially for larger projects. Good job!


Line range hint 1-20: Overall, these changes improve build efficiency and cross-platform compatibility

The modifications to this build script enhance its functionality in several ways:

  1. It potentially improves debugging capabilities (though this might need reconsideration for production use).
  2. It optimizes build speed through parallel compilation.
  3. It addresses a MacOS-specific issue, improving cross-platform compatibility.

These changes demonstrate attention to both performance and platform-specific considerations, which is commendable. The script now should work more efficiently across different environments.


Line range hint 13-17: Good fix for MacOS dynamic library loading issue

The modification to set DYLD_LIBRARY_PATH for MacOS builds addresses a specific issue with finding the bambamc dynamic library during the make check process.

This is a well-thought-out solution to a platform-specific problem. It ensures that the tests can run successfully on MacOS without affecting other platforms.

Let's verify if this change is consistent with other recipes that might have similar MacOS-specific adjustments:

✅ Verification successful

Verification Confirmed: Consistent DYLD_LIBRARY_PATH Adjustments Across Recipes

The modification in recipes/smalt/build.sh aligns with similar adjustments in other build scripts, ensuring consistent handling of MacOS dynamic library loading issues.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for similar MacOS-specific DYLD_LIBRARY_PATH adjustments in other recipes

# Test: Search for similar patterns in other build.sh files
rg --type sh 'DYLD_LIBRARY_PATH=\$\{PREFIX\}/lib' recipes

Length of output: 548

recipes/bambamc/config.sub (1)

62-62: Confirm the use of legacy command substitution for compatibility.

The script uses backticks `...` for command substitution:

me=`echo "$0" | sed -e 's,.*/,,'`

While the modern syntax $(...) is generally preferred for readability and nesting, the use of backticks here is intentional to maintain compatibility with older, pre-POSIX shells, as noted in the comments on lines 55-61. No changes are necessary unless there's a specific requirement to support only POSIX-compliant shells.

recipes/bambamc/config.guess (1)

1-1700: LGTM: config.guess Script Addition

The config.guess script has been correctly added and is up-to-date. It includes comprehensive system detection, covering a wide range of architectures, including aarch64/arm64, which aligns with the PR objectives to add builds for these architectures. The script follows best practices for shell scripting and includes proper licensing information.

recipes/bambamc/config.sub Show resolved Hide resolved
recipes/bambamc/config.sub Show resolved Hide resolved
recipes/bambamc/config.sub Show resolved Hide resolved
@martin-g
Copy link
Contributor Author

martin-g commented Oct 1, 2024

The CircleCI check is not listed for some reason but it passed for both Linux and OSX - https://app.circleci.com/pipelines/github/bioconda/bioconda-recipes?branch=pull%2F51067

@martin-g
Copy link
Contributor Author

martin-g commented Oct 1, 2024

@BiocondaBot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

Package(s) built are ready for inspection:

Arch Package Zip File / Repodata CI Instructions
linux-64 bambamc-0.0.50-he4a0461_8.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
osx-64 bambamc-0.0.50-h10309d6_8.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-arm64 bambamc-0.0.50-hcb76912_8.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/9deeb8c4-4f31-413c-a9b2-5f7dcf54cc2a/artifacts/0/tmp/artifacts/packages <package name>
linux-aarch64 bambamc-0.0.50-h73052cd_8.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/fd3a3c7d-b3a5-44ab-aa99-75b0ee3adf12/artifacts/0/tmp/artifacts/packages <package name>

Docker image(s) built:

Package Tag CI Install with docker
bambamc 0.0.50--he4a0461_8 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bambamc:0.0.50--he4a0461_8.tar.gz | docker load

@martin-g
Copy link
Contributor Author

martin-g commented Oct 1, 2024

mgrigorov in 🌐 euler-arm-22 in /tmp/bambamc via 🅒 test123 took 2s 
❯ tar xvf bambamc-0.0.50-h73052cd_8.tar.bz2                                                                                                                                                                (test123) 
info/run_exports.json
info/test/run_test.sh
info/hash_input.json
info/recipe/build.sh
info/has_prefix
info/index.json
info/licenses/COPYING
info/recipe/meta.yaml.template
info/files
info/recipe/meta.yaml
info/recipe/conda_build_config.yaml
info/paths.json
info/about.json
info/recipe/config.sub
info/recipe/config.guess
info/git
lib/libbambamc.so
lib/pkgconfig/bambamc.pc
lib/libbambamc.so.0
lib/libbambamc.a
include/bambamc/BamBam_Config.h
include/bambamc/BamBam_StrDup.h
include/bambamc/BamBam_CollatorState.h
include/bambamc/BamBam_Unused.h
include/bambamc/BamBam_MergeHeapEntry.h
include/bambamc/BamBam_Hash.h
include/bambamc/BamBam_FormatAlignment.h
include/bambamc/BamBam_Chromosome.h
include/bambamc/BamBam_FormatNumber.h
include/bambamc/BamBam_FastQRead.h
include/bambamc/BamBam_BamCollationHash.h
include/bambamc/BamBam_BamCollatorInterface.h
include/bambamc/BamBam_GzipFileDecoder.h
include/bambamc/BamBam_LineParsing.h
include/bambamc/BamBam_BamFileDecoder.h
include/bambamc/BamBam_BamFlagBase.h
include/bambamc/BamBam_SamBamFileDecoder.h
include/bambamc/BamBam_List.h
include/bambamc/BamBam_LineBuffer.h
include/bambamc/BamBam_GzipReader.h
include/bambamc/BamBam_GzipWriter.h
include/bambamc/BamBam_ListNode.h
include/bambamc/BamBam_SamFileDecoder.h
include/bambamc/BamBam_BamCollationTempFileGenerator.h
include/bambamc/BamBam_BamHeaderInfo.h
include/bambamc/BamBam_BamCollationVector.h
include/bambamc/BamBam_BamCollationOutputVector.h
include/bambamc/BamBam_BamCollationHashEntry.h
include/bambamc/BamBam_BamAlignmentPut.h
include/bambamc/BamBam_BamWriter.h
include/bambamc/BamBam_BamFileHeader.h
include/bambamc/BamBam_SamInfo.h
include/bambamc/BamBam_CharBuffer.h
include/bambamc/BamBam_BgzfCompressor.h
include/bambamc/BamBam_BamCollator.h
include/bambamc/BamBam_BamSingleAlignment.h
lib/libbambamc.so.0.0.50

mgrigorov in 🌐 euler-arm-22 in /tmp/bambamc via 🅒 test123 
❯ file lib/*                                                                                                                                                                                               (test123) 
lib/libbambamc.a:         current ar archive
lib/libbambamc.so:        symbolic link to libbambamc.so.0.0.50
lib/libbambamc.so.0:      symbolic link to libbambamc.so.0.0.50
lib/libbambamc.so.0.0.50: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, not stripped
lib/pkgconfig:            directory

LGTM!

@martin-g
Copy link
Contributor Author

martin-g commented Oct 1, 2024

@BiocondaBot please add label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
aarch64 Related to adding linux-aarch64 support osx-arm64 Related to adding osx-arm64 support please review & merge set to ask for merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants