add bolt #22204

h-vetinari · 2023-03-04T04:25:58Z

Fixes conda-forge/llvmdev-feedstock#147

Towards #13019

Checklist

conda-forge-webservices · 2023-03-04T04:26:04Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/bolt) and found some lint.

Here's what I've got...

For recipes/bolt:

requirements: build: clangdev == 15.0.7 should not contain a space between relational operator and the version, i.e. clangdev ==15.0.7
requirements: host: clangdev == 15.0.7 should not contain a space between relational operator and the version, i.e. clangdev ==15.0.7

For recipes/bolt:

Recipe with the same name exists in bioconda: please discuss with @conda-forge/bioconda-recipes.

conda-forge-webservices · 2023-03-04T04:26:39Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/bolt) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipes/bolt:

Recipe with the same name exists in bioconda: please discuss with @conda-forge/bioconda-recipes.

conda-forge-webservices · 2023-03-04T04:27:14Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/bolt) and found it was in an excellent condition.

h-vetinari · 2023-03-04T06:00:07Z

Looks like the pyston people have been packaging bolt for their conda channel already, but in the full build variant (i.e. using -DLLVM_ENABLE_PROJECTS=...), not against existing llvm & clang.

h-vetinari · 2023-04-15T06:17:33Z

I don't really understand why X86CommonTableGen can't be found in llvmdev, where it's built. Perhaps we're not installing whatever public_tablegen_target means correctly?

stale · 2023-12-31T20:56:14Z

Hi friend!

We really, really, really appreciate that you have taken the time to make a PR on conda-forge/staged-recipes! conda-forge only exists because people like you donate their time to build and maintain conda recipes for use by the community.

In an effort to maintain this repository and increase the signal-to-noise for open PRs, the maintainers of staged-recipes close excessively old PRs after six months. This PR will remain open for another month, and then will be closed.

If you'd like to keep it open, please comment/push and we will be happy to oblige! Note that very old PRs will likely need to be rebased on main so that they can be rebuilt with the most recent CI scripts. If you have any trouble, or we missed reviewing this PR in the first place (sorry!), feel free to ping the team using a special command in a comment on the PR to get the attention of the staged-recipes team.

Cheers and thank you for contributing to this community effort!

h-vetinari · 2024-01-02T03:02:32Z

not stale

conda-forge-webservices · 2024-07-24T05:53:23Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/bolt/meta.yaml) and found it was in an excellent condition.

h-vetinari · 2024-07-24T05:54:18Z

Updated based on discussion in conda-forge/llvmdev-feedstock#276; essentially using the state from conda-forge/llvmdev-feedstock#279, minus obviously the bits that package llvmdev

h-vetinari · 2024-07-24T06:40:54Z

This will fail on windows due to some missing components in the azure images; on the llvmdev feedstock we're working around this with conda-forge/llvmdev-feedstock@221c9f3 (and I can do the same for bolt once this is merged). If the situation persists, I guess we'll have to teach smithy how to do that.

h-vetinari · 2024-07-24T20:52:41Z

@isuruf, would you like to review? It should be a pretty simple recipe now. Another advantage of splitting this off is that we won't miss any other files that bolt might install (and I did miss one binary - merge-fdata).

recipes/bolt/meta.yaml

isuruf · 2024-07-24T20:56:27Z

recipes/bolt/meta.yaml

+      {% endfor %}
+        # only on linux-64
+        - test -f $PREFIX/lib/libbolt_rt_hugify.a                       # [linux64]
+        - test -f $PREFIX/lib/libbolt_rt_instr.a                        # [linux64]


Also check that the cmake files are here

Only way to do that currently is to clobber

lib/cmake/llvm/LLVM-Config.cmake lib/cmake/llvm/LLVMConfig.cmake lib/cmake/llvm/LLVMConfigVersion.cmake lib/cmake/llvm/LLVMExports-release.cmake lib/cmake/llvm/LLVMExports.cmake

from llvmdev, unfortunately. But I guess better than nothing.

bolt doesn't have different cmake files?

nope, it's built as part of llvm. I guess we could ask for separate CMake files in the "standalone" upstream PR...

Does this change your opinion somehow, or are you fine with the always_include_files: (0e96961)?

@isuruf, could you let me know how you'd like to proceed with the CMake metadata here?

Let's make libbolt-devel depend on llvmdev

Thanks. I'm planning to merge this once CI passes (well, on unix at least; the rest I'll fix on the feedstock).

Please don't. We require reviews from everyone including core team.

Then please don't drip-feed change requests. If there's been two rounds of review and no more open comments are left, it's not unreasonable to think that things are wrapped up.

h-vetinari · 2024-07-25T04:46:28Z

Ugh, I don't know why always_include_files: isn't working here

ERROR: Glob $PREFIX/lib/cmake/llvm/LLVM-Config.cmake did not match in root_dir $PREFIX
ERROR: Glob $PREFIX/lib/cmake/llvm/LLVMConfig.cmake did not match in root_dir $PREFIX
ERROR: Glob $PREFIX/lib/cmake/llvm/LLVMConfigVersion.cmake did not match in root_dir $PREFIX
ERROR: Glob $PREFIX/lib/cmake/llvm/LLVMExports-release.cmake did not match in root_dir $PREFIX
ERROR: Glob $PREFIX/lib/cmake/llvm/LLVMExports.cmake did not match in root_dir $PREFIX

I've double-checked the paths, and they're there

-- Installing: $SRC_DIR/temp_prefix/lib/cmake/llvm/LLVMExports.cmake
-- Installing: $SRC_DIR/temp_prefix/lib/cmake/llvm/LLVMExports-release.cmake
-- Installing: $SRC_DIR/temp_prefix/lib/cmake/llvm/LLVMConfig.cmake
-- Installing: $SRC_DIR/temp_prefix/lib/cmake/llvm/LLVMConfigVersion.cmake
-- Installing: $SRC_DIR/temp_prefix/lib/cmake/llvm/LLVM-Config.cmake
[...]
+ mkdir -p $PREFIX/lib/cmake/llvm
+ mv ./temp_prefix/lib/libLLVMBOLTCore.a ./temp_prefix/lib/libLLVMBOLTPasses.a ./temp_prefix/lib/libLLVMBOLTProfile.a ./temp_prefix/lib/libLLVMBOLTRewrite.a ./temp_prefix/lib/libLLVMBOLTRuntimeLibs.a ./temp_prefix/lib/libLLVMBOLTTargetAArch64.a ./temp_prefix/lib/libLLVMBOLTTargetX86.a ./temp_prefix/lib/libLLVMBOLTUtils.a $PREFIX/lib
+ mv ./temp_prefix/lib/libbolt_rt_hugify.a ./temp_prefix/lib/libbolt_rt_instr.a $PREFIX/lib
+ mv ./temp_prefix/lib/cmake/llvm $PREFIX/lib/cmake/llvm

recipes/bolt/install_bolt.sh

Co-authored-by: Marcel Bargull <[email protected]>

recipes/bolt/bld.bat

isuruf · 2024-07-26T22:43:19Z

recipes/bolt/install_bolt.bat

+    mkdir %LIBRARY_LIB%\cmake\llvm
+    move .\temp_prefix\lib\LLVMBOLT*.lib %LIBRARY_LIB%
+    REM copy CMake metadata
+    move .\temp_prefix\lib\cmake\llvm %LIBRARY_LIB%\cmake\llvm


What about include files?

I had been wondering about that too, but there's nothing I can find. The only occurrences of include/bolt in the logs (example) are during the build, not during installation. I've now done a regex-search (include/.*bolt) as well, and it only finds false positives

If there's anything that goes beyond what's in llvmdev, it's now being installed (4f17e53)

If there are no include files, then we might as well drop libbolt-devel as users can't use those libraries (not easily anyways).

I don't know sufficiently well how bolt works, but it's conceivable to me that bolt the binary needs those libraries to actually transform a binary (perhaps embedding some symbols from the support libraries along the way).

It seemed natural to split off the libraries, but if you want, I can just stuff everything into bolt itself.

isuruf · 2024-07-27T00:39:08Z

recipes/bolt/meta.yaml

+        - test -f $PREFIX/lib/libbolt_rt_hugify.a                       # [linux64]
+        - test -f $PREFIX/lib/libbolt_rt_instr.a                        # [linux64]


These two should be in bolt as they are used at runtime of bolt. The others are only needed if bolt is used as a library.

I'm not sure (between the two open threads) what you want me to do now. Here you say

The others are only needed if bolt is used as a library.

But in the other you say

If there are no include files, then we might as well drop libbolt-devel as users can't use those libraries

I understood that we'll move libbolt_rt_{hugify,instr} to bolt. But for the rest, do we want to:

ship them as libbolt-devel

ship them as part of bolt

not ship them at all until a use-case comes up

3 is preferred. 1 if not.

OK, I found the bolt headers: https://github.com/llvm/llvm-project/tree/main/bolt/include/bolt; no idea why they're not getting installed.

Also, I guess we can restrict this to linux until anything changes upstream w.r.t. llvm/llvm-project#72205

h-vetinari force-pushed the bolt branch from d45c04d to e26ee2f Compare April 15, 2023 05:42

h-vetinari added 13 commits May 18, 2023 15:06

add bolt

8886325

lint

e9097a6

rename to llvm-bolt

b888381

patch in minimal standalone changes for bolt CMakeLists

4d9a06b

simplify patch

c7c344f

include necessary CMake module

dd2dbeb

point CMAKE_MODULE_PATH to llvm's cmake/Modules

12d9023

patch in module search path like libcxx

46b34e2

try lld approach

350e74c

bump to 16.0.1

aedb4ee

use matching compiler on osx

61cc902

support libs

f79e231

bump to 16.0.3

dca8d45

h-vetinari force-pushed the bolt branch from 8211993 to dca8d45 Compare May 18, 2023 04:08

stale bot added the stale will be closed in 30 days label Dec 31, 2023

stale bot removed the stale will be closed in 30 days label Jan 2, 2024

h-vetinari added 4 commits July 10, 2024 14:55

Merge remote-tracking branch 'up/main' into bolt

d39eba5

bump to 17.0.6

19aebfc

rebase patches

828a41b

add stdlib

c605794

h-vetinari mentioned this pull request Jul 10, 2024

build bolt conda-forge/llvmdev-feedstock#276

Draft

Merge remote-tracking branch 'upstream/main' into bolt

e51d279

h-vetinari force-pushed the bolt branch from 12b2827 to bc22902 Compare July 24, 2024 06:04

build bolt together with llvmdev

53b59f6

h-vetinari force-pushed the bolt branch from bc22902 to 53b59f6 Compare July 24, 2024 06:38

fix clobber

cb2f1c4

isuruf reviewed Jul 24, 2024

View reviewed changes

recipes/bolt/meta.yaml Outdated Show resolved Hide resolved

isuruf reviewed Jul 24, 2024

View reviewed changes

h-vetinari added 5 commits July 25, 2024 08:05

-devel

f8d0234

package CMake metadata for libbolt-devel

0e96961

add test for overlooked binary

11472ee

more -devel

eebba24

copy CMake metadata

99ba39a

mbargull reviewed Jul 25, 2024

View reviewed changes

recipes/bolt/install_bolt.sh Outdated Show resolved Hide resolved

h-vetinari and others added 3 commits July 25, 2024 08:54

Properly copy into existing folder

6bb8bdc

Co-authored-by: Marcel Bargull <[email protected]>

remove superfluous prefix

a54de91

make libbolt-devel depend on llvmdev

bd1a6f1

isuruf reviewed Jul 26, 2024

View reviewed changes

recipes/bolt/bld.bat Outdated Show resolved Hide resolved

isuruf reviewed Jul 26, 2024

View reviewed changes

recipes/bolt/bld.bat Outdated Show resolved Hide resolved

isuruf reviewed Jul 26, 2024

View reviewed changes

h-vetinari added 2 commits July 27, 2024 10:04

remove extra target; build with MSVC on windows

004cdc9

install bolt includes, if any exist

4f17e53

isuruf reviewed Jul 27, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add bolt #22204

add bolt #22204

h-vetinari commented Mar 4, 2023

conda-forge-webservices bot commented Mar 4, 2023

conda-forge-webservices bot commented Mar 4, 2023

conda-forge-webservices bot commented Mar 4, 2023

h-vetinari commented Mar 4, 2023

h-vetinari commented Apr 15, 2023

stale bot commented Dec 31, 2023

h-vetinari commented Jan 2, 2024

conda-forge-webservices bot commented Jul 24, 2024

h-vetinari commented Jul 24, 2024

h-vetinari commented Jul 24, 2024

h-vetinari commented Jul 24, 2024

isuruf Jul 24, 2024

h-vetinari Jul 24, 2024

isuruf Jul 24, 2024

h-vetinari Jul 24, 2024

h-vetinari Jul 24, 2024

h-vetinari Jul 26, 2024

isuruf Jul 26, 2024

h-vetinari Jul 26, 2024

isuruf Jul 26, 2024

h-vetinari Jul 26, 2024

h-vetinari commented Jul 25, 2024

isuruf Jul 26, 2024

h-vetinari Jul 26, 2024

h-vetinari Jul 26, 2024

isuruf Jul 26, 2024

h-vetinari Jul 27, 2024

isuruf Jul 27, 2024

h-vetinari Aug 2, 2024

isuruf Aug 2, 2024

h-vetinari Aug 3, 2024

h-vetinari Aug 3, 2024

		- test -f $PREFIX/lib/libbolt_rt_hugify.a # [linux64]
		- test -f $PREFIX/lib/libbolt_rt_instr.a # [linux64]

add bolt #22204

Are you sure you want to change the base?

add bolt #22204

Conversation

h-vetinari commented Mar 4, 2023

conda-forge-webservices bot commented Mar 4, 2023

conda-forge-webservices bot commented Mar 4, 2023

conda-forge-webservices bot commented Mar 4, 2023

h-vetinari commented Mar 4, 2023

h-vetinari commented Apr 15, 2023

stale bot commented Dec 31, 2023

h-vetinari commented Jan 2, 2024

conda-forge-webservices bot commented Jul 24, 2024

h-vetinari commented Jul 24, 2024

h-vetinari commented Jul 24, 2024

h-vetinari commented Jul 24, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

h-vetinari commented Jul 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment