Convert circleci workflows to github actions #3325

robandpdx · 2024-03-27T22:55:43Z

This pull request converts the CircleCI workflows to GitHub actions workflows.

Notes

The dependencies between jobs seems to not be necessary, and I assume they exist only to limit spend on expensive jobs when less expensive jobs are failing. I have left the needs: * dependency definitions commented out in the workflow.

Some jobs in the workflow require GPU runners. Some jobs in the workflow require ARM runners. I have configured the following runner groups.

4-core-gpu
4-core-arm

Errors

I did my best to make all the jobs work, but lacking knowledge specific to this project makes it nearly impossible for me to get everything working. Someone with better knowledge of this project will need to address these error.

linux-x86_64-AVX512-cmake job fails with the following error:

Error

CMake Error at /home/runner/miniconda3/share/cmake-3.26/Modules/GoogleTestAddTests.cmake:112 (message):
make[2]: Leaving directory '/home/runner/work/faiss/faiss/build'
  Error running test executable.

    Path: '/home/runner/work/faiss/faiss/build/tests/faiss_test'
    Result: Illegal instruction

linux-x86_64-GPU-cmake job fails with the following error:

Error

/home/runner/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: ../faiss/libfaiss.so: undefined reference to `__libc_single_threaded'
collect2: error: ld returned 1 exit status

windows-x86_64-conda job fails with the following error:

Error

CMake Error at CMakeLists.txt:42 (project):
  Generator

    Visual Studio 16 2019

  could not find any instance of Visual Studio.



-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
  File "C:\Users\runneradmin\miniconda3\envs\test\Scripts\conda-build-script.py", line 10, in <module>
    sys.exit(execute())
             ^^^^^^^^^
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\cli\main_build.py", line 581, in execute
    api.build(
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\api.py", line 250, in build
    return build_tree(
           ^^^^^^^^^^^
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\build.py", line 3762, in build_tree
    packages_from_this = build(
                         ^^^^^^
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\build.py", line 2839, in build
    newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\build.py", line 1908, in bundle_conda
    utils.check_call_env(
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\utils.py", line 408, in check_call_env
    return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\utils.py", line 384, in _func_defaulting_env_to_os_environ
    raise subprocess.CalledProcessError(proc.returncode, _args)
subprocess.CalledProcessError: Command '['C:\\Windows\\system32\\cmd.exe', '/d', '/c', 'C:\\Users\\runneradmin\\miniconda3\\envs\\test\\conda-bld\\faiss-pkg_1711501857970\\work\\build-lib.bat']' returned non-zero exit status 1.
Error: Process completed with exit code 1.

linux-x86_64-GPU-packages-CUDA-11-4-4 job fails with the following error:

Error

CMake Error at /home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/share/cmake-3.26/Modules/CMakeDetermineCompilerId.cmake:751 (message):
  Compiling the CUDA compiler identification source file
  "CMakeCUDACompilerId.cu" failed.

  Compiler:
  /home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin/nvcc


  Build flags:

  Id flags: --keep;--keep-dir;tmp -v



  The output was:

  1

  #$ _NVVM_BRANCH_=nvvm

  #$ _SPACE_=

  #$ _CUDART_=cudart

  #$
  _HERE_=/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin


  #$
  _THERE_=/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin


  #$ _TARGET_SIZE_=

  #$ _TARGET_DIR_=

  #$ _TARGET_SIZE_=64

  #$
  TOP=/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin/..


  #$
  NVVMIR_LIBRARY_DIR=/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin/../nvvm/libdevice


  #$
  LD_LIBRARY_PATH=/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin/../lib:


  #$
  PATH=/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin/../nvvm/bin:/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin:/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin:/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/bin:/home/runner/miniconda3/condabin:/home/runner/miniconda3/condabin:/opt/ngc-cli:/opt/miniconda/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin


  #$
  INCLUDES="-I/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin/../include"


  #$ LIBRARIES=
  "-L/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin/../lib/stubs"
  "-L/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin/../lib"


  #$ CUDAFE_FLAGS=

  #$ PTXAS_FLAGS=

  #$ rm tmp/a_dlink.reg.c

  #$ gcc -D__CUDA_ARCH__=520 -E -x c++ -DCUDA_DOUBLE_MATH_FUNCTIONS
  -D__CUDACC__ -D__NVCC__
  "-I/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/bin/../include"
  -D__CUDACC_VER_MAJOR__=11 -D__CUDACC_VER_MINOR__=4
  -D__CUDACC_VER_BUILD__=152 -D__CUDA_API_VER_MAJOR__=11
  -D__CUDA_API_VER_MINOR__=4 -include "cuda_runtime.h" -m64
  "CMakeCUDACompilerId.cu" -o "tmp/CMakeCUDACompilerId.cpp1.ii"

  #$ cicc --c++17 --gnu_version=110400 --orig_src_file_name
  "CMakeCUDACompilerId.cu" --allow_managed -arch compute_52 -m64
  --no-version-ident -ftz=0 -prec_div=1 -prec_sqrt=1 -fmad=1
  --include_file_name "CMakeCUDACompilerId.fatbin.c" -tused
  --gen_module_id_file --module_id_file_name
  "tmp/CMakeCUDACompilerId.module_id" --gen_c_file_name
  "tmp/CMakeCUDACompilerId.cudafe1.c" --stub_file_name
  "tmp/CMakeCUDACompilerId.cudafe1.stub.c" --gen_device_file_name
  "tmp/CMakeCUDACompilerId.cudafe1.gpu" "tmp/CMakeCUDACompilerId.cpp1.ii" -o
  "tmp/CMakeCUDACompilerId.ptx"

  /usr/include/stdio.h(189): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdio.h(201): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdio.h(223): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdio.h(260): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdio.h(285): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdio.h(294): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdio.h(303): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdio.h(309): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdio.h(315): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdio.h(830): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdlib.h(566): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdlib.h(570): error: attribute "__malloc__" does not take
  arguments



  /usr/include/stdlib.h(799): error: attribute "__malloc__" does not take
  arguments



  13 errors detected in the compilation of "CMakeCUDACompilerId.cu".

  # --error 0x1 --





Call Stack (most recent call first):
  /home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/share/cmake-3.26/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
  /home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/share/cmake-3.26/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test)
  /home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/_build_env/share/cmake-3.26/Modules/CMakeDetermineCUDACompiler.cmake:307 (CMAKE_DETERMINE_COMPILER_ID)
  CMakeLists.txt:42 (project)


-- Configuring incomplete, errors occurred!

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/exception_handler.py", line 17, in __call__
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/cli/main.py", line 83, in main_subshell
        exit_code = do_call(args, parser)
                    ^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/cli/conda_argparse.py", line 175, in do_call
        result = plugin_subcommand.action(getattr(args, "_args", args))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/plugin.py", line 17, in build
        return execute(args)
               ^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/cli/main_build.py", line 581, in execute
        api.build(
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/api.py", line 250, in build
        return build_tree(
               ^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 3762, in build_tree
        packages_from_this = build(
                             ^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 2839, in build
        newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 1908, in bundle_conda
        utils.check_call_env(
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/utils.py", line 408, in check_call_env
        return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/utils.py", line 384, in _func_defaulting_env_to_os_environ
        raise subprocess.CalledProcessError(proc.returncode, _args)
    subprocess.CalledProcessError: Command '['/usr/bin/bash', '-e', '/home/runner/miniconda3/conda-bld/faiss-pkg_1711562812250/work/build-lib.sh']' returned non-zero exit status 1.

`$ /home/runner/miniconda3/condabin/conda build faiss-gpu --variants { "cudatoolkit": "11.4.4", "c_compiler_version": "11.2", "cxx_compiler_version": "11.2" } --user pytorch --label main -c pytorch -c nvidia/label/cuda-11.4.4 -c nvidia`

  environment variables:
                 CIO_TEST=<not set>
                    CONDA=/home/runner/miniconda3
    CONDA_ALLOW_SOFTLINKS=false
           CONDA_PKGS_DIR=/home/runner/conda_pkgs_dir
               CONDA_ROOT=/home/runner/miniconda3
           CURL_CA_BUNDLE=<not set>
       GITHUB_ACTION_PATH=/home/runner/work/faiss/faiss/./.github/actions/build_conda
        GITHUB_EVENT_PATH=/home/runner/work/_temp/_github_workflow/event.json
              GITHUB_PATH=/home/runner/work/_temp/_runner_file_commands/add_path_0d0cf8a2-a36c-
                          4ab0-ba22-e4d1421282bc
               LD_PRELOAD=<not set>
                     PATH=/home/runner/miniconda3/condabin:/opt/ngc-cli:/opt/miniconda/condabin:
                          /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/game
                          s:/usr/local/games:/snap/bin
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>

     active environment : None
       user config file : /home/runner/.condarc
 populated config files : /home/runner/.condarc
          conda version : 24.3.0
    conda-build version : 24.3.0
         python version : 3.12.1.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=zen2
                          __conda=24.3.0=0
                          __cuda=12.2=0
                          __glibc=2.35=0
                          __linux=6.2.0=0
                          __unix=0=0
       base environment : /home/runner/miniconda3  (writable)
      conda av data dir : /home/runner/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/runner/conda_pkgs_dir
       envs directories : /home/runner/miniconda3/envs
                          /home/runner/.conda/envs
               platform : linux-64
             user-agent : conda/24.3.0 requests/2.31.0 CPython/3.12.1 Linux/6.2.0-1011-azure ubuntu/22.04.3 glibc/2.35 solver/libmamba conda-libmamba-solver/23.12.0 libmambapy/1.5.3
                UID:GID : 1001:100
             netrc file : None
           offline mode : False


An unexpected error has occurred. Conda has prepared the above report.
If you suspect this error is being caused by a malfunctioning plugin,
consider using the --no-plugins option to turn off plugins.

Example: conda --no-plugins install <package>

Alternatively, you can set the CONDA_NO_PLUGINS environment variable on
the command line to run the command without plugins enabled.

Example: CONDA_NO_PLUGINS=true conda install <package>

Upload successful.
Error: Process completed with exit code 1.

linux-x86_64-GPU-RAFT-packages-CUDA11-8-0 job fails with the following error:

Error

CMake Error at /home/runner/miniconda3/conda-bld/faiss-pkg_1711571792861/_build_env/share/cmake-3.29/Modules/FindCUDAToolkit.cmake:855 (message):
  Could not find nvcc, please set CUDAToolkit_ROOT.
Call Stack (most recent call first):
  CMakeLists.txt:114 (find_package)


-- Configuring incomplete, errors occurred!

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/exception_handler.py", line 17, in __call__
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/cli/main.py", line 83, in main_subshell
        exit_code = do_call(args, parser)
                    ^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/cli/conda_argparse.py", line 175, in do_call
        result = plugin_subcommand.action(getattr(args, "_args", args))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/plugin.py", line 17, in build
        return execute(args)
               ^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/cli/main_build.py", line 581, in execute
        api.build(
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/api.py", line 250, in build
        return build_tree(
               ^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 3762, in build_tree
        packages_from_this = build(
                             ^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 2839, in build
        newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 1908, in bundle_conda
        utils.check_call_env(
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/utils.py", line 408, in check_call_env
        return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/utils.py", line 384, in _func_defaulting_env_to_os_environ
        raise subprocess.CalledProcessError(proc.returncode, _args)
    subprocess.CalledProcessError: Command '['/usr/bin/bash', '-e', '/home/runner/miniconda3/conda-bld/faiss-pkg_1711571792861/work/build-pkg.sh']' returned non-zero exit status 1.

`$ /home/runner/miniconda3/condabin/conda build faiss-gpu-raft --variants { "cudatoolkit": "11.8.0", "c_compiler_version": "11.2", "cxx_compiler_version": "11.2" } --user pytorch --label main -c pytorch -c nvidia/label/cuda-11.8.0 -c nvidia -c rapidsai -c rapidsai-nightly -c conda-forge`

  environment variables:
                 CIO_TEST=<not set>
                    CONDA=/home/runner/miniconda3
    CONDA_ALLOW_SOFTLINKS=false

linux-x86_64-GPU-packages-CUDA-12-1-1 job fails with the following error:

Error

CMake Error at /home/runner/miniconda3/conda-bld/faiss-pkg_1711571788943/_build_env/share/cmake-3.26/Modules/FindCUDAToolkit.cmake:777 (message):
  Could not find nvcc, please set CUDAToolkit_ROOT.
Call Stack (most recent call first):
  CMakeLists.txt:114 (find_package)


-- Configuring incomplete, errors occurred!

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/exception_handler.py", line 17, in __call__
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/cli/main.py", line 83, in main_subshell
        exit_code = do_call(args, parser)
                    ^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/cli/conda_argparse.py", line 175, in do_call
        result = plugin_subcommand.action(getattr(args, "_args", args))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/plugin.py", line 17, in build
        return execute(args)
               ^^^^^^^^^^^^^

linux-x86_64-GPU-RAFT-packages-CUDA12-1-1 job fails with the following error:

Error

CMake Error at /home/runner/miniconda3/conda-bld/faiss-pkg_1711571797816/_build_env/share/cmake-3.29/Modules/FindCUDAToolkit.cmake:855 (message):
  Could not find nvcc, please set CUDAToolkit_ROOT.
Call Stack (most recent call first):
  CMakeLists.txt:114 (find_package)


-- Configuring incomplete, errors occurred!

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/exception_handler.py", line 17, in __call__
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/cli/main.py", line 83, in main_subshell
        exit_code = do_call(args, parser)
                    ^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda/cli/conda_argparse.py", line 175, in do_call
        result = plugin_subcommand.action(getattr(args, "_args", args))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/plugin.py", line 17, in build
        return execute(args)
               ^^^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/cli/main_build.py", line 581, in execute
        api.build(
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/api.py", line 250, in build
        return build_tree(
               ^^^^^^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 3762, in build_tree
        packages_from_this = build(
                             ^^^^^^
      File "/home/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 2839, in build
        newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)

windows-x86_64-packages job fails with the following error:

Error

CMake Error at CMakeLists.txt:42 (project):
  Generator

    Visual Studio 16 2019

  could not find any instance of Visual Studio.



-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
  File "C:\Users\runneradmin\miniconda3\envs\test\Scripts\conda-build-script.py", line 10, in <module>
    sys.exit(execute())
             ^^^^^^^^^
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\cli\main_build.py", line 581, in execute
    api.build(
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\api.py", line 250, in build
    return build_tree(
           ^^^^^^^^^^^
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\build.py", line 3762, in build_tree
    packages_from_this = build(
                         ^^^^^^
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\build.py", line 2839, in build
    newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\build.py", line 1908, in bundle_conda
    utils.check_call_env(
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\utils.py", line 408, in check_call_env
    return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\runneradmin\miniconda3\envs\test\Lib\site-packages\conda_build\utils.py", line 384, in _func_defaulting_env_to_os_environ
    raise subprocess.CalledProcessError(proc.returncode, _args)
subprocess.CalledProcessError: Command '['C:\\Windows\\system32\\cmd.exe', '/d', '/c', 'C:\\Users\\runneradmin\\miniconda3\\envs\\test\\conda-bld\\faiss-pkg_1711571662616\\work\\build-lib.bat']' returned non-zero exit status 1.
Error: Process completed with exit code 1.

OSX-arm64-packages job fails with the following error:

Error

CMakeFiles/cmTC_8330e.dir/testCXXCompiler.cxx.o -o cmTC_8330e 
    ld: unsupported tapi file type '!tapi-tbd' in YAML file '/Applications/Xcode_15.0.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/libSystem.tbd' for architecture x86_64
    clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
    make[1]: *** [CMakeFiles/cmTC_8330e.dir/build.make:100: cmTC_8330e] Error 1
    make[1]: Leaving directory '/Users/runner/miniconda3/conda-bld/faiss-pkg_1711571588527/work/_build/CMakeFiles/CMakeScratch/TryCompile-Z5yjE3'
    make: *** [Makefile:127: cmTC_8330e/fast] Error 2
    
    

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:42 (project)



# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/exception_handler.py", line 17, in __call__
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/cli/main.py", line 83, in main_subshell
        exit_code = do_call(args, parser)
                    ^^^^^^^^^^^^^^^^^^^^^
      File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/cli/conda_argparse.py", line 175, in do_call
        result = plugin_subcommand.action(getattr(args, "_args", args))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/plugin.py", line 17, in build
        return execute(args)
               ^^^^^^^^^^^^^
      File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/cli/main_build.py", line 581, in execute
        api.build(
      File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/api.py", line 250, in build
        return build_tree(
               ^^^^^^^^^^^
      File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 3762, in build_tree
        packages_from_this = build(
                             ^^^^^^
      File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 2839, in build
        newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 1908, in bundle_conda
        utils.check_call_env(
      File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/utils.py", line 408, in check_call_env
        return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/utils.py", line 384, in _func_defaulting_env_to_os_environ
        raise subprocess.CalledProcessError(proc.returncode, _args)
    subprocess.CalledProcessError: Command '['/bin/bash', '-e', '/Users/runner/miniconda3/conda-bld/faiss-pkg_1711571588527/work/build-lib-osx.sh']' returned non-zero exit status 1.

`$ /Users/runner/miniconda3/condabin/conda build faiss --user pytorch --label main -c pytorch`

Here is the latest workflow run in my fork.

https://fburl.com/workplace/f6mz6tmw

facebook-github-bot · 2024-03-28T06:06:49Z

@junjieqi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

ramilbakhshyiev · 2024-04-24T05:28:11Z

Hi @robandpdx, I'm starting to look at this. Some early questions:

Are there needs with enabling job dependencies? I am ok to start without it and then add those in later. You're correct that it's for cost savings purposes.
Would we have SSH access to debuggers for all of our runners?
Is this PR an ok communication channel or should I use something else?

robandpdx · 2024-04-24T22:24:04Z

Are there needs with enabling job dependencies? I am ok to start without it and then add those in later. You're correct that it's for cost savings purposes.

I have uncommented lines 57 and 66, putting the job dependencies back into place. I have also updated the runner label needed for the GPU runners to 4-core-ubuntu-gpu-t4. This should align with what's available in your org, I think.

Would we have SSH access to debuggers for all of our runners?

Unfortunately, this is not as easily done with GitHub actions runners as with CircleCI ☹️ There are two options I've found. 1) I wrote these instructions to get a shell on a runner. Hacky, but it works. 2) Use tmate.

Is this PR an ok communication channel or should I use something else?

Communication via PR comments is good for me. If we need more, there is a slack channel @bigfootjon can help get you access to.

ramilbakhshyiev · 2024-04-24T22:34:44Z

Thanks @robandpdx !

Summary: This commit is the first in a series in an attempt to incrementally enable all jobs currenlty performed by CircleCI. It includes the main configuration files provided by GitHub team + 1 build. Original PR: #3325 Differential Revision: D56664307

Summary: This commit is the first in a series in an attempt to incrementally enable all jobs currenlty performed by CircleCI. It includes the main configuration files provided by GitHub team + 1 build. Original PR: #3325 Differential Revision: D56664619

Summary: This commit is the first in a series in an attempt to incrementally enable all jobs currenlty performed by CircleCI. It includes the main configuration files provided by GitHub team + 1 build. Original PR: #3325 Differential Revision: D56665402

Summary: This commit is the first in a series in an attempt to incrementally enable all jobs currenlty performed by CircleCI. It includes the main configuration files provided by GitHub team + 1 build. Original PR: #3325 Differential Revision: D56665616

Summary: This commit is the first in a series in an attempt to incrementally enable all jobs currenlty performed by CircleCI. It includes the main configuration files provided by GitHub team + 1 build. Original PR: facebookresearch#3325 Differential Revision: D56665616

Summary: This commit is the first in a series in an attempt to incrementally enable all jobs currenlty performed by CircleCI. It includes the main configuration files provided by GitHub team + 1 build. Original PR: #3325 Differential Revision: D56671582

Summary: This commit is the first in a series in an attempt to incrementally enable all jobs currenlty performed by CircleCI. It includes the main configuration files provided by GitHub team + 1 build. Original PR: #3325 Reviewed By: junjieqi Differential Revision: D56671582 fbshipit-source-id: c8a21cd69aabaf86134eb86753e90b1facf51bc3

ramilbakhshyiev · 2024-05-20T04:52:04Z

Closing this as we have fixed all issues here and have ported all pipelines to GHA. Thanks for getting us started, Rob!

Summary: This commit is the first in a series in an attempt to incrementally enable all jobs currenlty performed by CircleCI. It includes the main configuration files provided by GitHub team + 1 build. Original PR: facebookresearch#3325 Reviewed By: junjieqi Differential Revision: D56671582 fbshipit-source-id: c8a21cd69aabaf86134eb86753e90b1facf51bc3

facebook-github-bot added the CLA Signed label Mar 27, 2024

mdouze added the install label Mar 28, 2024

robandpdx force-pushed the convert-circleci-to-github-actions branch from 1d68d9b to 4877294 Compare April 24, 2024 22:11

Convert circleci workflows to github actions

ed4b32b

robandpdx force-pushed the convert-circleci-to-github-actions branch from 215487a to ed4b32b Compare April 24, 2024 22:12

ramilbakhshyiev mentioned this pull request Apr 27, 2024

Initial config and linux-x86_64-AVX2-cmake build job only #3389

Closed

ramilbakhshyiev mentioned this pull request Apr 27, 2024

Initial config and linux-x86_64-AVX2-cmake build job only #3390

Closed

ramilbakhshyiev mentioned this pull request Apr 27, 2024

Initial config and linux-x86_64-AVX2-cmake build job only #3391

Closed

ramilbakhshyiev mentioned this pull request Apr 27, 2024

Initial config and linux-x86_64-cmake build job only #3392

Closed

ramilbakhshyiev mentioned this pull request Apr 27, 2024

Initial config and linux-x86_64-cmake build job only #3395

Closed

ramilbakhshyiev mentioned this pull request Apr 27, 2024

Initial config and linux-x86_64-cmake build job only #3396

Closed

ramilbakhshyiev closed this May 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert circleci workflows to github actions #3325

Convert circleci workflows to github actions #3325

robandpdx commented Mar 27, 2024

facebook-github-bot commented Mar 28, 2024

ramilbakhshyiev commented Apr 24, 2024

robandpdx commented Apr 24, 2024

ramilbakhshyiev commented Apr 24, 2024

ramilbakhshyiev commented May 20, 2024

Convert circleci workflows to github actions #3325

Convert circleci workflows to github actions #3325

Conversation

robandpdx commented Mar 27, 2024

Notes

Errors

facebook-github-bot commented Mar 28, 2024

ramilbakhshyiev commented Apr 24, 2024

robandpdx commented Apr 24, 2024

ramilbakhshyiev commented Apr 24, 2024

ramilbakhshyiev commented May 20, 2024