convert circleci workflows to github actions

[robandpdx]

This pull request converts the CircleCI workflows to GitHub actions workflows.

Notes

I could not get the "Increase open file descriptor limit" job to work on macos. sudo is not available on macos runners.

Errors

The following jobs have errors that I am not able to solve. Someone with better knowledge fo the project will need to adress these error.

linux-build-examples error

Error

[2024-01-09T20:16:12.092+00:00] Action failed: root//ocaml/ppx:ppx (prelude//platforms:default#524f8da68ea2a374) (ocaml_link_native)
[2024-01-09T20:16:12.092+00:00] Local command returned non-zero exit code 2
[2024-01-09T20:16:12.092+00:00] Local command: env -- "BUCK_SCRATCH_PATH=buck-out/v2/tmp/root/524f8da68ea2a374/ocaml/ppx/__ppx__/ocaml_link_native" buck-out/v2/gen/root/524f8da68ea2a374/ocaml/ppx/__ppx__/ocamloptnative.sh -g -noautolink -cc buck-out/v2/gen/root/524f8da68ea2a374/ocaml/ppx/__ppx__/ld_native.sh -linkall -w -4-29-35-41-42-44-45-48-50-58-70 third-party/ocaml/opam/lib/ocaml-compiler-libs/shadow/ocaml_shadow.cmxa third-party/ocaml/opam/lib/ppx_derivers/ppx_derivers.cmxa third-party/ocaml/opam/lib/ocaml/compiler-libs/ocamlcommon.cmxa third-party/ocaml/opam/lib/ocaml-compiler-libs/common/ocaml_common.cmxa third-party/ocaml/opam/lib/ppxlib/astlib/astlib.cmxa third-party/ocaml/opam/lib/ppxlib/ast/ppxlib_ast.cmxa third-party/ocaml/opam/lib/ppxlib/print_diff/ppxlib_print_diff.cmxa third-party/ocaml/opam/lib/sexplib0/sexplib0.cmxa third-party/ocaml/opam/lib/stdlib-shims/stdlib_shims.cmxa third-party/ocaml/opam/lib/ppxlib/stdppx/stdppx.cmxa third-party/ocaml/opam/lib/ppxlib/traverse_builtins/ppxlib_traverse_builtins.cmxa third-party/ocaml/opam/lib/ppxlib/ppxlib.cmxa buck-out/v2/gen/root/524f8da68ea2a374/ocaml/ppx/__ppx-record-selectors__/libppx-record-selectors.cmxa buck-out/v2/gen/root/524f8da68ea2a374/ocaml/ppx/__ppx-record-selectors__/libppx-record-selectors.a -args buck-out/v2/gen/root/524f8da68ea2a374/ocaml/ppx/__ppx__/cmxs_order_native.lst -cclib -lpthread -o buck-out/v2/gen/root/524f8da68ea2a374/ocaml/ppx/__ppx__/ppx.opt
[2024-01-09T20:16:12.092+00:00] Stdout: <empty>
[2024-01-09T20:16:12.092+00:00] Stderr:
ld.lld: error: undefined symbol: ZSTD_decompress
>>> referenced by zstd.c:98 (runtime/zstd.c:98)
>>>               zstd.npic.o:(caml_zstd_decompress) in archive third-party/ocaml/opam/lib/ocaml/compiler-libs/ocamlcommon.a

ld.lld: error: undefined symbol: ZSTD_createCCtx
>>> referenced by zstd.c:39 (runtime/zstd.c:39)
>>>               zstd.npic.o:(caml_zstd_compress) in archive third-party/ocaml/opam/lib/ocaml/compiler-libs/ocamlcommon.a

ld.lld: error: undefined symbol: ZSTD_compressStream2
>>> referenced by zstd.c:72 (runtime/zstd.c:72)
>>>               zstd.npic.o:(caml_zstd_compress) in archive third-party/ocaml/opam/lib/ocaml/compiler-libs/ocamlcommon.a

ld.lld: error: undefined symbol: ZSTD_freeCCtx
>>> referenced by zstd.c:77 (runtime/zstd.c:77)
>>>               zstd.npic.o:(caml_zstd_compress) in archive third-party/ocaml/opam/lib/ocaml/compiler-libs/ocamlcommon.a
>>> referenced by zstd.c:89 (runtime/zstd.c:89)
>>>               zstd.npic.o:(caml_zstd_compress) in archive third-party/ocaml/opam/lib/ocaml/compiler-libs/ocamlcommon.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)
File "caml_startup", line 1:
Error: Error during linking (exit code 1)

macos-build-examples error

Error

[2024-01-09T20:35:41.701+00:00] Action failed: root//ocaml/ppx:ppx (prelude//platforms:default#6553ba90a552c2c0) (ocaml_link_native)
[2024-01-09T20:35:41.701+00:00] Local command returned non-zero exit code 2
[2024-01-09T20:35:41.701+00:00] Local command: env -- "BUCK_SCRATCH_PATH=buck-out/v2/tmp/root/6553ba90a552c2c0/ocaml/ppx/__ppx__/ocaml_link_native" buck-out/v2/gen/root/6553ba90a552c2c0/ocaml/ppx/__ppx__/ocamloptnative.sh -g -noautolink -cc buck-out/v2/gen/root/6553ba90a552c2c0/ocaml/ppx/__ppx__/ld_native.sh -linkall -w -4-29-35-41-42-44-45-48-50-58-70 third-party/ocaml/opam/lib/ocaml-compiler-libs/shadow/ocaml_shadow.cmxa third-party/ocaml/opam/lib/ppx_derivers/ppx_derivers.cmxa third-party/ocaml/opam/lib/ocaml/compiler-libs/ocamlcommon.cmxa third-party/ocaml/opam/lib/ocaml-compiler-libs/common/ocaml_common.cmxa third-party/ocaml/opam/lib/ppxlib/astlib/astlib.cmxa third-party/ocaml/opam/lib/ppxlib/ast/ppxlib_ast.cmxa third-party/ocaml/opam/lib/ppxlib/print_diff/ppxlib_print_diff.cmxa third-party/ocaml/opam/lib/sexplib0/sexplib0.cmxa third-party/ocaml/opam/lib/stdlib-shims/stdlib_shims.cmxa third-party/ocaml/opam/lib/ppxlib/stdppx/stdppx.cmxa third-party/ocaml/opam/lib/ppxlib/traverse_builtins/ppxlib_traverse_builtins.cmxa third-party/ocaml/opam/lib/ppxlib/ppxlib.cmxa buck-out/v2/gen/root/6553ba90a552c2c0/ocaml/ppx/__ppx-record-selectors__/libppx-record-selectors.cmxa buck-out/v2/gen/root/6553ba90a552c2c0/ocaml/ppx/__ppx-record-selectors__/libppx-record-selectors.a -args buck-out/v2/gen/root/6553ba90a552c2c0/ocaml/ppx/__ppx__/cmxs_order_native.lst -cclib -lpthread -o buck-out/v2/gen/root/6553ba90a552c2c0/ocaml/ppx/__ppx__/ppx.opt
[2024-01-09T20:35:41.701+00:00] Stdout: <empty>
[2024-01-09T20:35:41.701+00:00] Stderr:
Undefined symbols for architecture x86_64:
  "_ZSTD_compressStream2", referenced from:
      _caml_zstd_compress in ocamlcommon.a(zstd.npic.o)
  "_ZSTD_createCCtx", referenced from:
      _caml_zstd_compress in ocamlcommon.a(zstd.npic.o)
  "_ZSTD_decompress", referenced from:
      _caml_zstd_decompress in ocamlcommon.a(zstd.npic.o)
  "_ZSTD_freeCCtx", referenced from:
      _caml_zstd_compress in ocamlcommon.a(zstd.npic.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
File "caml_startup", line 1:
Error: Error during linking (exit code 1)

windows-build-examples error

Error

[2024-01-09T20:28:36.838+00:00] Action failed: toolchains//:go (<unspecified>) (extract_go_toolchain)
[2024-01-09T20:28:36.838+00:00] Local command returned non-zero exit code 2
[2024-01-09T20:28:36.838+00:00] Local command: env -- "BUCK_SCRATCH_PATH=buck-out\\v2\\tmp\\toolchains\\6dd044292ff31ae1\\__go__\\extract_go_toolchain" "buck-out\\v2\\gen\\toolchains\\6dd044292ff31ae1\\__go__\\unpack.bat"
[2024-01-09T20:28:36.838+00:00] Stdout:

D:\a\buck2\buck2\examples\no_prelude>mkdir buck-out\v2\gen\toolchains\6dd044292ff31ae1\__go__\go 

D:\a\buck2\buck2\examples\no_prelude>cd buck-out\v2\gen\toolchains\6dd044292ff31ae1\__go__\go 

D:\a\buck2\buck2\examples\no_prelude\buck-out\v2\gen\toolchains\6dd044292ff31ae1\__go__\go>tar -z -x -f ..\archive.zip 
[2024-01-09T20:28:36.838+00:00] Stderr:
gzip: stdin has more than one entry--rest ignored
tar: Child returned status 2
tar: Error is not recoverable: exiting now

Due the the errors above, I was not able to test workflow steps that followed these error.

Here is the latest workflow run in my fork.

---

https://fburl.com/workplace/f6mz6tmw

[ndmitchell]

Do you suggest we merge as is and iterate, or iterate here first?

For Windows, which tar/gzip are on the path?

For Linux/Mac its all in the OCaml zlib stuff, which I know @shayne-fletcher debugged.

Context for Shayne: We are switching to GitHub actions more broadly and @robandpdx is helping us.

[shayne-fletcher]

For Linux/Mac its all in the OCaml zlib stuff, which I know @shayne-fletcher debugged.

Context for Shayne: We are switching to GitHub actions more broadly and @robandpdx is helping us.

the issue appears to be a known ocaml 5.1.0 bug that's fixed in 5.1.1. PR https://github.com/robandpdx-org/buck2/pull/25 upgrades to 5.1.1 in the opam init action.

[shayne-fletcher]

For Linux/Mac its all in the OCaml zlib stuff, which I know @shayne-fletcher debugged.
Context for Shayne: We are switching to GitHub actions more broadly and @robandpdx is helping us.

the issue appears to be a known ocaml 5.1.0 bug that's fixed in 5.1.1. PR robandpdx-org#25 upgrades to 5.1.1 in the opam init action.

got that wrong. neither sufficient or necessary to upgrade to 5.1.1. in fact #534 is seen to fix it.

[robandpdx]

@ndmitchell Since the workflow is triggered by push to main, it's probably easier to merge first, then iterate. Iterating in a fork is also a good option, and suited me well during my work on this.

On windows bash, we have...

/usr/bin/gzip
gzip 1.13
/usr/bin/tar
tar (GNU tar) 1.35

... and pwsh...

/bin/gzip
gzip 1.13
/bin/tar
bsdtar 3.5.2 - libarchive 3.5.2 zlib/1.2.5.f-ipp

[robandpdx]

@shayne-fletcher Thank you! I've rebased my fork and am running the workflow again now. I'll report back.

[robandpdx]

@shayne-fletcher Progress. New errors with linux-build-examples and macos-build-examples. Same error with windows-build-examples.

I'm happy to keep iterating with your guidance. You are also welcome to push to the branch in my fork and PR/merge to my forks main as well.

[shayne-fletcher]

@shayne-fletcher Progress. New errors with linux-build-examples and macos-build-examples. Same error with windows-build-examples.

I'm happy to keep iterating with your guidance. You are also welcome to push to the branch in my fork and PR/merge to my forks main as well.

the 'buck2-examples' branch https://github.com/shayne-fletcher/buck2/tree/buck2-examples is based off 'robanpdx-org/convert-facebook-buck2-to-actions-20240105-214735'

on this branch the "Build and test" workflow runs the 'macos-build-examples' and 'linux-build-examples' jobs. see results here.

everything up to here works. in particular, the go examples on macos and the conan examples on linux now succeed.

i suggest incorporating the relevant fixes from buck2-examples into convert-facebook-buck2-to-actions-20240105-214735.

[robandpdx]

@shayne-fletcher Do we want to discard the jobs removed in this change? Let me know please.

[shayne-fletcher]

hi @robandpdx.

i think with that last push in which we got the linux-build-examples and macos-build-examples working that a good strategy now is to land this excellent work and iterate further in place.

Do we want to discard the jobs removed in this change? Let me know please.

so, in short yes.

i see two ways forward, (1) sync your branch with the changes in https://github.com/shayne-fletcher/buck2/tree/buck2-examples including the change that removes the jobs that don't work yet and i import it and get it landed or (2) abandon your branch and i import https://github.com/shayne-fletcher/buck2/tree/buck2-examples directly.

in either case, we can aim to have the linux & mac build examples working and we can resume work to get the other jobs going from there.

how does all this sit with you and if you are agreeable which of (1) and (2) do you prefer? (should mention i had been planning to do this today)

[robandpdx]

@shayne-fletcher With the linux and macos examples working, the only job not working is the windows examples. So we can keep the build and test jobs for linux, macos, and windows seen working here. That said, I'm happy to land these 2 jobs that build the examples, then follow up with another PR for the build and test jobs for linux, macos, and windows.

This PR is ready now with option 1. Building now here.

[facebook-github-bot]

@shayne-fletcher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[shayne-fletcher]

@robandpdx merged in 965dcc2. thank-you!