[BUG] Project build is failing on Mac M1

[vzez]

Describe the bug
Build is failing with below err
Could not determine the dependencies of task ':qa:full-cluster-restart:v2.1.1#upgradedClusterTest'.

Could not resolve all task dependencies for configuration ':qa:full-cluster-restart:opensearch_distro_extracted_testclusters-qa-full-cluster-restart-v2.1.1-1-2.1.1-'.
Could not resolve project :distribution:bwc:bugfix.
Required by:
project :qa:full-cluster-restart
> Project :qa:full-cluster-restart declares a dependency from configuration 'opensearch_distro_extracted_testclusters-qa-full-cluster-restart-v2.1.1-1-2.1.1-' to configuration 'expanded-darwin-arm64-tar' which is not declared in the descriptor for project :distribution:bwc:bugfix.

To Reproduce
pull latest code and run a gradle build on mac m1

Expected behavior
gradle build and gradle check must be completed without any errors

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: Mac M1
• Version: 12.3.1

Additional context
@saratvemulapalli

[saratvemulapalli]

Thanks for opening this up @vzez.
I dont have an M1 machine, the code says we do build a distribution for darwin-arm64[1].
Anybody here can help run this and confirm ?

[1] https://github.com/opensearch-project/OpenSearch/tree/main/distribution/archives/darwin-arm64-tar

[heemin32]

I see the same issue.

Could not determine the dependencies of task ':qa:full-cluster-restart:v2.3.1#upgradedClusterTest'.
Could not resolve all task dependencies for configuration ':qa:full-cluster-restart:opensearch_distro_extracted_testclusters-qa-full-cluster-restart-v2.3.1-1-2.3.1-'.
Could not resolve project :distribution:bwc:bugfix.
Required by:
project :qa:full-cluster-restart
Project :qa:full-cluster-restart declares a dependency from configuration 'opensearch_distro_extracted_testclusters-qa-full-cluster-restart-v2.3.1-1-2.3.1-' to configuration 'expanded-darwin-arm64-tar' which is not declared in the descriptor for project :distribution:bwc:bugfix.

Following is from the previous conversation. #1668

I rebased my changes. I did more debugging on my side from on an Apple Silicon Mac, and it looks like the BWC process is checking compatibility with previous versions (starting from 7.x) with the darwin-arm4 arch. The problem is that these distributions don't exist. I would need to explicitely say that the first darwin-arm64 version is this one for the backward compatibility checks, but couldn't find out how.

[dblock]

Related #1449 is still open, and would love someone who cares about Mac to contribute, even if it's just the precommit checks.

[peterzhuamazon]

Re-open as disable macos bwc only happens on main, not 2.x.

With this issue completed we now have both x64 and arm64 MacOS Core Snapshot tarball:

• Support aarch64 OpenSearch distribution for MacOS architecture opensearch-build#4670

Would love to see this get properly supported on MacOS.
I will take a look since I now have a M macbook.

Thanks.

[peterzhuamazon]

Current 2.x error:

$ ./gradlew check
Picked up JAVA_TOOL_OPTIONS: -Dlog4j2.formatMsgNoLookups=true

> Task :buildSrc:compileJava
Note: <>/opensearch-project/OpenSearch/buildSrc/src/main/java/org/opensearch/gradle/test/DistroTestPlugin.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

> Task :buildSrc:compileGroovy
Picked up JAVA_TOOL_OPTIONS: -Dlog4j2.formatMsgNoLookups=true
Note: <>/opensearch-project/OpenSearch/buildSrc/src/main/groovy/org/opensearch/gradle/test/TestWithSslPlugin.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: <>/opensearch-project/OpenSearch/buildSrc/src/main/groovy/org/opensearch/gradle/test/TestWithSslPlugin.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':qa:full-cluster-restart:v1.3.19#upgradedClusterTest'.
> Could not resolve all dependencies for configuration ':qa:full-cluster-restart:opensearch_distro_extracted_testclusters-qa-full-cluster-restart-v1.3.19-0-1.3.19-'.
   > Could not resolve project :distribution:bwc:maintenance.
     Required by:
         project :qa:full-cluster-restart
      > A dependency was declared on configuration 'expanded-darwin-arm64-tar' which is not declared in the descriptor for project :distribution:bwc:maintenance.

* Try:
> Review the variant matching algorithm at https://docs.gradle.org/8.9/userguide/variant_attributes.html#sec:abm_algorithm.
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.

BUILD FAILED in 27s
8 actionable tasks: 5 executed, 3 up-to-date

[peternied]

[Triage - attendees 1 2 3 4 5]
Thanks for engaging on this issue @vzez & @peterzhuamazon this was just fixed a little bit ago. Please create any new issues if you have troubles, thanks.

• Fix MacOS Mx (arm64) and Linux (arm64, ppc64le, s390x) checks #15036