Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI][MATLAB] Add job to build and test MATLAB Interface on macos-14 #41435

Closed
sgilmore10 opened this issue Apr 29, 2024 · 1 comment
Closed

Comments

@sgilmore10
Copy link
Member

Describe the enhancement requested

Once #41400 is closed, we should add a job to the .github/workflows/matlab.yml to build and test the MATLAB interface on macos-14.

Currently, the MATLAB interface is built and tested on macos-12 - not macos-14 - because the version of mathworks/libmexclass it depends on does not support macos-14. However, once #41400 is closed, the version of mathworks/libmexclass the MATLAB interface depends on will work on macos-14, so we will be able to build and test the MATLAB interface on macos-14.

Component(s)

Continuous Integration, MATLAB

@github-project-automation github-project-automation bot moved this to Backlog in Arrow MATLAB Apr 29, 2024
sgilmore10 added a commit that referenced this issue Apr 30, 2024
…1436)

### Rationale for this change

@ kevingurney and I recently resolved multiple issues related to `mathworks/libmexclass` not supporting ARM-based macOS builds (i.e. builds on `macos-14`):

- mathworks/libmexclass#76
- mathworks/libmexclass#77

We should bump the version of mathworks/libmexclass used by the MATLAB interface to the latest available commit ([ca3cea6](mathworks/libmexclass@ca3cea6) as of now) in order to enable building the MATLAB interface to Arrow on `macos-14` (which is ARM-based).

### What changes are included in this PR?

- Bumped version of `mathworks/libmexclass` used by the MATLAB interface to [ca3cea6](mathworks/libmexclass@ca3cea6)

### Are these changes tested?

- Yes. The existing test points verify verify upgrading `mathworks/libmexclass` does not break the MATLAB interface.

### Are there any user-facing changes?

- No. 

### Future Directions

- #41435
- #41385

* GitHub Issue: #41400

Authored-by: Sarah Gilmore <[email protected]>
Signed-off-by: Sarah Gilmore <[email protected]>
tolleybot pushed a commit to tmct/arrow that referenced this issue May 2, 2024
…6` (apache#41436)

### Rationale for this change

@ kevingurney and I recently resolved multiple issues related to `mathworks/libmexclass` not supporting ARM-based macOS builds (i.e. builds on `macos-14`):

- mathworks/libmexclass#76
- mathworks/libmexclass#77

We should bump the version of mathworks/libmexclass used by the MATLAB interface to the latest available commit ([ca3cea6](mathworks/libmexclass@ca3cea6) as of now) in order to enable building the MATLAB interface to Arrow on `macos-14` (which is ARM-based).

### What changes are included in this PR?

- Bumped version of `mathworks/libmexclass` used by the MATLAB interface to [ca3cea6](mathworks/libmexclass@ca3cea6)

### Are these changes tested?

- Yes. The existing test points verify verify upgrading `mathworks/libmexclass` does not break the MATLAB interface.

### Are there any user-facing changes?

- No. 

### Future Directions

- apache#41435
- apache#41385

* GitHub Issue: apache#41400

Authored-by: Sarah Gilmore <[email protected]>
Signed-off-by: Sarah Gilmore <[email protected]>
@sgilmore10 sgilmore10 assigned sgilmore10 and unassigned sgilmore10 May 2, 2024
sgilmore10 added a commit that referenced this issue May 3, 2024
…CI and crossbow packaging workflows (#41504)

### Rationale for this change

MATLAB `R2024a` is the latest available version of MATLAB as of April 2024.

We are currently building against MATLAB `R2023a` in CI and for the crossbow packaging workflow.

We should update the version of MATLAB we support to the latest available version.

We previously created an issue to use `R2023b` (#37809). However, `R2024a` has become publicly available since then.

### What changes are included in this PR?

1. Changed the `release` argument supplied to `matlab-actions/setup-matlab@ v2` to `R2024a` from `R2023a` in `.github/workflows/matlab.yml` and `dev/tasks/matlab/github.yml`.

2. Updated the script used to package the MLTBX file (`arrow/matlab/tools/packageMatlabInterface.m`) to specify the version of MATLAB currently running as the only compatible version of MATLAB for the interface. 

3.  Updated display tests to react to a change in how class names are formatted when an objected is displayed in the Command Window.

### Are these changes tested?

Yes. Existing tests used.

### Are there any user-facing changes?

There are no changes to the MATLAB Arrow Interface. However, the MATLAB release used to build, test, and package the interface has been upgraded.

### Future Work

1. We may want to consider building and packaging against multiple versions of MATLAB in parallel, rather than just the latest. This would require some more thought on how many releases back to support over time, however.

2. #41435

* GitHub Issue: #41385

Lead-authored-by: Sarah Gilmore <[email protected]>
Co-authored-by: Sarah Gilmore <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sarah Gilmore <[email protected]>
@sgilmore10 sgilmore10 self-assigned this May 8, 2024
sgilmore10 added a commit that referenced this issue May 8, 2024
…`macos-14` (#41592)

### Rationale for this change

Currently, the MATLAB interface is built and tested on `macos-12` - not `macos-14` - because the version of `mathworks/libmexclass` depends on used to not support `macos-14`. However, now that #41400 is closed, the version of `mathworks/libmexclass` the MATLAB interface depends on works on `macos-14`, so we will be able to build and test the MATLAB interface on `macos-14`.

**Note**: When adding support for ARM-based macOS builds, we discovered an issue with the way in which we package the MLTBX files for the MATLAB Interface to Arrow.
 
Currently, we bundle all shared libraries for all platforms (.dll, .dylib, and .so) into one large "monolithic" MLTBX file.
 
Unfortunately, putting all platform-specific files into one MLTBX file poses an issue when we support multiple ISAs (e.g. x86 and ARM) because builds for the same operating system with different ISAs will have the same shared library file names. In other words, we will have a library named libarrowproxy.dylib for both ARM and x86 macOS builds.
 
Therefore, we are going to hold off on adding ARM-based macOS builds to the crossbow packaging workflow for now until we have a chance to properly explore alternative packaging approaches. For example, we may want to consider having platform-specific MLTBX files. However, we still think it is worthwhile to add CI support for `macos-14` in the meantime.

### What changes are included in this PR?

1. Added workflow to build and test the MATLAB interface on `macos-14` as well as `macos-12`.

### Are these changes tested?

N/A. 

### Are there any user-facing changes?

No.

### Future Directions

1. Add crossbow packaging workflow on `macos-14` once we determine how to package the interface for both ARM-based and Intel-based mac ISAs.

* GitHub Issue: #41435

Authored-by: Sarah Gilmore <[email protected]>
Signed-off-by: Sarah Gilmore <[email protected]>
@sgilmore10 sgilmore10 added this to the 17.0.0 milestone May 8, 2024
@sgilmore10
Copy link
Member Author

Issue resolved by pull request 41592
#41592

@github-project-automation github-project-automation bot moved this from Backlog to Done in Arrow MATLAB May 8, 2024
vibhatha pushed a commit to vibhatha/arrow that referenced this issue May 25, 2024
…6` (apache#41436)

### Rationale for this change

@ kevingurney and I recently resolved multiple issues related to `mathworks/libmexclass` not supporting ARM-based macOS builds (i.e. builds on `macos-14`):

- mathworks/libmexclass#76
- mathworks/libmexclass#77

We should bump the version of mathworks/libmexclass used by the MATLAB interface to the latest available commit ([ca3cea6](mathworks/libmexclass@ca3cea6) as of now) in order to enable building the MATLAB interface to Arrow on `macos-14` (which is ARM-based).

### What changes are included in this PR?

- Bumped version of `mathworks/libmexclass` used by the MATLAB interface to [ca3cea6](mathworks/libmexclass@ca3cea6)

### Are these changes tested?

- Yes. The existing test points verify verify upgrading `mathworks/libmexclass` does not break the MATLAB interface.

### Are there any user-facing changes?

- No. 

### Future Directions

- apache#41435
- apache#41385

* GitHub Issue: apache#41400

Authored-by: Sarah Gilmore <[email protected]>
Signed-off-by: Sarah Gilmore <[email protected]>
vibhatha pushed a commit to vibhatha/arrow that referenced this issue May 25, 2024
…a` in CI and crossbow packaging workflows (apache#41504)

### Rationale for this change

MATLAB `R2024a` is the latest available version of MATLAB as of April 2024.

We are currently building against MATLAB `R2023a` in CI and for the crossbow packaging workflow.

We should update the version of MATLAB we support to the latest available version.

We previously created an issue to use `R2023b` (apache#37809). However, `R2024a` has become publicly available since then.

### What changes are included in this PR?

1. Changed the `release` argument supplied to `matlab-actions/setup-matlab@ v2` to `R2024a` from `R2023a` in `.github/workflows/matlab.yml` and `dev/tasks/matlab/github.yml`.

2. Updated the script used to package the MLTBX file (`arrow/matlab/tools/packageMatlabInterface.m`) to specify the version of MATLAB currently running as the only compatible version of MATLAB for the interface. 

3.  Updated display tests to react to a change in how class names are formatted when an objected is displayed in the Command Window.

### Are these changes tested?

Yes. Existing tests used.

### Are there any user-facing changes?

There are no changes to the MATLAB Arrow Interface. However, the MATLAB release used to build, test, and package the interface has been upgraded.

### Future Work

1. We may want to consider building and packaging against multiple versions of MATLAB in parallel, rather than just the latest. This would require some more thought on how many releases back to support over time, however.

2. apache#41435

* GitHub Issue: apache#41385

Lead-authored-by: Sarah Gilmore <[email protected]>
Co-authored-by: Sarah Gilmore <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sarah Gilmore <[email protected]>
vibhatha pushed a commit to vibhatha/arrow that referenced this issue May 25, 2024
…ce on `macos-14` (apache#41592)

### Rationale for this change

Currently, the MATLAB interface is built and tested on `macos-12` - not `macos-14` - because the version of `mathworks/libmexclass` depends on used to not support `macos-14`. However, now that apache#41400 is closed, the version of `mathworks/libmexclass` the MATLAB interface depends on works on `macos-14`, so we will be able to build and test the MATLAB interface on `macos-14`.

**Note**: When adding support for ARM-based macOS builds, we discovered an issue with the way in which we package the MLTBX files for the MATLAB Interface to Arrow.
 
Currently, we bundle all shared libraries for all platforms (.dll, .dylib, and .so) into one large "monolithic" MLTBX file.
 
Unfortunately, putting all platform-specific files into one MLTBX file poses an issue when we support multiple ISAs (e.g. x86 and ARM) because builds for the same operating system with different ISAs will have the same shared library file names. In other words, we will have a library named libarrowproxy.dylib for both ARM and x86 macOS builds.
 
Therefore, we are going to hold off on adding ARM-based macOS builds to the crossbow packaging workflow for now until we have a chance to properly explore alternative packaging approaches. For example, we may want to consider having platform-specific MLTBX files. However, we still think it is worthwhile to add CI support for `macos-14` in the meantime.

### What changes are included in this PR?

1. Added workflow to build and test the MATLAB interface on `macos-14` as well as `macos-12`.

### Are these changes tested?

N/A. 

### Are there any user-facing changes?

No.

### Future Directions

1. Add crossbow packaging workflow on `macos-14` once we determine how to package the interface for both ARM-based and Intel-based mac ISAs.

* GitHub Issue: apache#41435

Authored-by: Sarah Gilmore <[email protected]>
Signed-off-by: Sarah Gilmore <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

No branches or pull requests

1 participant