GH-38417: [MATLAB] Implement a TableTypeValidator class that validates a MATLAB cell array contains only tables that share the same schema
#38551
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2. Implement getElementLength in TableValidator to return the height of the table
kevingurney
requested changes
Nov 2, 2023
matlab/src/matlab/+arrow/+array/+internal/+list/TableValidator.m
Outdated
Show resolved
Hide resolved
matlab/src/matlab/+arrow/+array/+internal/+list/TableValidator.m
Outdated
Show resolved
Hide resolved
matlab/src/matlab/+arrow/+array/+internal/+list/TableValidator.m
Outdated
Show resolved
Hide resolved
matlab/src/matlab/+arrow/+array/+internal/+list/TableValidator.m
Outdated
Show resolved
Hide resolved
matlab/src/matlab/+arrow/+array/+internal/+list/TableValidator.m
Outdated
Show resolved
Hide resolved
github-actions
bot
added
awaiting changes
kevingurney
approved these changes
Nov 2, 2023
|
+1 |
|
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 56edb2d. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
loicalleyne
pushed a commit
to loicalleyne/arrow
that referenced
this pull request
Nov 13, 2023
…validates a MATLAB `cell` array contains only `table`s that share the same schema (apache#38551) ### Rationale for this change This is a followup to apache#38533. Adding this `TableValidator` class is a step towards implementing the `arrow.array.ListArray.fromMATLAB` method for creating `ListArray`s whose `ValueType` is a `StructArray`. This validator will ensure all `table`s in a `cell` array have the same schema when attempting to make a `ListArray` of `Struct`s. This is a requirement to ensure the `table`s in the `cell` array are vertcat'ble. For example, two `table`s with different `VariableNames` cannot be concatenated together: ```matlab >> t1 = table(1, 2, VariableNames=["A", "B"]); >> t2 = table(3, 4, VariableNames=["C", "D"]); >> vertcat(t1, t2) Error using tabular/vertcat All tables being vertically concatenated must have the same variable names. ``` ### What changes are included in this PR? Modified `arrow.array.internal.list.Validator` to inherit from `matlab.mixin.Heterogeneous`. Doing so enables creating an array whose elements are different subclasses of `arrow.array.internal.list.Validator`. Added a new MATLAB class `arrow.array.internal.list.TableValidator`, which inherits from `arrow.array.internal.list.Validator`. This class has two properties: `VariableNames` and `VariableValidators`. `VariableNames` is a `string` array containing the expected variable names of all `table`s. `VariableValidators` is an array of `arrow.array.internal.list.Validator`, in which each element represents one variable in a `table`. This array is used to validate `table` variables have the expected type and configuration. `TableValidator`'s `validateElement` method uses both its `VariableNames` and `VariableValidator` properties to validate the input argument provided is a `table` with the expected schema. If not, it throws an error. Lastly, I added a gateway function called `arrow.array.internal.list.createValidator`, which creates the appropriate `Validator` subclass based on the input. If no such `Validator` exists, an error is thrown. ### Are these changes tested? Yes. Added two new test classes: `tTableValidator.m` and `tCreateValidator.m`. ### Are there any user-facing changes? No. ### Future Directions: 1. apache#38354 * Closes: apache#38417 Authored-by: Sarah Gilmore <[email protected]> Signed-off-by: Kevin Gurney <[email protected]>
dgreiss
pushed a commit
to dgreiss/arrow
that referenced
this pull request
Feb 19, 2024
…validates a MATLAB `cell` array contains only `table`s that share the same schema (apache#38551) ### Rationale for this change This is a followup to apache#38533. Adding this `TableValidator` class is a step towards implementing the `arrow.array.ListArray.fromMATLAB` method for creating `ListArray`s whose `ValueType` is a `StructArray`. This validator will ensure all `table`s in a `cell` array have the same schema when attempting to make a `ListArray` of `Struct`s. This is a requirement to ensure the `table`s in the `cell` array are vertcat'ble. For example, two `table`s with different `VariableNames` cannot be concatenated together: ```matlab >> t1 = table(1, 2, VariableNames=["A", "B"]); >> t2 = table(3, 4, VariableNames=["C", "D"]); >> vertcat(t1, t2) Error using tabular/vertcat All tables being vertically concatenated must have the same variable names. ``` ### What changes are included in this PR? Modified `arrow.array.internal.list.Validator` to inherit from `matlab.mixin.Heterogeneous`. Doing so enables creating an array whose elements are different subclasses of `arrow.array.internal.list.Validator`. Added a new MATLAB class `arrow.array.internal.list.TableValidator`, which inherits from `arrow.array.internal.list.Validator`. This class has two properties: `VariableNames` and `VariableValidators`. `VariableNames` is a `string` array containing the expected variable names of all `table`s. `VariableValidators` is an array of `arrow.array.internal.list.Validator`, in which each element represents one variable in a `table`. This array is used to validate `table` variables have the expected type and configuration. `TableValidator`'s `validateElement` method uses both its `VariableNames` and `VariableValidator` properties to validate the input argument provided is a `table` with the expected schema. If not, it throws an error. Lastly, I added a gateway function called `arrow.array.internal.list.createValidator`, which creates the appropriate `Validator` subclass based on the input. If no such `Validator` exists, an error is thrown. ### Are these changes tested? Yes. Added two new test classes: `tTableValidator.m` and `tCreateValidator.m`. ### Are there any user-facing changes? No. ### Future Directions: 1. apache#38354 * Closes: apache#38417 Authored-by: Sarah Gilmore <[email protected]> Signed-off-by: Kevin Gurney <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
This is a followup to #38533.
Adding this
TableValidatorclass is a step towards implementing thearrow.array.ListArray.fromMATLABmethod for creatingListArrays whoseValueTypeis aStructArray.This validator will ensure all
tables in acellarray have the same schema when attempting to make aListArrayofStructs. This is a requirement to ensure thetables in thecellarray are vertcat'ble. For example, twotables with differentVariableNamescannot be concatenated together:What changes are included in this PR?
Modified
arrow.array.internal.list.Validatorto inherit frommatlab.mixin.Heterogeneous. Doing so enables creating an array whose elements are different subclasses ofarrow.array.internal.list.Validator.Added a new MATLAB class
arrow.array.internal.list.TableValidator, which inherits fromarrow.array.internal.list.Validator. This class has two properties:VariableNamesandVariableValidators.VariableNamesis astringarray containing the expected variable names of alltables.VariableValidatorsis an array ofarrow.array.internal.list.Validator, in which each element represents one variable in atable. This array is used to validatetablevariables have the expected type and configuration.TableValidator'svalidateElementmethod uses both itsVariableNamesandVariableValidatorproperties to validate the input argument provided is atablewith the expected schema. If not, it throws an error.Lastly, I added a gateway function called
arrow.array.internal.list.createValidator, which creates the appropriateValidatorsubclass based on the input. If no suchValidatorexists, an error is thrown.Are these changes tested?
Yes. Added two new test classes:
tTableValidator.mandtCreateValidator.m.Are there any user-facing changes?
No.
Future Directions:
fromMATLABmethod forarrow.array.ListArray. #38354TableTypeValidatorclass that validates a MATLABcellarray contains onlytables that share the same schema #38417