Skip to content

Commit

Permalink
chore(mlmd): Upgrade ML Metadata to 1.14.0. (kubeflow#9856)
Browse files Browse the repository at this point in the history
* chore(mlmd): Upgrade ML Metadata to 1.14.0.

* fix requirements, skip visualization.

* go mod tidy

* mlmd new enum in frontend

* mlmd license

* mlmd license again, using the failure test diff

* again
  • Loading branch information
zijianjoy authored and stijntratsaertit committed Feb 16, 2024
1 parent dcf8742 commit 22de188
Show file tree
Hide file tree
Showing 29 changed files with 26,167 additions and 11,434 deletions.
2 changes: 1 addition & 1 deletion .cloudbuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ steps:
# * manifests/kustomize/base/metadata/base/metadata-grpc-deployment.yaml
# * test/tag_for_hosted.sh
- name: 'gcr.io/cloud-builders/docker'
args: ['pull', 'gcr.io/tfx-oss-public/ml_metadata_store_server:1.5.0']
args: ['pull', 'gcr.io/tfx-oss-public/ml_metadata_store_server:1.14.0']
id: 'pullMetadataServer'
- name: 'gcr.io/cloud-builders/docker'
args: ['pull', 'gcr.io/ml-pipeline/minio:RELEASE.2019-08-14T20-37-41Z-license-compliance']
Expand Down
10 changes: 5 additions & 5 deletions .release.cloudbuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -259,14 +259,14 @@ steps:
# * manifests/kustomize/base/metadata/base/metadata-grpc-deployment.yaml
# * test/tag_for_hosted.sh
- name: 'gcr.io/cloud-builders/docker'
args: ['pull', 'gcr.io/tfx-oss-public/ml_metadata_store_server:1.5.0']
args: ['pull', 'gcr.io/tfx-oss-public/ml_metadata_store_server:1.14.0']
id: 'pullMetadataServer'
- name: 'gcr.io/cloud-builders/docker'
args: ['tag', 'gcr.io/tfx-oss-public/ml_metadata_store_server:1.5.0', 'gcr.io/ml-pipeline/google/pipelines/metadataserver:$TAG_NAME']
args: ['tag', 'gcr.io/tfx-oss-public/ml_metadata_store_server:1.14.0', 'gcr.io/ml-pipeline/google/pipelines/metadataserver:$TAG_NAME']
id: 'tagMetadataServerForMarketplace'
waitFor: ['pullMetadataServer']
- name: 'gcr.io/cloud-builders/docker'
args: ['tag', 'gcr.io/tfx-oss-public/ml_metadata_store_server:1.5.0', 'gcr.io/ml-pipeline/google/pipelines-test/metadataserver:$TAG_NAME']
args: ['tag', 'gcr.io/tfx-oss-public/ml_metadata_store_server:1.14.0', 'gcr.io/ml-pipeline/google/pipelines-test/metadataserver:$TAG_NAME']
id: 'tagMetadataServerForMarketplaceTest'
waitFor: ['pullMetadataServer']
- id: 'tagMetadataServerForMarketplaceMajorMinor'
Expand All @@ -276,8 +276,8 @@ steps:
args:
- -ceux
- |
docker tag gcr.io/tfx-oss-public/ml_metadata_store_server:1.5.0 gcr.io/ml-pipeline/google/pipelines/metadataserver:$(cat /workspace/mm.ver)
docker tag gcr.io/tfx-oss-public/ml_metadata_store_server:1.5.0 gcr.io/ml-pipeline/google/pipelines-test/metadataserver:$(cat /workspace/mm.ver)
docker tag gcr.io/tfx-oss-public/ml_metadata_store_server:1.14.0 gcr.io/ml-pipeline/google/pipelines/metadataserver:$(cat /workspace/mm.ver)
docker tag gcr.io/tfx-oss-public/ml_metadata_store_server:1.14.0 gcr.io/ml-pipeline/google/pipelines-test/metadataserver:$(cat /workspace/mm.ver)
docker push gcr.io/ml-pipeline/google/pipelines/metadataserver:$(cat /workspace/mm.ver)
docker push gcr.io/ml-pipeline/google/pipelines-test/metadataserver:$(cat /workspace/mm.ver)
Expand Down
2 changes: 1 addition & 1 deletion backend/metadata_writer/requirements.in
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
kubernetes>=8.0.0,<11.0.0
ml-metadata==1.5.0
ml-metadata==1.14.0
lru-dict>=1.1.7,<2.0.0
2 changes: 1 addition & 1 deletion backend/src/v2/test/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
-e ../../../../sdk/python
# TODO(chensun): remove the deprecated dependencies once migrated tests.
-r ../../../../sdk/python/requirements-deprecated.txt
ml-metadata==1.5.0
ml-metadata==1.14.0
minio==7.0.4
google-cloud-storage
fire
Expand Down
2 changes: 1 addition & 1 deletion backend/third_party_licenses/apiserver.csv
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ github.com/klauspost/cpuid,https://github.com/klauspost/cpuid/blob/v1.3.1/LICENS
github.com/klauspost/pgzip,https://github.com/klauspost/pgzip/blob/v1.2.5/LICENSE,MIT
github.com/kubeflow/pipelines/api/v2alpha1/go,https://github.com/kubeflow/pipelines/blob/758c91f76784/api/LICENSE,Apache-2.0
github.com/kubeflow/pipelines/backend,https://github.com/kubeflow/pipelines/blob/HEAD/LICENSE,Apache-2.0
github.com/kubeflow/pipelines/third_party/ml-metadata/go/ml_metadata,https://github.com/kubeflow/pipelines/blob/e78ed557ddcb/third_party/ml-metadata/LICENSE,Apache-2.0
github.com/kubeflow/pipelines/third_party/ml-metadata/go/ml_metadata,https://github.com/kubeflow/pipelines/blob/e1f0c010f800/third_party/ml-metadata/LICENSE,Apache-2.0
github.com/lann/builder,https://github.com/lann/builder/blob/47ae307949d0/LICENSE,MIT
github.com/lann/ps,https://github.com/lann/ps/blob/62de8c46ede0/LICENSE,MIT
github.com/lestrrat-go/strftime,https://github.com/lestrrat-go/strftime/blob/v1.0.4/LICENSE,MIT
Expand Down
14 changes: 14 additions & 0 deletions frontend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,8 +176,22 @@ If a file in [pipelines/third_party/ml-metadata/ml_metadata/proto](third_party/m

* Add `protoc` ([download](https://github.com/protocolbuffers/protobuf/releases)) to your system
PATH

```bash
# Example:
apt install -y protobuf-compiler=3.15.8
```

* Add `protoc-gen-grpc-web` ([download](https://github.com/grpc/grpc-web/releases)) to your system
PATH

```bash
# Example:
curl -LO https://github.com/grpc/grpc-web/releases/download/1.4.2/protoc-gen-grpc-web-1.4.2-linux-x86_64
mv protoc-gen-grpc-web-1.4.2-linux-x86_64 /usr/local/bin/protoc-gen-grpc-web
chmod +x /usr/local/bin/protoc-gen-grpc-web
```

* Replace `metadata_store.proto` and `metadata_store_service.proto` proto files with target mlmd version by running

```bash
Expand Down
1 change: 1 addition & 0 deletions frontend/src/mlmd/Utils.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -116,4 +116,5 @@ export function getMetadataValue(value?: Value): string | number | Struct | unde
case Value.ValueCase.VALUE_NOT_SET:
return '';
}
return '';
}
1 change: 1 addition & 0 deletions frontend/src/pages/ExecutionDetails.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,7 @@ function parseEventsByType(
[Event.Type.OUTPUT]: [],
[Event.Type.INTERNAL_INPUT]: [],
[Event.Type.INTERNAL_OUTPUT]: [],
[Event.Type.PENDING_OUTPUT]: [],
};

if (!response) {
Expand Down
Loading

0 comments on commit 22de188

Please sign in to comment.