Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Atari test #5796

Merged
merged 265 commits into from
Sep 20, 2022
Merged

Atari test #5796

merged 265 commits into from
Sep 20, 2022

Conversation

ForJadeForest
Copy link
Contributor

@ForJadeForest ForJadeForest commented Sep 16, 2022

Description

Because the license of atari package is GPL and an orca example rl_pong need this package.
After deleting the package the jenkins test failed. So we delete the rl_pong example test in run-example-test-ray.sh

Hanyu-Jin and others added 30 commits September 20, 2022 09:09
* add assembly

* Enable FL Sever in SGX

* Change the url in tutorial.md

* Add start-fl-server.py to Docker Image

* rm ppml-assebly.xml

* Add start-python-fl-server-sgx.sh to Docker Image

* Put start-fl-server.py into assembly

* Update 3.1.2 Start FL Server in SGX in pytorch-nn-lr-tutorial.md

* Change the location of start-fl-server.py

* Correct the path in start-python-fl-server-sgx.sh

* Delete -r in cp command

Co-authored-by: Jiaming <[email protected]>
* fix tf inc metric format

* re-run actions

* Apply suggestions from code review

Co-authored-by: Zhentao Chen <[email protected]>

* Update

Co-authored-by: Zhentao Chen <[email protected]>
* fix: update known-issues.

* fix: fix wording.
Migrate current jenkins CI/CD jobs to github action, this is chronos PR validation-Spark2.4-ray job
…ntel-analytics#5483)

When the input data to AutoformerForecaster.predict() is incorrect, the error message is confusing. Thus, a new error message is added.
* notebook with nano

* load with nano

* modify readme

* clear all outputs and enable benchmark_app

* fix typo
…cs#5509)

* add assembly

* Enable FL Sever in SGX

* Add start-fl-server.py to Docker Image

* rm ppml-assebly.xml

* Add start-python-fl-server-sgx.sh to Docker Image

* Put start-fl-server.py into assembly

Co-authored-by: Jiaming <[email protected]>
* rollback requirement-doc

* Update

* Update

* Update

* Update

* Fix ipex with amp error

* Fix unit test

* Update

* Remove redundant method

* Fix error

* remove redundant import

* Update ut to conver cases
* add assembly

* Enable FL Sever in SGX

* Add start-fl-server.py to Docker Image

* rm ppml-assebly.xml

* Add start-python-fl-server-sgx.sh to Docker Image

* Put start-fl-server.py into assembly

* Fix the target directory

Co-authored-by: Jiaming <[email protected]>
…cs#5465)

* remove pyarrow dependency in tf2 ray backend

* update ut to compare prediction results
…d and nightly_test (intel-analytics#5511)

* reorg workflows
* remove duplicate yamls
* add dependencies
…ntel-analytics#5504)

* feat: add save and load model doc to pytorch estimator quickstart.

* fix: fix typo.

* fix: fix typo

* fix: fix typo.

* feat: add doc to ray backend quickstart

* fix: fix typo.

* feat: add doc to ray backend quickstart
* feat: add shutdown API to tf2 pyspark estimator.

* fix: replace raise with log4Error.

* refactor: refactor the stop log server function.

* fix: fix code style issue.
* fix: use default value if no hdfs port specified.

* fix: add default hdfs port when saving pkl

* refactor: replace pyarrow deps in estimator.

* fix: add process wait()

* fix: resolve unnecessary change
* Update

* Fix code style

* re-run action

* Fix code style

* re-run action

* Update

* Fix code style

* support bf16 multi training

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* reduce ut time and re-run action

* track avx512

* Update lite bf16 training

* Update

* Update

* Update bf16 api

* Update

* Fix typo
…el-analytics#5508)

* feat: add get remote file to local decorator in orca file utils.

* feat: add unit-test for local and s3 FS.

* fix: update func name and comments.

* fix: fix code style.

* fix: fix code style.

* fix: fix typo.

* fix: fix import.
* fix save model in pyspark est

* use a function to replace

* use orca.data to replace dllib
* Add Xshards test to github actions

* add file path

* fix yml

* fix csv path

* fix csv path

* fix yml

* add wget

* fic typo

* fix typo

* change csv file path

* update pwd

* add new line

* delete a line

* add time

Co-authored-by: yansu <[email protected]>
* add assembly

* Enable FL Sever in SGX

* Add start-fl-server.py to Docker Image

* rm ppml-assebly.xml

* Add start-python-fl-server-sgx.sh to Docker Image

* Put start-fl-server.py into assembly

* Change some packages' edition

* Fix the problem of fl/start-fl-server.py

Co-authored-by: Jiaming <[email protected]>
Reason to change: tracker.py in ${BIGDL_HOME}/jars/bigdl-dllib-spark_${SPARK_VERSION}-${BIGDL_VERSION}.jar is not functional.

Instead of using tracker.py to build ${BIGDL_HOME}/jars/bigdl-dllib-spark_${SPARK_VERSION}-${BIGDL_VERSION}.jar,
we now use it to build xgboost4j_2.12-1.1.2.jar, which can be reached when executing xgboost examples.
* Create nightly-build-example-tests-ppml.yaml

* Update nightly-build-example-tests-ppml.yaml

Triggered by schedule

* Update nightly-build-example-tests-ppml.yaml
* feat: add k8s related files

* fix: change secure-argv to secure_argv in Gramine

* fix: check if secured_argv exists with -f

* fix: exit when not ready in init.sh

* fix: change path of encalve-key in k8s template

* feat: copy k8s directory to Gramine directory

* mod: version in development

* mod: enclave-key path

* del: remove tmp build action

* feat: Simplify deployment operations for users

* doc: update README.md

* doc: correct a typo

* doc: correct typo

* add spark pi on k8s scripts
* add composite actions

* add manually build
Le-Zheng and others added 28 commits September 20, 2022 09:09
* rollback requirement-doc

* Add Nano PyTorch BF16 Example

* Add comments and remove testing loop

* Update
* Nano: Add tensorflow quantization example.

* simplify quantize arguments

* add some extra comments

* delete concrete numbers

* change dataset to cifar10

* change file name to cifar10

* improve code organization

* remove unnecessary comments
* change to xgb to gbt

* use overwrite to save
* update multi-task

* fix

* minor

* minor
…mer (intel-analytics#5673)

* fix input feature num != output feature num case

* add updates
* update databricks doc

* update databricks doc

* update databricks doc

* update databricks doc

* update databricks doc

* update databricks doc

Co-authored-by: Zhou <[email protected]>
* update end to end gbt readme

* update header location
…y deserialize untrusted data (intel-analytics#5666)

* fix readObject deserialize untrusted data

* use classTag[T] and use classof[modue]

* remove double init inputStream

* add accept pattern

* accept Array

* add EncodeUtils ut

* add success case and refine fail case

* add headers
* update graphene ppml examples

* update graphene ppml examples

Co-authored-by: Jin Hanyu <[email protected]>
* update mtnet model to decouple with orca

* rename one of the method
…#5682)

* first commit, add predict_interval

* add return value of calculate

* modify parameter name, only calculate data noise once

* add ut for lstm, nbeats, s2s, tcn

* add paper link, change evaluate to predict

* modify data noise

* add ut for loader c and change to mean_yhat

* support confidence interval for autoformer
…alytics#5809)

* update nano and chronos action trigger domain

* update

* delete orca.learn as a trigger
* midterm push for working in cafe:)

* add updates

* remove confusing comments

* consider the conflict between tsdataset.label_len and label_len

* update according to comments, fix pep8

* add test cmt
@ForJadeForest
Copy link
Contributor Author

@hkvision hkvision merged commit 492e05c into intel-analytics:main Sep 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.