Skip to content

Commit

Permalink
adapt to Spark 3.1.1
Browse files Browse the repository at this point in the history
adapt to Spark 3.1.1
  • Loading branch information
Jutao-liu authored Jun 10, 2022
2 parents ca6c1c4 + a3f29a3 commit 875866b
Show file tree
Hide file tree
Showing 1,338 changed files with 358 additions and 276,503 deletions.
16 changes: 4 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
Introduction
============

The machine learning algorithm library running on Kunpeng processors is an acceleration library that provides a rich set of high-level tools for machine learning algorithms. It is based on the original APIs of Apache [Spark 2.3.2](https://github.com/apache/spark/tree/v2.3.2), [breeze 0.13.1](https://github.com/scalanlp/breeze/tree/releases/v0.13.1) and [xgboost 1.1.0](https://github.com/dmlc/xgboost/tree/release_1.0.0). The acceleration library for greatly improves the computing power in big data scenarios.
The machine learning algorithm library running on Kunpeng processors is an acceleration library that provides a rich set of high-level tools for machine learning algorithms. It is based on the original APIs of Apache [Spark 3.1.1](https://github.com/apache/spark/tree/v3.1.1). The acceleration library for greatly improves the computing power in big data scenarios.

The library provides 21 machine learning algorithms: support vector machine (SVM), random forest classifier (RFC), gradient boosting decision tree (GBDT), decision tree (DT), K-means clustering, linear regression, logistic regression algorithm, principal component analysis (PCA), principal component analysis for Sparse Matrix(SPCA), singular value decomposition (SVD), latent dirichlet allocation (LDA), prefix-projected pattern prowth (Prefix-Span), alternating least squares (ALS), K-nearest neighbors (KNN), Covariance, Density-based spatial clustering of applicaitons with noise (DBSCAN), Pearson, Spearman, XGboost, Inverse Document Frequency(IDF), and SimRank. You can find the latest documentation on the project web page. This README file contains only basic setup instructions.
The library provides 5 machine learning algorithms: latent dirichlet allocation (LDA), prefix-projected pattern prowth (Prefix-Span), alternating least squares (ALS), K-nearest neighbors (KNN), Density-based spatial clustering of applicaitons with noise (DBSCAN). You can find the latest documentation on the project web page. This README file contains only basic setup instructions.
You can find the latest documentation, including a programming guide, on the project web page. This README file only contains basic setup instructions.


Expand All @@ -21,17 +21,9 @@ Building And Packageing

mvn clean package

(2) Build XGBoost project under the "Spark-ml-algo-lib/ml-xgboost/jvm-packages" directory:
(2) Obtain "boostkit-ml-core_2.12-2.1.0-spark3.1.1.jar" under the "Spark-ml-algo-lib/ml-core/target" directory.

mvn clean package

(3) Obtain "boostkit-ml-core_2.11-2.1.0-spark2.3.2.jar" under the "Spark-ml-algo-lib/ml-core/target" directory.

Obtain "boostkit-ml-acc_2.11-2.1.0-spark2.3.2.jar" under the "Spark-ml-algo-lib/ml-accelerator/target" directory.

Obtain "boostkit-xgboost4j_2.11-2.1.0.jar" under the "Spark-ml-algo-lib/ml-xgboost/jvm-packages/boostkit-xgboost4j/target" directory.

Obtain "boostkit-xgboost4j-spark2.3.2_2.11-2.1.0.jar" under the "Spark-ml-algo-lib/ml-xgboost/jvm-packages/boostkit-xgboost4j-spark/target" directory.
Obtain "boostkit-ml-acc_2.12-2.1.0-spark3.1.1.jar" under the "Spark-ml-algo-lib/ml-accelerator/target" directory.


Contribution Guidelines
Expand Down
6 changes: 3 additions & 3 deletions ml-accelerator/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,22 +6,22 @@
</parent>

<modelVersion>4.0.0</modelVersion>
<artifactId>boostkit-ml-acc_2.11</artifactId>
<artifactId>boostkit-ml-acc_2.12</artifactId>
<version>2.1.0</version>
<name>${project.artifactId}</name>
<description>Spark ml algo accelerator</description>

<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>boostkit-ml-core_2.11</artifactId>
<artifactId>boostkit-ml-core_2.12</artifactId>
<version>${project.version}</version>
<classifier>${spark.version}</classifier>

</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>boostkit-ml-kernel-client_2.11</artifactId>
<artifactId>boostkit-ml-kernel-client_2.12</artifactId>
<version>${project.version}</version>
<classifier>${spark.version}</classifier>
<scope>compile</scope>
Expand Down

This file was deleted.

Loading

0 comments on commit 875866b

Please sign in to comment.