Skip to content

Commit

Permalink
Merge pull request #290 from AbsaOSS/prep_release_2_0_8
Browse files Browse the repository at this point in the history
Release Cobrix 2.0.8
  • Loading branch information
yruslan authored May 14, 2020
2 parents 39e493a + fbcacb7 commit a4b24ce
Show file tree
Hide file tree
Showing 9 changed files with 23 additions and 26 deletions.
33 changes: 15 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ You can link against this library in your program at the following coordinates:
```
groupId: za.co.absa.cobrix
artifactId: spark-cobol_2.11
version: 2.0.7
version: 2.0.8
```

### Scala 2.12
Expand All @@ -72,7 +72,7 @@ version: 2.0.7
```
groupId: za.co.absa.cobrix
artifactId: spark-cobol_2.12
version: 2.0.7
version: 2.0.8
```

## Using with Spark shell
Expand All @@ -81,22 +81,12 @@ This package can be added to Spark using the `--packages` command line option. F

### Spark compiled with Scala 2.11
```
$SPARK_HOME/bin/spark-shell --packages za.co.absa.cobrix:spark-cobol_2.11:2.0.7
$SPARK_HOME/bin/spark-shell --packages za.co.absa.cobrix:spark-cobol_2.11:2.0.8
```

### Spark compiled with Scala 2.12
```
$SPARK_HOME/bin/spark-shell --packages za.co.absa.cobrix:spark-cobol_2.12:2.0.7
```

### Linking legacy `spark-cobol`

[![Maven Central](https://maven-badges.herokuapp.com/maven-central/za.co.absa.cobrix/spark-cobol/badge.svg)](https://maven-badges.herokuapp.com/maven-central/za.co.absa.cobrix/spark-cobol)

```
groupId: za.co.absa.cobrix
artifactId: spark-cobol
version: 1.1.2
$SPARK_HOME/bin/spark-shell --packages za.co.absa.cobrix:spark-cobol_2.12:2.0.8
```

## Usage
Expand Down Expand Up @@ -213,16 +203,16 @@ to decode various binary formats.

The jars that you need to get are:

* spark-cobol_2.11-2.0.7.jar
* cobol-parser_2.11-2.0.7.jar
* spark-cobol_2.11-2.0.8.jar
* cobol-parser_2.11-2.0.8.jar
* scodec-core_2.11-1.10.3.jar
* scodec-bits_2.11-1.1.4.jar

After that you can specify these jars in `spark-shell` command line. Here is an example:
```
$ spark-shell --packages za.co.absa.cobrix:spark-cobol_2.11:2.0.7
$ spark-shell --packages za.co.absa.cobrix:spark-cobol_2.11:2.0.8
or
$ spark-shell --master yarn --deploy-mode client --driver-cores 4 --driver-memory 4G --jars spark-cobol_2.11-2.0.7.jar,cobol-parser_2.11-2.0.7.jar,scodec-core_2.11-1.10.3.jar,scodec-bits_2.11-1.1.4.jar
$ spark-shell --master yarn --deploy-mode client --driver-cores 4 --driver-memory 4G --jars spark-cobol_2.11-2.0.8.jar,cobol-parser_2.11-2.0.8.jar,scodec-core_2.11-1.10.3.jar,scodec-bits_2.11-1.1.4.jar
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Expand Down Expand Up @@ -1180,6 +1170,13 @@ For multisegment variable lengths tests:
![](performance/images/exp3_multiseg_wide_records_throughput.svg) ![](performance/images/exp3_multiseg_wide_mb_throughput.svg)

## Changelog
- #### 2.0.8 released 14 May 2020.
- [#184](https://github.com/AbsaOSS/cobrix/issues/184) Record extractors are made generic to be reusable for other targets in addition to Sspark Row. (Thanks [@tr11](https://github.com/tr11)).
- [#283](https://github.com/AbsaOSS/cobrix/issues/283) Added a custom JSON parser to mitigate jackson compatibility when `spark-cobol` is used in Spark 3.0. (Thanks [@tr11](https://github.com/tr11)).

- #### 2.0.7 released 14 April 2020.
- [#273](https://github.com/AbsaOSS/cobrix/issues/273) Fixed the class loader for custom code pages (Thanks [@schaloner-kbc](https://github.com/schaloner-kbc)).

- #### 2.0.7 released 14 April 2020.
- [#273](https://github.com/AbsaOSS/cobrix/issues/273) Fixed the class loader for custom code pages (Thanks [@schaloner-kbc](https://github.com/schaloner-kbc)).

Expand Down
2 changes: 1 addition & 1 deletion cobol-converters/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>za.co.absa.cobrix</groupId>
<artifactId>cobrix_2.11</artifactId>
<version>2.0.8-SNAPSHOT</version>
<version>2.0.9-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion cobol-parser/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>za.co.absa.cobrix</groupId>
<artifactId>cobrix_2.11</artifactId>
<version>2.0.8-SNAPSHOT</version>
<version>2.0.9-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion examples/examples-collection/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
<scala.compat.version>2.11</scala.compat.version>
<spark.version>2.4.4</spark.version>
<specs.version>2.4.16</specs.version>
<spark.cobol.version>2.0.7</spark.cobol.version>
<spark.cobol.version>2.0.8</spark.cobol.version>
</properties>

<dependencies>
Expand Down
2 changes: 1 addition & 1 deletion examples/spark-cobol-app/build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ ThisBuild / version := "0.1.0-SNAPSHOT"
ThisBuild / scalaVersion := "2.11.12"

val sparkVersion = "2.4.4"
val sparkCobolVersion = "2.0.7"
val sparkCobolVersion = "2.0.8"
val scalatestVersion = "3.0.1"

ThisBuild / libraryDependencies ++= Seq(
Expand Down
2 changes: 1 addition & 1 deletion examples/spark-cobol-app/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
<scalatest.version>3.0.0</scalatest.version>
<spark.version>2.4.4</spark.version>
<specs.version>2.4.16</specs.version>
<spark.cobol.version>2.0.7</spark.cobol.version>
<spark.cobol.version>2.0.8</spark.cobol.version>
</properties>

<dependencies>
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<groupId>za.co.absa.cobrix</groupId>
<artifactId>cobrix_2.11</artifactId>

<version>2.0.8-SNAPSHOT</version>
<version>2.0.9-SNAPSHOT</version>

<packaging>pom</packaging>

Expand Down
2 changes: 1 addition & 1 deletion spark-cobol/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>za.co.absa.cobrix</groupId>
<artifactId>cobrix_2.11</artifactId>
<version>2.0.8-SNAPSHOT</version>
<version>2.0.9-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion version.sbt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
version in ThisBuild := "2.0.8-SNAPSHOT"
version in ThisBuild := "2.0.9-SNAPSHOT"

0 comments on commit a4b24ce

Please sign in to comment.