Skip to content

Commit

Permalink
[improve][doc] Add doc for builtin functions (#18569)
Browse files Browse the repository at this point in the history
Co-authored-by: tison <[email protected]>
  • Loading branch information
cbornet and tisonkun authored Nov 24, 2022
1 parent fe5a8ce commit 7dc6700
Show file tree
Hide file tree
Showing 4 changed files with 119 additions and 2 deletions.
16 changes: 16 additions & 0 deletions site2/docs/functions-deploy-cluster-builtin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
id: functions-deploy-cluster-builtin
title: Use built-in functions
sidebar_label: "Use built-in functions"
---

Similar to built-in connectors, the code of Java functions [packaged as NAR](functions-package-java.md) that are placed in the `functions` directory of the function worker are loaded at startup and can be referenced when creating a function.

For instance if you have a built-in function with name `exclamation` in its `pulsar-io.yaml`, you can create a function instance with:

```bash
bin/pulsar-admin functions create \
--function-type exclamation \
--inputs persistent://public/default/input-1 \
--output persistent://public/default/output-1
```
1 change: 1 addition & 0 deletions site2/docs/functions-deploy-cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,5 @@ bin/pulsar-admin functions update \
* [Enable parallel processing](functions-deploy-cluster-parallelism.md)
* [Enable end-to-end encryption](functions-deploy-cluster-encryption.md)
* [Enable package management service](functions-deploy-cluster-package.md)
* [Use built-in functions](functions-deploy-cluster-builtin.md)

101 changes: 100 additions & 1 deletion site2/docs/functions-package-java.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,25 @@ title: Package Java Functions
sidebar_label: "Package Java Functions"
---

There are two methods to package Java Functions, that is [uber JAR](#package-as-jar) and [NAR](#package-as-nar).

:::note

If you plan to package and distribute your function for others to use, you are obligated to
license and copyright your own code properly. Remember to add the license and copyright to
all libraries your code uses and to your distribution.

If you use the [NAR](#package-as-nar) method, the NAR plugin
automatically creates a `DEPENDENCIES` file in the generated NAR package, including the proper
licensing and copyrights of all libraries of your function.

For the runtime Java version, refer to [Pulsar Runtime Java Version Recommendation](https://github.com/apache/pulsar/blob/master/README.md#pulsar-runtime-java-version-recommendation) according to your target Pulsar version.

:::

To package a Java function, complete the following steps.
## Package as JAR

To package a Java function as JAR, complete the following steps.

1. Create a new maven project with a pom file. In the following code sample, the value of `mainClass` is your package name.

Expand Down Expand Up @@ -106,3 +118,90 @@ To package a Java function, complete the following steps.
07:55:03.724 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - Started process successfully
...
```

## Package as NAR

To package a Java function as NAR, complete the following steps.

1. Create a new maven project with a pom file.

```xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>java-function</groupId>
<artifactId>java-function</artifactId>
<version>1.0-SNAPSHOT</version>

<dependencies>
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-functions-api</artifactId>
<version>@pulsar:version@</version>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-nar-maven-plugin</artifactId>
<version>1.2.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<release>17</release>
</configuration>
</plugin>
</plugins>
</build>

</project>
```

You must also create a `resources/META-INF/services/pulsar-io.yaml` file. In the following code sample, the value of `functionClass` is your function class name. The `name` is the one used when the Function is deployed as a [built-in](functions-deploy-cluster-builtin.md) one.

```yaml
name: java-function
description: my java function
functionClass: org.example.test.ExclamationFunction
```
2. Package your Java function.
```bash
mvn package
```

After the Java function is packaged, a `target` directory is created automatically. Open the `target` directory to check if there is a NAR package similar to `java-function-1.0-SNAPSHOT.nar`.

3. Copy the packaged NAR file to the Pulsar image.

```bash
docker cp <path of java-function-1.0-SNAPSHOT.nar> CONTAINER ID:/pulsar
```

4. Run the Java function using the following command.

```bash
./bin/pulsar-admin functions localrun \
--jar java-function-1.0-SNAPSHOT.nar \
--inputs persistent://public/default/my-topic-1 \
--output persistent://public/default/test-1 \
--tenant public \
--namespace default \
--name JavaFunction
```

The following log indicates that the Java function starts successfully.

```text
...
07:55:03.724 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - Started process successfully
...
```
3 changes: 2 additions & 1 deletion site2/website/sidebars.json
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,8 @@
"functions-deploy-cluster-resource",
"functions-deploy-cluster-parallelism",
"functions-deploy-cluster-encryption",
"functions-deploy-cluster-package"
"functions-deploy-cluster-package",
"functions-deploy-cluster-builtin"
]
},
"functions-deploy-trigger"
Expand Down

0 comments on commit 7dc6700

Please sign in to comment.