Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][doc] Add doc for builtin functions #18569

Merged
merged 7 commits into from
Nov 24, 2022
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions site2/docs/functions-deploy-cluster-builtin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
id: functions-deploy-cluster-builtin
momo-jun marked this conversation as resolved.
Show resolved Hide resolved
title: Built-in functions
sidebar_label: "Built-in functions"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a task-oriented topic for users. So it makes more sense to use an imperative title, like Use built-in functions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

---

Similar to built-in connectors, the code of Java functions [packaged as NAR](functions-package-java.md) that are placed in the `functions` directory of the function worker are loaded at startup and can be referenced when creating a function.

For instance if you have a built-in function with name `exclamation` in its `pulsar-io.yaml`, you can create a function instance with:

```bash
bin/pulsar-admin functions create \
--function-type exclamation \
--inputs persistent://public/default/input-1 \
--output persistent://public/default/output-1
```

To get the list of available built-in Functions, use the `available-functions` command:

```bash
bin/pulsar-admin functions available-functions
```

If you add or delete a nar file in a `functions` folder, reload the available built-in functions before using it.

```bash
bin/pulsar-admin functions reload
```
109 changes: 108 additions & 1 deletion site2/docs/functions-package-java.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,33 @@ title: Package Java Functions
sidebar_label: "Package Java Functions"
---

There are two methods to package Java Functions, that is [uber JAR](#package-as-jar) and [NAR](#package-as-nar).

:::note
momo-jun marked this conversation as resolved.
Show resolved Hide resolved

If you plan to package and distribute your function for others to use, you are obligated to
license and copyright your own code properly. Remember to add the license and copyright to
all libraries your code uses and to your distribution.

:::

:::tip

If you use the [NAR](#package-as-nar) method, the NAR plugin
automatically creates a `DEPENDENCIES` file in the generated NAR package, including the proper
licensing and copyrights of all libraries of your function.

:::

:::note

For the runtime Java version, refer to [Pulsar Runtime Java Version Recommendation](https://github.com/apache/pulsar/blob/master/README.md#pulsar-runtime-java-version-recommendation) according to your target Pulsar version.

:::

To package a Java function, complete the following steps.
## Package as JAR

To package a Java function as JAR, complete the following steps.

1. Create a new maven project with a pom file. In the following code sample, the value of `mainClass` is your package name.

Expand Down Expand Up @@ -106,3 +126,90 @@ To package a Java function, complete the following steps.
07:55:03.724 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - Started process successfully
...
```

## Package as NAR

To package a Java function as NAR, complete the following steps.

1. Create a new maven project with a pom file.

```xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>java-function</groupId>
<artifactId>java-function</artifactId>
<version>1.0-SNAPSHOT</version>

<dependencies>
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-functions-api</artifactId>
<version>@pulsar:version@</version>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-nar-maven-plugin</artifactId>
<version>1.2.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<release>17</release>
</configuration>
</plugin>
</plugins>
</build>

</project>
```

You must also create a `resources/META-INF/services/pulsar-io.yaml` file. In the following code sample, the value of `functionClass` is your function class name. The `name` is the one used when the Function is deployed as a [built-in](functions-deploy-cluster-builtin.md) one.

```yaml
name: java-function
description: my java function
functionClass: org.example.test.ExclamationFunction
```

2. Package your Java function.

```bash
mvn package
```

After the Java function is packaged, a `target` directory is created automatically. Open the `target` directory to check if there is a NAR package similar to `java-function-1.0-SNAPSHOT.nar`.

3. Copy the packaged NAR file to the Pulsar image.

```bash
docker cp <path of java-function-1.0-SNAPSHOT.nar> CONTAINER ID:/pulsar
```

4. Run the Java function using the following command.

```bash
./bin/pulsar-admin functions localrun \
--jar java-function-1.0-SNAPSHOT.nar \
--inputs persistent://public/default/my-topic-1 \
--output persistent://public/default/test-1 \
--tenant public \
--namespace default \
--name JavaFunction
```

The following log indicates that the Java function starts successfully.

```text
...
07:55:03.724 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - Started process successfully
...
```
3 changes: 2 additions & 1 deletion site2/website/sidebars.json
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,8 @@
"functions-deploy-cluster-resource",
"functions-deploy-cluster-parallelism",
"functions-deploy-cluster-encryption",
"functions-deploy-cluster-package"
"functions-deploy-cluster-package",
"functions-deploy-cluster-builtin"
]
},
"functions-deploy-trigger"
Expand Down