Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][doc] Add doc for builtin functions #18569

Merged
merged 7 commits into from
Nov 24, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions site2/docs/functions-deploy-cluster-builtin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
id: functions-deploy-cluster-builtin
momo-jun marked this conversation as resolved.
Show resolved Hide resolved
title: Built-in functions
sidebar_label: "Built-in functions"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a task-oriented topic for users. So it makes more sense to use an imperative title, like Use built-in functions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

---

Similar to built-in connectors, the code of Java functions [packaged as NAR](functions-package-java.md) that are placed in the `functions` directory of the function worker are loaded at startup and can be referenced when creating a function.

For instance if you have a built-in function with name `exclamation` in its `pulsar-io.yaml`, you can create a function instance with:

```bash
bin/pulsar-admin functions create \
--function-type exclamation \
--inputs persistent://public/default/input-1 \
--output persistent://public/default/output-1
```

To get the list of available built-in Functions, use the `available-functions` command:

```bash
bin/pulsar-admin functions available-functions
```

If you add or delete a nar file in a `functions` folder, reload the available built-in functions before using it.

```bash
bin/pulsar-admin functions reload
```


cbornet marked this conversation as resolved.
Show resolved Hide resolved
105 changes: 104 additions & 1 deletion site2/docs/functions-package-java.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,23 @@ For the runtime Java version, refer to [Pulsar Runtime Java Version Recommendati

:::

To package a Java function, complete the following steps.
There are two methods to package Java Functions, that is [uber JAR](#package-as-jar) and [NAR](#package-as-nar).

:::note
momo-jun marked this conversation as resolved.
Show resolved Hide resolved

If you plan to package and distribute your function for others to use, you are obligated to
license and copyright your own code properly. Remember to add the license and copyright to
all libraries your code uses and to your distribution.

:::

> If you use the [NAR](#package-as-nar) method, the NAR plugin
automatically creates a `DEPENDENCIES` file in the generated NAR package, including the proper
licensing and copyrights of all libraries of your function.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They seem similar notes/tips, why different syntax?

You may read the syntax guide (https://pulsar.apache.org/contribute/document-syntax/#admonitions) for what is expected here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


# Package as JAR

To package a Java function as JAR, complete the following steps.

1. Create a new maven project with a pom file. In the following code sample, the value of `mainClass` is your package name.

Expand Down Expand Up @@ -106,3 +122,90 @@ To package a Java function, complete the following steps.
07:55:03.724 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - Started process successfully
...
```

# Package as NAR

To package a Java function as NAR, complete the following steps.

1. Create a new maven project with a pom file.

```xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>java-function</groupId>
<artifactId>java-function</artifactId>
<version>1.0-SNAPSHOT</version>

<dependencies>
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-functions-api</artifactId>
<version>2.10.0</version>
cbornet marked this conversation as resolved.
Show resolved Hide resolved
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-nar-maven-plugin</artifactId>
<version>1.2.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<release>17</release>
</configuration>
</plugin>
</plugins>
</build>

</project>
```

You must also create a `resources/META-INF/services/pulsar-io.yaml` file. In the following code sample, the value of `functionClass` is your function class name. The `name` is the one used when the Function is deployed as a [built-in](functions-deploy-cluster-builtin.md) one.

```yaml
name: java-function
description: my java function
functionClass: org.example.test.ExclamationFunction
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also one-level ident?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


2. Package your Java function.

```bash
mvn package
cbornet marked this conversation as resolved.
Show resolved Hide resolved
```

After the Java function is packaged, a `target` directory is created automatically. Open the `target` directory to check if there is a NAR package similar to `java-function-1.0-SNAPSHOT.nar`.

3. Copy the packaged NAR file to the Pulsar image.

```bash
docker cp <path of java-function-1.0-SNAPSHOT.nar> CONTAINER ID:/pulsar
```

4. Run the Java function using the following command.

```bash
./bin/pulsar-admin functions localrun \
--jar java-function-1.0-SNAPSHOT.nar \
--inputs persistent://public/default/my-topic-1 \
--output persistent://public/default/test-1 \
--tenant public \
--namespace default \
--name JavaFunction
```

The following log indicates that the Java function starts successfully.

```text
...
07:55:03.724 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - Started process successfully
...
```