GH-38594: [Docs][C++][Gandiva] Document how to register Gandiva external functions

[niyue]

Rationale for this change

Provide a basic documentation on how to register and develop Gandiva external functions, including IR functions and C functions.

What changes are included in this PR?

A markdown doc providing an overview and detailed steps for integrating external functions into Gandiva.

Are these changes tested?

It is a doc change

Are there any user-facing changes?

No

• Closes: [Docs][C++][Gandiva] Document how to register external functions #38594

[github-actions]

⚠️ GitHub issue #38594 has been automatically assigned in GitHub to PR creator.

[niyue]

@kou @js8544
I draft a simple markdown doc to describe how Gandiva external function could be integrated. Could you please help to review? Thanks.

It may be easier to read in a markdown rendered format, like the GitHub markdown viewer: https://github.com/niyue/arrow/blob/feature/gdv-external-func-doc/cpp/src/gandiva/doc/external_func.md

I am not a native English speaker, and far from a decent doc writer, please feel free to let me know what should be revised. Thanks.

[llama90]

Hello.

While this is not about Gandiva, I see areas that seem to need changes based on the .md files written in Arrow. reference: README.md

[llama90]

I propose changes for the readability of Markdown documents in terms of code. As I enjoy writing Markdown documents, I have looked into the recommended aspects for readability.

[js8544]

Thank you for the PR! I noticed that the doc is in Markdown format. It's fine for internal documentation. However, if it's targeted to external users (i.e. displayed on Arrow website), it has to be in rst format. See the various docs in docs/source/cpp/ directory for examples.

[kou]

As js8544 said, could you write this as a Sphinx documentation?
We have a page for Gandiva: https://github.com/apache/arrow/blob/main/docs/source/cpp/gandiva.rst
We can extend it or create a new page and refer the new page from the existing page.

If we want to use Markdown in our Sphinx documentation, we can discuss it as a separated issue.
(https://github.com/apache/arrow-flight-sql-postgresql uses Sphinx https://arrow.apache.org/flight-sql-postgresql/current/ but it uses Markdown not reStructuredText. So it's not hard to use Markdown in Sphinx.)

[niyue]

Sure. Let me port this to rst today.

[niyue]

I re-format the document to be a rst document, and put it under the docs/source/cpp directory now. Please check it out. Thanks.

[llama90]

Based on the document available at https://github.com/apache/arrow/blob/main/docs/source/cpp/arrays.rst?plain=1, I would like to make a suggestion.

[niyue]

Hi there, is there anything that we need to revise for this doc? Please feel free to suggest and I will try my best to polish it. Thanks.

[js8544]

Sorry I totally forgot about this PR 🤦. I'll have a look tomorrow!

[kou]

Could you add a link to this page to docs/source/cpp/gandiva.rst?

@AlenkaF Do you have any suggestion how to organize Gandiva documentations?

[kou]

@github-actions crossbow submit preview-docs

[github-actions]

Revision: 892ae01

Submitted crossbow builds: ursacomputing/crossbow @ actions-8420909df5

Task	Status
preview-docs

[js8544]

Thanks! The doc looks good. Just a couple of nits.

[AlenkaF]

The structure already looks pretty good.
I would suggest, though, to organise it in a similar way to acero docs:

• index page in the docs/source/cpp folder https://github.com/apache/arrow/blob/main/docs/source/cpp/streaming_execution.rst this is docs/source/cpp/gandiva.rst currently for gandiva docs
• on the index page list all the subpages with toctree directive (.. toctree::)
• It would be good to maybe move more content from docs/source/cpp/gandiva.rst to subpages, save the subpages in the gandiva folder and add them to the index page toctree
• keep gandiva folder as is currently in the PR
• The overall flow chart from the PR (GH-37753: [C++][Gandiva] Add external function registry support #38116 (comment)) could also be added to docs/source/cpp/gandiva/external_func.rst subpage.

[niyue]

@AlenkaF

on the index page list all the subpages with toctree directive (.. toctree::)
It would be good to maybe move more content from docs/source/cpp/gandiva.rst to subpages, save the subpages in the gandiva folder and add them to the index page toctree

• I move the three sections in the index page to a new sub page
• I added the toctree to the index page, but currently there is not too much content for gandiva so I still keep the brief summary sections above, and added the toctree below. Let me know if this is not desirable.

The overall flow chart from the PR

I updated the overall flow chart, and remove some internal classes from the chart and add external C functions to the chart

[AlenkaF]

@github-actions crossbow submit preview-docs

[github-actions]

Revision: 90968bd

Submitted crossbow builds: ursacomputing/crossbow @ actions-11a52e7806

Task	Status
preview-docs

[niyue]

@AlenkaF I am new to authoring docs in arrow, could you please help how I could see the results of crossbow submit preview-docs? Thanks.

I expect this command to generate some docs for previewing, but I cannot find the link I could use to preview the doc in CI results.

Currently I use github's file preview function to preview the basic skeleton of the rst file, but I am not able to verify some special rst constructs like page links with that.

[AlenkaF]

Sure! When the build is finished you navigate to the Summary part of the Crossbow build and on the bottom you will find the preview of the docs. See link for better info: https://arrow.apache.org/docs/dev/developers/documentation.html#building-a-docs-preview-in-a-pull-request ;)

[AlenkaF]

Looking good! http://crossbow.voltrondata.com/pr_docs/38763/cpp/gandiva.html

[kou]

+1

[kou]

I'll merge this. Thanks!

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit f3cdd81.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.