Integrating Kuzu with GUI

[kamalikaray]

Hi,

Can a Kuzu graph be integrated into a graphical user interface like Dash or RShiny to create an application?

Thanks,
Kamalika

[prrao87]

Hi @kamalikaray, for RShiny, there isn't a way to embed Kùzu as we don't have an embeddable client library for R, so you can't generate query results if you want to work purely in R. For Plotly Dash in Python, you can extract the results of a Kùzu subgraph and pass them to a visualization library like Cytoscape.js (I believe Plotly Dash provides examples of this, which shouldn't be too hard). Alternatively, you can also look at example code for PyVis to render your query results to a graph - there are enough code examples online for this as well. Howeve, these all approaches tend to be used to visualize only small graphs and generally take a lot of custom code.

In general, it takes some JavaScript library to render the components of a graph in an aesthetically pleasing way - for our browser-based visualizer Kùzu Explorer, we use the G6 graph viz library that sits on top of a Node.js API. If you can point to some examples of Python-based graph visualizations you'd like to see, we can take a look and discuss further.

[kamalikaray]

Hi @prrao87,

Thanks a lot. So, Initially I was writing queries and extracting the subgraph as a networkx object and visualizing it using PyVis. However, that's when I found out the bug regarding the missing edges between same nodes (MultiGraph) in the networkx object. Next, I tried generating the query output as a pandas data frame, but due to the dynamic nature of this output with changing queries, it is difficult to generalize it for visualization.
I am not able to run the Kuzu explorer for my database due to firewall issues in my organization.
Could you please tell me which extraction method would be suitable to convert query results for visualization in PyVis or Plotly Dash/Cytoscape.js?

[prrao87]

Hi @kamalikaray apologies for that, @mewim and I discussed this and we're working on allowing multi-edge exports to NetworkX, so you should be able to export the correct graph that PyVis can then use. It should be done soon.

As for you not being able to run Kùzu Explorer due to firewall issues, maybe @mewim can suggest some alternate ideas? Could you explain a bit more about where your db sits, in relation to your application? Maybe there are ways you could use our REST API server (link below) to pass query requests to your database and then write a simple frontend that manipulates the output for one of these visualization libraries.
https://github.com/kuzudb/api-server

We do have a standalone explorer application (that doesn't require browsers or Docker) on our roadmap, but that's not been prioritized as we are adding a lot of visualization features on a weekly basis and it's a lot faster to deploy on DockerHub.

[prrao87]

Hi @kamalikaray, the update to allow MultiDiGraphs in NetworkX was just merged to master 🎉, see #3836. Apologies for the delay, we were prioritizing a number of other features.

I tested the workflow in NetworkX on a number of examples, and verified that the export to NetworkX using get_as_networkx() does indeed retain the multiple edges between the same pair of nodes, and the resulting object type is a MultiGraph (if undirected), or MultiDiGraph, which can be used in downstream operations.

You can try it tomorrow once the new nightly build is available using the following steps:

• pip uninstall kuzu to remove any existing installations
• pip install --pre kuzu to install the latest nightly build
• Build a graph in Kùzu that has multi-edges between the same pairs of nodes
• Use get_as_networkx() to output the results to a NetworkX graph
• Pass the NetworkX object to pyvis.networkx
• Render the graph in PyVis

You can leverage the PyVis + NetworkX integration this way to visualize your graphs in a Jupyter Notebook. I hope this answers your question. Sometime next week, we will release v0.5.0 of Kùzu that will allow you to get off the nightly build and use the stable release of Kùzu instead. Let us know if you have any further questions!

[kamalikaray]

Hi @prrao87
Thanks a lot! This is very helpful. I will also check the REST API and reach out to you in case of any questions.

[kamalikaray]

Hi @prrao87 ,

I am working on extracting the results of a Kùzu subgraph and pass them to a visualization library like Cytoscape.js in plotly dash in python. Could you please tell me which method of Kuzu subgraph extraction would be most suitable for doing this? Is it possible to pass a networkx object to it?

Thanks,
Kamalika

[prrao87]

Hi @kamalikaray, I think that regardless of what format you output to, rendering a Kùzu graph in any JavaScript library will require some custom manipulation of the underlying objects. From reading the contents of this thread on the Dash-Cytoscape forums, it seems that you could attempt to convert a NetworkX graph into a JSON graph, and then transform that to a Cytoscape graph. However this may come with some formatting problems and some custom code might be required to get it to render correctly.

How important is it that you use Cytoscape? I've seen some examples of people using Plotly Dash along with PyVis, and a cursory glance at the forum posts on this tells me that that this might be a promising option.

[kamalikaray]

Hi @prrao87

Thanks your reply. I tried using Pyvis with Dash app, but I am experiencing some javascript errors when integrating it with Dash.
I believe, I will have to try cytoscape for the visualization.
Could you please tell me if the multiDigraph property has been pushed to the main kuzu package. I have been working with the dev version with which I am getting the following error sometimes:
RuntimeError: Runtime exception: Trying to read a database file with a different version. Database file version: 30, Current build storage version: 31

[prrao87]

Hi @kamalikaray, yes, it's been around for our last two versions actually -- we're currently on 0.6.0. The error you're seeing is because our storage layer is still evolving as we add more features and optimizations. So the best way is to upgrade to the latest stable version of Kùzu and re-ingest the data (hopefully of the order of seconds or minutes, depending on the size of your data). We don't recommend staying on the dev version for any extended period of time as things evolve pretty rapidly on dev :).

Here's what I'd do:

# Remove existing dev version
pip uninstall kuzu
# Install new version
pip install kuzu
# Run DDL to get your dataset into a new Kùzu database
python workflow.py

For the last step (the script workflow.py), you could just write the data ingestion and the export to NetworkX in a single script, or break these down into separate scripts, depending on the complexity of what you want to do with the data downstream. I normally like to keep the DDL/ingestion scripts separate from the query/analysis scripts.