-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
ed38f73
commit 1c246be
Showing
29 changed files
with
1,570 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
# Read the Docs configuration file for Sphinx projects | ||
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details | ||
|
||
# Required | ||
version: 2 | ||
|
||
# Set the OS, Python version and other tools you might need | ||
build: | ||
os: ubuntu-22.04 | ||
tools: | ||
python: "3.12" | ||
# You can also specify other tool versions: | ||
# nodejs: "20" | ||
# rust: "1.70" | ||
# golang: "1.20" | ||
|
||
# Build documentation in the "docs/" directory with Sphinx | ||
sphinx: | ||
configuration: docs/source/conf.py | ||
# You can configure Sphinx to use a different builder, for instance use the dirhtml builder for simpler URLs | ||
# builder: "dirhtml" | ||
# Fail on all warnings to avoid broken references | ||
# fail_on_warning: true | ||
|
||
# Optional but recommended, declare the Python requirements required | ||
# to build your documentation | ||
# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html | ||
python: | ||
install: | ||
- requirements: doc-requirements.txt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Minimal makefile for Sphinx documentation | ||
# | ||
|
||
# You can set these variables from the command line, and also | ||
# from the environment for the first two. | ||
SPHINXOPTS ?= | ||
SPHINXBUILD ?= sphinx-build | ||
SOURCEDIR = source | ||
BUILDDIR = build | ||
|
||
# Put it first so that "make" without argument is like "make help". | ||
help: | ||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) | ||
|
||
.PHONY: help Makefile | ||
|
||
# Catch-all target: route all unknown targets to Sphinx using the new | ||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). | ||
%: Makefile | ||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
# Model | ||
|
||
|
||
## Properties | ||
Name | Type | Description | Notes | ||
------------ | ------------- | ------------- | ------------- | ||
**description** | **str** | | | ||
**name** | **str** | | | ||
**public** | **bool** | | | ||
**schema_box_id** | **int** | | | ||
**user_id** | **int** | | | ||
**version** | **int** | | | ||
|
||
## Example | ||
|
||
```python | ||
from structify.models.model import Model | ||
|
||
# TODO update the JSON string below | ||
json = "{}" | ||
# create an instance of Model from a JSON string | ||
model_instance = Model.from_json(json) | ||
# print the JSON string representation of the object | ||
print Model.to_json() | ||
|
||
# convert the object into a dict | ||
model_dict = model_instance.to_dict() | ||
# create an instance of Model from a dict | ||
model_form_dict = model.from_dict(model_dict) | ||
``` | ||
[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
@ECHO OFF | ||
|
||
pushd %~dp0 | ||
|
||
REM Command file for Sphinx documentation | ||
|
||
if "%SPHINXBUILD%" == "" ( | ||
set SPHINXBUILD=sphinx-build | ||
) | ||
set SOURCEDIR=source | ||
set BUILDDIR=build | ||
|
||
%SPHINXBUILD% >NUL 2>NUL | ||
if errorlevel 9009 ( | ||
echo. | ||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx | ||
echo.installed, then set the SPHINXBUILD environment variable to point | ||
echo.to the full path of the 'sphinx-build' executable. Alternatively you | ||
echo.may add the Sphinx directory to PATH. | ||
echo. | ||
echo.If you don't have Sphinx installed, grab it from | ||
echo.https://www.sphinx-doc.org/ | ||
exit /b 1 | ||
) | ||
|
||
if "%1" == "" goto help | ||
|
||
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% | ||
goto end | ||
|
||
:help | ||
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% | ||
|
||
:end | ||
popd |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
sphinxawesome_theme |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
<!-- sidebar.html --> | ||
|
||
<div class="sidebar"> | ||
<div class="sidebar-header"> | ||
<!-- Sidebar header content --> | ||
<h2>Documentation</h2> | ||
</div> | ||
<div class="sidebar-content"> | ||
<!-- Navigation links --> | ||
<ul class="sidebar-nav"> | ||
<li> | ||
<details> | ||
<summary>Getting Started</summary> | ||
<ul> | ||
<li><a href="{{ pathto('index') }}">Overview</a></li> | ||
<li><a href="{{ pathto('getting_started') }}">Intro</a></li> | ||
<li><a href="{{ pathto('example0') }}">Quickstart</a></li> | ||
</ul> | ||
</details> | ||
</li> | ||
<li> | ||
<details> | ||
<summary>Capabilities</summary> | ||
<ul> | ||
<li><a href="{{ pathto('datasets') }}">Creating Datasets</a></li> | ||
<li><a href="{{ pathto('populate') }}">Populating Datasets</a></li> | ||
<li><a href="{{ pathto('documents') }}">Using Documents</a></li> | ||
<li><a href="{{ pathto('search') }}">Searching Datasets</a></li> | ||
<li><a href="{{ pathto('notifications') }}">Notifications</a></li> | ||
<li><a href="{{ pathto('analysis') }}">Analysis</a></li> | ||
<li><a href="{{ pathto('workflows') }}">Workflows</a></li> | ||
</ul> | ||
</details> | ||
</li> | ||
<li><a href="{{ pathto('index') }}">Home</a></li> | ||
<li><a href="{{ pathto('installation') }}">Installation</a></li> | ||
<li><a href="{{ pathto('usage') }}">Usage</a></li> | ||
<li><a href="{{ pathto('usage') }}">Usage</a></li> | ||
<li><a href="{{ pathto('api') }}">API</a></li> | ||
<li><a href="{{ pathto('faq') }}">FAQ</a></li> | ||
<li><a href="{{ pathto('changelog') }}">Changelog</a></li> | ||
|
||
</ul> | ||
<!-- Table of Contents --> | ||
<div class="sidebar-toc"> | ||
{{ toctree(maxdepth=2, collapse=False) }} | ||
</div> | ||
<!-- Search box --> | ||
<div class="sidebar-search"> | ||
<form role="search" method="get" action="{{ pathto('search') }}"> | ||
<input type="text" name="q" placeholder="Search..." /> | ||
<button type="submit">Search</button> | ||
</form> | ||
</div> | ||
</div> | ||
</div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
.. _Analyzing Datasets: | ||
|
||
Analyzing Your Datasets | ||
======================= | ||
|
||
Overview | ||
-------- | ||
|
||
Part of the advantages to using Structify as your data infrastructure is the automatic powering of advanced analytics on top of your custom datasets. In our pipeline, Structify is developing the ability to power the following: | ||
|
||
#. :ref:`Creating Custom Tags for Data <tagging>` | ||
#. :ref:`Sorting Data along Any Axis <sorting>` | ||
#. :ref:`Getting Confidence Scores <confidence>` | ||
|
||
|
||
.. _tagging: | ||
|
||
Tagging | ||
------- | ||
We will allow you to tag data either via LLM generated tags or custom tags. This lets you to easily filter your data based on the tags you have created. | ||
|
||
A common practice is to sort datasets by industry. For example, if you are hiring a GTM specialist, you would want them to have deep knowledge and contacts within your vertical, so tagging your network by industry would allow you to easily filter for the right candidates. You can see a great example of this in `our tutorial <example/example3>`. | ||
|
||
.. code-block:: python | ||
industry_tags = ['healthcare', 'retail', 'finance', 'technology', 'education', 'government', 'non-profit', 'other'] | ||
structify.analysis.filter( | ||
dataset=candidates, | ||
tags=industry_tags, | ||
tag_description="a list of possible industries that the candidate has experience in" | ||
) | ||
.. _sorting: | ||
|
||
Sorting | ||
------- | ||
We allow for you to sort your data along any axis (subjective or objective). For example, you can sort news about clients along the sentiment to see how sentiment has changed over time, or you could cluster based on topic and sentiment to determine why audiences are reacting the way they are. | ||
|
||
.. code-block:: python | ||
structify.analysis.sort( | ||
dataset=news, | ||
axis=['sentiment', 'topic'], | ||
sort_description="sorts the news by sentiment in order of positive association with our client George Washington University" | ||
) | ||
.. _confidence: | ||
|
||
Confidence Scores | ||
----------------- | ||
We allow for you to get confidence scores for any given datapoint. This is useful for understanding the quality of the data, and for understanding how strongly our agents feel about the certainty of a given datapoint. | ||
|
||
If we wanted to get the confidence score for a datapoint, we would call the following: | ||
|
||
.. code-block:: python | ||
structify.source.get_confidence(id = [123456]) | ||
Note that you first have to use the ``structufy.dataset.view`` endpoint to retrieve the id(s) of the relevant entities. | ||
|
||
Now, you have the tools to be able to more deeply understand your datasets and derive insights from them. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
Searching through Datasets | ||
========================== | ||
|
||
Overview | ||
-------- | ||
When you have a large dataset, it can be difficult to find the specific piece of data you are looking for. And often, you will create the datasets as a reference backend for users or AI tools to reference in answering certain questions, which means you won't know immediately what to search for. In those cases, it will be crucial to set up a method to search through the datasets. This can be done via a couple different methods depending on how much specificity you want to allow in the search: | ||
|
||
#. :ref:`If you know the keywords to search <string-search>` | ||
#. :ref:`If you just have a question <natural-language-search>` | ||
|
||
|
||
.. _string-search: | ||
|
||
Searching for Specific Strings within Datasets | ||
----------------------------------------------- | ||
Another simple method is to allow users to search for a specific string within the dataset. This can be done by creating a function that takes in a string and returns all the rows that contain that string. This endpoint works best if the you've used enums in your dataset. | ||
|
||
If we wanted to power a search for employees who attended a certain school, we could create the following function: | ||
|
||
.. code-block:: python | ||
def search_schools(dataset_name, school_name): | ||
# We need to specify the table and columns the keyword search applies to | ||
search_target = { | ||
"table": [ | ||
"name": "education", | ||
"columns": ["name"] | ||
] | ||
} | ||
return client.dataset.query(name = dataset_name, search = search_target, keyword = school_name.lower()) | ||
This will return to us a subset of the dataset that contains just the entities whose education table contains the school name we are looking for. | ||
|
||
.. tip:: | ||
You can bulk search for multiple keywords by passing a list of keywords to the "keyword" parameter. You can also conduct a search across multiple tables by passing a list of search targets to the "search" parameter. | ||
|
||
.. _natural-language-search: | ||
|
||
Natural Language Search | ||
----------------------- | ||
The most powerful method is to allow users to ask questions in natural language and have the system return the relevant data. This endpoint is powered by Structify's LLM agents. While the most complex method, it is allows for the most flexible and user-friendly experience. | ||
|
||
If we wanted to power users to search for employees by describing the type of school they attended (e.g. "Ivy League tier schools" or "liberal arts colleges in California"), we could create the following function: | ||
|
||
.. code-block:: python | ||
def plaintext_school_search(dataset_name, query): | ||
return client.analysis.query(dataset = dataset_name, query = query) | ||
Using the ``client.analysis.query`` endpoint powers a more conversational experience for users, and typically, we see this endpoint powering chatbots or other conversational interfaces. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
Sharing Datasets | ||
================ | ||
Oftentimes, you will want to share your dataset with others. You can use the ``structify.dataset.share`` API call to share your dataset with others. This API call requires the following: | ||
|
||
* **name:** The name of the dataset you want to share | ||
* **share_method:** The method of sharing the dataset. This can be "public" or "private". | ||
* **restrictions**: (optional) A list of restrictions that you want to place on the dataset. This can be "view-only", "refresh-only", "edit", "no-delete", or "admin". Each successive option has more priviledges. The default is "view". | ||
* **users:** (optional) A list of user ids that you want to share the dataset with. | ||
* **emails:** (optional) A list of emails that you want to share the dataset with. | ||
|
||
.. note:: | ||
If you want to share the dataset with specific users, you can use the "private" method and pass a list of either ``user_ids`` to the "users" parameter. If the target recipients are not users, you can pass a list of emails to the "emails" parameter, which will send them an email link to create an account and view the dataset. | ||
|
||
Here's an example that walks through sharing the employees dataset with various co-workers who do not have Structify accounts: | ||
|
||
.. code-block:: python | ||
structify.dataset.share( | ||
name = "employees", | ||
share_method = "private", | ||
restrictions = "no-delete", | ||
emails = ["[email protected]", "[email protected]", "[email protected]"]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
# Configuration file for the Sphinx documentation builder. | ||
# | ||
# For the full list of built-in configuration values, see the documentation: | ||
# https://www.sphinx-doc.org/en/master/usage/configuration.html | ||
|
||
# -- Project information ----------------------------------------------------- | ||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information | ||
import sys | ||
|
||
sys.path.insert(0, "/home/dev/src/prospero/client/structify/") | ||
|
||
project = "Structify" | ||
copyright = "2024, YMTM Inc." | ||
author = "Alex Reichenbach & Ronak Gandhi" | ||
|
||
# -- General configuration --------------------------------------------------- | ||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration | ||
|
||
extensions = [ | ||
"sphinx.ext.doctest", | ||
"sphinx.ext.autodoc", | ||
"sphinx.ext.autosummary", | ||
# "autoapi.extension", | ||
# "sphinx.ext.autosectionlabel", | ||
# "sphinx.ext.napoleon", | ||
] | ||
|
||
templates_path = ["/docs/_templates"] | ||
exclude_patterns = [] | ||
pygments_style = "sphinx" | ||
|
||
# -- Options for HTML output ------------------------------------------------- | ||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output | ||
|
||
html_theme = "sphinxawesome_theme" | ||
extensions += ["sphinxawesome_theme.highlighting", "sphinxcontrib.details.directive"] | ||
html_title = "Structify" | ||
|
||
autoapi_dirs = ["../../../"] | ||
autoapi_options = [ | ||
"members", | ||
"show-inheritance", | ||
"show-module-summary", | ||
"imported-members", # This is the problematic option | ||
"undoc-members", | ||
] | ||
|
||
|
||
html_baseurl = "/docs/" | ||
html_static_path = ["_static/"] | ||
html_logo = "_static/webclip.png" | ||
html_favicon = "_static/favicon.png" | ||
|
||
html_js_files = ["/docs"] |
Oops, something went wrong.