Skip to content

Commit

Permalink
fixed documentation errors, move platforms from api to own section, c…
Browse files Browse the repository at this point in the history
…hanged left toc so it makes more sense, added documentation on how to use standard scripts
  • Loading branch information
trbKnl committed Nov 8, 2024
1 parent c2fe1dd commit cc32084
Show file tree
Hide file tree
Showing 9 changed files with 100 additions and 37 deletions.
11 changes: 1 addition & 10 deletions doc/source/api/api.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# API reference
# API Reference

## Props

Expand Down Expand Up @@ -27,12 +27,3 @@
.. automodule:: port.helpers.validate
:members:
```

## Platforms

### ChatGPT

```{eval-rst}
.. automodule:: port.platforms.chatgpt
:members:
```
4 changes: 2 additions & 2 deletions doc/source/api/index.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
API Documentation
=============================
API Reference
=============

.. toctree::
:maxdepth: 3
Expand Down
2 changes: 1 addition & 1 deletion doc/source/articles/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Articles
introduction-to-data-donation.md
installation.md
creating-your-own-data-donation-task.md
visualizations.md
data-donation-checklist.md
deployment.md
next-in-docker.md
visualizations.md
6 changes: 6 additions & 0 deletions doc/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,9 @@

html_theme = 'piccolo_theme'
html_static_path = ['_static']

html_sidebars = {
'**': [
'globaltoc.html',
]
}
15 changes: 15 additions & 0 deletions doc/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,19 @@ The Data Donation Task Documentation
Welcome to the Data Donation Task Documentation!

This is the place to learn about the data donation task.

Getting started
---------------

Checkout the following wiki articles to get started:

.. toctree::
:maxdepth: 2

articles/index

API Reference
-----------------

You can find the API documentation here:

Expand All @@ -19,6 +25,15 @@ You can find the API documentation here:

api/index

Standard scripts
----------------

We provide standard extraction scripts for a various platforms which you can find here:

.. toctree::
:maxdepth: 1

standard_scripts/index

Indices and tables
==================
Expand Down
32 changes: 32 additions & 0 deletions doc/source/standard_scripts/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
Platform Documentation
=============================

For various platforms we provide default extraction scripts, so you do not have to invent the wheel.

Freel free to use the extraction scripts as you see fit.

In order to use the scripts open the file `src/framework/processing/py/port/main.py` and change this line:

.. code-block:: python
from port.script import process
to:

.. code-block:: python
#from port.script import process
# Change to (in this case the standard script for instagram will be used):
from port.platforms.instagram import process
Available platforms
-------------------

.. automodule:: port.platforms.chatgpt


Instagram
---------

.. automodule:: port.platforms.instagram
20 changes: 11 additions & 9 deletions src/framework/processing/py/port/api/props.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,18 +107,18 @@ class PropsUIPromptConsentFormTable:
Attributes:
id (str): A unique string to identify the table after donation.
title (Translatable): Title of the table.
data_frame (pd.DataFrame | Dict[str, Dict[str, Any]]): Table to be shown can be a pandas data frame or a dictionary
data_frame (pd.DataFrame | Dict[str, Dict[str, Any]]): Table to be shown can be a pandas DataFrame or a dictionary.
description (Optional[Translatable]): Optional description of the table.
visualizations (Optional[list]): Optional visualizations to be shown.
folded (Optional[bool]): Whether the table should be initially folded.
delete_option (Optional[bool]): Whether to show a delete option for the table.
Examples:
# Using a pandas DataFrame for data_frame
Examples::
data_frame_df = pd.DataFrame([
{"column1": 1, "column2": 4},
{"column1": 2, "column2": 5},
{"column1": 3, "column2": 6}
{"column1": 3, "column2": 6},
])
example1 = PropsUIPromptConsentFormTable(
Expand All @@ -127,12 +127,12 @@ class PropsUIPromptConsentFormTable:
data_frame=data_frame_df,
)
# Using a dictionary for data_frame
data_frame_dict = {
"column1": {"0": 1, "1": 4},
"column2": {"0": 2, "1": 5},
"column3": {"0": 3, "1": 6}
"column3": {"0": 3, "1": 6},
}
example2 = PropsUIPromptConsentFormTable(
id="example2",
title=Translatable("Table as Dictionary"),
Expand Down Expand Up @@ -475,10 +475,12 @@ class PropsUIPageDonation:
A multi-purpose page that gets shown to the user.
Attributes:
platform (str): The platform name the user is currently in the process of donating data from.
platform (str):
The platform name the user is currently in the process of donating data from.
header (PropsUIHeader): Page header.
body (PropsUIPromptRadioInput | PropsUIPromptConsentForm | PropsUIPromptFileInput |
PropsUIPromptFileInputMultiple | PropsUIPromptConfirm | PropsUIPromptQuestionnaire):
body (PropsUIPromptRadioInput | PropsUIPromptConsentForm |
PropsUIPromptFileInput | PropsUIPromptFileInputMultiple |
PropsUIPromptConfirm | PropsUIPromptQuestionnaire):
Main body of the page.
footer (Optional[PropsUIFooter]): Optional page footer.
"""
Expand Down
44 changes: 30 additions & 14 deletions src/framework/processing/py/port/helpers/extraction_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ def dict_denester(inp: dict[Any, Any] | list[Any], new: dict[Any, Any] | None =
Returns:
dict[Any, Any]: A new denested dictionary.
Examples:
Examples::
>>> nested_dict = {"a": {"b": {"c": 1}}, "d": [2, 3]}
>>> dict_denester(nested_dict)
{"a-b-c": 1, "d-0": 2, "d-1": 3}
Expand Down Expand Up @@ -73,7 +74,8 @@ def find_item(d: dict[Any, Any], key_to_match: str) -> str:
Raises:
Exception: Logs an error message if an exception occurs during the search.
Examples:
Examples::
>>> d = {"asd-asd-asd": 1, "asd-asd": 2, "qwe": 3}
>>> find_item(d, "asd")
"2"
Expand Down Expand Up @@ -109,7 +111,8 @@ def find_items(d: dict[Any, Any], key_to_match: str) -> list:
Raises:
Exception: Logs an error message if an exception occurs during the search.
Examples:
Examples::
>>> d = {"asd-1": "a", "asd-2": "b", "qwe": "c"}
>>> find_items(d, "asd")
["a", "b"]
Expand Down Expand Up @@ -140,7 +143,8 @@ def json_dumper(zfile: str) -> pd.DataFrame:
Raises:
Exception: Logs an error message if an exception occurs during the process.
Examples:
Examples::
>>> df = json_dumper("data.zip")
>>> print(df.head())
"""
Expand Down Expand Up @@ -180,7 +184,8 @@ def fix_ascii_string(input: str) -> str:
Returns:
str: The fixed string with only ASCII characters, or the original string if an exception occurs.
Examples:
Examples::
>>> fix_ascii_string("Hello, 世界!")
"Hello, !"
"""
Expand All @@ -201,7 +206,8 @@ def replace_months(input_string: str) -> str:
Returns:
str: The input string with Dutch month abbreviations replaced by English equivalents.
Examples:
Examples::
>>> replace_months("15 mei 2023")
"15 may 2023"
"""
Expand Down Expand Up @@ -233,7 +239,8 @@ def epoch_to_iso(epoch_timestamp: str | int | float) -> str:
Raises:
Exception: Logs an error message if conversion fails.
Examples:
Examples::
>>> epoch_to_iso(1632139200)
"2021-09-20T12:00:00+00:00"
"""
Expand All @@ -257,7 +264,8 @@ def sort_isotimestamp_empty_timestamp_last(timestamp_series: pd.Series) -> pd.Se
Returns:
pd.Series: A Series of sorting keys, with -timestamp for valid dates and infinity for invalid/empty dates.
Examples:
Examples::
>>> df = df.sort_values(by="Date", key=sort_isotimestamp_empty_timestamp_last)
"""
def convert_timestamp(timestamp):
Expand Down Expand Up @@ -285,7 +293,8 @@ def fix_latin1_string(input: str) -> str:
Returns:
str: The fixed string after encoding and decoding, or the original string if an exception occurs.
Examples:
Examples::
>>> fix_latin1_string("café")
"café"
"""
Expand Down Expand Up @@ -319,7 +328,8 @@ def extract_file_from_zip(zfile: str, file_to_extract: str) -> io.BytesIO:
zipfile.BadZipFile: Logs an error if the zip file is invalid.
Exception: Logs any other unexpected errors.
Examples:
Examples::
>>> extracted_file = extract_file_from_zip("archive.zip", "data.txt")
>>> content = extracted_file.getvalue().decode('utf-8')
"""
Expand Down Expand Up @@ -385,7 +395,8 @@ def _json_reader_file(json_file: str, encoding: str) -> Any:
Returns:
Any: The parsed JSON data.
Examples:
Examples::
>>> data = _json_reader_file("data.json", "utf-8")
>>> print(data)
{'key': 'value'}
Expand Down Expand Up @@ -413,7 +424,8 @@ def _read_json(json_input: Any, json_reader: Callable[[Any, str], Any]) -> dict[
json.JSONDecodeError: Logs an error if JSON decoding fails.
Exception: Logs any other unexpected errors.
Examples:
Examples::
>>> data = _read_json(b'{"key": "value"}', _json_reader_bytes)
>>> print(data)
{'key': 'value'}
Expand Down Expand Up @@ -456,7 +468,8 @@ def read_json_from_bytes(json_bytes: io.BytesIO) -> dict[Any, Any] | list[Any]:
dict[Any, Any] | list[Any]: The parsed JSON data as a dictionary or list.
Returns an empty dictionary if parsing fails.
Examples:
Examples::
>>> buffer = io.BytesIO(b'{"key": "value"}')
>>> data = read_json_from_bytes(buffer)
>>> print(data)
Expand All @@ -483,7 +496,8 @@ def read_json_from_file(json_file: str) -> dict[Any, Any] | list[Any]:
dict[Any, Any] | list[Any]: The parsed JSON data as a dictionary or list.
Returns an empty dictionary if parsing fails.
Examples:
Examples::
>>> data = read_json_from_file("data.json")
>>> print(data)
{'key': 'value'}
Expand All @@ -504,6 +518,7 @@ def read_csv_from_bytes(json_bytes: io.BytesIO) -> list[dict[Any, Any]]:
Returns an empty list if parsing fails.
Examples:
>>> buffer = io.BytesIO(b'name,age\\nAlice,30\\nBob,25')
>>> data = read_csv_from_bytes(buffer)
>>> print(data)
Expand Down Expand Up @@ -536,6 +551,7 @@ def read_csv_from_bytes_to_df(json_bytes: io.BytesIO) -> pd.DataFrame:
pd.DataFrame: A pandas DataFrame containing the CSV data.
Examples:
>>> buffer = io.BytesIO(b'name,age\\nAlice,30\\nBob,25')
>>> df = read_csv_from_bytes_to_df(buffer)
>>> print(df)
Expand Down
3 changes: 2 additions & 1 deletion src/framework/processing/py/port/helpers/port_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,8 @@ def exit(code: int, info: str) -> CommandSystemExit:
Returns:
CommandSystemExit: A system command that initiates the exit process in Next.
Examples:
Examples::
yield exit(0, "Success")
"""
return CommandSystemExit(code, info)
Expand Down

0 comments on commit cc32084

Please sign in to comment.