From 371ef413fdfabc12e75e1bc58e003e2a1f03786a Mon Sep 17 00:00:00 2001 From: Wouter Beek Date: Wed, 13 Nov 2024 11:47:03 +0100 Subject: [PATCH] ADDED: More extensive documentation for adding datasets and adding data. --- docs/assets/add-data-from-a-url-field.png | 3 + ...from-an-existing-dataset-choose-graphs.png | 3 + ...dd-data-from-an-existing-dataset-field.png | 3 + docs/assets/add-data-from-files-dialog.png | 3 + docs/assets/add-data-from-files-job.png | 3 + docs/assets/add-data-pane.png | 3 + docs/assets/add-data.png | 3 - docs/assets/add-dataset-dialog.png | 3 + docs/assets/createdataset.png | 3 - docs/assets/graphs-page.png | 3 + docs/assets/home-screen-logged-in-raw.png | 3 + docs/assets/home-screen-logged-in.png | 3 + docs/assets/landing-page.png | 3 + docs/assets/login-page.png | 3 + docs/assets/user-screen-logged-in-raw.png | 3 + docs/assets/user-screen-logged-in.png | 3 + .../uploading-data/index.md | 169 +++++++++++++----- 17 files changed, 164 insertions(+), 53 deletions(-) create mode 100644 docs/assets/add-data-from-a-url-field.png create mode 100644 docs/assets/add-data-from-an-existing-dataset-choose-graphs.png create mode 100644 docs/assets/add-data-from-an-existing-dataset-field.png create mode 100644 docs/assets/add-data-from-files-dialog.png create mode 100644 docs/assets/add-data-from-files-job.png create mode 100644 docs/assets/add-data-pane.png delete mode 100644 docs/assets/add-data.png create mode 100644 docs/assets/add-dataset-dialog.png delete mode 100644 docs/assets/createdataset.png create mode 100644 docs/assets/graphs-page.png create mode 100644 docs/assets/home-screen-logged-in-raw.png create mode 100644 docs/assets/home-screen-logged-in.png create mode 100644 docs/assets/landing-page.png create mode 100644 docs/assets/login-page.png create mode 100644 docs/assets/user-screen-logged-in-raw.png create mode 100644 docs/assets/user-screen-logged-in.png diff --git a/docs/assets/add-data-from-a-url-field.png b/docs/assets/add-data-from-a-url-field.png new file mode 100644 index 00000000..540eb4a4 --- /dev/null +++ b/docs/assets/add-data-from-a-url-field.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3fae444e053d025c9c026d9207bf94171e07cc24df27dd69677cb327e0fe4660 +size 86207 diff --git a/docs/assets/add-data-from-an-existing-dataset-choose-graphs.png b/docs/assets/add-data-from-an-existing-dataset-choose-graphs.png new file mode 100644 index 00000000..29a80caa --- /dev/null +++ b/docs/assets/add-data-from-an-existing-dataset-choose-graphs.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ce5dd4d2d20a2e402e6eaa51ec52098b967110bb7f726c7f7972f294bde81f2 +size 100997 diff --git a/docs/assets/add-data-from-an-existing-dataset-field.png b/docs/assets/add-data-from-an-existing-dataset-field.png new file mode 100644 index 00000000..a8c3c7ec --- /dev/null +++ b/docs/assets/add-data-from-an-existing-dataset-field.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b8abeb040df7470cfc69e4f72fdefa87be732baa2a00a689581f6463d6e0b18f +size 104962 diff --git a/docs/assets/add-data-from-files-dialog.png b/docs/assets/add-data-from-files-dialog.png new file mode 100644 index 00000000..3380aebf --- /dev/null +++ b/docs/assets/add-data-from-files-dialog.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c52a49ea18c35db1640a67fa6aea0cae7238f223162cc42d4383c5243feef848 +size 155789 diff --git a/docs/assets/add-data-from-files-job.png b/docs/assets/add-data-from-files-job.png new file mode 100644 index 00000000..9be35da0 --- /dev/null +++ b/docs/assets/add-data-from-files-job.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7de3ebad69be72222fd04b0568fb2cea6d48bf5c2cbd9e71aae9f1adfc2e1e9e +size 96768 diff --git a/docs/assets/add-data-pane.png b/docs/assets/add-data-pane.png new file mode 100644 index 00000000..55a7f552 --- /dev/null +++ b/docs/assets/add-data-pane.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f5efdad27f9207df4382d4d7b33433b3a75d73d8dd251ea7970e99505c1cd1d +size 83388 diff --git a/docs/assets/add-data.png b/docs/assets/add-data.png deleted file mode 100644 index cda8ab2e..00000000 --- a/docs/assets/add-data.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:175add373d0a6f5133336ff368421976a846e1e4085aeb1ec917fc0e00e9228c -size 52316 diff --git a/docs/assets/add-dataset-dialog.png b/docs/assets/add-dataset-dialog.png new file mode 100644 index 00000000..c541ff1e --- /dev/null +++ b/docs/assets/add-dataset-dialog.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4e6ec55307b1bb03eb3e4d6287abfd78d3c9957bc57f877b103bb80fc0c17e52 +size 120502 diff --git a/docs/assets/createdataset.png b/docs/assets/createdataset.png deleted file mode 100644 index b8004f75..00000000 --- a/docs/assets/createdataset.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:d2b55093b2b542aea02c0d8e86009e013924bf9e8ec1a5314a50170dfa1d3296 -size 33648 diff --git a/docs/assets/graphs-page.png b/docs/assets/graphs-page.png new file mode 100644 index 00000000..9165b428 --- /dev/null +++ b/docs/assets/graphs-page.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b36d9b39faacb1607468f8ed756b231493bdcf928f89ca9b612fea6fa4da3b36 +size 160496 diff --git a/docs/assets/home-screen-logged-in-raw.png b/docs/assets/home-screen-logged-in-raw.png new file mode 100644 index 00000000..558c2a0e --- /dev/null +++ b/docs/assets/home-screen-logged-in-raw.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c7d692ad35fbfc2386df6b34f74b3805271d3bf08fdb0b8cab98669284b2b9a0 +size 364281 diff --git a/docs/assets/home-screen-logged-in.png b/docs/assets/home-screen-logged-in.png new file mode 100644 index 00000000..9df0c3b0 --- /dev/null +++ b/docs/assets/home-screen-logged-in.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:afee1af364416c97dcb66cdb2b1d376c2cdbfd98590dcd306f5257c2e9a6516d +size 365957 diff --git a/docs/assets/landing-page.png b/docs/assets/landing-page.png new file mode 100644 index 00000000..2051a948 --- /dev/null +++ b/docs/assets/landing-page.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2532d69b12527a43575480e3f95b7752bdb7fbc9fbd59e294ead45273104fd9 +size 550307 diff --git a/docs/assets/login-page.png b/docs/assets/login-page.png new file mode 100644 index 00000000..fc16f084 --- /dev/null +++ b/docs/assets/login-page.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ff5d97509c60e333dbce7123a9b6157926933247000fc6c3bce9498ccbc3a708 +size 97697 diff --git a/docs/assets/user-screen-logged-in-raw.png b/docs/assets/user-screen-logged-in-raw.png new file mode 100644 index 00000000..ffa9d8cf --- /dev/null +++ b/docs/assets/user-screen-logged-in-raw.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5aa7a2bfea6c9aeb9e9507e090769902257c9909a5e9bc75d6907e8223958561 +size 263797 diff --git a/docs/assets/user-screen-logged-in.png b/docs/assets/user-screen-logged-in.png new file mode 100644 index 00000000..239e8347 --- /dev/null +++ b/docs/assets/user-screen-logged-in.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:268415e94eeb477647be61d44b68d53275d13833f6410ff7522d84bba14bad01 +size 272181 diff --git a/docs/triply-db-getting-started/uploading-data/index.md b/docs/triply-db-getting-started/uploading-data/index.md index ba5e7b48..676c2ef4 100644 --- a/docs/triply-db-getting-started/uploading-data/index.md +++ b/docs/triply-db-getting-started/uploading-data/index.md @@ -8,86 +8,161 @@ This section explains how to create a linked dataset in TriplyDB. ## Creating a new dataset -The following steps allow a new linked datasets to be created: +You must be logged in before you can create a new dataset. + -1. Log into a TriplyDB instance. +### Opening the “Create dataset” dialog -2. Click the `+` button on the dataset pane that appears on the right-hand side of the home screen. +You can create a new dataset in either of the following two ways: -3. This brings up the dialog for creating a new dataset. Enter a dataset name that consists of alpha-numeric characters (`A-Za-z0-9`) and hyphens (`-`). +1. From the home screen (see [Figure 1a](#home-screen-logged-in)), click on the `+` button next to "Your datasets", on the right-hand side of the screen. -4. Optionally, enter a dataset display name. This name will be used in the GUI and will be exposed in dataset metadata. +2. From the user screen (see [Figure 1b](#user-screen-logged-in)), click on the “Create dataset” button on the right-hand side. -5. Optionally, enter a dataset description. You can use rich text formatting by using Markdown. See [our section about Markdown](../reference/index.md#markdown-support) for details. +
+ +
Figure 1a. The home screen for a logged in user.
+
-6. Optionally, change the access level for the dataset. By default this is set to “Private”. See [dataset access levels](../reference/index.md#access-levels) for more information. +
+ +
Figure 1b. The user screen for a logged in user.
+
-![The “Add dataset” dialog](../../assets/createdataset.png) +### Inside the “Create dataset” dialog -When datasets are Public (see [Access Levels](../reference/index.md#access-levels)), they -automatically expose metadata and are automatically crawled and -indexed by popular search engines (see [Metadata](../publishing-data/index.md#entering-metadata)). +This opens the “Create dataset” dialog (see [Figure 2](#add-dataset-dialog)). + +
+ +
Figure 2. The “Create dataset” dialog
+
+ +In the “Create dataset” dialog, perform the following steps: + +1. Required: Enter a dataset name. A dataset name can contain letters, number, and hyphens. + +2. Optional: Enter a dataset display name. The display name will be shown in the GUI and will be included in dataset metadata. + +3. Optional: Enter a dataset description. This description will be shown in the GUI, and will be included in dataset metadata. The description can be formatted with Markdown. See Section [Markdown](../reference/index.md#markdown-support) for details. + +4. Optional: Change the access level of the dataset. The standard access level is “Private”. See Section [Dataset Access Levels](../reference/index.md#access-levels) for more information. + +This creates a new dataset, and displays the “Add data” page (see Section [Adding data](#adding-data)). + + ## Adding data -Once the dataset is created, the “Add data” view is displayed (see screenshot). In this view data can be added in three ways: +You must first have a dataset, before you can add data. See Section [Creating a new dataset](#creating-a-new-dataset) for more information. + +### Opening the “Add data” pane + +You can open the “Add data” pane in either of the following two ways: + + +1. From the Graphs page, click on the "Import a new graph" button (see [Figure 3](#graphs-page)). This opens the "Add data" pane. +2. After creating a new dataset, the "Add data" pane is automatically opened. + +
+ +
Figure 3. The Graphs page of a dataset.
+
+ +### Inside the “Add data” pane + +The “Add data” pane is now displayed (see [Figure 4](#add-data-pane)). + +
+ +
Figure 4. The “Add data” pane.
+
+ +In the "Add data" pane, choose one of the following approaches for adding data:
-
File upload
-
Select files from your local machine. It is also possible to drag-and-drop local files on the cloud icon with the upward pointing arrow.
-
URL upload
-
Copy/paste a URL that points to an online RDF file.
-
Import
-
Import a dataset that is already published in the same TriplyDB instance.
+
"Add data from an existing dataset"
+
Search for data from a dataset that you have access to in the TriplyDB system. After you have found a dataset, you can choose which graphs to add. See Section Add data from an existing dataset for more details.
+
"Add data from URL"
+
Enter a URL to a data file that is published online. The URL must be publicly accessible. The URL must point to a file that contains RDF or CSV data. See Section Add data from a URL for more details.
+
"Add data from files"
+
Click the cloud icon to open a file explorer window, in which you can select one or more files from your computer. Alternatively, drag-and-drop the local files from your computer onto the cloud icon with the upward pointing arrow. Files must contain RDF or CSV data. See Section Add data from files for more details.
-![The “Add data” view. The three ways in which data can be added are displayed on top of one another.](../../assets/add-data.png) +### Add data from an existing dataset + +The first option for adding data is to add it from datasets that are already published in the same TriplyDB instance. This is done with the “Add data from an existing dataset” field. By typing in this field, a dropdown list of existing datasets is shown (see [Figure 5](#add-data-from-an-existing-dataset-field)). + +
+ +
Figure 5. The dropdown list that shows existing datasets.
+
+ +Once the correct dataset appears in the dropdown list, click it to select it. This will open the "Import from dataset" pane (see [Figure 6](#add-data-from-an-existing-dataset-choose-graphs)). You can choose which graphs to import from the existing dataset. Click "Import graphs" to start importing from an existing dataset. Moments later, the graphs are added to your dataset. + +
+ +
Figure 6. The "Import from dataset" pane.
+
-The “Add data” view is also available for existing datasets: +### Add data from URL -1. Go to the “Graphs” view by clicking on the graph icon in the left-hand sidebar. +The second option for adding data is to add it from an online URL. This is done by entering the URL inside the “Add data from a URL” text field (see [Figure 7](#add-data-from-a-url-field)). After you have entered the URL, click the orange button on the right to start adding data. The data is now being downloaded to your dataset. How long this takes depends on the size of the data and the speed of the remote server where the data is retrieved from. -2. Click the “Import a new graph” button. +
+ +
Figure 7. The "Add data from URL" field.
+
+Only URLs that contain supported data formats will be added. See Section [Supported data formats](#supported-data-formats) for more information. -### Adding data: File upload +### Add data from files -The file upload feature allows you to upload RDF files from your local machine. This can be done in either of two ways: +The third option for adding data is to add it from files that are on your computer. This can be done in two ways: -- Click on the cloud icons to open a dialog that allows local RDF files from your machine to be selected. +1. Click the cloud icon to open a file finder dialog. Here you can select one or more files from your computer (see [Figure 8](#add-data-from-files-dialog)). The file finder dialog that opens, depends on your Operating System. In [Figure 8](#add-data-from-files-dialog), the Windows file finder dialog is shown. +2. Drag-and-drop one or more files from your computer onto the cloud icon with the upward pointing arrow. -- Drag-and-drop RDF files from your local machine onto the cloud icon. +
+ +
Figure 8. The file finder dialog that is opened when adding data from files.
+
-The following RDF serialization formats are currently supported: +After you have added one of more files, a list of uploaded appears (see [Figure 9](#add-data-from-files-job)). You can add or remove more files, until you have the complete upload job configured. Once the list of files is complete, you can click "Import from files" to start adding data from files. How long this takes depends on the size of the data. -| **Serialization Format** | **File extension** | -| ------------------------ | ---------------------- | -| N-Quads | `.nq` | -| N-Triples | `.nt` | -| RDF/XML | `.rdf`, `.owl`, `.owx` | -| TriG | `.trig` | -| Turtle | `.ttl`, `.n3` | -| JSON-LD | `.jsonld`, `.json` | +
+ +
Figure 9. The list of uploaded files in the "Add data from files" pane.
+
-Up to 1,000 separate files can be uploaded in one go. It is also possible to upload compressed files and archives. When the number of files exceeds 1,000, it is more efficient to combine them in archives and upload those. This allows an arbitrary number of files to be uploaded. The following archive/compression formats are currently supported: +Only files that contain supported data formats will be added. See Section [Supported data formats](#supported-data-formats) for more information. -| **Format** | **File extensions** | -| ---------- | ------------------- | -| gzip | `.gz` | -| bzip2 | `.bz2` | -| tar | `tar` | -| XZ | `.xz` | -| ZIP | `.zip` | +### Supported data formats -### Adding data by URL upload +Files must contain RDF and/or CSV data, and must use one of the supported file name extensions: -The second option for adding data is to include it from an online URL location. This is done by entering the URL inside the “Add data from a URL” text field. +| **Serialization Format** | **File name extension** | +| ------------------------ | ----------------------- | +| N-Quads | `.nq` | +| N-Triples | `.nt` | +| RDF/XML | `.rdf`, `.owl`, `.owx` | +| TriG | `.trig` | +| Turtle | `.ttl`, `.n3` | +| JSON-LD | `.jsonld`, `.json` | -### Adding data by import +It is possible to upload up to 1,000 separate files in this way. When you have a lot of files and/or large files, it is better to compress them into an archive format. This allows an any number of files of any size to be uploaded. The following archive/compression formats are supported: -The third option for adding data is to import from datasets that are published in the same TriplyDB instance. This is done with the “Add data from an existing dataset” dropdown list (see screenshot). +| **Archive format** | **File name extension** | +| ------------------ | ----------------------- | +| gzip | `.gz` | +| bzip2 | `.bz2` | +| tar | `tar` | +| XZ | `.xz` | +| ZIP | `.zip` | ### Adding malformed data