Skip to content

Commit

Permalink
Merge pull request #280 from kuzudb/spill-to-disk-docs
Browse files Browse the repository at this point in the history
Add spill to disk setting doc
  • Loading branch information
ray6080 authored Nov 11, 2024
2 parents b7741b1 + 8cd0ea5 commit 490f21f
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 0 deletions.
8 changes: 8 additions & 0 deletions src/content/docs/cypher/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ configuration **cannot** be used with other query clauses, such as `RETURN`.
| `PROGRESS_BAR_TIME` | show progress bar after time in ms | 1000 |
| `CHECKPOINT_THRESHOLD` | the WAL size threshold in bytes at which to automatically trigger a checkpoint | 16777216 (16MB) |
| `WARNING_LIMIT` | The maximum number of warnings that can be stored in a single connection | 8192 |
| `SPILL_TO_DISK_TMP_FILE` | The location of the temporary file to use to store data if there is not enough memory during a copy | `copy.tmp` inside the database directory |

### Database configuration
| Option | Description | Default |
Expand Down Expand Up @@ -74,3 +75,10 @@ CALL checkpoint_threshold=16777216;
```cypher
CALL warning_limit=1024;
```

#### Configure Spill to disk temporary file
```cypher
CALL spill_to_disk_tmp_file="/path/to/tmp/file";
# Disables spilling to disk
CALL spill_to_disk_tmp_file="";
```
12 changes: 12 additions & 0 deletions src/content/docs/import/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -112,3 +112,15 @@ The above approach only works when the input data source contains *fewer* column
node table being copied into. If the node table had fewer columns than the data source, it would
lead to a Runtime error because of a column number mismatch.
:::

## Importing large REL tables

Importing large REL tables can use significant amounts of memory. If the memory used to prepare the
imported REL table data comes close to exceeding the buffer pool size,
Kùzu will spill some of this data to a `copy.tmp` file in the database directory.
The location of this file can be customized using the
[`spill_to_disk_tmp_file`](/cypher/configuration#configure-spill-to-disk-temporary-file) setting.
You can also disable spilling to disk by setting `spill_to_disk_tmp_file` to an empty string.

This feature is disabled by default for in-memory databases and databases using file system extensions,
but can be enabled for these cases by setting a custom `spill_to_disk_tmp_file` path.

0 comments on commit 490f21f

Please sign in to comment.