Docs: logging & troubleshooting pages

docs/internals.md → docs/advanced/internals.md

@@ -1,4 +1,4 @@
-# igir Internals
+# Internal Operations
 
 Information about the inner workings of `igir`.
 
@@ -8,20 +8,20 @@ Information about the inner workings of `igir`.
 
 1. Scan each DAT input path for every file and parse them, if provided (`--dat <path>`)
 2. Scan each ROM input path for every file (`--input <path>`)
-   - Detect headers in those files, if applicable (see [header docs](roms/headers.md))
-3. Scan each patch input path for every file (`--patch <path>`) (see [patching docs](roms/patching.md))
+   - Detect headers in those files, if applicable (see [header docs](../roms/headers.md))
+3. Scan each patch input path for every file (`--patch <path>`) (see [patching docs](../roms/patching.md))
 4. Then for each DAT:
-   - Parent/clone information is inferred if the DAT has none (see [DATs docs](./dats/processing.md#parentclone-inference))
-   - Parent/clone ROMs sets are merged or split (`--merge-roms <type>`) (see [arcade docs](./usage/arcade.md))
-   - ROMs in the DAT are filtered to only those desired (`--filter-*` options) (see [filtering & preference docs](roms/filtering-preferences.md))
+   - Parent/clone information is inferred if the DAT has none (see [DATs docs](../dats/processing.md#parentclone-inference))
+   - Parent/clone ROMs sets are merged or split (`--merge-roms <type>`) (see [arcade docs](../usage/arcade.md))
+   - ROMs in the DAT are filtered to only those desired (`--filter-*` options) (see [filtering & preference docs](../roms/filtering-preferences.md))
    - Input files are matched to ROMs in the DAT
    - Patch files are matched to ROMs found
-   - ROM preferences are applied (`--single`, see [filtering & preference docs](roms/filtering-preferences.md#preferences-for-1g1r))
+   - ROM preferences are applied (`--single`, see [filtering & preference docs](../roms/filtering-preferences.md#preferences-for-1g1r))
    - ROMs are combined (`--zip-dat-name`)
    - ROMs are written to the output directory, if specified (`copy`, `move`, `symlink`)
      - Written ROMs are tested for accuracy, if specified (`test`)
-   - A "dir2dat" DAT is created, if specified (`dir2dat`) (see [dir2dat docs](./dats/dir2dat.md))
-   - A "fixdat" is created, if specified (`fixdat`) (see [fixdats docs](./dats/fixdats.md))
+   - A "dir2dat" DAT is created, if specified (`dir2dat`) (see [dir2dat docs](../dats/dir2dat.md))
+   - A "fixdat" is created, if specified (`fixdat`) (see [fixdats docs](../dats/fixdats.md))
 5. "Moved" input ROMs are deleted (`move`)
-6. Unknown files are recycled from the output directory, if specified (`clean`, see [cleaning docs](output/cleaning.md))
-7. An output report is written to the output directory, if specified (`report`, see [reporting docs](output/reporting.md))
+6. Unknown files are recycled from the output directory, if specified (`clean`, see [cleaning docs](../output/cleaning.md))
+7. An output report is written to the output directory, if specified (`report`, see [reporting docs](../output/reporting.md))

docs/advanced/logging.md

@@ -0,0 +1,55 @@
+# Logging
+
+By default, `igir` will print the following log levels:
+
+- `ERROR`: an unexpected error has prevented an intended [command](../commands.md)
+- `WARN`: a preventable error has prevented an intended [command](../commands.md)
+
+There are additional levels of verbosity that can be enabled with the `-v` flag:
+
+- **`INFO` (`-v`): print information about actions taken.**
+
+  This includes:
+
+  - Files being copied, zipped, and symlinked
+  - [dir2dat](../dats/dir2dat.md) files being created
+  - [Fixdat](../dats/fixdats.md) files being created
+  - Input files deleted after being moved
+  - Output files being [cleaned](../output/cleaning.md)
+  - [Report](../output/reporting.md) files being created
+
+  Usage:
+
+  ```shell
+  igir [commands..] [options] -v
+  ```
+
+  This level is helpful to turn on if you want to know every action that is resulting in a file being created, modified, or deleted.
+
+- **`DEBUG` (`-vv`): print information about actions taken and skipped.**
+
+  This includes:
+
+  - Everything from the `INFO` level above
+  - Files skipped from being copied, zipped, or symlinked because the output file exists and an `--overwrite` option wasn't provided
+  - [Fixdat](../dats/fixdats.md) files skipped from being created because all games were found
+
+  Usage:
+
+  ```shell
+  igir [commands..] [options] -v
+  ```
+
+  This level is helpful to turn on if you want debug why an action didn't take place.
+
+- **`TRACE` (`-vvv`): print information about actions taken, skipped, and additional information that can be helpful to debug issues.**
+
+  Usage:
+
+  ```shell
+  igir [commands..] [options] -v
+  ```
+
+  !!! note
+
+      Trace logs are required when submitting [bug reports](https://github.com/emmercm/igir/issues/new/choose) as they include information that can help diagnose your unique situation!

docs/advanced/troubleshooting.md

@@ -0,0 +1,49 @@
+# Troubleshooting
+
+See the sibling page about [logging](./logging.md) for more information to help troubleshoot issues.
+
+## Fatal crashes
+
+### Heap space out-of-memory
+
+These types of crashes will print something similar to this to the console:
+
+```text
+<--- Last few GCs --->
+
+[5789:0x3f6eca0]    79353 ms: Scavenge 2020.1 (2075.5) -> 2016.5 (2076.5) MB, 9.57 / 0.00 ms  (average mu = 0.566, current mu = 0.422) allocation failure;
+[5789:0x3f6eca0]    79403 ms: Scavenge 2021.2 (2076.5) -> 2018.3 (2079.5) MB, 11.10 / 0.00 ms  (average mu = 0.566, current mu = 0.422) allocation failure;
+[5789:0x3f6eca0]    80721 ms: Scavenge 2024.0 (2079.5) -> 2020.6 (2097.0) MB, 1280.87 / 0.00 ms  (average mu = 0.566, current mu = 0.422) allocation failure;
+
+
+<--- JS stacktrace --->
+
+FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
+----- Native stack trace -----
+
+ 1: 0xbd3a28 node::Abort() [/nix/store/d6lkbndr98lcn8spbqxfq52f2ldvqhks-nodejs-20.11.0/bin/node]
+ 2: 0xaa1af8  [/nix/store/d6lkbndr98lcn8spbqxfq52f2ldvqhks-nodejs-20.11.0/bin/node]
+ 3: 0xdfa1a0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/nix/store/d6lkbndr98lcn8spbqxfq52f2ldvqhks-nodejs-20.11.0/bin/node]
+ 4: 0xdfa574 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/nix/store/d6lkbndr98lcn8spbqxfq52f2ldvqhks-nodejs-20.11.0/bin/node]
+ 5: 0x10197b7  [/nix/store/d6lkbndr98lcn8spbqxfq52f2ldvqhks-nodejs-20.11.0/bin/node]
+ 6: 0x1031839 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/nix/store/d6lkbndr98lcn8spbqxfq52f2ldvqhks-nodejs-20.11.0/bin/node]
+ 7: 0x100b547 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/nix/store/d6lkbndr98lcn8spbqxfq52f2ldvqhks-nodejs-20.11.0/bin/node]
+ 8: 0x100c184 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/nix/store/d6lkbndr98lcn8spbqxfq52f2ldvqhks-nodejs-20.11.0/bin/node]
+ 9: 0xfec37e v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/nix/store/d6lkbndr98lcn8spbqxfq52f2ldvqhks-nodejs-20.11.0/bin/node]
+10: 0x143b35c v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/nix/store/d6lkbndr98lcn8spbqxfq52f2ldvqhks-nodejs-20.11.0/bin/node]
+11: 0x7fe14fed9ef6
+```
+
+The issue is that `igir` ran out of memory likely due to low system limits, large DAT packs, or large ROM collections.
+
+You likely need to process your ROM collection in batches, just be careful when using the [`igir clean` command](../commands.md). If you don't need every DAT from a pack, you can try reducing the number of DATs being processed with the [`--dat-*-regex <pattern>` and `--dat-*-regex-exclude <pattern>` options](../dats/processing.md#dat-filtering) like this:
+
+```shell
+igir [commands..] --dat "*.dat" --dat-name-regex "/nintendo/i"
+```
+
+You can also try increasing V8's "old memory" limit above [Node.js' default (version dependent)](https://medium.com/geekculture/node-js-default-memory-settings-3c0fe8a9ba1) to ~75% of your system's available RAM in megabytes like this:
+
+```shell
+npx --node-options='--max-old-space-size=6144' igir@latest [commands..] [options]
+```

mkdocs.yml

@@ -96,7 +96,9 @@ nav:
           - output/reporting.md
           - output/cleaning.md
       - Advanced:
-          - internals.md
+          - advanced/logging.md
+          - advanced/troubleshooting.md
+          - advanced/internals.md
       - Misc:
           - rom-dumping.md
       - Terms and Conditions:
@@ -130,6 +132,7 @@ plugins:
         'examples.md': 'usage/collection-sorting.md'
         'input/archives.md': 'input/reading-archives.md'
         'input/dats.md': 'dats/introduction.md'
+        'internals.md': 'advanced/internals.md'
         'output/arcade.md': 'usage/arcade.md'
         'reporting.md': 'output/reporting.md'
         'rom-filtering.md': 'roms/filtering-preferences.md'