diff --git a/README.rst b/README.rst index f3a42fc095b..50313d9a372 100644 --- a/README.rst +++ b/README.rst @@ -5,7 +5,7 @@ gallery-dl *gallery-dl* is a command-line program to download image galleries and collections from several image hosting sites (see `Supported Sites`_). It is a cross-platform tool with many configuration options -and powerful filenaming capabilities. +and powerful `filenaming capabilities `_. |pypi| |build| |gitter| @@ -322,6 +322,7 @@ To authenticate with a ``mastodon`` instance, run *gallery-dl* with .. _gallery-dl-example.conf: https://github.com/mikf/gallery-dl/blob/master/docs/gallery-dl-example.conf .. _configuration.rst: https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst .. _Supported Sites: https://github.com/mikf/gallery-dl/blob/master/docs/supportedsites.md +.. _Formatting: https://github.com/mikf/gallery-dl/blob/master/docs/formatting.md .. _Python: https://www.python.org/downloads/ .. _PyPI: https://pypi.org/ diff --git a/docs/configuration.rst b/docs/configuration.rst index 027cc138255..7572f9b23dc 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -482,7 +482,7 @@ Default ``"None"`` Description Default value used for missing or undefined keyword names in - format strings. + `format strings`_. extractor.*.category-transfer @@ -542,7 +542,9 @@ Type Example ``"{id}_{offset}"`` Description - An alternative `format string`_ to build archive IDs with. + An alternative `format string`__ to build archive IDs with. + +.. __: https://docs.python.org/3/library/string.html#format-string-syntax extractor.*.postprocessors @@ -708,7 +710,7 @@ Description extractor.*.date-format ----------------------------- +----------------------- Type ``string`` Default @@ -3028,13 +3030,13 @@ Description .. _Last-Modified: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.29 .. _datetime: https://docs.python.org/3/library/datetime.html#datetime-objects .. _datetime.max: https://docs.python.org/3/library/datetime.html#datetime.datetime.max -.. _format string: https://docs.python.org/3/library/string.html#formatstrings -.. _format strings: https://docs.python.org/3/library/string.html#formatstrings .. _strptime: https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior .. _webbrowser.open(): https://docs.python.org/3/library/webbrowser.html .. _mature_content: https://www.deviantart.com/developers/http/v1/20160316/object/deviation .. _Authentication: https://github.com/mikf/gallery-dl#authentication .. _OAuth: https://github.com/mikf/gallery-dl#oauth +.. _format string: https://github.com/mikf/gallery-dl/blob/master/docs/formatting.md +.. _format strings: https://github.com/mikf/gallery-dl/blob/master/docs/formatting.md .. _youtube-dl: https://github.com/ytdl-org/youtube-dl .. _requests.request(): https://requests.readthedocs.io/en/master/api/#requests.request .. _timeout: https://requests.readthedocs.io/en/master/user/advanced/#timeouts diff --git a/docs/formatting.md b/docs/formatting.md new file mode 100644 index 00000000000..66948c127ca --- /dev/null +++ b/docs/formatting.md @@ -0,0 +1,62 @@ +# String Formatting + +Format strings in gallery-dl follow the general rules of [`str.format()`](https://docs.python.org/3/library/string.html#format-string-syntax) ([PEP 3101](https://www.python.org/dev/peps/pep-3101/)) plus several extras. + +The syntax for replacement fields is `{!:}`, where `!` and `:` are both optional and can be used to specify how the value selected by `` should be transformed. + + +## Field Names + +Field names select the metadata value to use in a replacement field. + +While simple names are usually enough, more complex forms like accessing values by attribute, element index, or slicing are also supported. + +| | Example | Result | +| --- | --- | --- | +| Name | `{title}` | `Hello World` | +| Element Index | `{title[6]}` | `W` | +| Slicing | `{title[3:8]}` | `lo Wo` | +| Alternatives | `{empty\|title}` | `Hello World` | +| Element Access | `{user[name]}` | `John Doe` | +| Attribute Access | `{extractor.url}` | `https://example.org/` | + +All of these methods can be combined as needed. For example `{title[24]|empty|extractor.url[15:-1]}` would result in `.org`. + + +## Conversions + +Conversion specifiers allow to *convert* the value to a different form or type. Such a specifier must only consist of 1 character. gallery-dl supports the default three (`s`, `r`, `a`) as well as several others: + +| Conversion | Description | Example | Result | +|:---:| --- | --- | --- | +| `l` | Convert a string to lowercase | `{foo!l}` | `foo bar` +| `u` | Convert a string to uppercase | `{foo!u}` | `FOO BAR` +| `c` | Capitalize a string, i.e. convert the first character to uppercase and all others to lowercase | `{foo!c}` | `Foo bar` +| `C` | Capitalize each word in a string | `{foo!C}` | `Foo Bar` +| `t` | Trim a string, i.e. remove leading and trailing whitespace characters | `{bar!t}` | `FooBar` +| `T` | Convert a `datetime` object to a unix timestamp | `{date!T}` | `1262304000` +| `d` | Convert a unix timestamp to a `datetime` object | `{created!d}` | `2010-01-01 00:00:00` +| `s` | Convert value to [`str`](https://docs.python.org/3/library/stdtypes.html#text-sequence-type-str) | `{tags!s}` | `['sun', 'tree', 'water']` +| `S` | Convert value to `str` and provide a human-readable representation for lists | `{tags!S}` | `sun, tree, water` +| `r` | Convert value to `str` using [`repr()`](https://docs.python.org/3/library/functions.html#repr) | +| `a` | Convert value to `str` using [`ascii()`](https://docs.python.org/3/library/functions.html#ascii) | + + +## Format Specifiers + +Format specifiers can be used for advanced formatting by using the options provided by Python (see [Format Specification Mini-Language](https://docs.python.org/3/library/string.html#format-specification-mini-language)) like zero-filling a number (`{num:>03}`) or formatting a [`datetime`](https://docs.python.org/3/library/datetime.html#datetime.datetime) object (`{date:%Y%m%d}`), or with gallery-dl's extra formatting specifiers: + +| Format Specifier | Description | Example | Result | +| --- | --- | --- | --- | +| `?//` | Adds `` and `` to the actual value if it evaluates to `True`. Otherwise the whole replacement field becomes an empty string. | `{foo:?[/]/}`
`{empty:?[/]/}` | `[Foo Bar]`
` ` +| `L//` | Replaces the entire output with `` if its length exceeds `` | `{foo:L15/long/}`
`{foo:L3/long/}` | `Foo Bar`
`long`| +| `J/` | Concatenates elements of a list with `` using [`str.join()`](https://docs.python.org/3/library/stdtypes.html#str.join) | `{tags:J - /}` | `sun - tree - water` | +| `R//` | Replaces all occurrences of `` with `` using [`str.replace()`](https://docs.python.org/3/library/stdtypes.html#str.replace) | `{foo:Ro/()/}` | `F()() Bar` | + +All special format specifiers (`?`, `L`, `J`, `R`) can be chained and combined with one another, but must always come before any standard format specifiers: + +For example `{foo:?//RF/B/Ro/e/> 10}` -> `   Bee Bar` +- `?//` - Test if `foo` has a value +- `RF/B/` - Replace `F` with `B` +- `Ro/e/` - Replace `o` with `e` +- `> 10` - Left-fill the string with spaces until it is 10 characters long