diff --git a/docs/api/index.md b/docs/api/index.md index 5e2f92491..4d02cd237 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -4,7 +4,7 @@ Most (but not all) endpoints require authentication. API endpoints requiring aut ## Accessing the API documentation -The entire API is documented at http://pi.hole/api/docs and self-hosted by your Pi-hole to match 100% the API versions your local Pi-hole has. Using this locally served API documentation is preferred. In case you don't have Pi-hole installed yet, you can also check out the documentation for all branches online, e.g., [Pi-hole API documentation](https://ftl.pi-hole.net/development-v6/docs/) (branch `development-v6`). Similarly, you can check out the documentation for a specific other branches by replacing `development-v6` with the corresponding branch name. +The entire API is documented at http://pi.hole/api/docs and self-hosted by your Pi-hole to match 100% the API versions your local Pi-hole has. Using this locally served API documentation is preferred. In case you don't have Pi-hole installed yet, you can also check out the documentation for all branches online, e.g., [Pi-hole API documentation](https://ftl.pi-hole.net/master/docs/) (branch `master`). Similarly, you can check out the documentation for a specific other branches by replacing `master` with the corresponding branch name. ## API endpoints diff --git a/docs/database/domain-database/groups.md b/docs/database/domain-database/groups.md index d81428d04..1368ece90 100644 --- a/docs/database/domain-database/groups.md +++ b/docs/database/domain-database/groups.md @@ -15,7 +15,7 @@ Group management is implemented using so-called linking tables. Hence, it is pos - associate domains (and clients!) with any number of groups, - manage lists together with groups, -- use the same groups for black- and whitelisted domains at the same time. +- use the same groups for denylist and allowlist domains at the same time. The linking tables are particularly simple, as they only link group `id`s with list `id`s. As an example, we describe the `domainlist_by_group` table. The `adlist` and `client` linking tables are constructed similarly. diff --git a/docs/database/domain-database/index.md b/docs/database/domain-database/index.md index f92c94da4..25888b725 100644 --- a/docs/database/domain-database/index.md +++ b/docs/database/domain-database/index.md @@ -10,14 +10,14 @@ Pi-hole uses the following priorities when deciding whether to block or allow a 1. Exact allowlist entries 2. Regex allowlist entries -3. Exact blocklist entries +3. Exact denylist entries 4. Subscribed allowlists -5. Subscribed blocklists -6. Regex blocklist entries +5. Subscribed denylist +6. Regex denylist entries ## Domain tables (`domainlist`) -The database stores allow-, and blocklists which are directly relevant for Pi-hole's domain blocking behavior. The `domainlist` table contains all domains on the allow- and blocklists. It has a few extra fields to store data related to a given domain such as the `enabled` state, the dates when the domain was added and when it was last modified, and an optional comment. +The database stores allow-, and denylists which are directly relevant for Pi-hole's domain blocking behavior. The `domainlist` table contains all domains on the allow- and denylists. It has a few extra fields to store data related to a given domain such as the `enabled` state, the dates when the domain was added and when it was last modified, and an optional comment. The date fields are defined as `INTEGER` fields as they expect numerical timestamps also known as *UNIX time*. The `date_added` and `date_modified` fields are initialized with the current timestamp converted to UNIX time. The `comment` field is optional and can be empty. @@ -26,7 +26,7 @@ Pi-hole's *FTL*DNS reads the tables through the various view, omitting any disab Label | Type | Uniqueness enforced | Content ----- | ---- | ------------------- | -------- `id` | integer | Yes | Unique ID for database operations -`type` | integer | No | `0` = exact allowlist,
`1` = exact blocklist,
`2` = regex allowlist,
`3` = regex blocklist +`type` | integer | No | `0` = exact allowlist,
`1` = exact denylist,
`2` = regex allowlist,
`3` = regex denylists `domain` | text | Yes | Domain `enabled` | boolean | No | Flag whether domain should be used by `pihole-FTL`
(`0` = disabled, `1` = enabled) `date_added` | integer | No | Timestamp when domain was added diff --git a/docs/database/query-database.md b/docs/database/query-database.md index aae8bbc8a..e07ab8f48 100644 --- a/docs/database/query-database.md +++ b/docs/database/query-database.md @@ -1,13 +1,13 @@ Pi-hole *FTL*DNS uses the well-known relational database management system SQLite3 as its long-term storage of query data. In contrast to many other database management solutions, *FTL*DNS does not need a server database engine as the database engine is directly embedded in *FTL*DNS. It seems an obvious choice as it is probably the most widely deployed database engine - it is used today by several widespread web browsers, operating systems, and embedded systems (such as mobile phones), among others. Hence, it is rich in supported platforms and offered features. SQLite implements most of the SQL-92 standard for SQL and can be used for high-level queries. The long-term query database was the first database that was added to the Pi-hole project. -We update this database periodically and on the exit of *FTL*DNS (triggered e.g. by a `service pihole-FTL restart`). The updating frequency can be controlled by the parameter [`DBINTERVAL`](../ftldns/configfile.md#dbinterval) and defaults to once per minute. We think this interval is sufficient to protect against data losses due to power failure events. *FTL*DNS needs the database to populate its internal history of the most recent 24 hours. If the database is disabled, *FTL*DNS will show an empty query history after a restart. +We update this database periodically and on the exit of *FTL*DNS (triggered e.g. by a `service pihole-FTL restart`). The updating frequency can be controlled by the parameter `database.DBinterval` and defaults to once per minute. We think this interval is sufficient to protect against data losses due to power failure events. *FTL*DNS needs the database to populate its internal history of the most recent 24 hours. If the database is disabled, *FTL*DNS will show an empty query history after a restart. -The location of the database can be configured by the config parameter [`DBFILE`](../ftldns/configfile.md#dbfile). It defaults to `/etc/pihole/pihole-FTL.db`. If the given file does not exist, *FTL*DNS will create a new (empty) database file. +The location of the database can be configured by the config parameter `files.database`. It defaults to `/etc/pihole/pihole-FTL.db`. If the given file does not exist, *FTL*DNS will create a new (empty) database file. -Another way of controlling the size of the long-term database is by setting a maximum age for log queries to keep using the config parameter [`MAXDBDAYS`](../ftldns/configfile.md#maxdbdays). It defaults to 365 days, i.e. queries that are older than one year get periodically removed to limit the growth of the long-term database file. +Another way of controlling the size of the long-term database is by setting a maximum age for log queries to keep using the config parameter `database.maxDBdays`. It defaults to 365 days, i.e. queries that are older than one year get periodically removed to limit the growth of the long-term database file. -The config parameter [`DBIMPORT`](../ftldns/configfile.md#dbimport) controls whether `FTL` loads information from the database on startup. It needs to do this to populate the internal data structure with the most recent history. However, as importing from the database on disk can delay FTL on very large deploys, it can be disabled using this option. +The config parameter `database.DBimport` controls whether `FTL` loads information from the database on startup. It needs to do this to populate the internal data structure with the most recent history. However, as importing from the database on disk can delay FTL on very large deploys, it can be disabled using this option. --- @@ -65,9 +65,9 @@ The content and type of the `additional_info` row depends on the status of the g If a query was blocked due to a CNAME inspection (status 9, 10, 11), this field contains the domain which was the reason for blocking the entire CNAME chain (text). -##### Query influenced by a black- or whitelist entry {#additional_info_list data-toc-label='domainlist_id'} +##### Query influenced by a deny or allowlist entry {#additional_info_list data-toc-label='domainlist_id'} -If a query was influenced by a black- or whitelist entry, this field contains the ID of the corresponding entry in the [`domainlist`](domain-database/index.md#domain-tables-domainlist) table. +If a query was influenced by a deny or allowlist entry, this field contains the ID of the corresponding entry in the [`domainlist`](domain-database/index.md#domain-tables-domainlist) table. ### Counters table @@ -118,18 +118,18 @@ ID | Status | | Details 1 | Blocked | ❌ | Domain contained in [gravity database](domain-database/index.md#gravity-tables-gravity-and-antigravity) 2 | Allowed | ✅ | Forwarded 3 | Allowed | ✅ | Replied from cache -4 | Blocked | ❌ | Domain matched by a [regex blacklist](domain-database/index.md#domain-tables-domainlist) filter -5 | Blocked | ❌ | Domain contained in [exact blacklist](domain-database/index.md#domain-tables-domainlist) +4 | Blocked | ❌ | Domain matched by a [regex denylist](domain-database/index.md#domain-tables-domainlist) filter +5 | Blocked | ❌ | Domain contained in [exact denylist](domain-database/index.md#domain-tables-domainlist) 6 | Blocked | ❌ | By upstream server (known blocking page IP address) 7 | Blocked | ❌ | By upstream server (`0.0.0.0` or `::`) 8 | Blocked | ❌ | By upstream server (`NXDOMAIN` with `RA` bit unset) 9 | Blocked | ❌ | Domain contained in [gravity database](domain-database/index.md#gravity-tables-gravity-and-antigravity)
*Blocked during deep CNAME inspection* -10 | Blocked | ❌ | Domain matched by a [regex blacklist](domain-database/index.md#domain-tables-domainlist) filter
*Blocked during deep CNAME inspection* -11 | Blocked | ❌ | Domain contained in [exact blacklist](domain-database/index.md#domain-tables-domainlist)
*Blocked during deep CNAME inspection* +10 | Blocked | ❌ | Domain matched by a [regex denylist](domain-database/index.md#domain-tables-domainlist) filter
*Blocked during deep CNAME inspection* +11 | Blocked | ❌ | Domain contained in [exact denylist](domain-database/index.md#domain-tables-domainlist)
*Blocked during deep CNAME inspection* 12 | Allowed | ✅ | Retried query 13 | Allowed | ✅ | Retried but ignored query (this may happen during ongoing DNSSEC validation) 14 | Allowed | ✅ | Already forwarded, not forwarding again -15 | Blocked | ❌ | Blocked (database is busy)
How these queries are handled can be [configured](../ftldns/configfile.md#reply_when_busy) +15 | Blocked | ❌ | Blocked (database is busy)
How these queries are handled can be configured (dns.replyWhenBusy) 16 | Blocked | ❌ | Blocked (special domain)
*E.g. Mozilla's canary domain and Apple's Private Relay domains*
Handling can be [configured](../ftldns/configfile.md) 17 | Allowed | ✅⌛ | Replied from *stale* cache 18 | Blocked | ❌ | By upstream server (EDE 15) diff --git a/docs/ftldns/blockingmode.md b/docs/ftldns/blockingmode.md index 7199fe2af..1ecd29580 100644 --- a/docs/ftldns/blockingmode.md +++ b/docs/ftldns/blockingmode.md @@ -8,12 +8,9 @@ Pi-hole *FTL*DNS currently supports the following modes for blocking queries: Each mode has their advantages and drawbacks which will be discussed in detail below. -!!! note - In order to configure a blocking mode, you must edit the *FTL*DNS configuration file (`/etc/pihole/pihole-FTL.conf`). Once you've made any changes to the blocking mode, you must restart Pi-hole with `pihole restartdns`. - ## Pi-hole's unspecified IP or NULL blocking mode -In `NULL` mode, which is both the default and recommended mode for Pi-hole *FTL*DNS, blocked queries will be answered with the "unspecified address" (`0.0.0.0` or `::`). The "unspecified address" is a reserved IP address specified by [RFC 3513 - Internet Protocol Version 6 (IPv6) Addressing Architecture, section 2.5.2](https://tools.ietf.org/html/rfc3513#section-2.5.2). If no mode is explicitly defined in the configuration file, Pi-hole will default to this mode. To set this mode explicitly, set `BLOCKINGMODE=NULL` in `/etc/pihole/pihole-FTL.conf`. +In `NULL` mode, which is both the default and recommended mode for Pi-hole *FTL*DNS, blocked queries will be answered with the "unspecified address" (`0.0.0.0` or `::`). The "unspecified address" is a reserved IP address specified by [RFC 3513 - Internet Protocol Version 6 (IPv6) Addressing Architecture, section 2.5.2](https://tools.ietf.org/html/rfc3513#section-2.5.2). If no mode is explicitly defined in the configuration file, Pi-hole will default to this mode. To set this mode explicitly, run `pihole-FTL --config dns.blocking.mode NULL` A blocked query would look like the following: @@ -39,7 +36,7 @@ doubleclick.net. 2 IN AAAA :: ## Pi-hole's IP (IPv6 NODATA) blocking mode -In `IP-NODATA-AAAA` mode, blocked queries will be answered with the local IPv4 addresses of your Pi-hole (see [BLOCK_IP4](configfile.md#block_ipv4) for additional options). Blocked AAAA queries will be answered with `NODATA-IPV6` and clients will only try to reach your Pi-hole over its static IPv4 address. To set this mode explicitly, set `BLOCKINGMODE=IP-NODATA-AAAA` in `/etc/pihole/pihole-FTL.conf`. +In `IP-NODATA-AAAA` mode, blocked queries will be answered with the local IPv4 addresses of your Pi-hole. Blocked AAAA queries will be answered with `NODATA-IPV6` and clients will only try to reach your Pi-hole over its static IPv4 address. To set this mode explicitly, run `pihole-FTL --config dns.blocking.mode IP_NODATA_AAAA`. Assuming your Pi-hole server is at `192.168.1.42`, then a blocked query would look like the following: @@ -61,7 +58,9 @@ doubleclick.net. 2 IN A 192.168.1.42 ## Pi-hole's full IP blocking mode -In `IP` mode, blocked queries will be answered with the local IP addresses of your Pi-hole (see [BLOCK_IP4](configfile.md#block_ipv4) and [BLOCK_IP6](configfile.md#block_ipv6) for additional options). To set this mode explicitly, set `BLOCKINGMODE=IP` in `/etc/pihole/pihole-FTL.conf`. +In `IP` mode, blocked queries will be answered with the local IP addresses of your Pi-hole. To set this mode explicitly, run `pihole-FTL --config dns.blocking.mode IP`. + +The IP that is returned is automatically determined by FTL, however it can also be set with `reply.blocking.IPv4` and `reply.blocking.IPv6` FTL options. A blocked query would look like the following: @@ -85,7 +84,7 @@ doubleclick.net. 2 IN AAAA fda2:2001:4756:0:ab27:beff:ef37: ## Pi-hole's NXDOMAIN blocking mode -In `NXDOMAIN` mode, blocked queries will be answered with an empty response (i.e., there won't be an *answer* section) and status `NXDOMAIN`. A `NXDOMAIN` response should indicate that there is *no such domain* to the client making the query. To set this mode explicitly, set `BLOCKINGMODE=NXDOMAIN` in `/etc/pihole/pihole-FTL.conf`. +In `NXDOMAIN` mode, blocked queries will be answered with an empty response (i.e., there won't be an *answer* section) and status `NXDOMAIN`. A `NXDOMAIN` response should indicate that there is *no such domain* to the client making the query. To set this mode explicitly, run `pihole-FTL --config dns.blocking.mode NXDOMAIN`. A blocked query would look like the following: @@ -98,7 +97,7 @@ A blocked query would look like the following: ## Pi-hole's NODATA blocking mode -In `NODATA` mode, blocked queries will be answered with an empty response (no answer section) and status `NODATA`. A `NODATA` response indicates that the domain exists, but there is no record for the requested query type. To set this mode explicitly, set `BLOCKINGMODE=NODATA` in `/etc/pihole/pihole-FTL.conf`. +In `NODATA` mode, blocked queries will be answered with an empty response (no answer section) and status `NODATA`. A `NODATA` response indicates that the domain exists, but there is no record for the requested query type. To set this mode explicitly, run `pihole-FTL --config dns.blocking.mode NODATA`. A blocked query would look like the following: diff --git a/docs/ftldns/configfile.md b/docs/ftldns/configfile.md index 57c4d1333..e13b32818 100644 --- a/docs/ftldns/configfile.md +++ b/docs/ftldns/configfile.md @@ -1,402 +1,7 @@ -You can create a file `/etc/pihole/pihole-FTL.conf` that will be read by *FTL*DNS on startup. +# Configuration -!!! info - Comments need to start with `#;` to avoid issues with PHP and `bash` reading this file. (See [https://github.com/pi-hole/pi-hole/pull/4081](https://github.com/pi-hole/pi-hole/pull/4081) for more details) +The configuration file can be found at `/etc/pihole/pihole.toml`, all of the options are documented within this file. -Possible settings (**the option shown first is the default**): +The file can be edited directly, however you can also use the command line option (`pihole-FTL --config`) or the web interface. ---- - -### DNS settings - -#### `BLOCKINGMODE=NULL|IP-NODATA-AAAA|IP|NXDOMAIN` {#blocking_mode data-toc-label='Blocking Mode'} - -How should `FTL` reply to blocked queries?
-**[More details](blockingmode.md)** - -#### `CNAME_DEEP_INSPECT=true|false` (PR [#663](https://github.com/pi-hole/FTL/pull/663)) {#cname_deep_inspect data-toc-label='Deep CNAME inspection'} - -Use this option to disable deep CNAME inspection. This might be beneficial for very low-end devices. - -#### `BLOCK_ESNI=true|false` (PR [#733](https://github.com/pi-hole/FTL/pull/733)) {#block_esni data-toc-label='ESNI blocking'} - -[Encrypted Server Name Indication (ESNI)](https://tools.ietf.org/html/draft-ietf-tls-esni-06) is certainly a good step into the right direction to enhance privacy on the web. It prevents on-path observers, including ISPs, coffee shop owners and firewalls, from intercepting the TLS Server Name Indication (SNI) extension by encrypting it. This prevents the SNI from being used to determine which websites users are visiting. - -ESNI will obviously cause issues for `pixelserv-tls` which will be unable to generate matching certificates on-the-fly when it cannot read the SNI. Cloudflare and Firefox are already enabling ESNI. -According to the IEFT draft (link above), we can easily restore `piselserv-tls`'s operation by replying `NXDOMAIN` to `_esni.` subdomains of blocked domains as this mimics a "not configured for this domain" behavior. - -#### `EDNS0_ECS=true|false` (PR [#851](https://github.com/pi-hole/FTL/pull/851)) {#block_edns0_ecs data-toc-label='EDNS ECS overwrite'} - -Should we overwrite the query source when client information is provided through EDNS0 client subnet (ECS) information? -This allows Pi-hole to obtain client IPs even if they are hidden behind the NAT of a router. - -This feature has been requested and discussed on [Discourse](https://discourse.pi-hole.net/t/support-for-add-subnet-option-from-dnsmasq-ecs-edns0-client-subnet/35940), where further information on how to use it can be found. - -#### `RATE_LIMIT=1000/60` (PR [#1052](https://github.com/pi-hole/FTL/pull/1052)) {#rate_limit data-toc-label='Query rate limiting'} - -Control FTL's query rate-limiting. Rate-limited queries are answered with a `REFUSED` reply and not further processed by FTL. - -The default settings for FTL's rate-limiting are to permit no more than `1000` queries in `60` seconds. Both numbers can be customized independently. -It is important to note that rate-limiting is happening on a *per-client* basis. Other clients can continue to use FTL while rate-limited clients are short-circuited at the same time. - -For this setting, both numbers, the maximum number of queries within a given time, **and** the length of the time interval (seconds) have to be specified. For instance, if you want to set a rate limit of 1 query per hour, the option should look like `RATE_LIMIT=1/3600`. -The time interval is relative to when FTL has finished starting (start of the daemon + possible delay by `DELAY_STARTUP`) then it will advance in steps of the rate-limiting interval. If a client reaches the maximum number of queries it will be blocked until **the end of the current interval**. This will be logged to `/var/log/pihole/FTL.log`, e.g. `Rate-limiting 10.0.1.39 for at least 44 seconds`. If the client continues to send queries while being blocked already and this number of queries during the blocking exceeds the limit the client will continue to be blocked **until the end of the next interval** (`FTL.log` will contain lines like `Still rate-limiting 10.0.1.39 as it made additional 5007 queries`). As soon as the client requests less than the set limit, it will be unblocked (`Ending rate-limitation of 10.0.1.39`). - -Rate-limiting may be disabled altogether by setting `RATE_LIMIT=0/0` (this results in the same behavior as before FTL v5.7). - -#### `LOCAL_IPV4=` (unset by default, PR [#1293](https://github.com/pi-hole/FTL/pull/1293)) {#local_ipv4 data-toc-label='Force local A reply'} - -By default, `FTL` determines the address of the interface a query arrived on and uses this address for replying to `A` queries with the most suitable address for the requesting client. This setting can be used to use a fixed, rather than the dynamically obtained, address when Pi-hole responds to the following names: - -- `pi.hole` -- `` -- `pi.hole.` -- `.` - -#### `LOCAL_IPV6=` (unset by default, PR [#1293](https://github.com/pi-hole/FTL/pull/1293)) {#local_ipv6 data-toc-label='Force local AAAA reply'} - -Used to overwrite the IP address for local `AAAA` queries. See [`LOCAL_IPV4`](#local_ipv4) for details on when this setting is used. - -#### `BLOCK_IPV4=` (unset by default, PR [#1293](https://github.com/pi-hole/FTL/pull/1293)) {#block_ipv4 data-toc-label='Force blocked A reply'} - -By default, `FTL` determines the address of the interface a query arrived on and uses this address for replying to `A` queries with the most suitable address for the requesting client. This setting can be used to use a fixed, rather than the dynamically obtained, address when Pi-hole responds in the following cases: - -- `IP` blocking mode is used and this query is to be blocked -- A regular expression with the [`;reply=IP` regex extension](../regex/pi-hole.md#specify-reply-type) is used - -#### `BLOCK_IPV6=` (unset by default, PR [#1293](https://github.com/pi-hole/FTL/pull/1293)) {#block_ipv6 data-toc-label='Force blocked AAAA reply'} - -Used to overwrite the IP address for blocked `AAAA` queries. See [`BLOCK_IPV4`](#block_ipv4) for details on when this setting is used. - -#### `REPLY_WHEN_BUSY=DROP|ALLOW|BLOCK|REFUSE` (PR [#1156](https://github.com/pi-hole/FTL/pull/1156) & PR [#1341](https://github.com/pi-hole/FTL/pull/1341)) {#reply_when_busy data-toc-label='Database busy reply'} - -When the gravity database is locked/busy, how should Pi-hole handle queries? - -- `ALLOW` - allow all queries when the database is busy -- `BLOCK` - block all queries when the database is busy. This uses the configured `BLOCKINGMODE` (default `NULL`) -- `REFUSE` - refuse all queries which arrive while the database is busy -- `DROP` - just drop the queries, i.e., never reply to them at all. - -Despite `REFUSE` sounding similar to `DROP`, it turned out that many clients will just immediately retry, causing up to several thousands of queries per second. This does not happen in `DROP` mode. - -#### `MOZILLA_CANARY=true|false` (PR [#1148](https://github.com/pi-hole/FTL/pull/1148)) {#mozilla_canary data-toc-label='Mozilla canary domain handling'} - -Should Pi-hole always replies with `NXDOMAIN` to `A` and `AAAA` queries of `use-application-dns.net` to disable Firefox's automatic DNS-over-HTTP? -This is following the recommendation on [https://support.mozilla.org/en-US/kb/configuring-networks-disable-dns-over-https](https://support.mozilla.org/en-US/kb/configuring-networks-disable-dns-over-https) - - -#### `BLOCK_TTL=2` (PR [#1173](https://github.com/pi-hole/FTL/pull/1173)) {#block_ttl data-toc-label='Blocked domains lifetime'} - -FTL's internal TTL to be handed out for blocked queries. This setting allows users to select a value different from the `dnsmasq` config option `local-ttl`. This seems useful in context of locally used hostnames that are known to stay constant over long times (printers, etc.). - -Note that large values may render whitelisting ineffective due to client-side caching of blocked queries. - -#### `BLOCK_ICLOUD_PR=true|false` (PR [#1171](https://github.com/pi-hole/FTL/pull/1171)) {#icloud_private_relay data-toc-label='iCloud Private Relay domain handling'} - -Should Pi-hole always reply with `NXDOMAIN` to `A` and `AAAA` queries of `mask.icloud.com` and `mask-h2.icloud.com` to disable Apple's iCloud Private Relay to prevent Apple devices from bypassing Pi-hole? -This is following the recommendation on [https://developer.apple.com/support/prepare-your-network-for-icloud-private-relay](https://developer.apple.com/support/prepare-your-network-for-icloud-private-relay) - ---- - -### Statistics settings - -#### `MAXLOGAGE=24.0` {#maxlogage data-toc-label='Max Log Age'} - -Up to how many hours of queries should be imported from the database and logs? Values greater than the hard-coded maximum of 24h need a locally compiled `FTL` with a changed compile-time value. - -#### `PRIVACYLEVEL=0|1|2|3` {#privacylevel data-toc-label='Privacy Level'} - -Which privacy level is used?
-**[More details](privacylevels.md)** - -#### `IGNORE_LOCALHOST=no|yes` {#ignore_localhost data-toc-label='Ignore localhost'} - -Should `FTL` ignore queries coming from the local machine? - -#### `AAAA_QUERY_ANALYSIS=yes|no` {#aaaa_query_analysis data-toc-label='AAAA Query Analysis'} - -Should FTL analyze `AAAA` queries? The DNS server will handle `AAAA` queries the same way, regardless of this setting. All this does is ignoring `AAAA` queries when computing the statistics of Pi-hole. This setting is considered obsolete and will be removed in a future version. - -#### `ANALYZE_ONLY_A_AND_AAAA=false|true` {#analyze_only_a_and_aaaa data-toc-label='Analyze A and AAAA Only'} - -Should `FTL` only analyze A and AAAA queries? - - -#### `SHOW_DNSSEC=true|false` {#show_dnssec data-toc-label='Show automatic DNSSEC queries'} - -Should FTL analyze and include automatically generated DNSSEC queries in the Query Log? - ---- - -### Other settings - -#### `SOCKET_LISTENING=localonly|all` {#socket_listening data-toc-label='Socket Listening'} - -Listen only for local socket connections or permit all connections - -#### `FTLPORT=4711` {#ftlport data-toc-label='FTLDNS Port'} - -On which port should FTL be listening? - -#### `RESOLVE_IPV6=yes|no` {#resolve_ipv6 data-toc-label='Resolve IPV6'} - -Should `FTL` try to resolve IPv6 addresses to hostnames? - -#### `RESOLVE_IPV4=yes|no` {#resolve_ipv4 data-toc-label='Resolve IPV4'} - -Should `FTL` try to resolve IPv4 addresses to hostnames? - -#### `PIHOLE_PTR=PI.HOLE|HOSTNAME|HOSTNAMEFQDN|NONE` (PR [#1111](https://github.com/pi-hole/FTL/pull/1111), [#1164](https://github.com/pi-hole/FTL/pull/1164)) {#pihole_ptr data-toc-label='Pi-hole PTR'} - -Controls whether and how FTL will reply with for address for which a local interface exists. Valid options are: - -- `PI.HOLE` (the default) respond with `pi.hole` -- `HOSTNAME` serve the machine's global hostname -- `HOSTNAMEFQDN` serve the machine's global hostname as fully qualified domain by adding the local suffix. See note below. -- `NONE` Pi-hole will **not** respond automatically on PTR requests to local interface addresses. Ensure `pi.hole` and/or hostname records exist elsewhere. - -Note about `HOSTNAMEFQDN`: If no local suffix has been defined, FTL appends the local domain `.no_fqdn_available`. In this case you should either add `domain=whatever.com` to a custom config file inside `/etc/dnsmasq.d/` (to set `whatever.com` as local domain) or use `domain=#` which will try to derive the local domain from `/etc/resolv.conf` (or whatever is set with `resolv-file`, when multiple `search` directives exist, the first one is used). - -#### `DELAY_STARTUP=0` (PR [#716](https://github.com/pi-hole/FTL/pull/716), PR [1349](https://github.com/pi-hole/FTL/pull/1349)) {#delay_startup data-toc-label='Delay resolver startup'} - -During startup, in some configurations, network interfaces appear only late during system startup and are not ready when FTL tries to bind to them. Therefore, you may want FTL to wait a given amount of time before trying to start the DNS revolver. This setting takes any integer value between 0 and 300 seconds. To prevent delayed startup while the system is already running and FTL is restarted, the delay only takes place within the first 180 seconds (hard-coded) after booting. - -#### `NICE=-10` (PR [#798](https://github.com/pi-hole/FTL/pull/798)) {#nice data-toc-label='Set niceness'} - -Can be used to change the niceness of Pi-hole FTL. Defaults to `-10` and can be -disabled altogether by setting a value of `-999`. - -The nice value is an attribute that can be used to influence the CPU scheduler -to favor or disfavor a process in scheduling decisions. The range of the nice -value varies across UNIX systems. On modern Linux, the range is `-20` (high -priority = not very nice to other processes) to `+19` (low priority). - -#### `MAXNETAGE=[MAXDBDAYS]` (PR [#871](https://github.com/pi-hole/FTL/pull/871)) {#maxnetage data-toc-label='Network table cleaning'} - -IP addresses (and associated host names) older than the specified number of days -are removed to avoid dead entries in the network overview table. This setting -defaults to the same value as `MAXDBDAYS` above but can be changed independently -if needed. - -#### `NAMES_FROM_NETDB=true|false` (PR [#784](https://github.com/pi-hole/FTL/pull/784)) {#names_from_netdb data-toc-label='Load names from network table'} - -Control whether FTL should use the fallback option to try to obtain client names -from checking the network table. This behavior can be disabled -with this option - -Assume an IPv6 client without a host names. However, the network table knows - -though the client's MAC address - that this is the same device where we have a -host name for another IP address (e.g., a DHCP server managed IPv4 address). In -this case, we use the host name associated to the other address as this is the -same device. - -#### `REFRESH_HOSTNAMES=IPV4|ALL|UNKNOWN|NONE` (PR [#953](https://github.com/pi-hole/FTL/pull/953)) {#refresh_hostnames data-toc-label='Refresh hostnames'} - -With this option, you can change how (and if) hourly PTR requests are made to check for changes in client and upstream server hostnames. The following options are available: - -- `REFRESH_HOSTNAMES=IPV4` - Do the hourly PTR lookups only for IPv4 addresses - This is the new default since Pi-hole FTL v5.3.2. It should resolve issues with more and more very short-lived PE IPv6 addresses coming up in a lot of networks. -- `REFRESH_HOSTNAMES=ALL` - Do the hourly PTR lookups for all addresses - This is the same as what we're doing with FTL v5.3(.1). This can create a lot of PTR queries for those with many IPv6 addresses in their networks. -- `REFRESH_HOSTNAMES=UNKNOWN` - Only resolve unknown hostnames - Already existing hostnames are never refreshed, i.e., there will be no PTR queries made for clients where hostnames are known. This also means that known hostnames will not be updated once known. -- `REFRESH_HOSTNAMES=NONE` - Don't do any hourly PTR lookups - This means we look host names up exactly once (when we first see a client) and never again. You may miss future changes of host names. - -#### `PARSE_ARP_CACHE=true|false` (PR [#445](https://github.com/pi-hole/FTL/pull/445)) {#parse_arp_cache data-toc-label='Parse ARP cache'} - -This setting can be used to disable ARP cache processing. When disabled, client identification and the network table will stop working reliably. - -#### `CHECK_LOAD=true|false` (PR [#1249](https://github.com/pi-hole/FTL/pull/1249)) {#check_load data-toc-label='Check system load'} - -Pi-hole is very lightweight on resources. Nevertheless, this does not mean that you should run Pi-hole on a server that is otherwise extremely busy as queuing on the system can lead to unnecessary delays in DNS operation as the system becomes less and less usable as the system load increases because all resources are permanently in use. To account for this, FTL regularly checks the system load. To bring this to your attention, FTL warns about excessive load when the 15 minute system load average exceeds the number of cores. - -This check can be disabled with this setting. - -#### `CHECK_SHMEM=90` (PR [#1249](https://github.com/pi-hole/FTL/pull/1249)) {#check_shmem data-toc-label='Check shared-memory limits'} - -FTL stores history in shared memory to allow inter-process communication with forked dedicated TCP workers. If FTL runs out of memory, it cannot continue to work as queries cannot be analyzed any further. Hence, FTL checks if enough shared memory is available on your system and warns you if this is not the case. - -By default, FTL warns if the shared-memory usage exceeds 90%. You can set any integer limit between `0` to `100` (interpreted as percentages) where `0` means that checking of shared-memory usage is disabled. - -#### `CHECK_DISK=90` (PR [#1249](https://github.com/pi-hole/FTL/pull/1249)) {#check_disk data-toc-label='Check disk space'} - -FTL stores its long-term history in a database file on disk (see [here](../database/index.md)). Furthermore, FTL stores log files (see, e.g., [here](#file_LOGFILE)). - -By default, FTL warns if usage of the disk holding any crucial file exceeds 90%. You can set any integer limit between `0` to `100` (interpreted as percentages) where `0` means that checking of disk usage is disabled. - ---- - -### Long-term database settings - -**[Further details concerning the database](../database/index.md)** - -#### `DBIMPORT=yes|no` {#dbimport data-toc-label='Use database'} - -Should `FTL` load information from the database on startup to be aware of the most recent history? - -#### `MAXDBDAYS=365` {#maxdbdays data-toc-label='Max age'} - -How long should queries be stored in the database? Setting this to `0` disables the database - -#### `DBINTERVAL=1.0` {#dbinterval data-toc-label='Storing Interval'} - -How often do we store queries in FTL's database [minutes]? - -#### `DBFILE=/etc/pihole/pihole-FTL.db` {#dbfile data-toc-label='Database Filename'} - -Specify the path and filename of FTL's SQLite3 long-term database. Setting this to `DBFILE=` disables the database altogether - ---- - -### File options - -#### `LOGFILE=/var/log/pihole/FTL.log` {#file_LOGFILE data-toc-label='Log file'} - -The location of FTL's log file. If you want to move the log file to a different place, also consider [this FAQ article](https://discourse.pi-hole.net/t/moving-the-pi-hole-log-to-another-location-device/2041). - -#### `PIDFILE=/run/pihole-FTL.pid` {#file_PIDFILE data-toc-label='Process identifier file'} - -The file which contains the PID of FTL's main process. - -#### `SOCKETFILE=/run/pihole/FTL.sock` {#file_SOCKETFILE data-toc-label='Socket file'} - -The file containing the socket FTL's API is listening on. - -#### `SETUPVARSFILE=/etc/pihole/setupVars.conf` {#file_SETUPVARSFILE data-toc-label='setupVars file'} - -The config file of Pi-hole containing, e.g., the current blocking status (do not change). - -#### `MACVENDORDB=/etc/pihole/macvendor.db` {#file_MACVENDORDB data-toc-label='MacVendor database file'} - -The database containing MAC -> Vendor information for the network table. - -#### `GRAVITYDB=/etc/pihole/gravity.db` {#file_GRAVITYDB data-toc-label='Gravity database'} - -Specify path and filename of FTL's SQLite3 gravity database. This database contains all domains relevant for Pi-hole's DNS blocking - ---- - -### Debugging options - -#### `DEBUG_ALL=false|true` {#debug_all data-toc-label='All'} - -Enable all debug flags. If this is set to true, all other debug config options are ignored. - -#### `DEBUG_DATABASE=false|true` {#debug_database data-toc-label='Database'} - -Print debugging information about database actions. This prints performed SQL statements as well as some general information such as the time it took to store the queries and how many have been saved to the database. - -#### `DEBUG_NETWORKING=false|true` {#debug_networking data-toc-label='Networking'} - -Prints a list of the detected interfaces on the startup of `pihole-FTL`. Also, prints whether these interfaces are IPv4 or IPv6 interfaces. - -#### `DEBUG_EDNS0=false|true` {#debug_edns0 data-toc-label='EDNS0'} - -Print debugging information about received EDNS(0) data. - -#### `DEBUG_LOCKS=false|true` {#debug_locks data-toc-label='Locks'} - -Print information about shared memory locks. Messages will be generated when waiting, obtaining, and releasing a lock. - -#### `DEBUG_QUERIES=false|true` {#debug_queries data-toc-label='Queries'} - -Print extensive query information (domains, types, replies, etc.). This has always been part of the legacy `debug` mode of `pihole-FTL`. - -#### `DEBUG_FLAGS=false|true` {#debug_flags data-toc-label='Flags'} - -Print flags of queries received by the DNS hooks. Only effective when `DEBUG_QUERIES` is enabled as well. - -#### `DEBUG_SHMEM=false|true` {#debug_shmem data-toc-label='Shared Memory'} - -Print information about shared memory buffers. Messages are either about creating or enlarging shmem objects or string injections. - -#### `DEBUG_GC=false|true` {#debug_gc data-toc-label='Garbage Collection'} - -Print information about garbage collection (GC): What is to be removed, how many have been removed and how long did GC take. - -#### `DEBUG_ARP=false|true` {#debug_arp data-toc-label='Neighbor parsing'} - -Print information about ARP table processing: How long did parsing take, whether read MAC addresses are valid, and if the `macvendor.db` file exists. - -#### `DEBUG_REGEX=false|true` {#debug_regex data-toc-label='Regular expressions'} - -Controls if *FTL*DNS should print extended details about regex matching into `FTL.log`. - -**[More details](../regex/index.md)** - -#### `DEBUG_API=false|true` {#debug_api data-toc-label='API'} - -Print extra debugging information during API calls. - -#### `DEBUG_OVERTIME=false|true` {#debug_overtime data-toc-label='Over Time Data'} - -Print information about overTime memory operations, such as initializing or moving overTime slots. - -#### `DEBUG_STATUS=false|true` {#debug_status data-toc-label='Query status'} - -Print information about status changes for individual queries. This can be useful to identify unexpected `unknown` queries. - -#### `DEBUG_CAPS=false|true` {#debug_caps data-toc-label='Linux capabilities'} - -Print information about capabilities granted to the pihole-FTL process. The current capabilities are printed on receipt of `SIGHUP`, i.e., the current set of capabilities can be queried without restarting `pihole-FTL` (by setting `DEBUG_CAPS=true` and thereafter sending `killall -HUP pihole-FTL`). - -#### `DEBUG_DNSMASQ_LINES=false|true` {#debug_dnsmasq_lines data-toc-label='Analyze dnsmasq log lines'} - -Print file and line causing a `dnsmasq` event into FTL's log files. This is handy to implement additional hooks missing from FTL. - -#### `DEBUG_VECTORS=false|true` (PR [#725](https://github.com/pi-hole/FTL/pull/725)) {#debug_vectors data-toc-label='Vectors'} - -FTL uses dynamically allocated vectors for various tasks. This config option enables extensive debugging information such as information about allocation, referencing, deletion, and appending. - -#### `DEBUG_RESOLVER=false|true` (PR [#728](https://github.com/pi-hole/FTL/pull/728)) {#debug_resolver data-toc-label='Resolver details'} - -Extensive information about hostname resolution like which DNS servers are used in the first and second hostname resolving tries (only affecting internally generated PTR queries). - -#### `DEBUG_EDNS0=false|true` (PR [#851](https://github.com/pi-hole/FTL/pull/851)) {#debug_edns0 data-toc-label='EDNS(0) data'} - -Verbose logging during EDNS(0) header analysis. - -#### `DEBUG_CLIENTS=false|true` (PR [#762](https://github.com/pi-hole/FTL/pull/762)) {#debug_clients data-toc-label='Clients'} - -Log various important client events such as change of interface (e.g., client switching from WiFi to wired or VPN connection), as well as extensive reporting about how clients were assigned to its groups. - -#### `DEBUG_ALIASCLIENTS=false|true` (PR [#880](https://github.com/pi-hole/FTL/pull/880)) {#debug_aliasclients data-toc-label='Aliasclients'} - -Log information related to alias-client processing. - -#### `DEBUG_EVENTS=false|true` (PR [#881](https://github.com/pi-hole/FTL/pull/881)) {#debug_events data-toc-label='Events'} - -Log information regarding FTL's embedded event handling queue. - -#### `DEBUG_HELPER=false|true` (PR [#914](https://github.com/pi-hole/FTL/pull/914)) {#debug_helper data-toc-label='Script helpers'} - -Log information about script helpers, e.g., due to `dhcp-script`. - -#### `ADDR2LINE=true|false` (PR [#774](https://github.com/pi-hole/FTL/pull/774)) {#addr2line data-toc-label='Addr2Line'} - -Should FTL translate its own stack addresses into code lines during the bug backtrace? This improves the analysis of crashed significantly. It is recommended to leave the option enabled. This option should only be disabled when `addr2line` is known to not be working correctly on the machine because, in this case, the malfunctioning `addr2line` can prevent from generating any backtrace at all. - -#### `DEBUG_EXTRA=false|true` (PR [#994](https://github.com/pi-hole/FTL/pull/994)) {#debug_extra data-toc-label='Misc.'} - -Temporary flag that may print additional information. This debug flag is meant to be used whenever needed for temporary investigations. The logged content may change without further notice at any time. - -### Deprecated options - -#### `REPLY_ADDR4=` (unset by default, PR [#965](https://github.com/pi-hole/FTL/pull/965)) {#reply_addr4 data-toc-label='Force A reply'} - -*This option is deprecated and may be removed in future versions, please use `BLOCK_IPV4` and `LOCAL_IPV4` instead* - -If neither `BLOCK_IPV4` nor `LOCAL_IPV4` are set, this setting is used to set both of them. If either of the two is set, this setting is ignored altogether. - -#### `REPLY_ADDR6=` (unset by default, PR [#965](https://github.com/pi-hole/FTL/pull/965)) {#reply_addr6 data-toc-label='Force AAAA reply'} - -*This option is deprecated and may be removed in future versions, please use `BLOCK_IPV6` and `LOCAL_IPV6` instead* - -If neither `BLOCK_IPV6` nor `LOCAL_IPV6` are set, this setting is used to set both of them. If either of the two is set, this setting is ignored altogether. - -#### `PORTFILE=/run/pihole-FTL.port` {#file_PORTFILE data-toc-label='Port file'} - -*This option is deprecated as FTL does not write any port file anymore. Please parse `pihole-FTL.conf` if you need to check if a custom API port is set.* - -The file containing the port FTL's API is listening on. +There is also an outline of the configuration options in the [api documentation](../api/index.md#accessing-the-api-documentation). diff --git a/docs/ftldns/dnsmasq_warn.md b/docs/ftldns/dnsmasq_warn.md index b92c9abb7..c6b5c37b4 100644 --- a/docs/ftldns/dnsmasq_warn.md +++ b/docs/ftldns/dnsmasq_warn.md @@ -133,7 +133,7 @@ Warnings commonly seen in `dnsmasq`'s log file (`/var/log/pihole/pihole.log`) an edns-packet-max=1232 ``` - After running `pihole restartdns` your Pi-hole will not even try larger packet sizes (the default is 4096). Check out our [unbound guide](../guides/dns/unbound.md) for a comment about the particular value of `1232` or reference [this comment](https://discourse.pi-hole.net/t/dnsmasq-warn-reducing-dns-packet-size/51803/31) regarding the various allowed packet sizes for the various upstream DNS servers. + After restarting `pihole-FTL` it will not even try larger packet sizes (the default is 4096). Check out our [`unbound` guide](../guides/dns/unbound.md) for a comment about the particular value of `1232` or reference [this comment](https://discourse.pi-hole.net/t/dnsmasq-warn-reducing-dns-packet-size/51803/31) regarding the various allowed packet sizes for the various upstream DNS servers. !!! warning "Ignoring query from non-local network" diff --git a/docs/ftldns/signals.md b/docs/ftldns/signals.md index 65331ac2a..aaee683ce 100644 --- a/docs/ftldns/signals.md +++ b/docs/ftldns/signals.md @@ -20,9 +20,9 @@ When FTL receives a `SIGHUP`, it clears the entire DNS cache, and then **Note:** No other `dnsmasq` config files are re-read. - The FTL database connection (`/etc/pihole/pihole-FTL.db`) is re-opened. -- The privacy level is re-read from `pihole-FTL.conf` (`PRIVACY_LEVEL`). +- The privacy level is re-read from `pihole.toml` (`misc.privacylevel`). - The blocking status is re-read from `setupVars.conf` (`BLOCKING_ENABLED`). -- The debug settings are re-read from `pihole-FTL.conf` (`DEBUG_*`). +- The debug settings are re-read from `pihole.toml` (`debug.*`). - The gravity database connection (`/etc/pihole/gravity.db`) is re-opened. - The number of blocked domains is updated. - All regular expression (RegEx) filters in `gravity.db` are re-read and pre-compiled for fast execution later on. @@ -49,7 +49,7 @@ This signal does: - The number of blocked domains is updated. - All regular expression (RegEx) filters in `gravity.db` are re-read and pre-compiled for fast execution later on. - The blocking cache (storing if a domain has already been analyzed and what the result was) is cleared. -- The privacy level is re-read from `pihole-FTL.conf` (`PRIVACY_LEVEL`). +- The privacy level is re-read from `pihole.toml` (`misc.privacylevel`). The most important difference to `SIGHUP` is that the DNS cache itself is **not** flushed. Merely the blocking cache (storing if a domain has already been analyzed and what the result was) is cleared. diff --git a/docs/group_management/example.md b/docs/group_management/example.md index a7f6e6bce..e77f7eec6 100644 --- a/docs/group_management/example.md +++ b/docs/group_management/example.md @@ -5,7 +5,7 @@ In this example, we describe how to set up a blocking rule for three specific cl Don't forget to run ```bash -pihole restartdns reload-lists +pihole reloadlists ``` after your database modifications to have FTL flush its internal domain-blocking cache (separate from the DNS cache). @@ -14,7 +14,7 @@ after your database modifications to have FTL flush its internal domain-blocking 1. **Add three groups.** - The `Default` group has a special meaning and cannot be deleted. All domains, clients, and adlists without a specific group assignment are automatically managed through this group. Disabling this group will disable Pi-hole blocking for all unmanaged devices. + The `Default` group has a special meaning and cannot be deleted. All domains, clients, and subscribed lists without a specific group assignment are automatically managed through this group. Disabling this group will disable Pi-hole blocking for all unmanaged devices. ![Adding three groups](../images/group_management/example-groups.png) @@ -70,13 +70,13 @@ Client | Group membership | Domain | Blocked 192.168.0.103 | Group 3 + Default | doubleclick.net | Yes -All three clients got automatically assigned to the default (`Default`) group when they were added. The default group includes all adlists and list domains (if not already changed by the user). When we remove the default group for client `192.168.0.101`, we effectively remove all associations to any adlists and domains. This leaves this client completely unblocked. +All three clients got automatically assigned to the default (`Default`) group when they were added. The default group includes all subscribed lists and list domains (if not already changed by the user). When we remove the default group for client `192.168.0.101`, we effectively remove all associations to any subscribed lists and domains. This leaves this client completely unblocked. ## Example 2: Blocklist management -**Task:** Assign adlist with ID 1 to group 1 (in addition to the default assignment to group 0). This results in client `192.168.0.101` using *only this* adlist (we removed the default association in the last step). +**Task:** Assign subscribed list with ID 1 to group 1 (in addition to the default assignment to group 0). This results in client `192.168.0.101` using *only this* subscribed list (we removed the default association in the last step). -![Change blocklist group assignment - Overview](../images/group_management/example-adlists-1.png) +![Change subscribed list group assignment - Overview](../images/group_management/example-adlists-1.png) ??? "Raw database instructions" ```sql @@ -93,33 +93,33 @@ Client | Group membership | Domain | Blocked 192.168.0.103 | Group 3 + Default | doubleclick.net | Yes -`192.168.0.101` gets `doubleclick.net` blocked as it uses an adlist including this domain. All other clients stay unchanged. +`192.168.0.101` gets `doubleclick.net` blocked as it uses a subscribed list including this domain. All other clients stay unchanged. -## Example 3: Blacklisting +## Example 3: Denylisting -**Task:** Add a single domain that should be **blacklisted only for group 1** (client `192.168.0.101`). +**Task:** Add a single domain that should be **denylisted only for group 1** (client `192.168.0.101`). ### Step 1 Add the domain to be blocked -![Add new exact blacklist domain](../images/group_management/example-new-black.png) +![Add new exact denylist domain](../images/group_management/example-new-black.png) ![Resulting row in the list of domains](../images/group_management/example-domain-1.png) ??? "Raw database instructions" ```sql - INSERT INTO domainlist (type, domain, comment) VALUES (1, 'blacklisted.com', 'Blacklisted for members of group 1'); + INSERT INTO domainlist (type, domain, comment) VALUES (1, 'denylisted.com', 'Denylisted for members of group 1'); ``` **Result** Client | Group membership | Domain | Blocked ------------- | ----- | ------ | ------- -*all other* | Default | blacklisted.com | **Yes** -192.168.0.101 | Group 1 | blacklisted.com | No -192.168.0.102 | Group 2 + Default | blacklisted.com | **Yes** -192.168.0.103 | Group 3 + Default | blacklisted.com | **Yes** +*all other* | Default | denylisted.com | **Yes** +192.168.0.101 | Group 1 | denylisted.com | No +192.168.0.102 | Group 2 + Default | denylisted.com | **Yes** +192.168.0.103 | Group 3 + Default | denylisted.com | **Yes** Note that Pi-hole is *not* blocking this domain for client `192.168.0.101` as we removed the default assignment through group 0 above. All remaining clients are linked through the Default group to this domain and see it as being blocked. @@ -140,10 +140,10 @@ Assign this domain to group 1 Client | Group membership | Domain | Blocked ------------- | ----- | ------ | ------- -*all other* | Default | blacklisted.com | Yes -192.168.0.101 | Group 1 | blacklisted.com | **Yes** -192.168.0.102 | Group 2 + Default | blacklisted.com | Yes -192.168.0.103 | Group 3 + Default | blacklisted.com | Yes +*all other* | Default | denylisted.com | Yes +192.168.0.101 | Group 1 | denylisted.com | **Yes** +192.168.0.102 | Group 2 + Default | denylisted.com | Yes +192.168.0.103 | Group 3 + Default | denylisted.com | Yes All clients see this domain as being blocked: Client 1 due to a direct assignment through group 1, all remaining clients through the default group 0 (unchanged). @@ -163,28 +163,28 @@ Remove default assignment to all clients not belonging to a group Client | Group membership | Domain | Blocked ------------- | ----- | ------ | ------- -*all other* | Default | blacklisted.com | **No** -192.168.0.101 | Group 1 | blacklisted.com | Yes -192.168.0.102 | Group 2 + Default | blacklisted.com | **No** -192.168.0.103 | Group 3 + Default | blacklisted.com | **No** +*all other* | Default | denylisted.com | **No** +192.168.0.101 | Group 1 | denylisted.com | Yes +192.168.0.102 | Group 2 + Default | denylisted.com | **No** +192.168.0.103 | Group 3 + Default | denylisted.com | **No** While client 1 keeps its explicit assignment through group 1, the remaining clients lost their unassignments when we removed group 0 from the assignment. -## Example 4: Whitelisting +## Example 4: Allowlisting -**Task:** Add a single domain that should be **whitelisted only for group 2** (client `192.168.0.102`). +**Task:** Add a single domain that should be **allowlisted only for group 2** (client `192.168.0.102`). ### Step 1 -Add the domain to be whitelisted +Add the domain to be allowlisted -![Add a new exact whitelist domain](../images/group_management/example-new-white.png) +![Add a new exact allowlist domain](../images/group_management/example-new-white.png) ![Resulting row in the list of domains](../images/group_management/example-domain-4.png) ??? "Raw database instructions" ```sql - INSERT INTO domainlist (type, domain, comment) VALUES (0, 'doubleclick.net', 'Whitelisted for members of group 2'); + INSERT INTO domainlist (type, domain, comment) VALUES (0, 'doubleclick.net', 'Allowlisted for members of group 2'); ``` **Result** @@ -196,7 +196,7 @@ Client | Group membership | Domain | Blocked 192.168.0.102 | Group 2 + Default | doubleclick.net | **No** 192.168.0.103 | Group 3 + Default | doubleclick.net | **No** -Client `192.168.0.101` is not whitelisting this domain as we removed the default assignment through group 0 above. All remaining clients are linked through the default group to this domain and see it as being whitelisted. Note that this is completely analog to step 1 of [example 3](#example-3-blacklisting). +Client `192.168.0.101` is not allowlisting this domain as we removed the default assignment through group 0 above. All remaining clients are linked through the default group to this domain and see it as being allowlisted. Note that this is completely analog to step 1 of [example 3](#example-3-denylisting). ### Step 2 @@ -218,7 +218,7 @@ Client | Group membership | Domain | Blocked 192.168.0.102 | Group 2 + Default | doubleclick.net | **Yes** 192.168.0.103 | Group 3 + Default | doubleclick.net | **Yes** -Requests from all clients are blocked as the new whitelist entry is not associated with any group and, hence, is not used by any client. +Requests from all clients are blocked as the new allowlist entry is not associated with any group and, hence, is not used by any client. ### Step 3 @@ -241,4 +241,4 @@ Client | Group membership | Domain | Blocked 192.168.0.102 | Group 2 + Default | doubleclick.net | **No** 192.168.0.103 | Group 3 + Default | doubleclick.net | Yes -Client 2 got the whitelist entry explicitly assigned to. Accordingly, client 2 does not get the domain blocked whereas all remaining clients still see this domain as blocked. +Client 2 got the allowlist entry explicitly assigned to. Accordingly, client 2 does not get the domain blocked whereas all remaining clients still see this domain as blocked. diff --git a/docs/group_management/index.md b/docs/group_management/index.md index 73c06b52b..664633f3d 100644 --- a/docs/group_management/index.md +++ b/docs/group_management/index.md @@ -1,4 +1,4 @@ -Any blocklist or domain on the white-/black-/regex-lists can be managed through groups. This allows not only grouping them to highlight their relationship, but also enabling/disabling them together if one, for instance, wants to visit a specific service only temporarily. +Any subscribed list or domain on the allowlist, denylist, or regex list can be managed through groups. This allows not only grouping them to highlight their relationship, but also enabling/disabling them together if one, for instance, wants to visit a specific service only temporarily. Group `Default` (`group_id` `0`) is special as it is automatically assigned to domains and clients not being a member of other groups. Each newly added client or domain gets assigned to group zero when being added. diff --git a/docs/guides/misc/allowlist-denylist.md b/docs/guides/misc/allowlist-denylist.md new file mode 100644 index 000000000..5c827401a --- /dev/null +++ b/docs/guides/misc/allowlist-denylist.md @@ -0,0 +1,47 @@ +### What Domains To Allow or Deny + +[This extension for Google Chrome](https://chrome.google.com/webstore/detail/adamone-assistant/fdmpekabnlekabjlimjkfmdjajnddgpc) can help you in finding out which domains you need to allow. + +### How to Allow or Deny Domains + +There are scripts to aid users in adding or removing domains to the allowlist or denylist from the CLI + +Each script accepts the following parameters: + +| Parameter | Description | +|------------|--------------------------------------------------------------------------------------------------| +| `[domain]` | Fully qualified domain name you wish to add or remove. You can pass any number of domains. | +| `remove` | Removal mode. Domains will be removed from the list, rather than added | +| `-q` | Quiet mode. Console output is minimal. Useful for calling from another script (see `gravity.sh`) | + +Domains passed are parsed by the script to ensure they are valid domains. If a domain is invalid it will be ignored. + +By default, allowed or denied domains are associated with the Default Group only. If the domain should be associated with other groups, these will need to be selected in **Group Management > Domains** within the Pi-Hole web frontend. + +#### Example `pihole allow` usages + +* Attempt to add one or more domains to the allowlist and reload pihole-FTL: + + ```bash + pihole allow domain1 [domain2...] + ``` + +To remove domains from the allowlist add `remove` as an additional argument, e.g: + +```bash +pihole allow remove domain1 [domain2...] +``` + +#### Example `pihole deny` usages + +* Attempt to add one or more domains to the denylist and reload pihole-FTL: + + ```bash + pihole deny domain1 [domain2...] + ``` + +To remove domains from the denylist add `remove` as an additional argument, e.g: + +```bash +pihole deny remove domain1 [domain2...] +``` diff --git a/docs/guides/misc/benchmark.md b/docs/guides/misc/benchmark.md index f0cd14957..1478ca360 100644 --- a/docs/guides/misc/benchmark.md +++ b/docs/guides/misc/benchmark.md @@ -21,21 +21,19 @@ They are extracted into the list file as they are recorded in the database. Ther We suggest disabling both logging and the long-term database during the benchmark run as both the log file and the database would otherwise unnecessarily grow, several hundred megabytes may be possible. Not only would your statistics be distorted by the artificial mass querying, but the benchmark could also be negatively affected by the writing speed of your SD card. -Logging can be disabled using `sudo pihole logging off`. +Logging can be disabled using `pihole-FTL --config dns.queryLogging false`. -The long-term database can be disabled by setting - -``` -DBFILE= -``` - -in `/etc/pihole/pihole-FTL.conf` and running `sudo pihole restartdns` (see also [here](../../ftldns/configfile.md/#dbfile)). +The long-term database can be disabled running `pihole-FTL --config files.database ""` ### 2.2 Increase DNS cache size We also suggest increasing the DNS cache for benchmarking. The rather low value is fine for typical use cases. Domains will expire at some point and make room for new domains. As the benchmark will artificially increase the querying rate, there will be no time for the domains to expire naturally. This would dramatically hit the caching performance while you would never see such performance penalties in real use cases. -Set `cache-size` to a rather high value (maybe 25,000 - by guess roughly one-eighth to one-fourth number of the domains you extracted from the database in step 1) in `/etc/dnsmasq.d/01-pihole.conf` and run `sudo pihole restartdns` afterward. +Set the cache size to a rather high value (maybe 25,000 - by guess roughly one-eighth to one-fourth number of the domains you extracted from the database in step 1) + +```bash +pihole-FTL --config dns.cache.size 25000 +``` ## 3. Query domains from the list diff --git a/docs/guides/misc/tor/setup.md b/docs/guides/misc/tor/setup.md index 36e81da62..6830fb227 100644 --- a/docs/guides/misc/tor/setup.md +++ b/docs/guides/misc/tor/setup.md @@ -28,7 +28,7 @@ If you want a recognizable hostname for the Tor DNS in your Pi-hole GUI statisti Restart Pi-hole DNS Server for the `/etc/hosts` changes to take effect ```bash -sudo pihole restartdns +sudo service pihole-FTL restart ``` ## Testing your configuration @@ -51,6 +51,6 @@ For Windows: nslookup -server= -q= api.mixpanel.com ``` -That should give you the Pi-hole IPv4 and/or IPv6 address as Answer and show up as "Pi-holed" in the WebGUI Query Log (assuming you have the default blocklist, otherwise replace `api.mixpanel.com` with any domain on your blocklist). +That should give you the Pi-hole IPv4 and/or IPv6 address as Answer and show up as "Pi-holed" in the WebGUI Query Log (assuming you have the default subscribed list, otherwise replace `api.mixpanel.com` with any domain on your subscribed list). If any of the queries don't show up in the Query Log you should make sure to configure your Pi-hole/network setup properly ([this thread might help](https://www.reddit.com/r/pihole/comments/7e0jg9/dns_over_tor/dq4kkvg/)). diff --git a/docs/guides/misc/whitelist-blacklist.md b/docs/guides/misc/whitelist-blacklist.md deleted file mode 100644 index ac5a69423..000000000 --- a/docs/guides/misc/whitelist-blacklist.md +++ /dev/null @@ -1,73 +0,0 @@ -### What to Whitelist or Blacklist - -[This extension for Google Chrome](https://chrome.google.com/webstore/detail/adamone-assistant/fdmpekabnlekabjlimjkfmdjajnddgpc) can help you in finding out which domains you need to whitelist. - -### How to Whitelist or Blacklist - -There are scripts to aid users in adding or removing domains to the whitelist or blacklist from the CLI - -Each script accepts the following parameters: - -| Parameter | Description | -|------------|--------------------------------------------------------------------------------------------------| -| `[domain]` | Fully qualified domain name you wish to add or remove. You can pass any number of domains. | -| `-d` | Removal mode. Domains will be removed from the list, rather than added | -| `-nr` | Update blacklist without refreshing pihole-FTL | -| `-f` | Force delete cached blocklist content | -| `-q` | Quiet mode. Console output is minimal. Useful for calling from another script (see `gravity.sh`) | - -Domains passed are parsed by the script to ensure they are valid domains. If a domain is invalid it will be ignored. - -By default, Whitelisted/Blacklisted domains are associated with the Default Group only. If the domain should be associated with other groups, these will need to be selected in **Group Management > Domains** within the Pi-Hole web frontend. - -#### Example `pihole -w` usages - -* Attempt to add one or more domains to the whitelist and reload pihole-FTL: - - ```bash - pihole -w domain1 [domain2...] - ``` - -* Attempt to add one or more domains to the whitelist, but do not reload pihole-FTL: - - ```bash - pihole -w -nr domain1 [domain2...] - ``` - -* Attempt to add one or more domains to the whitelist and force pihole-FTL to reload: - - ```bash - pihole -w -f domain1 [domain2...] - ``` - -To remove domains from the whitelist add `-d` as an additional argument, e.g: - -```bash -pihole -w -d domain1 [domain2...] -``` - -#### Example `pihole -b` usages - -* Attempt to add one or more domains to the blacklist and reload pihole-FTL: - - ```bash - pihole -b domain1 [domain2...] - ``` - -* Attempt to add one or more domains to the blacklist, but do not reload pihole-FTL: - - ```bash - pihole -b -nr domain1 [domain2...] - ``` - -* Attempt to add one or more domains to the blacklist and force pihole-FTL to reload: - - ```bash - pihole -b -f domain1 [domain2...] - ``` - -To remove domains from the blacklist add `-d` as an additional argument, e.g: - -```bash -pihole -b -d domain1 [domain2...] -``` diff --git a/docs/guides/vpn/wireguard/faq.md b/docs/guides/vpn/wireguard/faq.md index 04d8ef9ba..49db9d3a6 100644 --- a/docs/guides/vpn/wireguard/faq.md +++ b/docs/guides/vpn/wireguard/faq.md @@ -86,10 +86,4 @@ MTU = 1500 ## Pi-hole not listening on `wg0` after reboot -If, e.g., after reboot, the `wg0` interface isn't up before Pi-hole is ready (more precisely, the `pihole-FTL` service is started), you may experience that Pi-hole doesn't listen on the Wireguard interface. This can be mitigated by artificially delaying the start of Pi-hole using, e.g., the config option - -```plain -DELAY_STARTUP=5 -``` - -in `/etc/pihole/pihole-FTL.conf` to have Pi-hole delay the start of the DNS server by `5` seconds. +If, e.g., after reboot, the `wg0` interface isn't up before Pi-hole is ready (more precisely, the `pihole-FTL` service is started), you may experience that Pi-hole doesn't listen on the Wireguard interface. This can be mitigated by artificially delaying the start of Pi-hole using, e.g., `pihole-FTL --config mic.delay_startup 5` to have Pi-hole delay the start of the DNS server by `5` seconds. diff --git a/docs/guides/webserver/caddy.md b/docs/guides/webserver/caddy.md deleted file mode 100644 index 94bbeb3b5..000000000 --- a/docs/guides/webserver/caddy.md +++ /dev/null @@ -1,128 +0,0 @@ -**This is an unsupported configuration created by the community** - -If you'd like to use [Caddy](https://caddyserver.com/) as your main web server with Pi-hole, you'll need to make a few changes. - -!!! note - This guide only deals with setting up caddy as a reverse-proxy and not as a replacement for lighttpd (Although caddy is capable of doing so, but it is beyond the scope of this guide). - -## Modifying lighttpd configuration - -First, change the listen port in this file: `/etc/lighttpd/lighttpd.conf:` - -```lighttpd -server.port = 1080 -``` - -In this case, port 1080 was chosen at random. You can use a custom port. - -BUT ANY CHANGES MADE TO THIS FILE WILL BE LOST ON THE NEXT PI-HOLE UPDATE. - -So if you want a permanent method of changing the lighttpd port and your lighttpd version >= 1.4.46, you can overwrite the port in: `/etc/lighttpd/external.conf` (note the different syntax!): - -```lighttpd -server.port := 1080 -``` - -Next, restart the lighttpd server with either of these commands: - -```bash -sudo systemctl restart lighttpd -``` - -or - -```bash -sudo service lighttpd restart -``` - -## Installing Caddy - -Follow the instructions on the [Caddy download](https://caddyserver.com/docs/download) documentation page. - -## Setting up your Caddyfile - -Now set up a "virtual host" in your Caddyfile (default `/etc/caddy/Caddyfile`). There are many options you can add, but at a minimum, you need to make a "default" host by binding `:80`. This will accept requests for any interface on port `80`. - -### Caddyfile (for Caddy v2) - -``` -http://pi.hole { - reverse_proxy localhost:1080 -} -``` - -- If you'd like to enable HTTPS on your site, make sure your server is reachable via your domain name (ex: myawesomesite.com) and is pointing to the right IP address. -- Additionally you need to open ports :80 and :443 (Apart from the one's required specifically for pi-hole) for your server before setting up HTTPS. - -The following configuration will automatically fetch and setup HTTPS for your domain using Lets-Encrypt - -``` -myawesomesite.com { - reverse_proxy localhost:1080 -} -``` - -Additionally you can make pihole reachable via a subdomain and optionally can you enable Zstandard and Gzip compression as follows: - -``` -pihole.myawesomesite.com { - reverse_proxy localhost:1080 - encode zstd gzip -} -``` - -Finally, run `sudo systemctl caddy reload` to reload Caddy with the new configuration. - -### Caddyfile (for Caddy v1) - -Caddy v1 is no longer actively supported, but the following is a config example if you're running an old installation. - -``` -blackhole:80, pi.hole:80, 0.0.0.0:80 { - root /var/www/html/pihole - log /var/log/caddy/blackhole.log - - rewrite { - ext js - to index.js - } - - proxy / localhost:1080 { - transparent - } -} -``` - -In this example, `blackhole` and `pi.hole` are added as valid names with which to open the admin page. - -## Verifying your setup - -First, make sure that any other sites you're serving from caddy are still functioning. For example, if you have a block for `myawesomesite.com:80` or similar in your Caddyfile, open up a browser to `http://myawesomesite.com` (or `https://` if you have enabled it) and verify it still loads. - -Next, verify you can load the admin page. Open up `http://pi.hole/admin` (or use the IP address of your server) and verify that you can access the admin page. - -Finally, verify that requests for ads are being black holed: - -```console -$ curl -H "Host: badhost" pi.hole/ - - - - - - - -``` - -Replace the URL `pi.hole` with the IP address or alternate DNS name you're using if necessary. - -Lastly, ensure that requests for JavaScript files from advertisement domains are being served properly: - -```bash -curl -H "Host: badhost" pi.hole/malicious.js -var x = "Pi-hole: A black hole for Internet advertisements." -``` - -For more information visit Caddy's documentation [website](https://caddyserver.com/docs/). - -For usage questions or support, visit the [Caddy Community forums](https://caddy.community/). diff --git a/docs/guides/webserver/nginx.md b/docs/guides/webserver/nginx.md deleted file mode 100644 index 62fa6dca2..000000000 --- a/docs/guides/webserver/nginx.md +++ /dev/null @@ -1,132 +0,0 @@ -### Notes & Warnings - -- **This is an unsupported configuration created by the community** -- **Replace `7.3` with the PHP version you installed, e.g. if you're using Raspbian Stretch (Debian 9) replace `7.3` with `7.0`.** -- The `php7.3-sqlite3` package must be installed otherwise Networking and Querying will throw an error that it can't access the database. - -### Basic requirements - -1. Stop default lighttpd - - ```bash - service lighttpd stop - ``` - -2. Install necessary packages - - ```bash - apt-get -y install nginx php7.3-fpm php7.3-cgi php7.3-xml php7.3-sqlite3 php7.3-intl apache2-utils - ``` - -3. Disable lighttpd at startup - - ```bash - systemctl disable lighttpd - ``` - -4. Enable php7.3-fpm at startup - - ```bash - systemctl enable php7.3-fpm - ``` - -5. Enable nginx at startup - - ```bash - systemctl enable nginx - ``` - -6. Edit `/etc/nginx/sites-available/default` to: - - ```nginx - server { - listen 80 default_server; - listen [::]:80 default_server; - - root /var/www/html; - server_name _; - autoindex off; - - index pihole/index.php index.php index.html index.htm; - - location / { - expires max; - try_files $uri $uri/ =404; - } - - location ~ \.php$ { - include fastcgi_params; - fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name; - fastcgi_pass unix:/run/php/php7.3-fpm.sock; - fastcgi_param FQDN true; - auth_basic "Restricted"; # For Basic Auth - auth_basic_user_file /etc/nginx/.htpasswd; # For Basic Auth - } - - location /*.js { - index pihole/index.js; - auth_basic "Restricted"; # For Basic Auth - auth_basic_user_file /etc/nginx/.htpasswd; # For Basic Auth - } - - location /admin { - root /var/www/html; - index index.php index.html index.htm; - auth_basic "Restricted"; # For Basic Auth - auth_basic_user_file /etc/nginx/.htpasswd; # For Basic Auth - } - - location ~ /\.ht { - deny all; - } - } - ``` - -7. Create a username for authentication for the admin - we don't want other people in our network change our black and whitelist ;) - - ```bash - htpasswd -c /etc/nginx/.htpasswd exampleuser - ``` - -8. Change ownership of the html directory to nginx user - - ```bash - chown -R www-data:www-data /var/www/html - ``` - -9. Make sure the html directory is writable - - ```bash - chmod -R 755 /var/www/html - ``` - -10. Grant the admin panel access to the gravity database - - ```bash - usermod -aG pihole www-data - ``` - -11. Start php7.3-fpm daemon - - ```bash - service php7.3-fpm start - ``` - -12. Start nginx web server - - ```bash - service nginx start - ``` - -### Optional configuration - -- If you want to use your custom domain to access admin page (e.g.: `http://mydomain.internal/admin/settings.php` instead of `http://pi.hole/admin/settings.php`), make sure `mydomain.internal` is assigned to `server_name` in `/etc/nginx/sites-available/default`. E.g.: `server_name mydomain.internal;` - -- When using nginx to serve Pi-hole, Let's Encrypt can be used to directly configure nginx. Make sure to use your hostname instead of _ in `server_name _;` line above. - - ```bash - add-apt-repository ppa:certbot/certbot - apt-get install certbot python-certbot-nginx - - certbot --nginx -m "$email" -d "$domain" -n --agree-tos --no-eff-email - ``` diff --git a/docs/guides/webserver/traefik-nodocker.md b/docs/guides/webserver/traefik-nodocker.md deleted file mode 100644 index 8ec1cf37f..000000000 --- a/docs/guides/webserver/traefik-nodocker.md +++ /dev/null @@ -1,77 +0,0 @@ -### Notes & Warnings - -- **This is an unsupported configuration created by the community** -- This describes how to use traefik on a (possibly remote) machine to serve pi-hole via https and a different domain, not how to do this in docker (via docker-compose). - -### Basic requirements - -1. Have a traefik server running anywhere where it can access port 80 of the pihole server. Technically it can run in a docker container though. For LetsEncrypt to work traefik must be reachable on port 80 and 443 from the internet and have the domain.tld pointed at its external address. - -2. The following traefik config (traefik.toml) - - ```toml - debug = false - checkNewVersion = true - logLevel = "INFO" - defaultEntryPoints = ["https","http"] - - [entryPoints] - [entryPoints.http] - address = ":80" - [entryPoints.http.redirect] - entryPoint = "https" - [entryPoints.https] - address = ":443" - [entryPoints.https.tls] - # Optional Security Settings - [retry] - - [docker] - endpoint = "unix:///var/run/docker.sock" - domain = "domain.tld" - watch = true - exposedbydefault = false - - [acme] - email = "emailForLetsEncryptACME" - storage = "acme.json" - storageFile = "/etc/traefik/acme/acme.json" - entryPoint = "https" - OnHostRule = true - [acme.tlsChallenge] - [[acme.domains]] - main = "pihole.domain.tld" - - [file] - watch = true - - [backends] - [backends.pihole] - [backends.pihole.servers.server1] - url = "http://IP-Of-Pihole:80" - - [frontends] - [frontends.pihole] - backend = "pihole" - passHostHeader = true - [frontends.pihole.headers] - STSSeconds = 31536000 - [frontends.pihole.routes.route1] - rule = "Host:pihole.domain.tld" - ``` - -3. Edit your /etc/lighttpd/external.conf to - - ```lighttpd - $SERVER["socket"] == ":80" { - # Ensure the Pi-hole Block Page knows that this is not a blocked domain - setenv.add-environment = ("fqdn" => "true") - - - $HTTP["host"] =~ "^pi\.hole" { - url.redirect = ("^/(.*)" => "https://pihole.domain.tld/$1") - } - } - ``` - -4. Restart pi-hole's lighttpd and traefik, then you should be able to access your pihole via `https://pihole.domain.tld/` diff --git a/docs/guides/webserver/traefik-v2-docker.md b/docs/guides/webserver/traefik-v2-docker.md deleted file mode 100644 index e5e7a50c5..000000000 --- a/docs/guides/webserver/traefik-v2-docker.md +++ /dev/null @@ -1,69 +0,0 @@ -### Notes & Warnings - -- **This is an unsupported configuration created by the community** -- This describes how to use [Traefik](https://doc.traefik.io/traefik/) v2 in a Docker container (via docker-compose.yml) to serve the Pi-hole web admin interface via https and includes a permenent http -> https redirect. -- This does not describe how to proxy DNS or DHCP requests to Pi-hole, which is not recommended. -- For ACME challenges, the Traefik container may need to be able to resolve the desired Pi-hole hostname without relying on Pi-hole to do so. Provide this via the `extra_hosts` parameter in your Traefik container's config in docker-compose.yml if needed. -- For LetsEncrypt to work Traefik must be reachable on port 80 and 443 from the Internet and have `domain.tld` pointed at its external address. - -### Basic requirements - -1. Have a Traefik v2 Docker container running where it can access port 80 of the Pi-hole server. - -1. The following Traefik static config (passed as `command` arguments to the Traefik container in docker-compose.yml): - - ``` - - "--providers.docker=true" - - "--providers.docker.network=traefik-net" # replace with your configured Docker network name - - "--entrypoints.web.address=:80" - - "--entrypoints.web.http.redirections.entrypoint.to=websecure" - - "--entrypoints.websecure.address=:443" - - "--certificatesresolvers.letsencrypt.acme.httpchallenge=true" - - "--certificatesresolvers.letsencrypt.acme.email=your-email@example.com" - - "--certificatesresolvers.letsencrypt.acme.storage=acme.json" - - "--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web" - ``` - -1. The next step has 2 scenarios: - - - If Pi-hole is running in a container on the same Docker host as Traefik, put the following `labels` in your Pi-hole container's config in docker-compose.yml: - - ``` - - "traefik.http.routers.pihole.rule=Host(`pihole.domain.tld`)" - - "traefik.http.routers.pihole.entrypoints=websecure" - - "traefik.http.routers.pihole.tls=true" - - "traefik.http.routers.pihole.tls.certresolver=letsencrypt" - - "traefik.http.routers.pihole.tls.domains[0].main=pihole.domain.tld" - - "traefik.http.routers.pihole.tls.domains[0].sans=pihole.domain.tld" - - "traefik.http.services.pihole.loadbalancer.server.port=80" - ``` - - - If Pi-hole is running on a different host, you need to provide the Pi-hole (dynamic) config via a `traefik.yml` file to Traefik. This is best done by bind mounting the local directory containing this file to the `/etc/traefik` directory within the container: - - ``` - # Traefik container config: - volumes: - - './traefik/fileproviders:/etc/traefik' - ``` - - ``` - # traefik.yml dynamic config for Pi-hole: - http: - routers: - pihole: - rule: Host(`pihole.domain.tld`) - entrypoints: websecure - tls: - certresolver: letsencrypt - domains: - - main: pihole.domain.tld - sans: - - pihole.domain.tld - services: - pihole: - loadbalancer: - servers: - - url: "http://pihole.domain.tld/" - ``` - -1. Restart the Traefik and Pi-hole containers, then you should be able to access your pihole via `https://pihole.domain.tld/` diff --git a/docs/main/faq.md b/docs/main/faq.md index a352c4bf2..2f503edfb 100644 --- a/docs/main/faq.md +++ b/docs/main/faq.md @@ -54,58 +54,6 @@ Ask the list maintainer to convert the IDNs to their punycode representation. Internationalizing Domain Names in Applications (IDNA) was conceived to allow client-side use of language-specific characters in domain names without requiring any existing infrastructure (DNS servers, mall servers, etc., including associated protocols) to change. Accordingly, the corresponding original [RFC 3490](https://tools.ietf.org/html/rfc3490) clearly states that IDNA is employed at application level, not on the server side. Hence, DNS servers never see any IDN domain name, which means DNS records do not store IDN domain names at all, only their [Punycode](https://en.wikipedia.org/wiki/Punycode) representations. -### Error while loading data from the long-term database - -If requesting a lot of data from the long-term database you get this error - -```code -An unknown error occurred while loading the data. - -Check the server's log files (/var/log/lighttpd/error-pihole.log) for details. - -You may need to increase PHP memory limit. - -You can find more info in pi-hole's FAQ: -https://docs.pi-hole.net/main/faq/#error-while-loading-data-from-the-long-term-database -``` - -**Solution:** - -You need to increase PHP's memory and restart the server. - -The amount of memory needed depends on many factors: - -- available system RAM, -- other processes running on your device, -- the amount of data you want to process. - -One approach would be to increase the limit by 128M and check if it was enough to retrieve the data. -If not, add another 128M, check again. If not, add another 128M, check again, until you find the best value. - -_Note:_ -Do not assign all available memory as this can freeze your system. -Please consider the possibility that your system does not have enough memory at all to load all the needed data. - -_Steps to increase `memory_limit`:_ - -Open or create `.user.ini` file: - -```bash -sudo nano /var/www/html/.user.ini -``` - -Add (or change) the memory limit (common abbreviation M=megabyte, G=gigabyte): - -```ini -memory_limit = 256M -``` - -Restart the web server: - -```bash -sudo service lighttpd restart -``` - ### In which order are locally defined DNS records used? **Answer:** diff --git a/docs/main/origins.md b/docs/main/origins.md index 08a61287a..4db3a56bd 100644 --- a/docs/main/origins.md +++ b/docs/main/origins.md @@ -8,8 +8,6 @@ Pi-hole being a **advertising-aware DNS/Web server**, makes use of the following - [`dnsmasq`](https://www.thekelleys.org.uk/dnsmasq/doc.html) - a lightweight DNS and DHCP server - [`curl`](https://curl.haxx.se/) - A command-line tool for transferring data with URL syntax -- [`lighttpd`](https://www.lighttpd.net/) - web server designed and optimized for high performance -- [`php`](https://www.php.net/) - a popular general-purpose web scripting language - [AdminLTE Dashboard](https://github.com/ColorlibHQ/AdminLTE) - premium admin control panel based on Bootstrap 3.x - [`sqlite3`](https://www.sqlite.org/index.html) - SQL Database engine diff --git a/docs/main/pihole-command.md b/docs/main/pihole-command.md index 1a0a31aa5..2f5f53ba9 100644 --- a/docs/main/pihole-command.md +++ b/docs/main/pihole-command.md @@ -11,7 +11,7 @@ Pi-hole makes use of many commands, and here we will break down those required t | Feature | Invocation | | -------------- | -------------- | | [Core](#core-script) | `pihole` | -| [Whitelisting, Blacklisting and Regex](#whitelisting-blacklisting-and-regex) | `pihole -w`, `pihole -b`, `pihole --regex`, `pihole --wild` | +| [Allowlisting, Denylisting and Regex](#allowlisting-denylisting-and-regex) | `pihole allow`, `pihole deny`, `pihole --regex`, `pihole --wild`, `pihole --allow-regex`, `pihole --allow-wild` | | [Debugger](#debugger) | `pihole debug` | | [Log Flush](#log-flush) | `pihole flush` | | [Reconfigure](#reconfigure) | `pihole reconfigure` | @@ -25,7 +25,6 @@ Pi-hole makes use of many commands, and here we will break down those required t | [Uninstall](#uninstall) | `pihole uninstall` | | [Status](#status) | `pihole status` | | [Enable & Disable](#enable-disable) | `pihole enable` | -| [Restart DNS](#restart-dns) | `pihole restartdns` | | [Checkout](#checkout) | `pihole checkout` | ### Core Script @@ -38,11 +37,11 @@ Pi-hole makes use of many commands, and here we will break down those required t The core script of Pi-hole provides the ability to tie many DNS related functions into a simple and user-friendly management system, so that one may easily block unwanted content such as advertisements. For both the Command-line Interface (CLI) and Web Interface, we achieve this through the `pihole` command (this helps minimize code duplication, and allows users to read exactly what's happening using `bash` scripting). This "wrapper" elevates the current user (whether it be your own user account, or `www-data`) using `sudo`, but restricts the elevation to solely what can be called through the wrapper. -### Whitelisting, Blacklisting and Regex +### Allowlisting, Denylisting and Regex | | | | -------------- | -------------- | -| Help Command | `pihole -w --help`, `pihole -b --help`, `pihole --regex --help`, `pihole --wild --help` | +| Help Command | `pihole allow --help`, `pihole deny --help`, `pihole --regex --help`, `pihole --wild --help`, `pihole --allow-regex --help`, `pihole --allow-wild --help` | | Script Location | [`/opt/pihole/list.sh`](https://github.com/pi-hole/pi-hole/blob/master/advanced/Scripts/list.sh) | | Example Usage | [`pihole --regex '^example.com$' '.*\.example2.net'`](https://discourse.pi-hole.net/t/the-pihole-command-with-examples/738#white-black-list) | @@ -111,7 +110,7 @@ Since Pi-hole will log DNS queries by default, using this command to watch the l | Script Location | [`/opt/pihole/gravity.sh`](https://github.com/pi-hole/pi-hole/blob/master/gravity.sh) | | Example Usage | [`pihole -g`](https://discourse.pi-hole.net/t/the-pihole-command-with-examples/738#gravity) | -Gravity is one of the most important scripts of Pi-hole. Its main purpose is to retrieve blocklists, and then consolidate them into one unique list for the built-in DNS server to use, but it also serves to complete the process of manual whitelisting, blacklisting and wildcard update. It is run automatically each week, but it can be invoked manually at any time. +Gravity is one of the most important scripts of Pi-hole. Its main purpose is to retrieve subscribed lists, and then consolidate them into one unique list for the built-in DNS server to use, but it also serves to complete the process of manual allowlisting, denylisting and wildcard update. It is run automatically each week, but it can be invoked manually at any time. **Basic Script Process**: @@ -131,7 +130,7 @@ Gravity is one of the most important scripts of Pi-hole. Its main purpose is to | Script Location | [`/usr/local/bin/pihole`](https://github.com/pi-hole/pi-hole/blob/master/pihole) | | Example Usage | [`pihole logging off`](https://discourse.pi-hole.net/t/the-pihole-command-with-examples/738#logging) | -This command specifies whether the Pi-hole log should be used, by commenting out `log-queries` within `/etc/dnsmasq.d/01-pihole.conf` and flushing the log. +This command specifies whether the Pi-hole log should be used. ### Query @@ -141,13 +140,13 @@ This command specifies whether the Pi-hole log should be used, by commenting out | Script Location | [`/usr/local/bin/pihole`](https://github.com/pi-hole/pi-hole/blob/master/pihole) | | Example Usage | [`pihole -q -exact -adlist example.domain.com`](https://discourse.pi-hole.net/t/the-pihole-command-with-examples/738#adlist-query) | -This command will query your whitelist, blacklist, wildcards and adlists for a specified domain. +This command will query your allowlist, denylist, wildcards and subscribed lists for a specified domain. **Basic Script Process**: * User-specified options are handled * Using `idn`, it will convert [Internationalized domain names](https://en.wikipedia.org/wiki/Internationalized_domain_name) into [punycode](https://en.wikipedia.org/wiki/Punycode) -* Database at [`/etc/pihole/gravity.db`](../database/domain-database/index.md) is queried to return a list of adlists in which the queried domain exists. +* Database at [`/etc/pihole/gravity.db`](../database/domain-database/index.md) is queried to return a list of subscribed lists in which the queried domain exists. ### Update @@ -205,16 +204,6 @@ Display the running status of Pi-hole's DNS and blocking services. Toggle Pi-hole's ability to block unwanted domains. The disable option has the option to set a specified time before blocking is automatically re-enabled. -### Restart DNS - -| | | -| -------------- | -------------- | -| Help Command | N/A | -| Script Location | [`/usr/local/bin/pihole`](https://github.com/pi-hole/pi-hole/blob/master/pihole) | -| Example Usage | [`pihole restartdns`](https://discourse.pi-hole.net/t/the-pihole-command-with-examples/738#restartdns) | - -Restart Pi-hole's DNS service. - ### Checkout | | | @@ -236,3 +225,24 @@ Switch Pi-hole subsystems to a different GitHub branch. An admin can specify rep | Example Usage | `pihole setpassword` | Set the Web Interface an API password. Password can be entered as an option (e.g: `pihole setpassword secretpassword`), or separately as to not display on the screen (e.g: `pihole setpassword`). + +### Reload Lists + +| | | +| -------------- | -------------- | +| Help Command | N/A | +| Script Location | [`/usr/local/bin/pihole`](https://github.com/pi-hole/pi-hole/blob/master/pihole) | +| Example Usage | `pihole reloadlists` | + +Reload DNS lists. Note: This will NOT re-read any *.conf files + +### Reload DNS + +| | | +| -------------- | -------------- | +| Help Command | N/A | +| Script Location | [`/usr/local/bin/pihole`](https://github.com/pi-hole/pi-hole/blob/master/pihole) | +| Example Usage | `pihole reloaddns` | + +Flush and reload the pihole-FTL DNS cache. Note: This will NOT re-read any *.conf files + diff --git a/docs/main/prerequisites.md b/docs/main/prerequisites.md index 445d54bbd..a53bc11a5 100644 --- a/docs/main/prerequisites.md +++ b/docs/main/prerequisites.md @@ -73,14 +73,13 @@ Pi-hole needs a static IP address to properly function (a DHCP reservation is ju | Service | Port | Protocol | Notes | | --------------------|:-------------|:---------| --------------------| -| pihole-FTL | 53 (DNS) | TCP/UDP | If you happen to have another DNS server running, such as BIND, you will need to turn it off in order for Pi-hole to respond to DNS queries. | -| pihole-FTL | 67 (DHCP) | IPv4 UDP | The DHCP server is an optional feature that requires additional ports. | -| pihole-FTL | 547 (DHCPv6) | IPv6 UDP | The DHCP server is an optional feature that requires additional ports. | -| lighttpd | 80 (HTTP) | TCP | If you have another Web server already running, such as Apache, Pi-hole's Web server will not work. You can either disable the other Web server or change the port on which `lighttpd` listens, which allows you keep both Web servers running. | +| pihole-FTL | 53 (DNS) | TCP/UDP | If you happen to have another DNS server running, such as BIND, you will need to turn it off in order for Pi-hole to respond to DNS queries. | +| pihole-FTL | 67 (DHCP) | IPv4 UDP | The DHCP server is an optional feature that requires additional ports. | +| pihole-FTL | 547 (DHCPv6) | IPv6 UDP | The DHCP server is an optional feature that requires additional ports. | +| pihole-FTL | 80 (HTTP)
443 (HTTPS) | TCP | If you have another webserver already listening on port `80`/`443`, then `pihole-FTL` will attempt to bind to `8080`/`8443` instead. If neither of these ports are available, `pihole-FTL`'s webserver will be unavailable until ports are configured manually (see configuration option `webserver.port`) | | pihole-FTL | 4711 | TCP | FTL is our API engine and uses port 4711 on the localhost interface. This port should not be accessible from any other interface.| !!! info - The use of lighttpd on port _80_ is optional if you decide not to install the Web dashboard during installation. The use of pihole-FTL on ports _67_ or _547_ is optional, but required if you use the DHCP functions of Pi-hole. ### Firewalls diff --git a/docs/regex/index.md b/docs/regex/index.md index 206c99f55..ec01db8b2 100644 --- a/docs/regex/index.md +++ b/docs/regex/index.md @@ -5,32 +5,26 @@ Our implementation is light and fast as each domain is only checked once for a m ## Hierarchy of regex filters in *FTL*DNS -*FTL*DNS uses a specific hierarchy to ensure regex filters work as you expect them to. Whitelisting always has priority over blacklisting. +*FTL*DNS uses a specific hierarchy to ensure regex filters work as you expect them to. Allowlisting always has priority over denylisting. There are two locations where regex filters are important: -1. On loading the blocking domains form the `gravity` database table, *FTL*DNS skips not only exactly whitelisted domains but also those that match enabled whitelist regex filters. -2. When a queried domain matches a blacklist regex filter, the query will *not* be blocked if the domain *also* matches an exact or a regex whitelist entry. +1. On loading the blocking domains form the `gravity` database table, *FTL*DNS skips not only exactly allowlisted domains but also those that match enabled allowlist regex filters. +2. When a queried domain matches a denylist regex filter, the query will *not* be blocked if the domain *also* matches an exact or a regex allowlist entry. ## How to use regular expressions for filtering domains *FTL*DNS reads in regular expression filters from the two [`regex` database views](../database/domain-database/index.md). To tell *FTL*DNS to reload the list of regex filters, either: -- Execute `pihole restartdns reload-lists` or +- Execute `pihole reloadlists` or - Send `SIGHUP` to `pihole-FTL` (`sudo killall -SIGHUP pihole-FTL`) or -- Restart the service (`sudo service pihole-FTL restart`) +- Restart the service (`sudo service pihole-FTL restart` or `sudo systemctl restart pihole-FTL`) The first command is to be preferred as it ensures that the DNS cache itself remains intact. Hence, it is also the fastest of the available options. ## Pi-hole Regex debugging mode -To ease the usage of regular expression filters in *FTL*DNS, we offer a regex debugging mode. Set - -```plain -DEBUG_REGEX=true -``` - -in your `/etc/pihole/pihole-FTL.conf` and restart `pihole-FTL` to enable or disable this mode. +To ease the usage of regular expression filters in *FTL*DNS, we offer a regex debugging mode, run `pihole-FTL --config debug.regex true` to enable this mode. Once the debugging mode is enabled, each match will be logged to `/var/log/pihole/FTL.log` in the following format: @@ -40,4 +34,4 @@ Once the debugging mode is enabled, each match will be logged to `/var/log/pihol The given DB ID corresponds to the ID of the corresponding row in the `domainlist` database table. -Note that validation is only done on the first occurrence of a domain to increase the computational efficiency of *FTL*DNS. The result of this evaluation is stored in an internal DNS cache that is separate from `dnsmasq`'s own DNS cache. This allows us to only flush this special cache when modifying the black- and whitelists *without* having to flush the entire DNS cache collected so far. +Note that validation is only done on the first occurrence of a domain to increase the computational efficiency of *FTL*DNS. The result of this evaluation is stored in an internal DNS cache that is separate from `dnsmasq`'s own DNS cache. This allows us to only flush this special cache when modifying the denylist or allowlist *without* having to flush the entire DNS cache collected so far. diff --git a/docs/regex/pi-hole.md b/docs/regex/pi-hole.md index 904d507f3..7fb317a8e 100644 --- a/docs/regex/pi-hole.md +++ b/docs/regex/pi-hole.md @@ -22,25 +22,25 @@ but not dig A abc ``` -This allows you to do query type based black-/whitelisting. +This allows you to do query type based denylisting and allowlisting. Some user-provided examples are: - `.*;querytype=!A` - A regex blacklist entry for blocking `AAAA` (in fact, everything else than `A`, call it "anti-`A`") requests for all clients assigned to the same group. This has been mentioned to be beneficial for devices like Chromecast. You may want to fine-tune this further to specific domains. + A regex denylist entry for blocking `AAAA` (in fact, everything else than `A`, call it "anti-`A`") requests for all clients assigned to the same group. This has been mentioned to be beneficial for devices like Chromecast. You may want to fine-tune this further to specific domains. - `.*;querytype=PTR` - A regex whitelist entry used to permit `PTR` lookups with the above "anti-`A`" regex + A regex allowlist entry used to permit `PTR` lookups with the above "anti-`A`" regex - `.*;querytype=ANY` - A regex blacklist entry to block `ANY` request network wide. + A regex denylist entry to block `ANY` request network wide. - `.*;querytype=OTHER` - A regex blacklist entry to block `OTHER` request network wide. This rule will match, for instance, proprietary DNS requests using custom query types in the reserved range or queries for seldom used DNS record types like `IXFR` or `AXFR`. + A regex denylist entry to block `OTHER` request network wide. This rule will match, for instance, proprietary DNS requests using custom query types in the reserved range or queries for seldom used DNS record types like `IXFR` or `AXFR`. Note that multiple (comma-separated) query types can be specified at the same time, e.g., `.*;querytype=A,AAAA` will match both `A` and `AAAA` requests. In a similar fashion, an inverted (`!` modifier) list, e.g., `.*;querytype=!A,AAAA` will match everything *except* `A` and `AAAA` requests. @@ -72,7 +72,7 @@ Valid options are: - `;reply=nxdomain` ("no such domain" will be provided, can cause unintended side-effects) - `;reply=refused` (the query will be refused) - `;reply=none` (the query will be silently dropped) -- `;reply=ip` (the Pi-hole's IP address if not overwritten by [`REPLY_ADDR4`](../ftldns/configfile.md#reply_addr4) and/or [`REPLY_ADDR6`](../ftldns/configfile.md#reply_addr6)) +- `;reply=ip` (the Pi-hole's IP address if not overwritten by configuration options: `reply.blocking.IPv4`/`reply.blocking.IPv6`) - `;reply=1.2.3.4` (any valid IPv4 address) - `;reply=fe80::1234` (any valid IPv6 address) diff --git a/docs/regex/tutorial.md b/docs/regex/tutorial.md index 22308e2c9..cc5de8fee 100644 --- a/docs/regex/tutorial.md +++ b/docs/regex/tutorial.md @@ -1,6 +1,6 @@ # Pi-hole regular expressions tutorial -We provide a short but thorough introduction to our regular expressions implementation. This may come in handy if you are designing blocking or whitelisting rules (see also our cheat sheet below!). In our implementation, all characters match themselves except for the following special characters: `.[{}()\*+?|^$`. If you want to match those, you need to escape them like `\.` for a literal period, but no rule without exception (see character groups below for further details). +We provide a short but thorough introduction to our regular expressions implementation. This may come in handy if you are designing rules to deny or allow domains (see also our cheat sheet below!). In our implementation, all characters match themselves except for the following special characters: `.[{}()\*+?|^$`. If you want to match those, you need to escape them like `\.` for a literal period, but no rule without exception (see character groups below for further details). ## Anchors (`^` and `$`) diff --git a/mkdocs.yml b/mkdocs.yml index c8eea4e42..5242ecd82 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -207,12 +207,7 @@ nav: - 'Using Tor': guides/misc/tor/using-tor.md - 'Performance and other issues': guides/misc/tor/performance-issues.md - 'Using DNSSEC': guides/misc/tor/dnssec.md - - 'Whitelist and Blacklist editing': guides/misc/whitelist-blacklist.md - - 'Web server': - - 'NGINX': guides/webserver/nginx.md - - 'Caddy': guides/webserver/caddy.md - - 'Traefik v1 (not in Docker)': guides/webserver/traefik-nodocker.md - - 'Traefik v2 (with Docker)': guides/webserver/traefik-v2-docker.md + - 'Allowlist and Denylist editing': guides/misc/allowlist-denylist.md - 'Router setup': - 'ASUS router': routers/asus.md - 'Fritz!Box (EN)': routers/fritzbox.md @@ -264,10 +259,7 @@ plugins: 'guides/tor/using-tor.md': guides/misc/tor/using-tor.md 'guides/tor/performance-issues.md': guides/misc/tor/performance-issues.md 'guides/tor/dnssec.md': guides/misc/tor/dnssec.md - 'guides/whitelist-blacklist.md': guides/misc/whitelist-blacklist.md - 'guides/nginx-configuration.md': 'guides/webserver/nginx.md' - 'guides/caddy-configuration.md': 'guides/webserver/caddy.md' - 'guides/traefik-configuration-nodocker.md': 'guides/webserver/traefik-nodocker.md' + 'guides/whitelist-blacklist.md': guides/misc/allowlist-denylist.md 'ftldns/regex/index.md': regex/index.md 'ftldns/regex/overview.md': regex/index.md 'ftldns/regex/tutorial.md': regex/tutorial.md @@ -283,3 +275,4 @@ plugins: 'guides/vpn/openvpn/overview.md': guides/vpn/openvpn/index.md 'guides/misc/tor/overview.md': guides/misc/tor/index.md 'guides/github/contributing.md': guides/github/index.md + 'guides/misc/whitelist-blacklist.md': guides/misc/allowlist-denylist.md