Native DNS caching for lookups

[robertschultz]

Will we ever see caching in the native dns module? We have noticed a decent amount of overhead when making many parallel calls to the same dns and the dns module is doing lookups.

I understand the idea around this module is to be unopinionated but it might be a feature typical users might want to enable w/o using something like bind.

[mscdex]

What about using dnsmasq? Also for net.Sockets for example, you can specify your own lookup() function when connecting which could utilize a custom cache.

[robertschultz]

@mscdex In regards to custom lookup() that's true. My request really is for the general masses, and the overhead 99.99% of people would have using the stock dns module. My main argument here is that the request module is using things like url.resolve so I'd have to monkey patch this or something.

[bompus]

+1

[justsml]

While this sounds nice, and i'd love to have an option to just 'turn on fast DNS'.
This would really burden Node's core. DNS is a well established tech, with really significant thought put into cache, expiration, failure handling, etc. Making this builtin would fight the way it's supposed to be utilized: as @mscdex mentioned, run a local dnsmasq cache - or any other DNS cache - again this should be a system wide design decision.
For example, It would be VERY frustrating if your nodejs process could use a different IP when debugging side-by-side using a curl cmd.
The reasons against this are numerous, and could open the system up to security issues, race conditions, inconsistent data, etc.

Just set up a local resolver, let it do what it does best. As a bonus your whole system will operate faster.

[mscdex]

@justsml You wouldn't have to use it system-wide unless you wanted to. node's dns module supports setting explicit nameservers (at least for dns.resolve()).

[robertschultz]

@justsml I'm in favor of a local resolver as well, just calling this out if it was worth bringing it up. Thanks.

[benjamingr]

I'm -1 on this, this is something that userland modules should solve and caching bugs are worse than performance bugs.

I think the caching should be done outside the dns module anywy - as in - the DNS module always works "as advertised" and performs a clean lookup to the nameservers.

[benjamingr]

Here is a userland solution that works https://www.npmjs.com/package/dnscache

[jasnell]

Given the responses it's unlikely that we'll end up doing this. Going to close. Feel free to reopen if necessary.

[reallistic]

Just to add my two cents in, it is very dangerous to use the dnscache package because it does not respect the ttl set by the dns server. This means that you could potentially have a bad dns cache for some time. By implementing a OS level solution, you'll get all the benefits mentioned above in addition to the reassurance that your dns entries will always be correct.

[jbergstroem]

@reallistic couldn't dnscache be patched then? Reading above it seems unlikely to land in core.

[reallistic]

@jbergstroem what I mean is that I agree with the comments above that this shouldn't fall on Node.js' hands and also warning that the dnscache module suggested could cause more problems.

[PaquitoSoft]

Based on the comments exposed here about not implementing DNS cache on nodejs core and the comment from @reallistic about potential problems using dnscache module, I would like to know how can we get the TTL set from the DNS server so we can correctly implement DNS caching in our applications.
The dns.lookup function will only return and address (and a family) but not that value. So, if we want to honor the TTL the DNS server sets, what would be the best approach?

Thanks!

[bnoordhuis]

@PaquitoSoft dns.lookup() uses getaddrinfo(3) which doesn't report the TTL. For the A and AAAA flavors of dns.resolve() we can retrieve it though. I've opened #9296 for that.

[PaquitoSoft]

@bnoordhuis thanks!
It sounds great.

[reallistic]

@PaquitoSoft I highly recommend fixing this at the OS level. Using bind, dnsmasq, or something similar.

[eduardbme]

Take a look at https://www.npmjs.com/package/lookup-dns-cache

[benjamingr]

@bnoordhuis can this be closed given that #9296 landed?

[bnoordhuis]

@benjamingr It's closed already.

[titanism]

We released 🍊 Tangerine today, which is a userland package tangerine that solves DNS caching for Node.js. It's a ⚡ faster ⚡ 1:1 drop-in replacement for dns.promises.Resolver using DNS over HTTPS ("DoH") via undici with built-in retries, timeouts, smart server rotation, AbortControllers, and caching support for multiple backends via Keyv.

npm install tangerine

-import dns from 'dns';
+import Tangerine from 'tangerine';

- const resolver = new dns.promises.Resolver();
+const resolver = new Tangerine();

Documentation, API, options, source code, tests, benchmarks, and more available at https://github.com/forwardemail/tangerine.