Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[1.15.x] grpc: ensure grpc resolver correctly uses lan/wan addresses on servers #17285

Merged
merged 4 commits into from
May 11, 2023
Merged

[1.15.x] grpc: ensure grpc resolver correctly uses lan/wan addresses on servers #17285

merged 4 commits into from
May 11, 2023

Commits on May 10, 2023

  1. [1.15.x] grpc: ensure grpc resolver correctly uses lan/wan addresses … 

    …on servers

The grpc resolver implementation is fed from changes to the
router.Router. Within the router there is a map of various areas storing
the addressing information for servers in those areas. All map entries
are of the WAN variety except a single special entry for the LAN.

Addressing information in the LAN "area" are local addresses intended
for use when making a client-to-server or server-to-server request.

The client agent correctly updates this LAN area when receiving lan serf
events, so by extension the grpc resolver works fine in that scenario.

The server agent only initially populates a single entry in the LAN area
(for itself) on startup, and then never mutates that area map again.
For normal RPCs a different structure is used for LAN routing.

Additionally when selecting a server to contact in the local datacenter
it will randomly select addresses from either the LAN or WAN addressed
entries in the map.

Unfortunately this means that the grpc resolver stack as it exists on
server agents is either broken or only accidentally functions by having
servers dial each other over the WAN-accessible address. If the operator
disables the serf wan port completely likely this incidental functioning
would break.

This PR enforces that local requests for servers (both for stale reads
or leader forwarded requests) exclusively use the LAN "area" information
and also fixes it so that servers keep that area up to date in the
router.

A test for the grpc resolver logic was added, as well as a higher level
full-stack test to ensure the externally perceived bug does not return.
    @rboyer
    rboyer committed May 10, 2023
    Configuration menu
    Copy the full SHA
    37653c7 View commit details
    Browse the repository at this point in the history

  2. add changelog

    @rboyer
    rboyer committed May 10, 2023
    Configuration menu
    Copy the full SHA
    a2735b8 View commit details
    Browse the repository at this point in the history

  3. add comment

    @rboyer
    rboyer committed May 10, 2023
    Configuration menu
    Copy the full SHA
    f5a9f7e View commit details
    Browse the repository at this point in the history

  4. simplify fix

    @rboyer
    rboyer committed May 10, 2023
    Configuration menu
    Copy the full SHA
    d3107be View commit details
    Browse the repository at this point in the history