[1/3] Bump grpc from 1.26.0 to 1.31.1

PART 1: prepare third_party/grpc files for new version
Composed PR: bazelbuild#12226
Note: generate_cc.bzl and protobuf.bzl are modified in place and
already affect the build. But the change seems to be harmless
(adding explicit ProtoInfo load from @rules_proto).

Fixes having external dependencies without checksum
- boringssl (each download was timestamped, but otherwise stable)
- bazel_skylark was overriden to be master (sic!)

There doesn't seem to be many breaking/big changes up to grpc 1.31.1
- removal of xds-experimental URI scheme
- removal of MAX_EPOLL_EVENTS_HANDLED_EACH_POLL_CALL
- enable TLS 1.3 in the C-core and all wrapped languages
- some of bazel-related patches got merged in
https://github.com/grpc/grpc/releases

How to check whether certain dependency has a checksum
bazel query //external:bazel_skylib --output build
bazel query //external:boringssl --output build

How to find (almost?) all problematic dependencies
compare output of
bazel query 'kind(http_archive, //external:all) + kind(http_file, //external:all) + kind(distdir_tar, //external:all)' --output xml | xq '.query.rule[] | ."@name"'
vs
bazel query 'kind(http_archive, //external:all) + kind(http_file, //external:all) + kind(distdir_tar, //external:all)' --output xml | xq '.query.rule[] | select (.string[]."@name" | contains("sha256")) | ."@name"'

Note that it looks for string sha256 and misses dict sha256 for
distdir_tar rules - those are false positive currently.

Closes bazelbuild#12235

third_party/grpc/BUILD

@@ -18,7 +18,7 @@ load("//tools/distributions:distribution_rules.bzl", "distrib_java_import", "dis
 
 licenses(["notice"])  # Apache v2
 
-exports_files(["grpc_1.26.0.patch", "upb_gcc10_fix.patch"])
+exports_files(["grpc_1.26.0.patch", "upb_gcc10_fix.patch", "grpc_1.31.1.patch"])
 
 package(default_visibility = ["//visibility:public"])
 

third_party/grpc/README.bazel.md

@@ -1,18 +1,18 @@
 # How to update the C++ sources of gRPC:
 
 1. Update the gRPC definitions in WORKSPACE file, currently we use 
-   https://github.com/grpc/grpc/archive/v1.26.0.tar.gz
+   https://github.com/grpc/grpc/archive/v1.31.1.tar.gz
 2. Update the gRPC patch file if necessary, it mostly helps avoid unnecessary dependencies.
 3. Update third_party/grpc/BUILD to redirect targets to @com_github_grpc_grpc if necessary.
 
 # How to update the BUILD/bzl sources of gRPC:
 
 1. `git clone http://github.com/grpc/grpc.git` in a convenient directory
-2. `git checkout <tag>` (current is `v1.26.0`, commithash `de893acb`)
+2. `git checkout <tag>` (current is `v1.31.1`, commithash `7d7e456762`)
 3. `mkdir -p third_party/grpc/bazel`
 4. `cp <gRPC git tree>/bazel/{BUILD,cc_grpc_library.bzl,generate_cc.bzl,protobuf.bzl} third_party/grpc/bazel`
 5. In the `third_party/grpc` directory, apply local patches:
-   `patch -p3 < bazel.patch`
+   `patch -p3 < bazel_1.31.1.patch`
 
 # How to update the Java plugin:
 

third_party/grpc/bazel.patch

@@ -31,7 +31,7 @@ diff --git a/third_party/grpc/bazel/BUILD b/third_party/grpc/bazel/BUILD
 index c3c82c9c0c..6f451a6f24 100644
 --- a/third_party/grpc/bazel/BUILD
 +++ b/third_party/grpc/bazel/BUILD
-@@ -17,3 +17,8 @@ licenses(["notice"])  # Apache v2
+@@ -17,3 +17,18 @@ licenses(["notice"])  # Apache v2
  package(default_visibility = ["//:__subpackages__"])
 
  load(":cc_grpc_library.bzl", "cc_grpc_library")
@@ -40,6 +40,16 @@ index c3c82c9c0c..6f451a6f24 100644
 +    name = "srcs",
 +    srcs = glob(["**"]),
 +)
++
++filegroup(
++    name = "embedded_tools_srcs",
++    srcs = [
++        "BUILD.tools",
++        "cc_grpc_library.bzl",
++        "generate_cc.bzl",
++        "protobuf.bzl",
++    ],
++)
 diff --git a/third_party/grpc/bazel/cc_grpc_library.bzl b/third_party/grpc/bazel/cc_grpc_library.bzl
 index dea493eaf2..0470a294fc 100644
 --- a/third_party/grpc/bazel/cc_grpc_library.bzl

third_party/grpc/bazel/generate_cc.bzl

@@ -4,6 +4,7 @@ This is an internal rule used by cc_grpc_library, and shouldn't be used
 directly.
 """
 
+load("@rules_proto//proto:defs.bzl", "ProtoInfo")
 load(
     ":protobuf.bzl",
     "get_include_directory",

third_party/grpc/bazel/protobuf.bzl

@@ -1,5 +1,7 @@
 """Utility functions for generating protobuf code."""
 
+load("@rules_proto//proto:defs.bzl", "ProtoInfo")
+
 _PROTO_EXTENSION = ".proto"
 _VIRTUAL_IMPORTS = "/_virtual_imports/"
 

third_party/grpc/bazel_1.31.1.patch

@@ -0,0 +1,312 @@
+From 6eb4a396945bbe89831ce1d205106c1348f376e3 Mon Sep 17 00:00:00 2001
+From: Mark Schaller <[email protected]>
+Date: Thu, 9 Jan 2020 19:50:57 -0500
+Subject: [PATCH] Adapt gRPC bazel files for use in Bazel
+
+This simplifies the cc_grpc_library macro and its deps:
+* removes cc_grpc_library's grpc_only parameter (now always true)
+* removes its proto_only, well_known_protos parameters (now always
+    false)
+* removes its use_external parameter (it was previously unused)
+* adds to it an extra_cc_library_kwargs parameter (forwarded to its
+    generated cc_library rule)
+* removes "well_known_proto" functionality from generate_cc.bzl and
+    protobuf.bzl
+* changes generate_cc's _protoc attribute to protoc
+* sets cc_grpc_library's generate_cc call's plugin and protoc arguments
+    to Bazel's targets ("//third_party/grpc:cpp_plugin" and
+    "//third_party/protobuf:protoc", respectively)
+* relativizes load statements throughout
+* adds "srcs" filegroup to the package's BUILD file and includes it in
+    //third_party/grpc:srcs
+
+---
+ third_party/grpc/bazel/BUILD               |   5 +
+ third_party/grpc/bazel/cc_grpc_library.bzl | 127 +++++++--------------
+ third_party/grpc/bazel/generate_cc.bzl     |  39 +------
+ third_party/grpc/bazel/protobuf.bzl        |  16 ---
+ 4 files changed, 51 insertions(+), 136 deletions(-)
+
+diff --git a/third_party/grpc/bazel/BUILD b/third_party/grpc/bazel/BUILD
+index c3c82c9c0c..6f451a6f24 100644
+--- a/third_party/grpc/bazel/BUILD
++++ b/third_party/grpc/bazel/BUILD
+@@ -17,3 +17,18 @@ licenses(["notice"])  # Apache v2
+ package(default_visibility = ["//:__subpackages__"])
+
+ load(":cc_grpc_library.bzl", "cc_grpc_library")
++
++filegroup(
++    name = "srcs",
++    srcs = glob(["**"]),
++)
++
++filegroup(
++    name = "embedded_tools_srcs",
++    srcs = [
++        "BUILD.tools",
++        "cc_grpc_library.bzl",
++        "generate_cc.bzl",
++        "protobuf.bzl",
++    ],
++)
+diff --git a/third_party/grpc/bazel/cc_grpc_library.bzl b/third_party/grpc/bazel/cc_grpc_library.bzl
+index dea493eaf2..0470a294fc 100644
+--- a/third_party/grpc/bazel/cc_grpc_library.bzl
++++ b/third_party/grpc/bazel/cc_grpc_library.bzl
+@@ -1,105 +1,58 @@
+ """Generates and compiles C++ grpc stubs from proto_library rules."""
+
+-load("@rules_proto//proto:defs.bzl", "proto_library")
+-load("//bazel:generate_cc.bzl", "generate_cc")
+-load("//bazel:protobuf.bzl", "well_known_proto_libs")
++load(":generate_cc.bzl", "generate_cc")
+
++# Simplified version of gRPC upstream's cc_grpc_library.
+ def cc_grpc_library(
+         name,
+         srcs,
+         deps,
+-        proto_only = False,
+-        well_known_protos = False,
+         generate_mocks = False,
+-        use_external = False,
+-        grpc_only = False,
++        extra_cc_library_kwargs = {},
+         **kwargs):
+-    """Generates C++ grpc classes for services defined in a proto file.
++    """Generates C++ grpc classes for services defined in proto_library rules.
+
+-    If grpc_only is True, this rule is compatible with proto_library and
+-    cc_proto_library native rules such that it expects proto_library target
+-    as srcs argument and generates only grpc library classes, expecting
+-    protobuf messages classes library (cc_proto_library target) to be passed in
+-    deps argument. By default grpc_only is False which makes this rule to behave
+-    in a backwards-compatible mode (trying to generate both proto and grpc
+-    classes).
++    This rule expects a singleton list containing a proto_library target for its
++    srcs argument, and expects a list (of arbitrary size) of cc_proto_library
++    targets for its deps argument.
++
++    It generates only grpc library classes.
+
+     Assumes the generated classes will be used in cc_api_version = 2.
+
+     Args:
+         name (str): Name of rule.
+-        srcs (list): A single .proto file which contains services definitions,
+-          or if grpc_only parameter is True, a single proto_library which
+-          contains services descriptors.
+-        deps (list): A list of C++ proto_library (or cc_proto_library) which
+-          provides the compiled code of any message that the services depend on.
+-        proto_only (bool): If True, create only C++ proto classes library,
+-          avoid creating C++ grpc classes library (expect it in deps).
+-          Deprecated, use native cc_proto_library instead. False by default.
+-        well_known_protos (bool): Should this library additionally depend on
+-          well known protos. Deprecated, the well known protos should be
+-          specified as explicit dependencies of the proto_library target
+-          (passed in srcs parameter) instead. False by default.
++        srcs (list): A single proto_library which contains services descriptors.
++        deps (list): A list of cc_proto_library targets which
++          provide the compiled code of any message that the services depend on.
+         generate_mocks (bool): when True, Google Mock code for client stub is
+           generated. False by default.
+-        use_external (bool): Not used.
+-        grpc_only (bool): if True, generate only grpc library, expecting
+-          protobuf messages library (cc_proto_library target) to be passed as
+-          deps. False by default (will become True by default eventually).
+-        **kwargs: rest of arguments, e.g., compatible_with and visibility
++        extra_cc_library_kwargs (map): extra arguments to pass to the cc_library
++          rule
++        **kwargs: extra arguments to pass to all rules instantiated by this
++          macro. Must be common to all build rules. See
++          https://docs.bazel.build/versions/master/be/common-definitions.html#common-attributes
+     """
+-    if len(srcs) > 1:
+-        fail("Only one srcs value supported", "srcs")
+-    if grpc_only and proto_only:
+-        fail("A mutualy exclusive configuration is specified: grpc_only = True and proto_only = True")
+-
+-    extra_deps = []
+-    proto_targets = []
+-
+-    if not grpc_only:
+-        proto_target = "_" + name + "_only"
+-        cc_proto_target = name if proto_only else "_" + name + "_cc_proto"
+-
+-        proto_deps = ["_" + dep + "_only" for dep in deps if dep.find(":") == -1]
+-        proto_deps += [dep.split(":")[0] + ":" + "_" + dep.split(":")[1] + "_only" for dep in deps if dep.find(":") != -1]
+-        if well_known_protos:
+-            proto_deps += well_known_proto_libs()
+-        proto_library(
+-            name = proto_target,
+-            srcs = srcs,
+-            deps = proto_deps,
+-            **kwargs
+-        )
+-
+-        native.cc_proto_library(
+-            name = cc_proto_target,
+-            deps = [":" + proto_target],
+-            **kwargs
+-        )
+-        extra_deps.append(":" + cc_proto_target)
+-        proto_targets.append(proto_target)
+-    else:
+-        if not srcs:
+-            fail("srcs cannot be empty", "srcs")
+-        proto_targets += srcs
+-
+-    if not proto_only:
+-        codegen_grpc_target = "_" + name + "_grpc_codegen"
+-        generate_cc(
+-            name = codegen_grpc_target,
+-            srcs = proto_targets,
+-            plugin = "@com_github_grpc_grpc//src/compiler:grpc_cpp_plugin",
+-            well_known_protos = well_known_protos,
+-            generate_mocks = generate_mocks,
+-            **kwargs
+-        )
+-
+-        native.cc_library(
+-            name = name,
+-            srcs = [":" + codegen_grpc_target],
+-            hdrs = [":" + codegen_grpc_target],
+-            deps = deps +
+-                   extra_deps +
+-                   ["@com_github_grpc_grpc//:grpc++_codegen_proto"],
+-            **kwargs
+-        )
++    if len(srcs) != 1:
++        fail("The srcs attribute must be a singleton list but was " + str(srcs),
++             "srcs")
++
++    codegen_grpc_target = "_" + name + "_grpc_codegen"
++    generate_cc(
++        name = codegen_grpc_target,
++        srcs = srcs,
++        plugin = "//third_party/grpc:cpp_plugin",
++        generate_mocks = generate_mocks,
++        protoc = "//third_party/protobuf:protoc",
++        **kwargs
++    )
++
++    cc_library_kwargs = dict(**extra_cc_library_kwargs)
++    cc_library_kwargs.update(**kwargs)
++    native.cc_library(
++        name = name,
++        srcs = [":" + codegen_grpc_target],
++        hdrs = [":" + codegen_grpc_target],
++        deps = deps + ["//third_party/grpc:grpc++_codegen_proto"],
++        **cc_library_kwargs
++    )
+diff --git a/third_party/grpc/bazel/generate_cc.bzl b/third_party/grpc/bazel/generate_cc.bzl
+index 484959ebb7..38a5b460f9 100644
+--- a/third_party/grpc/bazel/generate_cc.bzl
++++ b/third_party/grpc/bazel/generate_cc.bzl
+@@ -5,7 +5,7 @@ directly.
+ """
+
+ load(
+-    "//bazel:protobuf.bzl",
++    ":protobuf.bzl",
+     "get_include_directory",
+     "get_plugin_args",
+     "get_proto_root",
+@@ -117,34 +117,17 @@ def generate_cc_impl(ctx):
+     arguments += ["--proto_path={0}{1}".format(dir_out, proto_root)]
+     arguments += [_get_srcs_file_path(proto) for proto in protos]
+
+-    # create a list of well known proto files if the argument is non-None
+-    well_known_proto_files = []
+-    if ctx.attr.well_known_protos:
+-        f = ctx.attr.well_known_protos.files.to_list()[0].dirname
+-        if f != "external/com_google_protobuf/src/google/protobuf":
+-            print(
+-                "Error: Only @com_google_protobuf//:well_known_protos is supported",
+-            )
+-        else:
+-            # f points to "external/com_google_protobuf/src/google/protobuf"
+-            # add -I argument to protoc so it knows where to look for the proto files.
+-            arguments += ["-I{0}".format(f + "/../..")]
+-            well_known_proto_files = [
+-                f
+-                for f in ctx.attr.well_known_protos.files.to_list()
+-            ]
+-
+     ctx.actions.run(
+-        inputs = protos + includes + well_known_proto_files,
++        inputs = protos + includes,
+         tools = tools,
+         outputs = out_files,
+-        executable = ctx.executable._protoc,
++        executable = ctx.executable.protoc,
+         arguments = arguments,
+     )
+
+     return struct(files = depset(out_files))
+
+-_generate_cc = rule(
++generate_cc = rule(
+     attrs = {
+         "srcs": attr.label_list(
+             mandatory = True,
+@@ -160,13 +143,12 @@ _generate_cc = rule(
+             mandatory = False,
+             allow_empty = True,
+         ),
+-        "well_known_protos": attr.label(mandatory = False),
+         "generate_mocks": attr.bool(
+             default = False,
+             mandatory = False,
+         ),
+-        "_protoc": attr.label(
+-            default = Label("//external:protocol_compiler"),
++        "protoc": attr.label(
++            mandatory = True,
+             executable = True,
+             cfg = "host",
+         ),
+@@ -175,12 +157,3 @@ _generate_cc = rule(
+     output_to_genfiles = True,
+     implementation = generate_cc_impl,
+ )
+-
+-def generate_cc(well_known_protos, **kwargs):
+-    if well_known_protos:
+-        _generate_cc(
+-            well_known_protos = "@com_google_protobuf//:well_known_protos",
+-            **kwargs
+-        )
+-    else:
+-        _generate_cc(**kwargs)
+diff --git a/third_party/grpc/bazel/protobuf.bzl b/third_party/grpc/bazel/protobuf.bzl
+index 7af27a8b30..0d6a4135f0 100644
+--- a/third_party/grpc/bazel/protobuf.bzl
++++ b/third_party/grpc/bazel/protobuf.bzl
+@@ -3,22 +3,6 @@
+ _PROTO_EXTENSION = ".proto"
+ _VIRTUAL_IMPORTS = "/_virtual_imports/"
+
+-def well_known_proto_libs():
+-    return [
+-        "@com_google_protobuf//:any_proto",
+-        "@com_google_protobuf//:api_proto",
+-        "@com_google_protobuf//:compiler_plugin_proto",
+-        "@com_google_protobuf//:descriptor_proto",
+-        "@com_google_protobuf//:duration_proto",
+-        "@com_google_protobuf//:empty_proto",
+-        "@com_google_protobuf//:field_mask_proto",
+-        "@com_google_protobuf//:source_context_proto",
+-        "@com_google_protobuf//:struct_proto",
+-        "@com_google_protobuf//:timestamp_proto",
+-        "@com_google_protobuf//:type_proto",
+-        "@com_google_protobuf//:wrappers_proto",
+-    ]
+-
+ def get_proto_root(workspace_root):
+     """Gets the root protobuf directory.
+
+-- 
+2.25.0.rc1.283.g88dfdc4193-goog
+