kv: avoid internal retries for permanent errors

[erikgrinaker]

The KV layer often retries RPC errors internally, but tries to error out early on permanent errors, e.g.:

cockroach/pkg/kv/kvclient/rangecache/range_cache.go

Lines 1326 to 1329 in d5f73c6

	func IsRangeLookupErrorRetryable(err error) bool {
	// For now, all errors are retryable except authentication/authorization.
	return !grpcutil.IsAuthError(err)
	}

However, coverage of this is often spotty, which can cause operations to hang indefinitely. This can often be seen when decommissioning a node and then trying to run operations via that node -- these will often error internally with codes.PermissionDenied, but since the error isn't always considered permanent and returned early, it is retried internally and the operation hangs.

This problem is exacerbated by the RPC layer often wrapping errors in roachpb.Error internally:

cockroach/pkg/kv/kvclient/kvcoord/dist_sender.go

Lines 1446 to 1448 in d5f73c6

	if err != nil {
	return response{pErr: roachpb.NewError(err)}
	}

These errors do not propagate gRPC error codes because of cockroachdb/errors#63. As mentioned in #56208, we also often rely on checking the error message because of the lack of structured errors, which is brittle and should also be avoided.