Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NPM] Add Enhanced TLS Tags #31464

Merged
merged 60 commits into from
Dec 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
ae9b4e5
init
akarpz Oct 16, 2024
a2ec958
rework collection logic to use existing parsing
akarpz Oct 17, 2024
2768c3f
fix usm only map init
akarpz Oct 18, 2024
8b3f437
rework tests and collection
akarpz Oct 18, 2024
69ff889
store tls tags in map
akarpz Oct 21, 2024
3648687
fix processing of server hello packets
akarpz Oct 22, 2024
2c5bf65
fall back to TLS version field if extensions aren't present for client
akarpz Oct 23, 2024
5e7e0e1
Merge branch 'main' into akarpowich/tls_expanded_tags
akarpz Nov 6, 2024
d80f672
userspace changes to add tags to proto conn
akarpz Nov 11, 2024
152ae54
clean up and test for presence of version tag
akarpz Nov 12, 2024
9e31074
add comments
akarpz Nov 12, 2024
ad4d31c
package comment
akarpz Nov 12, 2024
0cee274
cleanup
akarpz Nov 12, 2024
9f834c5
move helper
akarpz Nov 13, 2024
60a1d92
Merge branch 'main' into akarpowich/tls_expanded_tags
akarpz Nov 13, 2024
c26c229
unroll loops and fix windows build
akarpz Nov 13, 2024
9576add
convert to for loops
akarpz Nov 14, 2024
344993d
reduce max extensions parsed
akarpz Nov 14, 2024
d3b426c
further attempts to limit stack usage
akarpz Nov 15, 2024
3b197d9
comment
akarpz Nov 18, 2024
0889697
test with 2 loops to validate old kernels
akarpz Nov 19, 2024
708d6b1
try defining loop variables outside unroll
akarpz Nov 19, 2024
69e0d3d
unroll inner loop
akarpz Nov 19, 2024
3e65bac
set max tags to 8 again
akarpz Nov 19, 2024
aabd648
6 extensions parsed
akarpz Nov 20, 2024
dc42d08
Merge branch 'main' into akarpowich/tls_expanded_tags
akarpz Nov 20, 2024
fc930a8
stop unrolling loops
akarpz Nov 26, 2024
c9da0b3
raise min version
akarpz Nov 26, 2024
af58525
fix routing and tagging, increase max extensions
akarpz Dec 2, 2024
91b33df
bail early for appdata
akarpz Dec 3, 2024
dc80849
make test less brittle
akarpz Dec 3, 2024
502f0b6
fix existing usm classification
akarpz Dec 3, 2024
a424632
reduce max extensions
akarpz Dec 3, 2024
1506f87
use go tls library for constants
akarpz Dec 3, 2024
c5a48a2
stab at disabling RC
akarpz Dec 3, 2024
064cc82
Revert "stab at disabling RC"
akarpz Dec 3, 2024
3d15f66
set max extensions to 1
akarpz Dec 3, 2024
253dadd
Merge branch 'main' into akarpowich/tls_tail_calls
akarpz Dec 3, 2024
8e98fa2
increase extension count
akarpz Dec 3, 2024
52a9e5f
refactor tls packet parsing
akarpz Dec 4, 2024
c9079c5
add release note
akarpz Dec 4, 2024
8bb5cb4
add UT for failure scenario
akarpz Dec 4, 2024
1e9c820
reduce diff slightly
akarpz Dec 4, 2024
4bd3505
revert some non-functional changes
akarpz Dec 5, 2024
0e55408
update releasenotes
akarpz Dec 5, 2024
300686d
use map cleaner for tls tags
akarpz Dec 9, 2024
8ab745f
optimize memory allocation pattern for tls tags
akarpz Dec 10, 2024
600f0d8
Merge branch 'main' into akarpowich/tls_tail_calls
akarpz Dec 10, 2024
8f65b8a
improve comments
akarpz Dec 11, 2024
58b6912
use skb_info.data_end instead of skb_len, clean up tests, more comments
akarpz Dec 12, 2024
f6670a7
Merge branch 'main' into akarpowich/tls_tail_calls
akarpz Dec 12, 2024
8cb4e43
store just content type byte instead of tls record header
akarpz Dec 12, 2024
d97e272
appease linter
akarpz Dec 13, 2024
b063757
fix typo and unexport vars
akarpz Dec 16, 2024
7b42eb0
subset of review comments addressed
akarpz Dec 19, 2024
1cefe66
numerous changes to docs and removal of change to usm_ctx
akarpz Dec 19, 2024
c52e48e
fix typos
akarpz Dec 19, 2024
26f867d
Merge branch 'main' into akarpowich/tls_tail_calls
akarpz Dec 20, 2024
fd3800b
even stricter validation of tls handshake classification, ascii art f…
akarpz Dec 20, 2024
e547542
use u32s in tls parsing, fix tests
akarpz Dec 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions pkg/network/ebpf/c/protocols/classification/defs.h
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ typedef enum {
// Each `protocol_t` entry is implicitly associated to a single
// `protocol_layer_t` value (see notes above).
//
//In order to determine which `protocol_layer_t` a `protocol_t` belongs to,
// In order to determine which `protocol_layer_t` a `protocol_t` belongs to,
// users can call `get_protocol_layer`
typedef enum {
LAYER_UNKNOWN,
Expand All @@ -103,7 +103,7 @@ typedef struct {
// `protocol_stack_t` is embedded in the `conn_stats_t` type, which is used
// across the whole NPM kernel code. If we added the 64-bit timestamp field
// directly to `protocol_stack_t`, we would go from 4 bytes to 12 bytes, which
// bloats the eBPF stack size of some NPM probes. Using the wrapper type
// bloats the eBPF stack size of some NPM probes. Using the wrapper type
// prevents that, because we pretty much only store the wrapper type in the
// connection_protocol map, but elsewhere in the code we're still using
// protocol_stack_t, so this is change is "transparent" to most of the code.
Expand All @@ -123,6 +123,8 @@ typedef enum {
CLASSIFICATION_GRPC_PROG,
__PROG_ENCRYPTION,
// Encryption classification programs go here
CLASSIFICATION_TLS_CLIENT_PROG,
CLASSIFICATION_TLS_SERVER_PROG,
CLASSIFICATION_PROG_MAX,
} classification_prog_t;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -160,15 +160,26 @@ __maybe_unused static __always_inline void protocol_classifier_entrypoint(struct

protocol_t app_layer_proto = get_protocol_from_stack(protocol_stack, LAYER_APPLICATION);

if ((app_layer_proto == PROTOCOL_UNKNOWN || app_layer_proto == PROTOCOL_POSTGRES) && is_tls(buffer, usm_ctx->buffer.size, skb_info.data_end)) {
tls_record_header_t tls_hdr = {0};

if ((app_layer_proto == PROTOCOL_UNKNOWN || app_layer_proto == PROTOCOL_POSTGRES) && is_tls(skb, skb_info.data_off, skb_info.data_end, &tls_hdr)) {
guyarb marked this conversation as resolved.
Show resolved Hide resolved
protocol_stack = get_or_create_protocol_stack(&usm_ctx->tuple);
if (!protocol_stack) {
return;
}
// TLS classification
update_protocol_information(usm_ctx, protocol_stack, PROTOCOL_TLS);
// The connection is TLS encrypted, thus we cannot classify the protocol
// using the socket filter and therefore we can bail out;
if (tls_hdr.content_type != TLS_HANDSHAKE) {
// We can't classify TLS encrypted traffic further, so return early
update_protocol_information(usm_ctx, protocol_stack, PROTOCOL_TLS);
return;
}

// Parse TLS handshake payload
tls_info_t *tags = get_or_create_tls_enhanced_tags(&usm_ctx->tuple);
if (tags) {
// The packet is a TLS handshake, so trigger tail calls to extract metadata from the payload
goto next_program;
}
return;
}

Expand Down Expand Up @@ -200,6 +211,58 @@ __maybe_unused static __always_inline void protocol_classifier_entrypoint(struct
classification_next_program(skb, usm_ctx);
}

__maybe_unused static __always_inline void protocol_classifier_entrypoint_tls_handshake_client(struct __sk_buff *skb) {
usm_context_t *usm_ctx = usm_context(skb);
if (!usm_ctx) {
return;
}
tls_info_t* tls_info = get_tls_enhanced_tags(&usm_ctx->tuple);
if (!tls_info) {
goto next_program;
}
__u32 offset = usm_ctx->skb_info.data_off + sizeof(tls_record_header_t);
__u32 data_end = usm_ctx->skb_info.data_end;
if (!is_tls_handshake_client_hello(skb, offset, usm_ctx->skb_info.data_end)) {
goto next_program;
}
if (!parse_client_hello(skb, offset, data_end, tls_info)) {
return;
guyarb marked this conversation as resolved.
Show resolved Hide resolved
}

next_program:
classification_next_program(skb, usm_ctx);
}

__maybe_unused static __always_inline void protocol_classifier_entrypoint_tls_handshake_server(struct __sk_buff *skb) {
usm_context_t *usm_ctx = usm_context(skb);
if (!usm_ctx) {
return;
}
tls_info_t* tls_info = get_tls_enhanced_tags(&usm_ctx->tuple);
if (!tls_info) {
goto next_program;
}
__u32 offset = usm_ctx->skb_info.data_off + sizeof(tls_record_header_t);
__u32 data_end = usm_ctx->skb_info.data_end;
if (!is_tls_handshake_server_hello(skb, offset, data_end)) {
goto next_program;
}
if (!parse_server_hello(skb, offset, data_end, tls_info)) {
return;
}

protocol_stack_t *protocol_stack = get_protocol_stack_if_exists(&usm_ctx->tuple);
if (!protocol_stack) {
return;
}
update_protocol_information(usm_ctx, protocol_stack, PROTOCOL_TLS);
// We can't classify TLS encrypted traffic further, so return early
return;

next_program:
classification_next_program(skb, usm_ctx);
}

__maybe_unused static __always_inline void protocol_classifier_entrypoint_queues(struct __sk_buff *skb) {
usm_context_t *usm_ctx = usm_context(skb);
if (!usm_ctx) {
Expand Down
31 changes: 4 additions & 27 deletions pkg/network/ebpf/c/protocols/classification/routing-helpers.h
Original file line number Diff line number Diff line change
Expand Up @@ -18,29 +18,7 @@ static __always_inline bool has_available_program(classification_prog_t current_
return true;
}

#pragma clang diagnostic push
// The following check is ignored because *currently* there are no API or
// Encryption classification programs registerd.
// Therefore the enum containing all BPF programs looks like the following:
//
// typedef enum {
// CLASSIFICATION_PROG_UNKNOWN = 0,
// __PROG_APPLICATION,
// APPLICATION_PROG_A
// APPLICATION_PROG_B
// APPLICATION_PROG_C
// ...
// __PROG_API,
// // No programs here
// __PROG_ENCRYPTION,
// // No programs here
// CLASSIFICATION_PROG_MAX,
// } classification_prog_t;
//
// Which means that the following conditionals will always evaluate to false:
// a) current_program > __PROG_API && current_program < __PROG_ENCRYPTION
// b) current_program > __PROG_ENCRYPTION && current_program < CLASSIFICATION_PROG_MAX
#pragma clang diagnostic ignored "-Wtautological-overlap-compare"
// get_current_program_layer returns the layer bit of the current program
static __always_inline u16 get_current_program_layer(classification_prog_t current_program) {
if (current_program > __PROG_APPLICATION && current_program < __PROG_API) {
return LAYER_APPLICATION_BIT;
Expand All @@ -56,20 +34,19 @@ static __always_inline u16 get_current_program_layer(classification_prog_t curre

return 0;
}
#pragma clang diagnostic pop

static __always_inline classification_prog_t next_layer_entrypoint(usm_context_t *usm_ctx) {
u16 to_skip = usm_ctx->routing_skip_layers;

if (!(to_skip&LAYER_ENCRYPTION_BIT)) {
return __PROG_ENCRYPTION+1;
}
guyarb marked this conversation as resolved.
Show resolved Hide resolved
if (!(to_skip&LAYER_APPLICATION_BIT)) {
return __PROG_APPLICATION+1;
}
if (!(to_skip&LAYER_API_BIT)) {
return __PROG_API+1;
}
if (!(to_skip&LAYER_ENCRYPTION_BIT)) {
return __PROG_ENCRYPTION+1;
}

return CLASSIFICATION_PROG_UNKNOWN;
}
Expand Down
Loading
Loading