Ssprod 23814 add addl event init method #1052

Signed-off-by: Andrea Terzolo <[email protected]>

This change increases the number of retries to retrieve container information from CRI API from 3 to 5, as several failures were observed with the maximum number of attempts set to 3. Signed-off-by: Iacopo Rozzo <[email protected]>

Signed-off-by: Melissa Kilby <[email protected]>

Co-authored-by: Andrea Terzolo <[email protected]> Signed-off-by: Melissa Kilby <[email protected]>

Signed-off-by: Melissa Kilby <[email protected]>

This change makes sure that 5 maximum retries to retrieve container information are used with CRI only. It puts back the number of retries to 3 for all the other container runtimes. It also adjusts the maximum time to complete all their attempts to take into account the increased of retries. Signed-off-by: Iacopo Rozzo <[email protected]>

Signed-off-by: Andrea Terzolo <[email protected]>

Signed-off-by: Andrea Terzolo <[email protected]> Co-authored-by: Hendrik Brueckner <[email protected]> Co-authored-by: Mauro Ezequiel Moltrasio <[email protected]>

Signed-off-by: Andrea Terzolo <[email protected]> Co-authored-by: Hendrik Brueckner <[email protected]>

Signed-off-by: Grzegorz Nosek <[email protected]>

Signed-off-by: Andrea Terzolo <[email protected]>

Signed-off-by: Jason Dellaluce <[email protected]>

…m target Signed-off-by: Andrea Terzolo <[email protected]>

Signed-off-by: Andrea Terzolo <[email protected]>

… seen by drivers Signed-off-by: Andrea Terzolo <[email protected]>

Signed-off-by: Andrea Terzolo <[email protected]>

Signed-off-by: Luca Guerra <[email protected]>

Signed-off-by: Adnan Ali <[email protected]>

Signed-off-by: Luca Guerra <[email protected]>

Signed-off-by: Andrea Terzolo <[email protected]>

…name Signed-off-by: Jason Dellaluce <[email protected]>

Signed-off-by: Andrea Terzolo <[email protected]>

Signed-off-by: Andrea Terzolo <[email protected]> Co-authored-by: Hendrik Brueckner <[email protected]>

Signed-off-by: Andrea Terzolo <[email protected]>

Signed-off-by: Federico Di Pierro <[email protected]>

Signed-off-by: Adnan Ali <[email protected]>

Signed-off-by: Jason Dellaluce <[email protected]>

Signed-off-by: Federico Di Pierro <[email protected]>

Signed-off-by: Aldo Lacuku <[email protected]>

… memory management, allocations Signed-off-by: Luca Guerra <[email protected]> Co-authored-by: Roberto Scolaro <[email protected]> Co-authored-by: Federico Di Pierro <[email protected]>

Signed-off-by: Luca Guerra <[email protected]> Co-authored-by: Federico Di Pierro <[email protected]> Co-authored-by: Roberto Scolaro <[email protected]>

… cleaned by the destructor Signed-off-by: Luca Guerra <[email protected]>

Signed-off-by: Luca Guerra <[email protected]> Co-authored-by: Federico Di Pierro <[email protected]> Co-authored-by: Roberto Scolaro <[email protected]>

Co-authored-by: Jason Dellaluce <[email protected]> Signed-off-by: Luca Guerra <[email protected]>

Signed-off-by: Luca Guerra <[email protected]> Co-authored-by: Jason Dellaluce <[email protected]> Co-authored-by: Roberto Scolaro <[email protected]>

Signed-off-by: Melissa Kilby <[email protected]>

Signed-off-by: Luca Guerra <[email protected]>

Attributes in "struct device*" are now const, so add a matching prototype for ppm_devnode(). Fixes #918 Signed-off-by: Holger Hoffstätte <[email protected]>

NOTE: this needed a small fix in release branch because `bpf_probe_read_kernel` is not present there. Signed-off-by: Federico Di Pierro <[email protected]>

NOTE: it needed some fixes because `__bpf_val_to_ring` is now taking an enum as last param in master. Signed-off-by: Federico Di Pierro <[email protected]> Co-authored-by: Andrea Terzolo <[email protected]>

Signed-off-by: Federico Di Pierro <[email protected]>

Signed-off-by: Roberto Scolaro <[email protected]>

… memory management, allocations Signed-off-by: Luca Guerra <[email protected]> Co-authored-by: Roberto Scolaro <[email protected]> Co-authored-by: Federico Di Pierro <[email protected]>

- Support terminating scan after specified timeout - Support periodic log messages to report progress - API to specify timeout, log interval, and log function - Add last PID and total FDs processed, to /proc scan progress messages - Enhance scap_open args and logic to record debug_log_fn and parameters - Reworked /proc scan to reduce complexity and nesting depth - Pass through API to specify log/timeout parameters to libscap /proc scan Signed-off-by: Joseph Pittman <[email protected]>

Changes to support tracking how a container was configured with an initial user and make that info available as user.name for CONTAINER_JSON events: 1. Add a "container user" field m_container_user to container_info. By default, the value is "<NA>". 2. In the docker and cri container engine resolvers, parse any configured user info out of the json response and set m_container_user. 3. Serialize the parsed username to the json blob that comprises a CONTAINER_JSON event, and parse it out of the json blob when parsing a CONTAINER_JSON event. 4. When creating the fake threadinfo that is attached to a container event, also set m_exe to "container:<id>". 5. For the proc.name filtercheck, if the event type is container_json, return not the thread uid but the container user. This ends up being more robust in the face of containers where the initial process might exec and then setuid than a different user. This tracks the configured user rather than the uids of processes in the container, which might change. Signed-off-by: Mark Stemm <[email protected]>

Signed-off-by: vadim.zyarko <[email protected]>

Signed-off-by: Grzegorz Nosek <[email protected]>

This reverts a part of c4370e7

* Make sinsp remove_inactive_threads() method public In the Sysdig agent's Secure_Light mode, the agent_diagnostic_metrics_sender functions as a lightweight version of the sinsp_analyzer, and needs similar access to the private method remove_inactive_threads(). This PR makes the method public.

New functionality pulled from falcosecurity/libs, introduces the ability for the scap_open_* callers to specify initial system calls of interest; if none are specified, default behavior is supposed to match revert to 'capture all system calls'. But due to a bug in the falcosecurity/libs code, the 'default' behavior omits the capturing of some system calls, including execve. This change implements a workaround to the bug, forcing the correct default.

…390x (zLinux) architectures - Fix Little-Endian byte-ordering assumptions in HTTP parsing and IP address validation code - Adjust set of LUA APIs used, to allow interoperability with the different LUA versions available for the different architectures [SMAGENT-3292] Fix unterminated comment block to re-enable RAW_BREAKPOINTS for x86

Sysdig agent containers use clang-7 to build the eBPF probe. That version of clang is too old to support some of the compiler attributes which appear in the source code for more modern kernel versions. So disable warnings associated with unknown attributes.

… eliminate repeated string concats.

such messages might end up flooding kernel logs. Silence them unless the "verbose" module parameter is enabled. Also, add information about the consumer. Signed-off-by: Gerlando Falauto <[email protected]>

When we take the image info from imageRef and it does not contain the tag, we properly set `m_imagetag` but `m_image` is left without the tag. Fix that by appending `:tag` to m_image when needed.

On ARM, Linux does not generate EXECVE_EXIT events, nor does it generate CLONE_EXIT events to the child process. libsinsp relies upon these events to maintain complete and up-to-date threadinfo state. Workaround this issue by using /proc scanning to fill in threadinfo state after CLONE and EXEC.

zLinux (Linux on s390x) does not generate CLONE_EXIT_TO_CHILD ptrace events. Enable the existing workaround for this misbehavior, for the s390x platform.

On ARM and s390x platforms, Linux ptrace fails to report CLONE_EXIT_TO_CHILD events. Workaround logic may trigger TID collision logic in CLONE_EXIT_TO_PARENT handler, for certain expected cases. In these expected cases, we want the benefits of the TID collision logic -- deleting and replacing partially-populated child threadinfo, but not the TID collision logging and watchdog timer behavior. The logic to detect these expected cases and avoid the logging, had a bug that caused some expected cases to not be detected.

…se)` See actions/checkout#760 for context

procfs_utils.ut.cpp was being included in the main list and also added for not-MINIMAL builds. It should only be added.

Not needed in draios builds for the moment.

…#93) - Falcosecurity/libs #416: Support execve exit and clone child exit events on ARM64 - Falcosecurity/libs #418: Enable 64BIT_ARGS_SINGLE_REGISER on ARM64 - Also, disable userspace workarounds ARM, which attempted to compensate for the missing execve/clone exit events

Have sinsp_container_lookup with what was sinsp_container_lookup_state inside. Also introduce convenience methods. Signed-off-by: Angelo Puglisi <[email protected]>

This is still used in analyzer_thread.cpp so keep it in our fork.

* fix(container_engine): Only return on success or all retries failed Instead of always returning a result on the first attempt, only return results on success or when all retries have failed. This prevents spurious "container" events for incomplete results. This is especially important when both docker and cri are enabled, when both must be tried due to the cgroup pattern overlapping, but only one actually holds the container. Signed-off-by: Mark Stemm <[email protected]> * Log a warning when empty container infos are returned When empty container infos are passed up due to all attempts failing, log a warining. This will help highlight cases when the communication with the container runtime isn't working properly. Signed-off-by: Mark Stemm <[email protected]> * Add debug log to note when a lookup is async or sync The "async_xxx" refers to the code that performs the lookup (we used to have a separate "docker" engine, but it's been removed. To make it more clear about whether a lookup is synchronous or asynchronous, add a debug log. Signed-off-by: Mark Stemm <[email protected]> * Use bundled valijson for "regular" build valijson doesn't really have an ubuntu package, so it can't be preinstalled. Use the bundled valijson instead. * Add RE2 to container used for builds + tests This way it will be present when building with -DUSE_BUNDLED_DEPS=False Signed-off-by: Mark Stemm <[email protected]>

This reverts commit 35d80de. It was probably causing some container runtime tests to fail.

Userspace workaround for Linux kernel behaviors on ARM and zLinux, was not fully effective, and has since been obviated by kernel driver/eBPF probe logic to generate missing scap events by other means. So this changeset removes the userspace workaround.

* Revert "Revert "Merge upstream pr 688 (#121)" (#122)" This reverts commit c8dbbf3. This adds the fix back. I'll test with an agent PR that updates/removes the tests. * Add the ability to "defer" an async lookup In some cases, the "server" code running run_impl might want to retry its work until later. The current version can't do that--once a key is dequeued using deque_next_key, it has to call store_value or lose the request. To make retries easier, add a method defer_lookup that pushes the key (and optional value) back onto the request queue with a configurable delay. After delay, the key will be pulled again with a call to dequeue_next_key(). Signed-off-by: Mark Stemm <[email protected]> * Use defer_lookup for container info retry instead of lookup_delayed When the container async lookup class wants to retry a lookup, the current version tries to use lookup_delayed to initiate a new request. It turns out that that doesn't work--if there's already an existing request in m_value_map, it assumes that the "server" doing run_impl will eventually return an answer, and doesn't add a request to the queue. The solution is to use the newly added lookup_delayed instead, which pushes the request back onto the queue with a short delay. Signed-off-by: Mark Stemm <[email protected]> * Use a separate max_wait_ms instead of re-using s_cri_timeout Now that timeouts are working, it may take several seconds for subsequent retries to complete. However, s_cri_timeout (typically 1 second) was being used for the max_wait_ms in cri_async_source. That would mean that a lookup would expire before the server side had retried the lookup. The solution is to use a separate 10 second max_wait_ms, which matches docker. Signed-off-by: Mark Stemm <[email protected]> Signed-off-by: Mark Stemm <[email protected]>

See falcosecurity/libs#677

It isn't on Windows. Signed-off-by: Grzegorz Nosek <[email protected]>

The function sinsp_thread_manager::reinit_thread_from_proc() was added to draios/agent-libs as part of a now-obsolete workaround for an ARM/zLinux platform bug. That workaround has been removed, so now we need to remove this no-longer-used function from sinsp_thread_manager.

In some cases, we want to ensure that a visitor does *not* change the ast. This includes cases where the ast pointer used by the visitor is read-only. To support these use cases, add a const_expr_visitor interface where all the visit() methods take a const argument. Also add variants of accept() that take const_expr_visitor arguments and call the const_expr_visitors visit() method. Compiling, cloning, and stringifying asts are all cases that should not change the underlying ast, so switch those to use const_expr_visitor instead of expr_visitor. A couple of compile private methods had to be changed to take const arguments. They already didn't modify those arguments, so it was a safe change. Signed-off-by: Mark Stemm <[email protected]> Signed-off-by: Mark Stemm <[email protected]>

We need them when building with hayabusa

We can't prevent losing setuid events completely and the uid is pretty important for some execve-related rules, so explicitly pass the uid in execve/at exit events Signed-off-by: Grzegorz Nosek <[email protected]> Co-authored-by: Angelo Puglisi <[email protected]> Co-authored-by: Andrea Terzolo <[email protected]>

`PPM_SC_PIDFD_GETFD` was introduced by syscalls-bumper in libs 0.10.x; `PPM_SC_PIDFD_GET_FD` was instead an uncorrect name being used by libs. Dropped the latter and renamed to correct name the former. Signed-off-by: Federico Di Pierro <[email protected]>

Add a way to skip PPM_SC_NA_X codes while populating syscall_info_table. Signed-off-by: Federico Di Pierro <[email protected]>

… user and group container lookup (#142) Partial port of falcosecurity#803 (e2e test update skipped)

When node is using NIS / nss_compat for user management, /etc/passwd entries can refer to NIS groups or users, which causes parser to return null pointers instead of c-strings. This change adds checks against those. --------- Signed-off-by: Wiktor Gołgowski <[email protected]>

**What type of PR is this?** /kind feature **Any specific area of the project related to this PR?** /area libsinsp **Does this PR require a change in the driver versions?** **What this PR does / why we need it**: Change usage of the library TBB so that it can cope with a newer version of it. More specifically it removes the usage of `tbb::tbb_hash` (nno longer available in newer versions of TBB) in favour of a hash function composed from `std::hash`. **Which issue(s) this PR fixes**: **Special notes for your reviewer**: The change affects code that is only present in `draios/agent-libs` and **not** in `falcosecurity/libs`. **Does this PR introduce a user-facing change?**: ```release-note NONE ```

* Add method to retrieve argid from thread filterchecks Thread filterchecks can have a numeric arg for some fields like proc.aname/proc.apid. This allows returning an argid for a filtercheck object. It's -1 if the field doesn't support or doesn't have an arg. This can be used in unit tests to print/compare filtercheck objects. Signed-off-by: Mark Stemm <[email protected]> * Also save (pointers) to filtercheck values in order Currently, a filtercheck saves the raw values in m_val_storages and a pointer + parsed len in m_val_storages_members. Because m_val_storages_members is an unordered_set, the original order is lost. This makes it difficult to print out a filter expression and compare it to the original input string, as the order of checks like "field in (val1, val2, val3, ...)" are lost. To help retain this order, add a vector m_vals that saves the pointer + length, but in a vector instead of in an ordered_set. Signed-off-by: Mark Stemm <[email protected]> * Add a const base_expr_visitor Following on the changes in falcosecurity/libs#837, add a const variant of base_expr_visitor. This allows definining subclasses that want to traverse an ast read-only without implementng all the methods. Signed-off-by: Mark Stemm <[email protected]> --------- Signed-off-by: Mark Stemm <[email protected]>

Signed-off-by: Luca Guerra <[email protected]>

tid != vtid is not necessary (though sufficient) to check if a thread is in a child pid namespace. This leads to pidns_start_ts being wrong occasionally (when a thread happens to have tid == vtid by chance, even if it's in a child pidns). Signed-off-by: Grzegorz Nosek <[email protected]>

We can significantly simplify the logic of checking the pidns start time (during the initial /proc scan), based on the following observations: * the task's start time is (apparently) accessible as simply the timestamp (any timestamp) on its /proc/ * for every task, its root filesystem is accessible via /proc/<pid>/root * the first task in every pidns has pid==1 (as seen from the pidns) Putting these together, it seems it's enough to stat("/proc/<pid>/root/proc/1") and pick whichever you want out of [acm]time. Ref: falcosecurity/libs#860 (comment) Signed-off-by: Grzegorz Nosek <[email protected]>

…reation time See falcosecurity/libs#932 for more context Change occurrences of `/proc/1` to `/proc/1/cmdline` in * userspace/libscap/linux/scap_procs.c * userspace/libscap/scap.c Previous: ```c snprintf(proc_dir, sizeof(proc_dir), "%s/proc/1/", scap_get_host_root()); ``` This PR: ```c snprintf(proc_cmdline, sizeof(proc_cmdline), "%s/proc/1/cmdline", scap_get_host_root()); ``` Co-authored-by: Grzegorz Nosek <[email protected]> Co-authored-by: Melissa Kilby <[email protected]> Signed-off-by: Stanley Chan <[email protected]>

Get boot time from btime value in /proc/stat ref: falcosecurity/libs#932 /proc/uptime and btime in /proc/stat are fed by the same kernel sources. Multiple ways to get boot time: * btime in /proc/stat * calculation via clock_gettime(CLOCK_REALTIME - CLOCK_BOOTTIME) * calculation via time(NULL) - sysinfo().uptime Maintainers preferred btime in /proc/stat because: * value does not depend on calculation using current timestamp * btime is "static" and doesn't change once set * btime is available in kernels from 2008 * CLOCK_BOOTTIME is available in kernels from 2011 (2.6.38) By scraping btime from /proc/stat, it is both the heaviest and most likely to succeed Co-authored-by: Grzegorz Nosek <[email protected]> Co-authored-by: Melissa Kilby <[email protected]> Signed-off-by: Stanley Chan <[email protected]>

Co-authored-by: Grzegorz Nosek <[email protected]> Co-authored-by: Melissa Kilby <[email protected]> Signed-off-by: Stanley Chan <[email protected]>

Co-authored-by: Angelo Puglisi <[email protected]> Signed-off-by: Angelo Puglisi <[email protected]> Signed-off-by: Grzegorz Nosek <[email protected]>

The nodriver engine is still the only user of limited /proc scan but we no longer check handle->m_mode. Instead we have a dedicated flag. This lets us have the nodriver engine with a full /proc scan if we want. Signed-off-by: Grzegorz Nosek <[email protected]>

The patch is somewhat weird because it introduces an option for the nodriver engine which is used only by the main libscap code (the /proc scan does not live in the engine). Still, we're (logically) configuring the nodriver engine, I believe the flag belongs in the engine config. Signed-off-by: Grzegorz Nosek <[email protected]>

This new flag means that we're *not* going to get any events from the engine and are using it just for the secondary effects of scap_open (mostly getting the process table). This is needed to safely open a new inspector while another one exists. Otherwise we'd overwrite the gVisor socket with a new one (which would become inactive the moment we close the second inspector), breaking all future gVisor connections. Co-Authored-By: Angelo Puglisi <[email protected]> Signed-off-by: Grzegorz Nosek <[email protected]>

In dba2e32 we switched the way we determine the boot time from CLOCK_BOOTTIME based to /proc/stat based. The new way is more compatible (including compatibility with ancient kernels) but it only has a full second accuracy (the fractional part is lost). Unfortunately, we need the extra precision in BPF engines since we only get timestamps since boot from the kernel. Without the subsecond part, all events get their timestamps shifted by up to a second to the past (the exact value depends on the fractional part of the second the machine booted). Since BPF engines do not need compatibility with prehistoric kernels (they don't support eBPF anyway), switch them to use CLOCK_BOOTTIME to get the boot time. Signed-off-by: Grzegorz Nosek <[email protected]>

Some tests rely on creating fake events without all of the overhead of an inspector, etc. To support new tests that rely on m_errorcode (generally maps to the res field of events), add a new initializer that passes in a scap + ppm header and errorcode, and just directly sets m_errorcode. Signed-off-by: Mark Stemm <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ssprod 23814 add addl event init method #1052

Ssprod 23814 add addl event init method #1052

Commits on Dec 15, 2022

Commits on Jan 17, 2023

Commits on Jan 25, 2023

Commits on Jan 31, 2023

Commits on Feb 17, 2023

Commits on Feb 28, 2023

Commits on Mar 9, 2023

Commits on Mar 16, 2023

Commits on Mar 19, 2023

Commits on Mar 24, 2023

Commits on Mar 28, 2023

Commits on Mar 31, 2023

Commits on Apr 7, 2023

Commits on Apr 17, 2023