general protection fault, probably for non-canonical address 0x25b5f6bb1a24827e: 0000 [#1] SMP NOPTI

[zsksy123]

Our machine restarted abnormally (CST) on 2024.09.02 15:41, and the following information was found after checking the log

grep -i pstore /var/log/syslog
Sep  2 15:41:09 gtxSep  2 15:45:21 gtx3090-3 systemd-pstore[2961]: PStore dmesg-erst-7409947607948066822 moved to /var/lib/systemd/pstore/7409947607948/dmesg-erst-7409947607948066822
Sep  2 15:45:21 gtx3090-3 systemd-pstore[2961]: PStore dmesg-erst-7409947607948066821 moved to /var/lib/systemd/pstore/7409947607948/dmesg-erst-7409947607948066821
Sep  2 15:45:21 gtx3090-3 systemd-pstore[2961]: PStore dmesg-erst-7409947607948066820 moved to /var/lib/systemd/pstore/7409947607948/dmesg-erst-7409947607948066820
Sep  2 15:45:21 gtx3090-3 systemd-pstore[2961]: PStore dmesg-erst-7409947607948066819 moved to /var/lib/systemd/pstore/7409947607948/dmesg-erst-7409947607948066819
Sep  2 15:45:21 gtx3090-3 systemd-pstore[2961]: PStore dmesg-erst-7409947607948066818 moved to /var/lib/systemd/pstore/7409947607948/dmesg-erst-7409947607948066818
Sep  2 15:45:21 gtx3090-3 systemd-pstore[2961]: PStore dmesg-erst-7409947607948066817 moved to /var/lib/systemd/pstore/7409947607948/dmesg-erst-7409947607948066817
Sep  2 15:45:21 gtx3090-3 kernel: [    3.289572] pstore: Registered erst as persistent store backend
Sep  2 15:45:21 gtx3090-3 kernel: [    3.667289] pstore: Using crash dump compression: deflate
Sep  2 15:45:21 gtx3090-3 kernel: [   10.366856] systemd[1]: Starting Load Kernel Module efi_pstore...
Sep  2 15:45:21 gtx3090-3 kernel: [   10.373055] pstore: ignoring unexpected backend 'efi'
Sep  2 15:45:21 gtx3090-3 kernel: [   10.377532] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
Sep  2 15:45:21 gtx3090-3 kernel: [   10.377873] systemd[1]: Finished Load Kernel Module efi_pstore.

According to the log in/var/log/syslog tip into the/var/lib/systemd/pstore / 7409947607948 / directory, found the following log information

<4>[4898512.496515] general protection fault, probably for non-canonical address 0x25b5f6bb1a24827e: 0000 [#1] SMP NOPTI
<4>[4898512.496901] CPU: 86 PID: 30498 Comm: python Tainted: P           OE     5.15.0-91-generic #101-Ubuntu
<4>[4898512.497163] Hardware name: Supermicro AS -4124GS-TNR/H12DSG-O-CPU, BIOS 2.4 04/22/2022
<4>[4898512.497428] RIP: 0010:__kmalloc+0x111/0x330
<4>[4898512.497700] Code: 8b 50 08 49 8b 00 49 83 78 10 00 48 89 45 c8 0f 84 c5 01 00 00 48 85 c0 0f 84 bc 01 00 00 41 8b 4c 24 28 49 8b 3c 24 48 01 c1 <48> 8b 19 48 89 ce 49 33 9c 24 b8 00 00 00 48 8d 4a 01 48 0f ce 48
<4>[4898512.498275] RSP: 0018:ffffa79c7eaef790 EFLAGS: 00010206
<4>[4898512.498574] RAX: 25b5f6bb1a24825e RBX: 0000000000006cc0 RCX: 25b5f6bb1a24827e
<4>[4898512.498877] RDX: 0000000004f4abc2 RSI: 0000000000006cc0 RDI: 00000000000360a0
<4>[4898512.499187] RBP: ffffa79c7eaef7d0 R08: ffff9a468dfb60a0 R09: ffff9a33561477b0
<4>[4898512.499498] R10: 0000000000000246 R11: 00000000ffffffff R12: ffff99c840044500
<4>[4898512.499814] R13: ffffffffc0c5f7be R14: 0000000000006cc0 R15: 0000000000000000
<4>[4898512.500133] FS:  00007f4aa93eb280(0000) GS:ffff9a468df80000(0000) knlGS:0000000000000000
<4>[4898512.500461] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[4898512.500792] CR2: 00007f4aa626d3b0 CR3: 0000005e483d8004 CR4: 0000000000770ee0
<4>[4898512.501129] PKRU: 55555554
<4>[4898512.501467] Call Trace:
<4>[4898512.501805]  <TASK>
<4>[4898512.502143]  ? show_trace_log_lvl+0x1d6/0x2ea
<4>[4898512.502491]  ? show_trace_log_lvl+0x1d6/0x2ea
<4>[4898512.502838]  ? os_alloc_mem+0xce/0xe0 [nvidia]
<4>[4898512.503581]  ? show_regs.part.0+0x23/0x29
<4>[4898512.503929]  ? __die_body.cold+0x8/0xd
<4>[4898512.504278]  ? die_addr+0x3e/0x60
<4>[4898512.504632]  ? exc_general_protection+0x1c5/0x410
<4>[4898512.504990]  ? asm_exc_general_protection+0x27/0x30
<4>[4898512.505350]  ? os_alloc_mem+0xce/0xe0 [nvidia]
<4>[4898512.506094]  ? __kmalloc+0x111/0x330
<4>[4898512.506457]  os_alloc_mem+0xce/0xe0 [nvidia]
<4>[4898512.507206]  _nv012724rm+0x34/0x50 [nvidia]
<4>[4898512.507994] WARNING: kernel stack frame pointer at 00000000a2cc99b1 in python:30498 has bad value 00000000b9d21f3f
<4>[4898512.507998] unwind stack type:0 next_sp:0000000000000000 mask:0x2 graph_idx:0
<4>[4898512.507999] 00000000f73efbe9: ffffa79c7eaef7f0 (0xffffa79c7eaef7f0)
<4>[4898512.508001] 00000000c9282c1b: ffffffffc0c5f7be (os_alloc_mem+0xce/0xe0 [nvidia])
<4>[4898512.508388] 00000000dd127a2b: ffff99c8ec897688 (0xffff99c8ec897688)
<4>[4898512.508389] 000000003ee412b7: 0000000000000038 (0x38)
<4>[4898512.508390] 00000000a2cc99b1: ffff9a3774e9d9e0 (0xffff9a3774e9d9e0)
<4>[4898512.508391] 00000000947f66de: ffffffffc152d724 (_nv012724rm+0x34/0x50 [nvidia])
<4>[4898512.508809] 0000000045e91962: ffffffffc152cc40 (_nv042286rm+0x40/0x40 [nvidia])
<4>[4898512.509224] 000000009d1531ea: ffffffffc152d56b (_nv012726rm+0x2b/0xd0 [nvidia])
<4>[4898512.509639] 000000007b16bb59: ffffffffc152cc40 (_nv042286rm+0x40/0x40 [nvidia])
<4>[4898512.510056] 00000000febc8e00: ffff99d1d99318b0 (0xffff99d1d99318b0)

The complete log file under /var/lib/systemd/store/7409947607948 / is in the compressed file

pstorelog.tar.gz

[max0x7ba]

I encounter a similar crash:

Sep 06 23:00:02 kernel: general protection fault, probably for non-canonical address 0x40b5ff3c61e2aed3: 0000 [#1] PREEMPT SMP NOPTI
Sep 06 23:00:02 kernel: CPU: 4 PID: 1089479 Comm: ray::ImplicitFu Tainted: P           OE      6.8.0-40-lowlatency #40.1~22.04.1-Ubuntu
Sep 06 23:00:02 kernel: Hardware name: Gigabyte Technology Co., Ltd. X570 AORUS MASTER/X570 AORUS MASTER, BIOS F39b 07/11/2024
Sep 06 23:00:02 kernel: RIP: 0010:__kmalloc+0x15b/0x480
Sep 06 23:00:02 kernel: Code: 83 78 10 00 48 8b 38 0f 84 7c 02 00 00 48 85 ff 0f 84 73 02 00 00 41 8b 44 24 28 49 8b 9c 24 b8 00 00 00 49 8b 34 24 48 01 f8 <48> 33 18 48 89 c1 48 89 f8 48 0f c9 48 31 cb 48 8d 8a 00 20 00 00
Sep 06 23:00:02 kernel: RSP: 0018:ffffa001d34d36d0 EFLAGS: 00010202
Sep 06 23:00:02 kernel: RAX: 40b5ff3c61e2aed3 RBX: 5fdff377e64e254c RCX: 0000000000000000
Sep 06 23:00:02 kernel: RDX: 00000006a5372004 RSI: 000000000003aa60 RDI: 40b5ff3c61e2aeb3
Sep 06 23:00:02 kernel: RBP: ffffa001d34d3720 R08: 0000000000000000 R09: 0000000000000000
Sep 06 23:00:02 kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff8c45c0047300
Sep 06 23:00:02 kernel: R13: 0000000000000040 R14: 0000000000006cc0 R15: ffffffffc10874ed
Sep 06 23:00:02 kernel: FS:  00007b4d46000640(0000) GS:ffff8c647e400000(0000) knlGS:0000000000000000
Sep 06 23:00:02 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Sep 06 23:00:02 kernel: CR2: 0000744a12e81000 CR3: 0000000207304000 CR4: 0000000000f50ef0
Sep 06 23:00:02 kernel: PKRU: 55555554
Sep 06 23:00:02 kernel: Call Trace:
Sep 06 23:00:02 kernel:  <TASK>
Sep 06 23:00:02 kernel:  ? show_regs+0x6d/0x80
Sep 06 23:00:02 kernel:  ? die_addr+0x37/0xa0
Sep 06 23:00:02 kernel:  ? exc_general_protection+0x1db/0x480
Sep 06 23:00:02 kernel:  ? asm_exc_general_protection+0x27/0x30
Sep 06 23:00:02 kernel:  ? os_alloc_mem+0xdd/0x100 [nvidia]
Sep 06 23:00:02 kernel:  ? __kmalloc+0x15b/0x480
Sep 06 23:00:02 kernel:  ? os_alloc_mem+0xdd/0x100 [nvidia]
Sep 06 23:00:02 kernel:  os_alloc_mem+0xdd/0x100 [nvidia]
Sep 06 23:00:02 kernel:  ? _raw_spin_lock_irqsave+0xe/0x20
Sep 06 23:00:02 kernel:  ? os_alloc_mem+0xdd/0x100 [nvidia]
Sep 06 23:00:02 kernel:  _nv013863rm+0x34/0x50 [nvidia]
Sep 06 23:00:02 kernel: WARNING: kernel stack frame pointer at 00000000f6cfd779 in ray::ImplicitFu:1089479 has bad value 000000003c185112
Sep 06 23:00:02 kernel: unwind stack type:0 next_sp:0000000000000000 mask:0x2 graph_idx:0
Sep 06 23:00:02 kernel: 00000000c168a5d8: ffffa001d34d3750 (0xffffa001d34d3750)
Sep 06 23:00:02 kernel: 0000000041eae51e: ffffffffc10874ed (os_alloc_mem+0xdd/0x100 [nvidia])
Sep 06 23:00:02 kernel: 00000000a201e012: ffffffff85c1d44e (_raw_spin_lock_irqsave+0xe/0x20)
Sep 06 23:00:02 kernel: 00000000fdfa4485: ffffffffc10874ed (os_alloc_mem+0xdd/0x100 [nvidia])
Sep 06 23:00:02 kernel: 00000000d862a94e: ffff8c45ea7cc008 (0xffff8c45ea7cc008)
Sep 06 23:00:02 kernel: 0000000002eed74b: 0000000000000038 (0x38)
Sep 06 23:00:02 kernel: 00000000f6cfd779: ffff8c508b092ca0 (0xffff8c508b092ca0)
Sep 06 23:00:02 kernel: 000000002b1fc6c9: ffffffffc19d6684 (_nv013863rm+0x34/0x50 [nvidia])
Sep 06 23:00:02 kernel: 00000000d3cb4f60: ffffffff85c1d44e (_raw_spin_lock_irqsave+0xe/0x20)
Sep 06 23:00:02 kernel: 0000000042b5e8a7: ffffffffc19d64cb (_nv013865rm+0x2b/0xd0 [nvidia])
Sep 06 23:00:02 kernel: 00000000a21f9c35: ffffffffc1088b92 (os_acquire_spinlock+0x12/0x30 [nvidia])
Sep 06 23:00:02 kernel: 00000000c35126a6: ffff8c508b092cd0 (0xffff8c508b092cd0)
Sep 06 23:00:02 kernel: 00000000585c3c36: 0000000000000000 ...
Sep 06 23:00:03 kernel: 000000009e6b12c2: ffffffffc19d53cd (_nv014874rm+0x8d/0xe0 [nvidia])
Sep 06 23:00:03 kernel: 000000004d20e074: ffff8c508b092ca0 (0xffff8c508b092ca0)
Sep 06 23:00:03 kernel: 00000000d3d0488d: ffff8c508b092cd8 (0xffff8c508b092cd8)
Sep 06 23:00:03 kernel: 00000000126ff78b: ffffffffc1d9bf60 (_nv000456rm+0xaf0/0xfffffffffff1eb90 [nvidia])
Sep 06 23:00:03 kernel: 00000000eb01eba3: ffff8c45c12c7410 (0xffff8c45c12c7410)
Sep 06 23:00:03 kernel: 000000001efd1240: ffff8c49f7870430 (0xffff8c49f7870430)
Sep 06 23:00:03 kernel: 000000004aa7cfce: ffffffffc19d99e0 (_nv045657rm+0x20/0x90 [nvidia])
Sep 06 23:00:03 kernel: 0000000040860fe6: 0000000000000000 ...
Sep 06 23:00:03 kernel: 000000006ba33fd1: ffff8c508b092e48 (0xffff8c508b092e48)
Sep 06 23:00:03 kernel: 0000000086f72da4: 0000000000000000 ...
Sep 06 23:00:03 kernel: 00000000752f07a1: ffffffffc19de9b0 (_nv017451rm+0xa0/0x2c0 [nvidia])
Sep 06 23:00:03 kernel: 00000000d035746b: ffff8c508b092e48 (0xffff8c508b092e48)
Sep 06 23:00:03 kernel: 00000000f03acd1d: 0000000000000000 ...
Sep 06 23:00:03 kernel: 00000000f0fe9b73: ffffffffc1d9bf60 (_nv000456rm+0xaf0/0xfffffffffff1eb90 [nvidia])
Sep 06 23:00:03 kernel: 00000000e3070cf0: ffff8c508b092e10 (0xffff8c508b092e10)
Sep 06 23:00:03 kernel: 00000000f1cd4f72: ffff8c45c12c7410 (0xffff8c45c12c7410)
Sep 06 23:00:03 kernel: 0000000078c9c508: ffffffffc19d9142 (_nv047622rm+0x212/0x270 [nvidia])
Sep 06 23:00:03 kernel: 0000000067027637: ffff8c508b092f48 (0xffff8c508b092f48)
Sep 06 23:00:03 kernel: 0000000074ce907a: ffffffffc1d9bd80 (_nv000456rm+0x910/0xfffffffffff1eb90 [nvidia])
Sep 06 23:00:03 kernel: 00000000bc46dc22: 00000000c1d0000c (0xc1d0000c)
Sep 06 23:00:03 kernel: 0000000065ce28d8: 00000000c1d2c3c5 (0xc1d2c3c5)
Sep 06 23:00:03 kernel: 000000008e4cb47e: 0000000000000000 ...
Sep 06 23:00:03 kernel: 00000000bb146ed4: ffffffffc11372f2 (_nv045797rm+0xe2/0x1a0 [nvidia])
Sep 06 23:00:03 kernel: 000000008000ecd1: ffff8c46c783ae08 (0xffff8c46c783ae08)
Sep 06 23:00:03 kernel: 00000000b8bd03e8: ffff8c45e07e4000 (0xffff8c45e07e4000)
Sep 06 23:00:03 kernel: 00000000b5e11a7b: ffff8c45cf664808 (0xffff8c45cf664808)
Sep 06 23:00:03 kernel: 00000000d0a23e95: ffff8c4852a13008 (0xffff8c4852a13008)
Sep 06 23:00:03 kernel: 0000000001f097c0: ffff8c4aa3d21bc8 (0xffff8c4aa3d21bc8)
Sep 06 23:00:03 kernel: 00000000dc03bf0f: ffffffffc11371fc (_nv045796rm+0x2c/0x40 [nvidia])
Sep 06 23:00:03 kernel: 00000000bce45496: 0000000000000001 (0x1)
Sep 06 23:00:03 kernel: 000000001d6f50e1: ffffffffc1d9be38 (_nv000456rm+0x9c8/0xfffffffffff1eb90 [nvidia])
Sep 06 23:00:03 kernel: 000000007a18d381: ffff8c45cef78008 (0xffff8c45cef78008)
Sep 06 23:00:03 kernel: 000000009c0970f6: ffffffffc11471e5 (_nv039446rm+0xed5/0x15a0 [nvidia])
Sep 06 23:00:03 kernel: 00000000eaf64c70: 0000000000000001 (0x1)
Sep 06 23:00:03 kernel: 000000005bc8381e: 0000000000000000 ...
Sep 06 23:00:03 kernel: 00000000e84c0d84: ffff8c508b090000 (0xffff8c508b090000)
Sep 06 23:00:03 kernel: 0000000029d99a89: 00000000c1d2c3c5 (0xc1d2c3c5)
Sep 06 23:00:03 kernel: 00000000fed9e101: ffffa001d34d3920 (0xffffa001d34d3920)
Sep 06 23:00:03 kernel: 00000000f9571c36: ffffa001d34d39e0 (0xffffa001d34d39e0)
Sep 06 23:00:03 kernel: 00000000908b156a: ffff8c45e07e4000 (0xffff8c45e07e4000)
Sep 06 23:00:03 kernel: 0000000079614272: ffffffffc1bbfc18 (rm_gpu_ops_retain_channel+0x28/0x70 [nvidia])
Sep 06 23:00:03 kernel: 0000000097e25b8e: ffff8c46bc2c6b48 (0xffff8c46bc2c6b48)
Sep 06 23:00:03 kernel: 00000000777233fa: 00000000c1d2c3c5 (0xc1d2c3c5)
Sep 06 23:00:03 kernel: 0000000026b2b4c4: 000000005c000036 (0x5c000036)
Sep 06 23:00:03 kernel: 000000009333d7d9: ffffffffc108fffb (nvUvmInterfaceRetainChannel+0xab/0xe0 [nvidia])
Sep 06 23:00:03 kernel: 00000000df954b81: ffff8c508b090000 (0xffff8c508b090000)
Sep 06 23:00:03 kernel: 000000008b8c1771: ffffa001d34d3ab0 (0xffffa001d34d3ab0)
Sep 06 23:00:03 kernel: 000000008fa21953: ffff8c47fb99c000 (0xffff8c47fb99c000)
Sep 06 23:00:03 kernel: 000000005dc3407d: ffff8c45d81c5c00 (0xffff8c45d81c5c00)
Sep 06 23:00:03 kernel: 00000000c7e0e6e7: ffff8c45e07e4000 (0xffff8c45e07e4000)
Sep 06 23:00:03 kernel: 00000000f4d04a99: ffffa001d4ab1008 (0xffffa001d4ab1008)
Sep 06 23:00:03 kernel: 000000002fb6b975: ffffa001d34d3a50 (0xffffa001d34d3a50)
Sep 06 23:00:03 kernel: 00000000aa958b91: ffffffffc48594f7 (uvm_api_register_channel+0x1e7/0xfe0 [nvidia_uvm])
Sep 06 23:00:03 kernel: 000000005a86d7c4: 0000000000000000 ...
Sep 06 23:00:03 kernel: 00000000ee3d9f0d: 8ae9a5d2aaa51d00 (0x8ae9a5d2aaa51d00)
Sep 06 23:00:03 kernel: 000000000f38b2d9: 95103e7083fd7baa (0x95103e7083fd7baa)
Sep 06 23:00:03 kernel: 00000000cd44bf01: ffff8c45d81c5c10 (0xffff8c45d81c5c10)
Sep 06 23:00:03 kernel: 0000000005c9396f: ffff8c45d81c5cd8 (0xffff8c45d81c5cd8)
Sep 06 23:00:03 kernel: 00000000a75a006f: 0000000002c1a000 (0x2c1a000)
Sep 06 23:00:03 kernel: 00000000b53f2947: 000000025c000036 (0x25c000036)
Sep 06 23:00:03 kernel: 00000000e18c0ae3: 00000000c1d2c3c5 (0xc1d2c3c5)
Sep 06 23:00:03 kernel: 00000000107260d3: 0000000203600000 (0x203600000)
Sep 06 23:00:03 kernel: 000000003743dc22: ffffa001d4ab1008 (0xffffa001d4ab1008)
Sep 06 23:00:03 kernel: 000000009a573ab1: ffffa001d4ab18b0 (0xffffa001d4ab18b0)
Sep 06 23:00:03 kernel: 00000000c3fa2813: ffffa001d4ab1888 (0xffffa001d4ab1888)
Sep 06 23:00:03 kernel: 00000000a374f6d7: 000000ab00000000 (0xab00000000)
Sep 06 23:00:03 kernel: 000000003ba25225: 5c000036c1d2c3c5 (0x5c000036c1d2c3c5)
Sep 06 23:00:03 kernel: 00000000b4610e5c: ffffa001d34d39d0 (0xffffa001d34d39d0)
Sep 06 23:00:03 kernel: 0000000059614977: ffffa001d34d39d0 (0xffffa001d34d39d0)
Sep 06 23:00:03 kernel: 000000005a096049: 0000000000000000 ...
Sep 06 23:00:03 kernel: 0000000023d4b095: ffff8c4992fd03c0 (0xffff8c4992fd03c0)
Sep 06 23:00:03 kernel: 00000000bf8b9d75: ffffffffc1d9bd80 (_nv000456rm+0x910/0xfffffffffff1eb90 [nvidia])
Sep 06 23:00:03 kernel: 00000000f2c96198: 95103e7083fd7baa (0x95103e7083fd7baa)
Sep 06 23:00:03 kernel: 000000006a1187b0: 7408669ce118d4dd (0x7408669ce118d4dd)
Sep 06 23:00:03 kernel: 00000000aff78ce2: c1d2c3c5000000ab (0xc1d2c3c5000000ab)
Sep 06 23:00:03 kernel: 0000000010526db2: 8ae9a5d2aaa51d00 (0x8ae9a5d2aaa51d00)
Sep 06 23:00:03 kernel: 00000000a4aca32f: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 000000008a5bce7d: ffffa001d34d3a70 (0xffffa001d34d3a70)
Sep 06 23:00:03 kernel: 00000000001e08b3: ffff8c55ee4ef000 (0xffff8c55ee4ef000)
Sep 06 23:00:03 kernel: 000000006fcbeaf3: ffffa001d34d3ab0 (0xffffa001d34d3ab0)
Sep 06 23:00:03 kernel: 00000000af5abd90: ffffa001d34d3c10 (0xffffa001d34d3c10)
Sep 06 23:00:03 kernel: 00000000ed036748: 0000000000000001 (0x1)
Sep 06 23:00:03 kernel: 000000004bf96be7: ffffa001d34d3c00 (0xffffa001d34d3c00)
Sep 06 23:00:03 kernel: 000000007168bd63: ffffffffc47d3e69 (uvm_ioctl+0x16c9/0x1cd0 [nvidia_uvm])
Sep 06 23:00:03 kernel: 00000000f3b1e060: ffffa001d34d3c00 (0xffffa001d34d3c00)
Sep 06 23:00:03 kernel: 000000001236aa31: ffffffffc47d3e69 (uvm_ioctl+0x16c9/0x1cd0 [nvidia_uvm])
Sep 06 23:00:03 kernel: 0000000090b4a453: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 00000000cd7c784d: ffffa001d34d3ac8 (0xffffa001d34d3ac8)
Sep 06 23:00:03 kernel: 00000000bbec8ade: 0000000000000001 (0x1)
Sep 06 23:00:03 kernel: 00000000bee1bc2e: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 00000000be6558ac: ffffa001d34d3ac8 (0xffffa001d34d3ac8)
Sep 06 23:00:03 kernel: 0000000035de23e8: ffffa001d34d3ab8 (0xffffa001d34d3ab8)
Sep 06 23:00:03 kernel: 0000000099c144da: ffffffffc483ad99 (uvm_thread_context_remove+0x39/0x50 [nvidia_uvm])
Sep 06 23:00:03 kernel: 00000000db623fe0: 0000000000000000 ...
Sep 06 23:00:03 kernel: 00000000868eccf2: 95103e7083fd7baa (0x95103e7083fd7baa)
Sep 06 23:00:03 kernel: 00000000f11d2a1b: 7408669ce118d4dd (0x7408669ce118d4dd)
Sep 06 23:00:03 kernel: 0000000083da79c5: c1d2c3c5000000ab (0xc1d2c3c5000000ab)
Sep 06 23:00:03 kernel: 00000000779cf2e8: 000000005c000036 (0x5c000036)
Sep 06 23:00:03 kernel: 000000002cfdab47: 0000000203600000 (0x203600000)
Sep 06 23:00:03 kernel: 0000000041078165: 0000000002c1a000 (0x2c1a000)
Sep 06 23:00:03 kernel: 000000001dbf8e2c: 0000000000000000 ...
Sep 06 23:00:03 kernel: 00000000f6e9d8e9: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 000000001301c173: ffffa001d34d3b58 (0xffffa001d34d3b58)
Sep 06 23:00:03 kernel: 000000002be069cd: 0000000000000001 (0x1)
Sep 06 23:00:03 kernel: 000000009bcd5214: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 00000000eff9d42b: ffffa001d34d3b58 (0xffffa001d34d3b58)
Sep 06 23:00:03 kernel: 000000008b583680: ffffa001d34d3b48 (0xffffa001d34d3b48)
Sep 06 23:00:03 kernel: 00000000a91f745d: ffffffffc483ad99 (uvm_thread_context_remove+0x39/0x50 [nvidia_uvm])
Sep 06 23:00:03 kernel: 000000008598e728: 0000000000000000 ...
Sep 06 23:00:03 kernel: 0000000001c56c6b: 000000000000001b (0x1b)
Sep 06 23:00:03 kernel: 000000008046b159: ffffa001d34d3c20 (0xffffa001d34d3c20)
Sep 06 23:00:03 kernel: 000000004930c5fb: ffffffffc47d4708 (uvm_unlocked_ioctl_entry.part.0+0xd8/0xf0 [nvidia_uvm])
Sep 06 23:00:03 kernel: 000000004348b148: ffff8c4c48cea900 (0xffff8c4c48cea900)
Sep 06 23:00:03 kernel: 0000000089841394: 8ae9a5d2aaa51d00 (0x8ae9a5d2aaa51d00)
Sep 06 23:00:03 kernel: 00000000f787fe05: ffff8c55ee4ef000 (0xffff8c55ee4ef000)
Sep 06 23:00:03 kernel: 00000000be5e3a1b: 0000000000000246 (0x246)
Sep 06 23:00:03 kernel: 000000006a362e69: 0000000000000008 (0x8)
Sep 06 23:00:03 kernel: 00000000d34cac07: ffffa001d34d3c10 (0xffffa001d34d3c10)
Sep 06 23:00:03 kernel: 00000000eb50a6fe: ffffa001d34d3b98 (0xffffa001d34d3b98)
Sep 06 23:00:03 kernel: 000000009642017a: ffffffff85c1d44e (_raw_spin_lock_irqsave+0xe/0x20)
Sep 06 23:00:03 kernel: 0000000078281d68: ffffa001d34d3bd0 (0xffffa001d34d3bd0)
Sep 06 23:00:03 kernel: 000000003440e191: ffffffffc483a78a (thread_context_non_interrupt_add+0x13a/0x250 [nvidia_uvm])
Sep 06 23:00:03 kernel: 00000000eab811e6: ffffa001d34d3c10 (0xffffa001d34d3c10)
Sep 06 23:00:03 kernel: 0000000099211120: 0000000000000001 (0x1)
Sep 06 23:00:03 kernel: 0000000092575cc8: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 00000000f6e98505: ffffa001d34d3c10 (0xffffa001d34d3c10)
Sep 06 23:00:03 kernel: 000000006beaad27: 8ae9a5d2aaa51d00 (0x8ae9a5d2aaa51d00)
Sep 06 23:00:03 kernel: 0000000089324831: ffffa001d34d3c00 (0xffffa001d34d3c00)
Sep 06 23:00:03 kernel: 000000007f367518: ffff8c55ee4ef000 (0xffff8c55ee4ef000)
Sep 06 23:00:03 kernel: 00000000e9cca1b6: 000000000000001b (0x1b)
Sep 06 23:00:03 kernel: 0000000001ce83ce: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 00000000ce652d63: ffffa001d34d3c10 (0xffffa001d34d3c10)
Sep 06 23:00:03 kernel: 00000000ce5e12b7: 0000000000000001 (0x1)
Sep 06 23:00:03 kernel: 00000000e013dc0b: ffffa001d34d3cd8 (0xffffa001d34d3cd8)
Sep 06 23:00:03 kernel: 000000008368abe9: ffffffffc47d46ab (uvm_unlocked_ioctl_entry.part.0+0x7b/0xf0 [nvidia_uvm])
Sep 06 23:00:03 kernel: 00000000ea511b95: ffff8c4c48cea900 (0xffff8c4c48cea900)
Sep 06 23:00:03 kernel: 00000000cb892036: 0000000000000000 ...
Sep 06 23:00:03 kernel: 00000000d3224570: ffffa001d34d3c80 (0xffffa001d34d3c80)
Sep 06 23:00:03 kernel: 00000000441b0893: ffffffff84efe91b (__x64_sys_ioctl+0xbb/0xf0)
Sep 06 23:00:03 kernel: 0000000084ceb06f: ffffa001d34d3f58 (0xffffa001d34d3f58)
Sep 06 23:00:03 kernel: 00000000a0970032: 0000000000000010 (0x10)
Sep 06 23:00:03 kernel: 0000000070208f90: ffffa001d34d3c90 (0xffffa001d34d3c90)
Sep 06 23:00:03 kernel: 00000000fd18b147: ffffffff85c04b29 (syscall_exit_to_user_mode+0x89/0x260)
Sep 06 23:00:03 kernel: 000000002c4c3b17: ffffa001d34d3f58 (0xffffa001d34d3f58)
Sep 06 23:00:03 kernel: 0000000033f1f988: 0000000000000010 (0x10)
Sep 06 23:00:03 kernel: 00000000f1017b73: 0000000000000010 (0x10)
Sep 06 23:00:03 kernel: 0000000085060b6a: ffffa001d34d3f48 (0xffffa001d34d3f48)
Sep 06 23:00:03 kernel: 00000000adc33a95: ffffffff85bfcccd (do_syscall_64+0x8d/0x170)
Sep 06 23:00:03 kernel: 00000000236a9120: ffffa001d34d3cb8 (0xffffa001d34d3cb8)
Sep 06 23:00:03 kernel: 000000003d771f38: 8ae9a5d2aaa51d00 (0x8ae9a5d2aaa51d00)
Sep 06 23:00:03 kernel: 00000000e88d1f84: ffff8c55ee4ef000 (0xffff8c55ee4ef000)
Sep 06 23:00:03 kernel: 00000000091b3f82: 000000000000001b (0x1b)
Sep 06 23:00:03 kernel: 0000000056478b6c: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 00000000af4eab76: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 00000000ef9bdc3b: ffff8c55ee4ef001 (0xffff8c55ee4ef001)
Sep 06 23:00:03 kernel: 00000000425cabc1: ffffa001d34d3d00 (0xffffa001d34d3d00)
Sep 06 23:00:03 kernel: 0000000025fbe8bd: ffffffffc47d479b (uvm_unlocked_ioctl_entry+0x6b/0x90 [nvidia_uvm])
Sep 06 23:00:03 kernel: 000000009e348487: 00000000000000ac (0xac)
Sep 06 23:00:03 kernel: 0000000071ddca44: ffff8c55ee4ef000 (0xffff8c55ee4ef000)
Sep 06 23:00:03 kernel: 00000000c6db95f3: 000000000000001b (0x1b)
Sep 06 23:00:03 kernel: 00000000297bf395: ffffa001d34d3d38 (0xffffa001d34d3d38)
Sep 06 23:00:03 kernel: 00000000107bcfd3: ffffffff84efe900 (__x64_sys_ioctl+0xa0/0xf0)
Sep 06 23:00:03 kernel: 00000000fa98d686: ffffa001d34d3f58 (0xffffa001d34d3f58)
Sep 06 23:00:03 kernel: 000000003de78fbc: 0000000000000010 (0x10)
Sep 06 23:00:03 kernel: 00000000e10d7815: 0000000000000010 (0x10)
Sep 06 23:00:03 kernel: 0000000095f7f3dc: 0000000000000000 ...
Sep 06 23:00:03 kernel: 00000000f127ba84: ffffa001d34d3d48 (0xffffa001d34d3d48)
Sep 06 23:00:03 kernel: 0000000075ca1d31: ffffffff84a05f18 (x64_sys_call+0xa68/0x24b0)
Sep 06 23:00:03 kernel: 00000000e4f48946: ffffa001d34d3f48 (0xffffa001d34d3f48)
Sep 06 23:00:03 kernel: 000000001bd20210: ffffffff85bfccc1 (do_syscall_64+0x81/0x170)
Sep 06 23:00:03 kernel: 00000000645f8ba4: ffff8c4c48cea900 (0xffff8c4c48cea900)
Sep 06 23:00:03 kernel: 00000000c472eeb9: 0000000000000000 ...
Sep 06 23:00:03 kernel: 00000000bc72701f: ffffa001d34d3db0 (0xffffa001d34d3db0)
Sep 06 23:00:03 kernel: 000000009b582ec3: ffffffffc483ad99 (uvm_thread_context_remove+0x39/0x50 [nvidia_uvm])
Sep 06 23:00:03 kernel: 00000000dc6190df: 0000000000000000 ...
Sep 06 23:00:03 kernel: 00000000e679b3c4: 000000000000001b (0x1b)
Sep 06 23:00:03 kernel: 00000000dad46262: ffffa001d34d3e88 (0xffffa001d34d3e88)
Sep 06 23:00:03 kernel: 0000000090a8bebb: ffffffffc47d4708 (uvm_unlocked_ioctl_entry.part.0+0xd8/0xf0 [nvidia_uvm])
Sep 06 23:00:03 kernel: 00000000c125f91d: ffff8c4c48cea900 (0xffff8c4c48cea900)
Sep 06 23:00:03 kernel: 00000000baea3931: 0000000000000000 ...
Sep 06 23:00:03 kernel: 000000008dd3ac02: 8ae9a5d2aaa51d00 (0x8ae9a5d2aaa51d00)
Sep 06 23:00:03 kernel: 00000000be826bfc: ffff8c55ee4ef000 (0xffff8c55ee4ef000)
Sep 06 23:00:03 kernel: 0000000072bd2cfa: 000000000000001b (0x1b)
Sep 06 23:00:03 kernel: 00000000095407a9: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 00000000c944d9ef: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 000000001dc61722: ffff8c55ee4ef001 (0xffff8c55ee4ef001)
Sep 06 23:00:03 kernel: 00000000c1d1f2d2: ffffa001d34d3e48 (0xffffa001d34d3e48)
Sep 06 23:00:03 kernel: 00000000c86c55c1: ffffffffc47d479b (uvm_unlocked_ioctl_entry+0x6b/0x90 [nvidia_uvm])
Sep 06 23:00:03 kernel: 00000000e7f1fde9: 00000000000000ac (0xac)
Sep 06 23:00:03 kernel: 000000004ae4d3f1: ffff8c55ee4ef000 (0xffff8c55ee4ef000)
Sep 06 23:00:03 kernel: 0000000075b18b64: 000000000000001b (0x1b)
Sep 06 23:00:03 kernel: 00000000d69f435a: ffffa001d34d3e80 (0xffffa001d34d3e80)
Sep 06 23:00:03 kernel: 000000003cefe457: ffffffff84efe91b (__x64_sys_ioctl+0xbb/0xf0)
Sep 06 23:00:03 kernel: 000000009c6a6fa8: ffffa001d34d3f58 (0xffffa001d34d3f58)
Sep 06 23:00:03 kernel: 00000000b76247ee: 0000000000000010 (0x10)
Sep 06 23:00:03 kernel: 00000000e1794acb: ffffa001d34d3e90 (0xffffa001d34d3e90)
Sep 06 23:00:03 kernel: 000000000cf78bd0: ffffffff85c04b29 (syscall_exit_to_user_mode+0x89/0x260)
Sep 06 23:00:03 kernel: 000000003d42b568: ffffa001d34d3f58 (0xffffa001d34d3f58)
Sep 06 23:00:03 kernel: 00000000df6c9757: 0000000000000010 (0x10)
Sep 06 23:00:03 kernel: 000000006b8bf3ed: 0000000000000010 (0x10)
Sep 06 23:00:03 kernel: 0000000083c91766: ffffa001d34d3f48 (0xffffa001d34d3f48)
Sep 06 23:00:03 kernel: 0000000068b7bac4: ffffffff85bfcccd (do_syscall_64+0x8d/0x170)
Sep 06 23:00:03 kernel: 0000000017add5cd: 0000000000000004 (0x4)
Sep 06 23:00:03 kernel: 00000000e4e84e70: ffffa001d34d3ed0 (0xffffa001d34d3ed0)
Sep 06 23:00:03 kernel: 000000002dc8d841: ffffffff84a6a745 (switch_fpu_return+0x55/0xf0)
Sep 06 23:00:03 kernel: 000000000d5ec839: 0000000000004000 (0x4000)
Sep 06 23:00:03 kernel: 00000000c1ca1d30: ffffa001d34d3f58 (0xffffa001d34d3f58)
Sep 06 23:00:03 kernel: 00000000c660e001: ffff8c4c48cea900 (0xffff8c4c48cea900)
Sep 06 23:00:03 kernel: 00000000546a794f: ffffa001d34d3ef8 (0xffffa001d34d3ef8)
Sep 06 23:00:03 kernel: 000000004dbfb2d5: ffffffff85c04b29 (syscall_exit_to_user_mode+0x89/0x260)
Sep 06 23:00:03 kernel: 000000001f4d7790: ffffa001d34d3f58 (0xffffa001d34d3f58)
Sep 06 23:00:03 kernel: 00000000c82715e9: 0000000000000010 (0x10)
Sep 06 23:00:03 kernel: 00000000042bd6eb: 0000000000000010 (0x10)
Sep 06 23:00:03 kernel: 000000007a0977e3: ffffa001d34d3f48 (0xffffa001d34d3f48)
Sep 06 23:00:03 kernel: 000000003f1b996f: ffffffff85bfcccd (do_syscall_64+0x8d/0x170)
Sep 06 23:00:03 kernel: 00000000489ad336: 0000000000000101 (0x101)
Sep 06 23:00:03 kernel: 000000002a3ea7d8: ffffa001d34d3f48 (0xffffa001d34d3f48)
Sep 06 23:00:03 kernel: 00000000b0ffb738: ffffffff85bfcccd (do_syscall_64+0x8d/0x170)
Sep 06 23:00:03 kernel: 0000000096dcec5d: ffffffff85c046a4 (exc_page_fault+0x94/0x1b0)
Sep 06 23:00:03 kernel: 0000000054b796d4: 0000000000000000 ...
Sep 06 23:00:03 kernel: 000000000ae44502: ffffffff85e00130 (entry_SYSCALL_64_after_hwframe+0x78/0x80)
Sep 06 23:00:03 kernel: 00000000e966eb58: 00007b443c7923a0 (0x7b443c7923a0)
Sep 06 23:00:03 kernel: 00000000cc0c9d7f: 00000000000000ac (0xac)
Sep 06 23:00:03 kernel: 000000002a81ca65: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 00000000f61110ff: 00007b443c09e6e6 (0x7b443c09e6e6)
Sep 06 23:00:03 kernel: 00000000995fac95: 00007b4d45ffb7b0 (0x7b4d45ffb7b0)
Sep 06 23:00:03 kernel: 0000000000acb965: 00007b443c7cd810 (0x7b443c7cd810)
Sep 06 23:00:03 kernel: 00000000b2f67c80: 0000000000000246 (0x246)
Sep 06 23:00:03 kernel: 000000000c440c4b: 00007b443c09dae0 (0x7b443c09dae0)
Sep 06 23:00:03 kernel: 00000000ca92a7cd: 0000000000000000 ...
Sep 06 23:00:03 kernel: 000000006ba9af59: ffffffffffffffda (0xffffffffffffffda)
Sep 06 23:00:03 kernel: 000000004f089ec9: 00007b4ded51a94f (0x7b4ded51a94f)
Sep 06 23:00:03 kernel: 0000000093d7de02: 00007b4d45ffb750 (0x7b4d45ffb750)
Sep 06 23:00:03 kernel: 00000000efeea9f3: 000000000000001b (0x1b)
Sep 06 23:00:03 kernel: 00000000654fa602: 00000000000000ac (0xac)
Sep 06 23:00:03 kernel: 00000000ab380379: 0000000000000010 (0x10)
Sep 06 23:00:03 kernel: 00000000a4c4bb98: 00007b4ded51a94f (0x7b4ded51a94f)
Sep 06 23:00:03 kernel: 0000000094e54192: 0000000000000033 (0x33)
Sep 06 23:00:03 kernel: 0000000030f6bb0c: 0000000000000246 (0x246)
Sep 06 23:00:03 kernel: 00000000d8afdb87: 00007b4d45ffb6e0 (0x7b4d45ffb6e0)
Sep 06 23:00:03 kernel: 00000000d5a85b07: 000000000000002b (0x2b)
Sep 06 23:00:03 kernel:  ? _raw_spin_lock_irqsave+0xe/0x20
Sep 06 23:00:03 kernel:  ? _nv013865rm+0x2b/0xd0 [nvidia]
Sep 06 23:00:03 kernel:  ? os_acquire_spinlock+0x12/0x30 [nvidia]
Sep 06 23:00:03 kernel:  ? _nv014874rm+0x8d/0xe0 [nvidia]
Sep 06 23:00:03 kernel:  ? _nv045657rm+0x20/0x90 [nvidia]
Sep 06 23:00:03 kernel:  ? _nv017451rm+0xa0/0x2c0 [nvidia]
Sep 06 23:00:03 kernel:  ? _nv047622rm+0x212/0x270 [nvidia]
Sep 06 23:00:03 kernel:  ? _nv045797rm+0xe2/0x1a0 [nvidia]
Sep 06 23:00:03 kernel:  ? _nv045796rm+0x2c/0x40 [nvidia]
Sep 06 23:00:03 kernel:  ? _nv039446rm+0xed5/0x15a0 [nvidia]
Sep 06 23:00:03 kernel:  ? rm_gpu_ops_retain_channel+0x28/0x70 [nvidia]
Sep 06 23:00:03 kernel:  ? nvUvmInterfaceRetainChannel+0xab/0xe0 [nvidia]
Sep 06 23:00:03 kernel:  ? uvm_api_register_channel+0x1e7/0xfe0 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? uvm_ioctl+0x16c9/0x1cd0 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? uvm_ioctl+0x16c9/0x1cd0 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? uvm_thread_context_remove+0x39/0x50 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? uvm_thread_context_remove+0x39/0x50 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? uvm_unlocked_ioctl_entry.part.0+0xd8/0xf0 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? _raw_spin_lock_irqsave+0xe/0x20
Sep 06 23:00:03 kernel:  ? thread_context_non_interrupt_add+0x13a/0x250 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? uvm_unlocked_ioctl_entry.part.0+0x7b/0xf0 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? __x64_sys_ioctl+0xbb/0xf0
Sep 06 23:00:03 kernel:  ? syscall_exit_to_user_mode+0x89/0x260
Sep 06 23:00:03 kernel:  ? do_syscall_64+0x8d/0x170
Sep 06 23:00:03 kernel:  ? uvm_unlocked_ioctl_entry+0x6b/0x90 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? __x64_sys_ioctl+0xa0/0xf0
Sep 06 23:00:03 kernel:  ? x64_sys_call+0xa68/0x24b0
Sep 06 23:00:03 kernel:  ? do_syscall_64+0x81/0x170
Sep 06 23:00:03 kernel:  ? uvm_thread_context_remove+0x39/0x50 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? uvm_unlocked_ioctl_entry.part.0+0xd8/0xf0 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? uvm_unlocked_ioctl_entry+0x6b/0x90 [nvidia_uvm]
Sep 06 23:00:03 kernel:  ? __x64_sys_ioctl+0xbb/0xf0
Sep 06 23:00:03 kernel:  ? syscall_exit_to_user_mode+0x89/0x260
Sep 06 23:00:03 kernel:  ? do_syscall_64+0x8d/0x170
Sep 06 23:00:03 kernel:  ? switch_fpu_return+0x55/0xf0
Sep 06 23:00:03 kernel:  ? syscall_exit_to_user_mode+0x89/0x260
Sep 06 23:00:03 kernel:  ? do_syscall_64+0x8d/0x170
Sep 06 23:00:03 kernel:  ? do_syscall_64+0x8d/0x170
Sep 06 23:00:03 kernel:  ? exc_page_fault+0x94/0x1b0
Sep 06 23:00:03 kernel:  ? entry_SYSCALL_64_after_hwframe+0x78/0x80
Sep 06 23:00:03 kernel:  </TASK>
Sep 06 23:00:03 kernel: Modules linked in: wireguard curve25519_x86_64 libchacha20poly1305 chacha_x86_64 poly1305_x86_64 libcurve25519_generic libchacha ip6_udp_tunnel udp_tunnel xt_nat dm_crypt xt_conntrack nf_conntrack_netlink xfrm_user xfrm_algo xt_addrtype br_netfilter bridge stp llc xt_MASQUERADE rfcomm snd_seq_dummy snd_hrtimer nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip6t_REJECT nf_reject_ipv6 xt_cgroup xt_mark xt_owner xt_tcpudp ipt_REJECT nf_reject_ipv4 xt_multiport nft_compat nf_tables nfnetlink cmac algif_hash overlay algif_skcipher af_alg bnep corsair_cpro nvidia_uvm(POE) btusb btrtl btintel btbcm btmtk joydev input_leds bluetooth ecdh_generic ecc nvidia_drm(POE) nvidia_modeset(POE) binfmt_misc nvidia(POE) snd_hda_codec_hdmi snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec nls_iso8859_1 snd_hda_core snd_hwdep snd_pcm iwlmvm exfat snd_seq_midi intel_rapl_msr snd_seq_midi_event mac80211 intel_rapl_common snd_rawmidi snd_seq libarc4 snd_seq_device snd_timer iwlwifi edac_mce_amd snd
Sep 06 23:00:03 kernel:  it87(OE) rapl soundcore hwmon_vid video k10temp ccp gigabyte_wmi wmi_bmof mxm_wmi cfg80211 mac_hid sch_fq_codel msr parport_pc ppdev lp parport efi_pstore ip_tables x_tables autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 hid_logitech_hidpp raid1 hid_logitech hid_logitech_dj ff_memless hid_generic usbhid hid crct10dif_pclmul crc32_pclmul polyval_clmulni polyval_generic ghash_clmulni_intel sha256_ssse3 nvme igb sha1_ssse3 i2c_piix4 ahci xhci_pci nvme_core dca xhci_pci_renesas libahci i2c_algo_bit nvme_auth wmi aesni_intel crypto_simd cryptd
Sep 06 23:00:03 kernel: ---[ end trace 0000000000000000 ]---

[Petemir]

Having the same problem for the past months (occurs when server is on pressure) with the same hardware as op.

• Hardware name: Supermicro AS -4124GS-TNR/H12DSG-O-CPU, BIOS 2.5 11/02/2022
• Ubuntu 22.04 - 5.15.0-124-generic
• Nvidia Driver Version: 535.183.01
• GPUs: 4 x NVIDIA RTX A6000
• CUDA Version: 12.2

We don't have k8s at all. I've seen lots of reports on reddit and nvidia developer forums with people having the same error (general protection fault) and similar traces pointing to problems with nvidia drivers, rather than the specific k8s-device-plugin. Hopefully somebody at Nvidia notices...