-
Notifications
You must be signed in to change notification settings - Fork 5.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
USB-modem + USB-webcam on RPi3 causes kernel oops #2551
Comments
I would recommend trying a newer kernel - we are currently shipping Raspbian based on 4.14.34. Amongst a slew of improvements is a back-port of a patch that fixes a memory corruption bug caused by the USB driver - see 8960ba6 . You can either grab the current image, update the relevant Raspbian packages ( |
@pelwell We are currently testing the latest update for which we used
See also: #1272 |
Perhaps - it's worth a try. |
@pelwell As I suspected - unfortunately, the latest update did not fix the problem. The kernel oops still occurs, even after latest
|
Here are the
|
@pelwell We installed a fresh Arch Linux for the RPi 3B, which still led to the problem (I figure the kernel is just the same). Note that it is unlikely due to a bug in the USB-devices themselves, since the problem only occurs when a combination of both is used. Power supply is definitively not an issue either, as stated before. It applies to all ten RPi 3B devices, so it is not a coincidental manufacturing error either. We are wasting a lot of time on this issue, if you have any idea to what might cause this issue or help to prevent it, please let us know. I also see that in RPi 3B+ model some changes were made to the USB architecture, including an additional USB controller. Why was this changed? Could this have anything to do with this issue? |
@jetibest The USB connection on the SOC itself is the same. The new ethernet chip (to give us gig ethernet) now has two hubs internally, one hanging off the other, to give enough ports. Not sure why it was done like that, but that's how it is. We just use the chip! |
@JamesH65 Thank you. We are now testing with a RPi 3B+, to see if the issue still occurs, just to make sure we tried everything before we start looking for alternatives to the RPi. If there is something else we could or should try, I am open to suggestions. Will this issue be investigated? How do you typically deal with issues like these? |
The problem here is that you are using two external devices. We are currently working full time on ensuring the on board devices work as expected, so there is little time for trying combinations of third party stuff. As for how we deal, generally, the bugs that affect a lot of people get investigated first. And of course, we always have new products to develop as well, and as we don't have a dedicated bug fixing team, there is a certain amount of flitting around between new product and bug fixing. We also like to have easy ways of replicating issues, on Raspbian. Easily replicated bugs get fixed faster. Bugs reported on other distro's, because we cannot be sure they also go wrong on our distro, tend to be lower on the priority list. |
@JamesH65 Thank you for your swift reply. I understand, fair enough. As far as reproducibility goes, when you have the time, you can do the following which has been very consistent for us:
Set a crontab for every 5 minutes running the following script:
Make sure to connect to the internet through the usbmodem, and keep an ssh-connection open (e.g. use Of course, for many people, an occasional kernel oops may only be a slight annoyance, and it is hard to track down the problem. Many times it may be indistinguishable from a bad power supply. I wouldn't be surprised if this is not reported too often, if at all. But I found these that might be similar: ajohn370 commented on Dec 14, 2017 "Having a similar issue with an rpi 3 b. System runs smoothly but it freezes hangs and I loose ssh and VNC access." - raspberrypi/firmware#247 l-spitz commented on Sep 26, 2017 "I'm getting a lot of issues similar to what you describe, when running webcams over the usb, namely sporadically get dmesg messages, and eventually get following, which requires a power cycle to recover from." - #1272 zerwes commented on Mar 29 "same issue here" - #2481 |
Actually I could reproduce it in usually less than an hour, typically 20 mins (rarely a day long), by running two webcams, taking stills as fast as they could. Using a python script (calling linux using os.system), but suspect bash script would do the same . Each webcam was in its own process. Each time it was same thing (kernel NULL pointer dereference at virtual address 0...) I came to the conclusion that trying to do more than one large file transfer over the usb at the same time was the trigger. I re-arranged my application to avoid this situation. |
Thanks, both useful data points that will help when we get the time. @P33M Any thoughts? |
I can now confirm that the problem is only reproducable when the webcam is repeatedly turned on and off again to take pictures (using avconv). When the webcam is permanently streaming (avconv mjpeg), there is no issue. It seems like turning the webcam on or off is the cause (however, only in combination with another usb-device). @l-spitz do you still have your python script, or know what it consisted of? I would like to test your script on our system. |
Sure, my script is constantly turning two usb webcams on and off (as an accelerated test) so fits with what you found. Hope it helps `
` |
@l-spitz Thank you for the script. Ran it for an hour or so, and later again for 5 hours straight. Unfortunately could not reproduce any problem within a short period of time (even while the usb-modem was also still plugged in). I used two ELP camera modules with USB connectors, on the RPi 3 Model B. After that I ran the same script for 3 hours, but now with a sleep of 3 seconds (to ensure the webcams are suspended momentarily, which I verified with So the current status is that it is still not reproducable within hours, but only within days. It's very peculiar that the problem does not occur sooner for a higher frequency of accessing the USB webcams. |
What resolution was the camera, I was using a genius HD webcam F100, and also tried with others in the 5MP range (all on RPi 3 Model B). I re-tried the script before posting, and had three crashes in a 6 hr period. I had a hunch larger MP caused more crashes. |
I think I am having a similar issue. I am using ELP 5MP webcams ELP-USB500W05G, currently on Rpi 3B but hopefully soon on 3B+ models. We are using thousands of Pi's so this is a big deal for us. Here is the initial trace from kernel.... As you can see, there is some sort of USB bus crash, followed by a USB restart.
We have been able to see this with variable frequency on pretty much all Pi's. Kernel is 4.14.34-v7+ #1110 In the past we were able to trigger the disconnect 100% of the time by closing the camera port. We switched from doing that to leaving it open to mitigate the problem. We have several issues at this point.
we have tried it both stock and with dwc_otg.fiq_enable=1 dwc_otg.fiq_fsm_enable=1 dwc_otg.fiq_fsm_mask=0xf. No difference. If you would like our exercise script, please let me know, and I will send it. |
@noafterglow Thank you for your contribution to this issue! We worked around the problem exactly the same way. Since we need to grab frames, and stream video with different settings etc. I wrote a service that keeps the camera always running on MJPEG stream mode. Then another program can pipe these frames as input, using ffmpeg to generate either individual JPEG files, or e.g. a live H264 in TS stream - even at the same time. Using this modular approach, the webcam never needs to be disabled (and Even though this workaround is stable, we would really prefer to keep power consumption down as we are using solar panels to power the system. On the longer term, I also found that sometimes there are suddenly Could you provide your exercise script? Not sure if your issue is actually related though, seeing your kernel logs. We did not find USB disconnects in our logs. I would lilke to try your script. We are using ELP-USBFHD06H-L28 Sony IMX322 ( |
basically the camera was operated from cv2, with the following sequence (It was spread out over two files...: self._capture = cv2.VideoCapture(port)
for i in xrange(10): _, image = self._capture.retrieve() |
The disconnect happens at release(). |
@noafterglow this is a separate issue to the crash described above.
External influence has caused a disconnect event on the root port of the USB controller. Poor quality power supplies, RF interference or electrostatic discharge events can all create conditions that result in a spurious disconnect. Note that turning a webcam on/off will cause a significant change in power draw. A PSU with a badly tuned feedback arrangement may overshoot/undershoot a considerable amount. It would be worth monitoring the 5V and 3v3 pins on the GPIO header with an oscilloscope to rule out a gross PSU issue. The null pointer dereference is more likely to be a genuine bug in uvcvideo and friends. |
The power supply is a 3A 5.25V supply with with a great feedback arrangement, and class B EMI components. I know because I designed it, and the test labs tested it, which is what I do all the time. This is NOT and EMI issue either, unless the Pi board itself is poorly designed. There is no way for the Pi to see any change in voltage as a result of the webcam being turned on and off. There is way too much capacitance there for the webcam's measly 100mA draw to make any difference. The camera is out of range of people touching it, so ESD is not it either. This is a bug. I don't know what the Pi can do to make the root hub disconnect, but its doing it internally. The issue happens at different rates on different Pi's, some have a couple of times a day, some every few minutes, some go days between events. Mostly is a couple of times per day. It seems to follow the Pi, and not the location. |
In another attempt to fix this problem, I am going to modify the USB-driver configuration - apparently there are many options to try. More specifically, I will be changing the dwc_otg module (using modprobe or in /boot/cmdline.txt) parameters a.k.a. "DesignWare USB 2.0 OTG Controller (DWC_otg)". References: Obviously, messing with these parameters will require thorough testing, and many modifications may not work at all. Will report back in a few months. @noafterglow this information may also be interesting to you. |
Did the USB driver modifications work for resolving the issue? |
I have not investigated the issue in the last 6 months. We have just
loaded the latest kernel and will go looking at the logs again to see if
the problem persists.
Stay tuned.
Ihor Lys
617-470-2740
…On Sat, Mar 23, 2019 at 7:42 AM ysirotin ***@***.***> wrote:
Did the USB driver modifications work for resolving the issue?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2551 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AE1wwyvHY03JsJ-ejm9Dp_3QMPdaAaGsks5vZj1MgaJpZM4T8TOh>
.
|
We have narrowed a similar issue down specifically to the USB port used. When plugging the webcam into the 1-1.3 port on the 3B, we consistently get errors/resets like those documented above. However, when plugging a webcam into 1-1.4 port, we do not see the errors (or the errors are much more infrequent). I am honestly not sure what the difference is between these ports. |
Is it the specific ports or the relative position of the webcam compared to the other devices? With 2 devices and 4 ports you have 12 possible combinations. |
We only have one thing (Logitech C920) plugged into the USB port of the Pi (no keyboard/mouse). We did not exhaustively test each port yet. |
I suspect that this is actually driven by an issue in the hub itself. This
may be why the Pi foundation dumped that part on the latest pi. The only
real way to know is to hook up a USB analyzer to both the internal port
which hooks to the hub as well as the port used for the webcam and look at
what happens during the event.
Now that that Pi is EOL, its over, and we only have the next version to
deal with.
…On Fri, Mar 29, 2019, 05:56 ysirotin ***@***.***> wrote:
We only have one thing (Logitech C920) plugged into the USB port of the Pi
(no keyboard/mouse). We did not exhaustively test each port yet.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2551 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AE1ww0zKzy1Wwij5l8mHsl_CCF2U32Yzks5vbg2RgaJpZM4T8TOh>
.
|
We didn't dump the part - still used on the Pi1, 2, and 3 which are still in production, not EOL. Yes, you can still get Pi 1 if you really want! For the Pi3B+ we simply moved to a part that supported gig ethernet. |
Ok. Then perhaps setting up a test system is worth it. As i
mentioned earlier, i have a 100% method for causing this problem, using
the shutoff of a webcam from user mode. I dont know if its the same
problem here, but it definitely has the same symptom. I dont have the
proper equipment to snoop the intermediate buses to see what actually
happens, but id bet that a day of work on that would probably be
very useful. Otherwise i fear that this is really nothing more than
guessing as to parameters i certainly dont understand.
I will set up my system and run it and let you know if the latest kernel
solves the problem. However, we are upgrading to to latest pi on our next
build, as well as moving the camera to the csi connector, so we wont see
this exact issue again.
…On Fri, Mar 29, 2019, 08:38 James Hughes ***@***.***> wrote:
We didn't dump the part - still used on the Pi1, 2, and 3 which are still
in production, not EOL. Yes, you can still get Pi 1 if you really want! For
the Pi3B+ we simply moved to a part that supported gig ethernet.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2551 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AE1wwyjT1sELQ3rks2jBGd0TWekFF8gjks5vbjNrgaJpZM4T8TOh>
.
|
In case it can be useful, I am having something very similar with an old Hauppage WinTV-USB (analog TV tuner, works as I tried pretty much everything:
All of it on different combinations, including everthing at the same time. My only conclusion is that, being a USB 1.0 (according to I'd rule any kind of real EMI problem as if I place the USB cable following the same path, but connected to a PC instead of connected to a Raspberry, things work fine, with or without ferrite beads. The exact same setup using a Terratec Cinergy Hybrid T USB XS (USB 2.0, includes an analog TV tuner supported as Not complaining! But if you need any more info, and you think it can be useful for this issue, just ping :-) I wonder, is any one of the persons reporting problems using any USB 1.X device? |
Hi I know this is old but same issue here for months (Fedora Linux x86, not just Rpi). Happens randomly between boots, so I think it must be some kind of race condition. That and the C920 webcam isn't officially supported by Linux. Still nothing should ever cause a null dereference in the kernel. Linux 5.8.18-200.fc32.x86_64 #1 SMP Mon Nov 2 19:49:11 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
Fixing issue with Logitech webcams and NULL deref in kernel. raspberrypi#2551
That's in unmodified upstream code - have you passed it on to the correct upstream Linux mailing list? |
I figured it was. Still trying to hunt it down upstream. Hold on the PR for now thanks! |
There is some kind of race condition affecting Logitech webcams that crash USB with a null dereference. Affects raspberry pi devices as well. No check on dev before dereference. Simple fix. Signed-off-by: John Boero <[email protected]> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1827452 raspberrypi/linux#2551 [ 5312.470363] BUG: kernel NULL pointer dereference, address: 0000000000000000 [ 5312.470370] #PF: supervisor read access in kernel mode [ 5312.470372] #PF: error_code(0x0000) - not-present page [ 5312.470374] PGD 8000001a1f7c2067 P4D 8000001a1f7c2067 PUD 0 [ 5312.470380] Oops: 0000 [#1] SMP PTI [ 5312.470385] CPU: 18 PID: 47381 Comm: v4l2src0:src Tainted: P OE 5.8.18-200.fc32.x86_64 #1 [ 5312.470387] Hardware name: Hewlett-Packard HP Z640 Workstation/212A, BIOS M60 v02.50 11/07/2019 [ 5312.470394] RIP: 0010:usb_ifnum_to_if+0x3a/0x50 [ 5312.470398] Code: 34 41 0f b6 50 04 84 d2 74 2f 83 ea 01 49 8d 80 98 00 00 00 49 8d 8c d0 a0 00 00 00 eb 09 48 83 c0 08 48 39 c8 74 12 4c 8b 00 <49> 8b 10 0f b6 52 02 39 f2 75 e9 4c 89 c0 c3 45 31 c0 4c 89 c0 c3 [ 5312.470401] RSP: 0018:ffffac3683143bb0 EFLAGS: 00010206 [ 5312.470404] RAX: ffff8d63f1463498 RBX: 0000000000000000 RCX: ffff8d63f14634b8 [ 5312.470406] RDX: 0000000000000003 RSI: 0000000000000001 RDI: ffff8d63fda0f000 [ 5312.470408] RBP: ffff8d63f8f15398 R08: 0000000000000000 R09: ffffffff98bca248 [ 5312.470409] R10: ffff8d6407531328 R11: 0000000000000000 R12: ffff8d63f8f15398 [ 5312.470411] R13: ffff8d63fda0f000 R14: ffff8d63fda0f000 R15: ffff8d640851a000 [ 5312.470415] FS: 00007ff500ff9700(0000) GS:ffff8d640f880000(0000) knlGS:0000000000000000 [ 5312.470417] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5312.470419] CR2: 0000000000000000 CR3: 0000001a1e700004 CR4: 00000000001606e0 [ 5312.470421] Call Trace: [ 5312.470430] usb_hcd_alloc_bandwidth+0x23d/0x360 [ 5312.470438] usb_set_interface+0x120/0x360 [ 5312.470452] uvc_video_start_transfer+0x19c/0x4f0 [uvcvideo] [ 5312.470461] uvc_video_start_streaming+0x7b/0xd0 [uvcvideo] [ 5312.470467] uvc_start_streaming+0x2d/0xf0 [uvcvideo] [ 5312.470478] vb2_start_streaming+0x63/0x100 [videobuf2_common] [ 5312.470484] vb2_core_streamon+0x54/0xb0 [videobuf2_common] [ 5312.470490] uvc_queue_streamon+0x2a/0x40 [uvcvideo] [ 5312.470496] uvc_ioctl_streamon+0x3a/0x60 [uvcvideo] [ 5312.470518] __video_do_ioctl+0x377/0x3b0 [videodev] [ 5312.470529] ? do_futex+0x87d/0xcb0 [ 5312.470534] ? __mod_lruvec_state+0x41/0xf0 [ 5312.470544] video_usercopy+0x177/0x570 [videodev] [ 5312.470555] ? v4l_reqbufs+0x60/0x60 [videodev] [ 5312.470560] ? selinux_file_ioctl+0x122/0x1c0 [ 5312.470570] v4l2_ioctl+0x48/0x50 [videodev] [ 5312.470577] ksys_ioctl+0x82/0xc0 [ 5312.470581] __x64_sys_ioctl+0x16/0x20 [ 5312.470588] do_syscall_64+0x4d/0x90 [ 5312.470593] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 5312.470598] RIP: 0033:0x7ff52a45e3bb [ 5312.470602] Code: 0f 1e fa 48 8b 05 dd aa 0c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ad aa 0c 00 f7 d8 64 89 01 48 [ 5312.470604] RSP: 002b:00007ff500ff88f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 5312.470607] RAX: ffffffffffffffda RBX: 00007ff4e4026260 RCX: 00007ff52a45e3bb [ 5312.470609] RDX: 000056430202cff0 RSI: 0000000040045612 RDI: 0000000000000027 [ 5312.470611] RBP: 000056430202cfe0 R08: 00000000000005e7 R09: 0000000000000000 [ 5312.470613] R10: 00000000fffffffe R11: 0000000000000246 R12: 0000000000000000 [ 5312.470614] R13: 0000000000000004 R14: 00007ff4e400c8c0 R15: 0000000000000001 [ 5312.470618] Modules linked in: snd_seq_dummy snd_hrtimer rfcomm xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT nf_nat_tftp nft_objref nf_conntrack_tftp tun bridge stp llc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_raw iptable_security vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) ip_set nf_tables nfnetlink ip6table_filter ip6_tables iptable_filter overlay cmac bnep lm75 rpcrdma sunrpc ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod xfs ib_srp scsi_transport_srp ib_ipoib rdma_ucm vfat fat squashfs ib_umad rdma_cm loop ib_cm iw_cm btusb btrtl btbcm btintel bluetooth intel_rapl_msr intel_rapl_common ecdh_generic ecc sb_edac x86_pkg_temp_thermal intel_powerclamp [ 5312.470671] snd_hda_codec_realtek coretemp snd_hda_codec_generic snd_hda_codec_hdmi ledtrig_audio kvm_intel snd_hda_intel snd_intel_dspcfg dm_cache_smq ocrdma uvcvideo snd_hda_codec snd_usb_audio kvm videobuf2_vmalloc iTCO_wdt ucsi_ccg intel_pmc_bxt typec_ucsi snd_hda_core typec snd_usbmidi_lib ib_uverbs videobuf2_memops nvidia_drm(POE) iTCO_vendor_support pktcdvd nvidia_modeset(POE) irqbypass dm_cache snd_hwdep snd_rawmidi videobuf2_v4l2 rapl videobuf2_common intel_cstate dm_persistent_data nvidia_uvm(OE) ib_core dm_bio_prison snd_seq snd_seq_device intel_uncore hp_wmi videodev joydev pcspkr sparse_keymap snd_pcm wmi_bmof mc rfkill snd_timer i2c_i801 nvidia(POE) i2c_smbus lpc_ich snd soundcore i2c_nvidia_gpu tpm_infineon binfmt_misc nbd ip_tables amdgpu iommu_v2 gpu_sched i2c_algo_bit ttm drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel cec ghash_clmulni_intel drm serio_raw nvme e1000e be2net nvme_core wmi fuse [ 5312.470722] CR2: 0000000000000000 [ 5312.470726] ---[ end trace 1df6e1f93d1754fc ]--- [ 5312.470729] RIP: 0010:usb_ifnum_to_if+0x3a/0x50 [ 5312.470732] Code: 34 41 0f b6 50 04 84 d2 74 2f 83 ea 01 49 8d 80 98 00 00 00 49 8d 8c d0 a0 00 00 00 eb 09 48 83 c0 08 48 39 c8 74 12 4c 8b 00 <49> 8b 10 0f b6 52 02 39 f2 75 e9 4c 89 c0 c3 45 31 c0 4c 89 c0 c3 [ 5312.470734] RSP: 0018:ffffac3683143bb0 EFLAGS: 00010206 [ 5312.470737] RAX: ffff8d63f1463498 RBX: 0000000000000000 RCX: ffff8d63f14634b8 [ 5312.470739] RDX: 0000000000000003 RSI: 0000000000000001 RDI: ffff8d63fda0f000 [ 5312.470740] RBP: ffff8d63f8f15398 R08: 0000000000000000 R09: ffffffff98bca248 [ 5312.470742] R10: ffff8d6407531328 R11: 0000000000000000 R12: ffff8d63f8f15398 [ 5312.470744] R13: ffff8d63fda0f000 R14: ffff8d63fda0f000 R15: ffff8d640851a000 [ 5312.470747] FS: 00007ff500ff9700(0000) GS:ffff8d640f880000(0000) knlGS:0000000000000000 [ 5312.470749] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5312.470751] CR2: 0000000000000000 CR3: 0000001a1e700004 CR4: 00000000001606e0
get_maintainer.pl is your friend.
[email protected] is the mailing list of last resort, so go for Greg KH and linux-usb initially. |
Yep thanks I've submitted and sent the email to the usb list. |
Hi, Any suggestion what i could do ? [Sa Mai 22 21:23:18 2021] uvcvideo: Failed to resubmit video URB (-1). |
Ask on a Jetson forum? |
Fixed upstream. |
did this solution help you solve the problem with the usb camera? |
Good question, forgot all about this issue, since the workaround of keeping the webcam on at all times has been working flawlessly. But I might be testing this again, to check. However, I am skeptical about the solution ( @jboero ). From my limited understanding, the In my view, the function should indicate in its usage documentation that the The function in question is called as |
I keep the camera on all the time, but that doesn't solve the problem I increased the buffers of the uvcvideo.ko driver as suggested here and that didn't solve the problem either. Checking for NULL, there is no linux kernel branch in 5.x or 6.x, so this is also not a solution. P.S. I don't just keep the camera turned on, but I'm constantly reading 1920 x 1080 30fps MJPEG from it |
I have ten Raspberry Pi 3's (Model B Rev 1.2) with a USB 4G-modem and a USB webcam plugged in. And I managed to get a log which is similar to what @l-spitz describes at #1272 (but I have not been able to reproduce the issue with the dwc_otg buspower toggle) - see down below.
It should also be mentioned that we have ten additional Raspberry Pi 3's with an identical setup and hardware, except that these ten have no USB webcam, but instead use the RPi camera module. These ten work fine, stable for at least months.
When the webcam is used every 5 minutes using crontab, the problem occurs every 24~48 hours. A kernel oops shows up, and after half an hour of no 4G connection the system is rebooted by an external watchdog that checks if the RPi is still functional (without this, presumably the RPi would stay 'frozen' or off). There is definitively no power supply issue, especially since we've recently tested using an externally powered USB-hub which still does not solve the problem.
Interestingly, the problem only occurs when both the webcam and modem are plugged in (and used). I have another ten RPi's that have been up and running for about a year now 24/7, using the exact same modem and power supply etc, but without the webcam, and those RPi's have never had any problems.
Unfortunately I haven't been able to reproduce the problem, it seems to have the properties of a race condition of some sorts. Using the webcam every 5 minutes instead of every hour gives a higher probability of the problem showing up. However, any script where I have tried to force the problem with a shorter interval did not increase the frequency of the problem. I have also tried stressing the RPi with high CPU and other things, but to no avail. Additionally, when the webcam is plugged in but it is not used, the problem does not occur.
Since at first I thought the problem was only related to the webcam, I have tried several uvcvideo modprobe options such as quirks mode and nodrop and several other combinations. These did not have an effect on the problem.
Some more information on the RPi's (Raspbian GNU/Linux 8):
Additionally, in february I have fully updated/upgraded the system, but the problem persisted. Therefore, it does not seem to be related to a bug that occurs in only a specific kernel version. This problem spans about a year of updates.
Here are the kernel logs I managed to extract from one of the RPi's (I don't have too many logs, since I have been using
tmpfs
on/var/log
to reduce SD-card corruption):The text was updated successfully, but these errors were encountered: