edgeHub crashes after changing config.yaml when configured for additional offline storage #1177
Labels
Comments
|
Thank you for the bug report. You are running into this bug: #1082 This fix is in the |
|
Thanks for the heads up @myagley. Looking forward to the next release. |
|
The release is out. The fix for this specific bug is in the daemon. To upgrade, you will need to update your local apt cache and upgrade: |
|
@myagley - Unfortunately updating does not work for me.
gives me the following output: I get the same output for
which is recommended on https://docs.microsoft.com/de-de/azure/iot-edge/how-to-update-iot-edge#update-the-security-daemon |
|
You will need to update the local package repository before installing: The first command contacts the remote repository to download the list of all new available packages. The second will actually download and install the new version. |
|
@myagley - I used both commands but just added the output of the second. |
|
@myagley - Doing another installation as described here seems to do the trick. |
|
The commands you run from here installed 1.0.7 version, so when you run Was the original issue with the storage solved after the update? |
|
I'm going to close this issue, as it is fixed in 1.0.7. Please feel to reopen if you are still facing issues. |
|
@myagley We just ran into this issue running iotedge 1.0.8 on the host with the 1.0.7.1 version of the edgeHub and edgeAgent modules: Version: I've tried but that doesn't help (results in the same error) |
|
Well, looks like the actual issue is different, just the symptom is the same: I forgot we bind-mount a named volume for the edgeHub storage instead of |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Expected Behavior
Changing RuntimeLogLevel to “debug” in /etc/config.yaml and restarting the iotedge runtime (systemctl restart iotedge) when the edgeHub is configured for additional storage should change the log level of the edgeAgent and edgeHub and not cause the edgeHub to repeatedly crash.
Current Behavior
If I configure the edgeHub to use non-container storage (https://docs.microsoft.com/en-us/azure/iot-edge/offline-capabilities) and then edit the config.yaml file (
/etc/iotedge/config.yaml) to change the RuntimeLogLevel and restartsystemctl restart iotedgethe edgeHub fails to come back up completely, logging that it received 500 Internal Server Error when making a call to the workload API to decrypt.Changing the config.yaml file back to the way it was before and restarting again does not resolve the issue.
Steps to Reproduce
Create & change permissions on the directory for offline/non-container edgeHub storage if you haven’t already done so
sudo mkdir /etc/iotedge/strorage && sudo chown -R 1000:1000 /etc/iotedge/storageUpdate your deployment.json to use this directory for storage. My systemModules section looks like the following:
Deploy and make sure everything is working/that you are using the additional storage.
Edit
/etc/iotedge/config.yamland change RuntimeLogLevel to debug (or if you have it at debug, you can change it to info or whatever your favorite log level happens to be). Save your changes and then restart the edge runtimesystemctl restart itoedgeso the changes are applied.If you follow the edgeHub and edge daemon logs, you should now see errors.
journalctl -u iotedge:
docker logs edgeHub:
Context (Environment)
Device (Host) Operating System
Ubuntu 18.04 LTS
Container Operating System
Linux Containers
Runtime Versions
iotedged
iotedge 1.0.6.1 (3fa6cbe)
Edge Agent
1.0.6
Edge Hub
1.0.6
Docker
3.0.5
Logs
edgedlogs.log
Additional Information
When the edgeHub is not configured for additional storage, I don’t see this issue. I can change the RuntimeLogLevel repeatedly and restart the runtime without issue.
After getting to this bad state, I can return to a workings state by stopping the runtime, clearing out the storage directory, and restarting the runtime.
systemctl stop iotedgesudo rm -rf /etc/iotedge/storage/*systemctl restart iotedgeThe text was updated successfully, but these errors were encountered: