Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nightscout crash: "MongoDB connection failed! Double check the MONGODB_URI setting in Heroku" #6775

Open
philipgo opened this issue Jan 20, 2021 · 25 comments

Comments

@philipgo
Copy link
Contributor

philipgo commented Jan 20, 2021

Describe the bug
Every day or so my Nightscout crashes with the error message "MongoDB connection failed! Double check the MONGODB_URI setting in Heroku."

The problem appears to be with Nightscout (or AndroidAPS??), I do not need to restart the MongoDB instance and other Nightscout instances continue to work with the same MONGODB_URI without any issues. However I do need to manually restart the Nightscout instance.

Expected behavior
Nightscout should not crash or if it does it should recover automatically.

Screenshots
NS crash

Your setup information
MongoDB self hosted on a private server, Nightscout Docker (latest dev, but same issue on master) hosted on a different private server. My uploader is AndroidAPS 2.8.1.1

Logs

nightscout_1  | DEDUPE: Devicestatus exact match
nightscout_1  | /opt/app/node_modules/mongodb/lib/utils.js:668
nightscout_1  |           throw error;
nightscout_1  |           ^
nightscout_1  |
nightscout_1  | TypeError: Cannot read property '0' of undefined
nightscout_1  |     at findResult (/opt/app/lib/server/websocket.js:414:32)
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/utils.js:665:9
nightscout_1  |     at handleCallback (/opt/app/node_modules/mongodb/lib/utils.j                                                                                                                                                             s:102:55)
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/cursor.js:834:20
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/core/cursor.js:736:9
nightscout_1  |     at Object.callback (/opt/app/node_modules/mongodb/lib/core/c                                                                                                                                                             ursor.js:432:11)
nightscout_1  |     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/cor                                                                                                                                                             e/sdam/topology.js:443:25)
nightscout_1  |     at listOnTimeout (internal/timers.js:554:17)
nightscout_1  |     at processTimers (internal/timers.js:497:7)
nightscout_1  | Executing startBoot
nightscout_1  | Executing checkNodeVersion
nightscout_1  | Node LTS version v14.15.3 is supported
nightscout_1  | Executing checkEnv
nightscout_1  | Executing augmentSettings
nightscout_1  | Parsing config URL from IMPORT_CONFIG failed
nightscout_1  | Executing checkSettings
nightscout_1  | Checking settings
nightscout_1  | Executing setupStorage
nightscout_1  | Setting up new connection to MongoDB
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 3 sec
nightscout_1  | ERROR CONNECTING TO MONGO Error: MongoDB connection failed! Doub                                                                                                                                                             le check the MONGODB_URI setting in Heroku.
nightscout_1  |     at connect_with_retry (/opt/app/lib/storage/mongo-storage.js                                                                                                                                                             :70:28)
nightscout_1  |     at processTicksAndRejections (internal/process/task_queues.j                                                                                                                                                             s:93:5)
nightscout_1  | Mongo Storage system ready
nightscout_1  | Executing setupAuthorization
nightscout_1  | Executing setupInternals
nightscout_1  | Executing ensureIndexes
nightscout_1  | Executing setupListeners
nightscout_1  | Executing setupBridge
nightscout_1  | Executing setupMMConnect
nightscout_1  | Executing finishBoot
nightscout_1  | Boot event processing completed
nightscout_1  | Security settings: INSECURE_USE_HTTP= true , SECURE_HSTS_HEADER=                                                                                                                                                              false
nightscout_1  | Listening on port 1337 b2d084333cfc
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 6 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 9 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 12 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 15 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 18 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 21 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 24 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 27 sec
nightscout_1  | Successfully established a connected to MongoDB
nightscout_1  | Mongo user role seems ok: [ { role: 'readWrite', db: 'nightscout'                                                                                                                                                              } ]
nightscout_1  | Mongo Storage system ready
@sulkaharo
Copy link
Member

Right - would it be possible for you to deploy this branch, so we'll see what the error is. Looks like you're hitting a previously unreported bug and sadly the current build doesn't log the error that's happening. #6777

@sulkaharo
Copy link
Member

@philipgo
Copy link
Contributor Author

I have deployed the websocket_dedupe_error branch and will wait for it to crash again and get back to you. Thank you for looking into this!

@philipgo
Copy link
Contributor Author

Here are the logs of a crash running your websocket_dedupe_errorbranch:
(The MongoDB logs show nothing exceptional)

nightscout_1  | devicestatus insertion error:  getaddrinfo EAI_AGAIN mongo.xyz.de
nightscout_1  | DEDUPE: Exact match
nightscout_1  | /opt/app/node_modules/mongodb/lib/utils.js:668
nightscout_1  |           throw error;
nightscout_1  |           ^
nightscout_1  |
nightscout_1  | TypeError: Cannot read property '0' of undefined
nightscout_1  |     at findResult (/opt/app/lib/server/websocket.js:326:32)
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/utils.js:665:9
nightscout_1  |     at handleCallback (/opt/app/node_modules/mongodb/lib/utils.j                                                                                                                                                             s:102:55)
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/cursor.js:834:20
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/core/cursor.js:736:9
nightscout_1  |     at Object.callback (/opt/app/node_modules/mongodb/lib/core/c                                                                                                                                                             ursor.js:432:11)
nightscout_1  |     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/cor                                                                                                                                                             e/sdam/topology.js:443:25)
nightscout_1  |     at listOnTimeout (internal/timers.js:554:17)
nightscout_1  |     at processTimers (internal/timers.js:497:7)
nightscout_1  | Executing startBoot
nightscout_1  | Executing checkNodeVersion
nightscout_1  | Node LTS version v14.15.3 is supported
nightscout_1  | Executing checkEnv
nightscout_1  | Executing augmentSettings
nightscout_1  | Parsing config URL from IMPORT_CONFIG failed
nightscout_1  | Executing checkSettings
nightscout_1  | Checking settings
nightscout_1  | Executing setupStorage
nightscout_1  | Setting up new connection to MongoDB
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 3 sec
nightscout_1  | ERROR CONNECTING TO MONGO Error: MongoDB connection failed! Doub                                                                                                                                                             le check the MONGODB_URI setting in Heroku.
nightscout_1  |     at connect_with_retry (/opt/app/lib/storage/mongo-storage.js                                                                                                                                                             :70:28)
nightscout_1  |     at processTicksAndRejections (internal/process/task_queues.j                                                                                                                                                             s:93:5)
nightscout_1  | Mongo Storage system ready
nightscout_1  | Executing setupAuthorization
nightscout_1  | Executing setupInternals
nightscout_1  | Executing ensureIndexes
nightscout_1  | Executing setupListeners
nightscout_1  | Executing setupBridge
nightscout_1  | Executing setupMMConnect
nightscout_1  | Executing finishBoot
nightscout_1  | Boot event processing completed
nightscout_1  | Security settings: INSECURE_USE_HTTP= true , SECURE_HSTS_HEADER=                                                                                                                                                              false
nightscout_1  | Listening on port 1337 dace99d2d2ce
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 6 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 9 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 12 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 15 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 18 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 21 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 24 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 27 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 30 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 33 sec
nightscout_1  | Successfully established a connected to MongoDB
nightscout_1  | Mongo user role seems ok: [ { role: 'readWrite', db: 'nightscout'                                                                                                                                                              } ]
nightscout_1  | Mongo Storage system ready

@ghost
Copy link

ghost commented Jan 21, 2021

I have just deployed the latest and got the same error straight away.

@sulkaharo
Copy link
Member

Dammit, @philipgo can you update to latest in the branch, there were two additional error spots like this, which now also capture the error. Btw how do you host the site? Based on the URL this is your own setup? Which Mongo version do you have? @lukebrogan Same to you - how are you hosting the site?

@ghost
Copy link

ghost commented Jan 21, 2021

Hello!
I am deploying to Heroku 20 stack.
URL is https://cgm-luke.herokuapp.com/
Let me know if you need anything else.
Many thanks.

@philipgo
Copy link
Contributor Author

@sulkaharo Thank you, Sulka, already deployed.
My MongoDB version is v4.2.11. MongoDB is self hosted on a private server, Nightscout Docker hosted on a different private server behind an nginx reverse proxy. My uploader is AndroidAPS 2.8.1.1. I have never found anything in the MongoDB logs which could point to the cause.

@lukebrogan Your error may not be the same as mine. If your site never works: are you sure your MongoDB_URI is correct? My site works fine for hours or sometimes days, then it crashes.

@ghost
Copy link

ghost commented Jan 21, 2021

Yeah you are right - it was an error in the MongoDB_URI string, all working for me now 👍

@philipgo
Copy link
Contributor Author

philipgo commented Jan 23, 2021

Here are the logs of another crash running the latest websocket_dedupe_error branch.

Some googling showed others are seeing similar errors in connection with useUnifiedTopology: true which was changed here recently.

�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'treatments',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     eventType: 'Temp Basal',
�[36mnightscout_1  |�[0m     duration: 60,
�[36mnightscout_1  |�[0m     absolute: 0,
�[36mnightscout_1  |�[0m     rate: 0,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T09:00:13Z',
�[36mnightscout_1  |�[0m     enteredBy: 'openaps://AndroidAPS',
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392413159'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m devicestatus insertion error:  getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m got data-received event, requesting reload
�[36mnightscout_1  |�[0m Problem loading entries
�[36mnightscout_1  |�[0m Problem loading activity data
�[36mnightscout_1  |�[0m Problem loading database stats
�[36mnightscout_1  |�[0m Load Complete:
�[36mnightscout_1  |�[0m 	 sgvs:576, profiles:1, food:2
�[36mnightscout_1  |�[0m data loaded: reloading sandbox data and updating plugins
�[36mnightscout_1  |�[0m OpenAPS hasn't reported a loop yet
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m devicestatus insertion error:  getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'devicestatus',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     device: 'openaps://samsung SM-G960F',
�[36mnightscout_1  |�[0m     pump: {
�[36mnightscout_1  |�[0m       clock: '2021-01-23T08:55:30Z',
�[36mnightscout_1  |�[0m       reservoir: 187,
�[36mnightscout_1  |�[0m       status: [Object],
�[36mnightscout_1  |�[0m       extended: [Object],
�[36mnightscout_1  |�[0m       battery: [Object]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     openaps: { suggested: [Object], iob: [Object] },
�[36mnightscout_1  |�[0m     uploaderBattery: 88,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T08:59:58Z',
�[36mnightscout_1  |�[0m     configuration: {},
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392398671'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'treatments',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     eventType: 'Temp Basal',
�[36mnightscout_1  |�[0m     duration: 60,
�[36mnightscout_1  |�[0m     absolute: 0,
�[36mnightscout_1  |�[0m     rate: 0,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T09:00:13Z',
�[36mnightscout_1  |�[0m     enteredBy: 'openaps://AndroidAPS',
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392413159'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m tick 2021-01-23T08:58:41.228Z
�[36mnightscout_1  |�[0m Problem loading entries
�[36mnightscout_1  |�[0m Problem loading activity data
�[36mnightscout_1  |�[0m Problem loading database stats
�[36mnightscout_1  |�[0m Load Complete:
�[36mnightscout_1  |�[0m 	 sgvs:576, profiles:1, food:2
�[36mnightscout_1  |�[0m data loaded: reloading sandbox data and updating plugins
�[36mnightscout_1  |�[0m OpenAPS hasn't reported a loop yet
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'devicestatus',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     device: 'openaps://samsung SM-G960F',
�[36mnightscout_1  |�[0m     pump: {
�[36mnightscout_1  |�[0m       clock: '2021-01-23T08:55:30Z',
�[36mnightscout_1  |�[0m       reservoir: 187,
�[36mnightscout_1  |�[0m       status: [Object],
�[36mnightscout_1  |�[0m       extended: [Object],
�[36mnightscout_1  |�[0m       battery: [Object]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     openaps: { suggested: [Object], iob: [Object] },
�[36mnightscout_1  |�[0m     uploaderBattery: 88,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T08:59:58Z',
�[36mnightscout_1  |�[0m     configuration: {},
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392398671'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'treatments',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     eventType: 'Temp Basal',
�[36mnightscout_1  |�[0m     duration: 60,
�[36mnightscout_1  |�[0m     absolute: 0,
�[36mnightscout_1  |�[0m     rate: 0,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T09:00:13Z',
�[36mnightscout_1  |�[0m     enteredBy: 'openaps://AndroidAPS',
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392413159'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m devicestatus insertion error:  getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'devicestatus',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     device: 'openaps://samsung SM-G960F',
�[36mnightscout_1  |�[0m     pump: {
�[36mnightscout_1  |�[0m       clock: '2021-01-23T08:55:30Z',
�[36mnightscout_1  |�[0m       reservoir: 187,
�[36mnightscout_1  |�[0m       status: [Object],
�[36mnightscout_1  |�[0m       extended: [Object],
�[36mnightscout_1  |�[0m       battery: [Object]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     openaps: { suggested: [Object], iob: [Object] },
�[36mnightscout_1  |�[0m     uploaderBattery: 88,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T08:59:58Z',
�[36mnightscout_1  |�[0m     configuration: {},
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392398671'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'treatments',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     eventType: 'Temp Basal',
�[36mnightscout_1  |�[0m     duration: 60,
�[36mnightscout_1  |�[0m     absolute: 0,
�[36mnightscout_1  |�[0m     rate: 0,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T09:00:13Z',
�[36mnightscout_1  |�[0m     enteredBy: 'openaps://AndroidAPS',
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392413159'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m Problem loading entries
�[36mnightscout_1  |�[0m Problem loading activity data
�[36mnightscout_1  |�[0m Problem loading database stats
�[36mnightscout_1  |�[0m Load Complete:
�[36mnightscout_1  |�[0m 	 sgvs:576, profiles:1, food:2
�[36mnightscout_1  |�[0m data loaded: reloading sandbox data and updating plugins
�[36mnightscout_1  |�[0m OpenAPS hasn't reported a loop yet
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m devicestatus insertion error:  getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }

@sulkaharo
Copy link
Member

@philipgo Given the logged error is simply that the connection has failed, I'm assuming this has to be something in your Mongo setup, as this hasn't been reported by others. Not the Mongo Driver in Nightscout has been configured to auto-reconnect and this seems to be also fail in this case. FWIW I have a similar setup in Digital Ocean and have never had the Mongo connection fail in this manner. Wonder if there's a Mongo forum somewhere that could help you?

@philipgo
Copy link
Contributor Author

I am still experimenting with different setups.

My MongoDB instance is super fast, so I would have no idea where timeouts might come from. However, is there a reason setting them to much shorter values than their defaults? Just removing the two lines will let them default to socketTimeoutMS = 360000 and connectTimeoutMS=10000.

One note: One of your changes in https://github.com/nightscout/cgm-remote-monitor/pull/6777/files causes the MONGODB_URI string to be displayed in full in the html error message on the web, including user name and password.

@sulkaharo
Copy link
Member

Right, increasing the timeouts makes sense. Note the connection time can be different from the query performance and if what you're seeing is caused by a connection or socket timing out, that indicates there's something odd either in the networking between the servers or the Mongo server is misbehaving and sleeping, or some other issue that slows down the handshake.

@inventor96
Copy link
Contributor

I was having a very similar issue on my local machine doing development work. The error message said the type was ReplicaSetNoPrimary. I had it pointing to my live database on MongoDB Cloud. While troubleshooting, I determined that the DNS record for my MongoDB instance didn't exist (I checked locally and by using some DNS lookup services online). This missing DNS record issue has come and gone periodically.

I chatted MongoDB support, and the support rep gave me another connection string to work with instead of the one I was using on my live Nightscout instance on Heroku, which specifically set the name of each shard (rather than referencing the cluster as a whole) and changed a couple options.

Original connection string:

mongodb+srv://username:[email protected]/heroku_bp7z8gq4?retryWrites=true&w=majority

New connection string:

mongodb://username:[email protected]:27017,cluster-bp7z8gq4-shard-00-01.ngvbd.mongodb.net:27017,cluster-bp7z8gq4-shard-00-02.ngvbd.mongodb.net:27017/heroku_bp7z8gq4?ssl=true&replicaSet=atlas-llw8hn-shard-0&authSource=admin&retryWrites=true&w=majority

Using this new connection string, I have been able to connect to my database on MongoDB Cloud from my local machine.

@peligrims
Copy link

I installed linux ubuntu 20.04.2 and nightscout After starting i have logo on black screen "loading the client" .what am i doing wrong?

@bewest
Copy link
Member

bewest commented May 6, 2022

On ubuntu, make sure to unset the LANGUAGE environment variable, or set it to one of the values Nightscout supports. Recent versions of Ubuntu set this in my case to en_US:en when Nightscout needs en. Hope this helps.

@marcirene
Copy link

Hello,I ve made 7 attempts to add my Mongo connection string into my Heroku application code (need it for my diabetes application). Keep getting the message: 'Unable to connect to Mongo, MongoDB connection failed! Double check the MONGODB_URI setting in Heroku'.

I can think of 2 things going wrong (see attachment)

  1. Do i have to make a copy of the string by 'including full driver code example' (red arrow)...or not necessary? (have used the short one till know)

  2. Also don't know which combination of username & pasword to ad. Is this the combination for the account from Mongo?....or the account from Heroku (where I try to deploy my app?) (green stripe!)

Thanks for your help!

Dear regards Marc

1

@psonnera
Copy link

psonnera commented May 17, 2022

@marcirene you need to replace password (remove bracket!) with the one you decided in step 3e https://nightscout.github.io/nightscout/new_user/#step-3-create-an-atlas-account Please don't select anything that's not specified in the documentation. If you need to change your Atlas password look at this https://nightscout.github.io/troubleshoot/connection_string/#change-your-atlas-database-password

@marcirene
Copy link

@marcirene you need to replace password (remove bracket!) with the one you decided in step 3e https://nightscout.github.io/nightscout/new_user/#step-3-create-an-atlas-account Please don't select anything that's not specified in the documentation. If you need to change your Atlas password look at this https://nightscout.github.io/troubleshoot/connection_string/#change-your-atlas-database-password

Thanks for your help psonnera! Don't get it yet ;) (see attachment 2). My nightscout username is my gmail adres. However Nightscout calls me 'Flieren123' as the username. Do you think this creates the problem?

1

@marcirene
Copy link

...forgot something....do i use the 'big connection string' (see bleu 'check mark') or the small one (whiteout using the check mark?)

@psonnera
Copy link

Please follow the documentation. Your Atlas login is different from your database username and password. Make sure to create a database user and a password you write down and use those for the connection string. Flieren123 is your database user. Do not include full code example to obtain the connection string.

@Maceysmom
Copy link

@marcirene you need to replace password (remove bracket!) with the one you decided in step 3e https://nightscout.github.io/nightscout/new_user/#step-3-create-an-atlas-account Please don't select anything that's not specified in the documentation. If you need to change your Atlas password look at this https://nightscout.github.io/troubleshoot/connection_string/#change-your-atlas-database-password

Thank you so much for the simple "remove bracket" instruction! I've been losing my mind for 2 hours trying to figure out what I could possibly be doing wrong! Working now! Thank you SO much!!!!!

@robster7674
Copy link

Just had this today on Northflank, with MongoDB as add-on, also in Northflank. The logs weren't very helpful, so what I first did was re-copy MONGO_SRV contents to MONGODB_URI. After the container with the updated config started, NightScout worked again.

@szantos
Copy link

szantos commented Sep 13, 2023

Same occured here 4-6 times during the last 3 weeks.
A "rollout restart" seems to solve it for some days, but missing data (during downtime) won't get backfilled.

Host: Northflank
DB: mongoDB (Atlas)
NS dev 15 d602de1
AAPS dev with NSClient v3 + websockets

I had aa4faa6 running without any problems in April-July (maybe August), then something happened, I've started to experience this failure. A newer NS build did not solved it.

@robster7674
Copy link

Just had this again some hours ago, running da3b2ee on Northflank.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants