Unhandled promise error when unable to reach Wazuh API #3887
Comments
ResearchI tried to replicate the reported problem that it states the plugin platform crashed due to the @asteriscos could replicate the problem and we were doing peer to locate and analyze the unhandled promise. The problem was a supposed logic to build a chain of promises and resolves it with an The try {
const hosts = await this.getApiObjects();
const data = await hosts.reduce(async (acc: Promise<object[]>, host) => {
const { status } = configuredJobs({ host, jobName: this.jobName })[this.jobName];
if (!status) return acc;
const response = await this.getResponses(host);
const accResolve = await Promise.resolve(acc)
return [
...accResolve,
...response,
];
}, Promise.resolve([]));
!!data.length && await this.saveDocument.save(data, index);
} catch (error) {
ErrorHandler(error, this.logger);
}The supposed chain of promises was generated with @asteriscos suggested replacing the chain of promises with running them in parallel because: |
|
After forcing an invalid state in a custom decoder, I was able to replicate the issue. We traced the exception to promises rejected when trying to execute cron-jobs.
After analyzing the error handling of reducing the promises, we decided a simultaneous execution of the promises would be faster and more stable. In case one of the promises fails it would not affect the others, thus we took a |
ResearchI was researching the configuration of NodeJS with the Causing an unhandled rejectionI edited the server-side code of the Wazuh plugin to cause an unhandled promise and restart the server. // Added to the `start` function
// /usr/share/wazuh-dashboard/plugins/wazuh/server/plugin.js
// Reject the promise in 5 seconds
(async () => new Promise((res,rej) => setTimeout(rej,5000)))()I tested without and with the Digging the cause the process existsTaking into account the log of the unhandled rejection: I searched it in the OpenSearch Dashboards production code and I found a file where it defines the logging of the unhandled promise and exits the process. https://github.com/opensearch-project/OpenSearch-Dashboards/blob/1.2.0/src/setup_node_env/exit_on_warning.js#L61. There is a handler for The The OpenSearch Dashboards app is initialized with: I don't know the reason, but in my testings, if the NODE_OPTIONS="--no-warnings --max-http-header-size=65536 $OSD_NODE_OPTS $NODE_OPTIONS" NODE_ENV=production exec "${NODE}" "--no-warnings" "${DIR}/src/cli/dist" ${@}
#!/bin/sh
SCRIPT=$0
# SCRIPT may be an arbitrarily deep series of symlinks. Loop until we have the concrete path.
while [ -h "$SCRIPT" ] ; do
ls=$(ls -ld "$SCRIPT")
# Drop everything prior to ->
link=$(expr "$ls" : '.*-> \(.*\)$')
if expr "$link" : '/.*' > /dev/null; then
SCRIPT="$link"
else
SCRIPT=$(dirname "$SCRIPT")/"$link"
fi
done
DIR="$(dirname "${SCRIPT}")/.."
CONFIG_DIR=${OSD_PATH_CONF:-"/etc/wazuh-dashboard"}
if [ -x "${DIR}/node/bin/node" ]; then
NODE="${DIR}/node/bin/node"
else
NODE="$(which node)"
fi
if [ ! -x "$NODE" ]; then
echo "unable to find usable node.js executable."
exit 1
fi
if [ -f "${CONFIG_DIR}/node.options" ]; then
OSD_NODE_OPTS="$(grep -v ^# < ${CONFIG_DIR}/node.options | xargs)"
fi
NODE_OPTIONS="--no-warnings --max-http-header-size=65536 $OSD_NODE_OPTS $NODE_OPTIONS" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli/dist" ${@}Other tests:Wazuh dashboard Docker image ✔️I added the same unhandled promise to the Wazuh server-side code and the server doesn't crash due to it. I checked the value of
#!/bin/sh
SCRIPT=$0
# SCRIPT may be an arbitrarily deep series of symlinks. Loop until we have the concrete path.
while [ -h "$SCRIPT" ] ; do
ls=$(ls -ld "$SCRIPT")
# Drop everything prior to ->
link=$(expr "$ls" : '.*-> \(.*\)$')
if expr "$link" : '/.*' > /dev/null; then
SCRIPT="$link"
else
SCRIPT=$(dirname "$SCRIPT")/"$link"
fi
done
DIR="$(dirname "${SCRIPT}")/.."
CONFIG_DIR=${OSD_PATH_CONF:-"$DIR/config"}
if [ -x "${DIR}/node/bin/node" ]; then
NODE="${DIR}/node/bin/node"
else
NODE="$(which node)"
fi
if [ ! -x "$NODE" ]; then
echo "unable to find usable node.js executable."
exit 1
fi
if [ -f "${CONFIG_DIR}/node.options" ]; then
OSD_NODE_OPTS="$(grep -v ^# < ${CONFIG_DIR}/node.options | xargs)"
fi
NODE_OPTIONS="--no-warnings --max-http-header-size=65536 $OSD_NODE_OPTS $NODE_OPTIONS" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli/dist" ${@}
Kibana Docker image ✔️I added the same unhandled promise to the Wazuh server-side code and the server doesn't crash. I checked the value of
#!/bin/sh
SCRIPT=$0
# SCRIPT may be an arbitrarily deep series of symlinks. Loop until we have the concrete path.
while [ -h "$SCRIPT" ] ; do
ls=$(ls -ld "$SCRIPT")
# Drop everything prior to ->
link=$(expr "$ls" : '.*-> \(.*\)$')
if expr "$link" : '/.*' > /dev/null; then
SCRIPT="$link"
else
SCRIPT=$(dirname "$SCRIPT")/"$link"
fi
done
DIR="$(dirname "${SCRIPT}")/.."
CONFIG_DIR=${KBN_PATH_CONF:-"$DIR/config"}
NODE="${DIR}/node/bin/node"
test -x "$NODE"
if [ ! -x "$NODE" ]; then
echo "unable to find usable node.js executable."
exit 1
fi
if [ -f "${CONFIG_DIR}/node.options" ]; then
KBN_NODE_OPTS="$(grep -v ^# < ${CONFIG_DIR}/node.options | xargs)"
fi
NODE_OPTIONS="--no-warnings --max-http-header-size=65536 $KBN_NODE_OPTS $NODE_OPTIONS" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli/dist" ${@}ConclustionIt seems there is some problem in the package or environment of the Wazuh dashboard, that causes the options set in the |
ResearchI was trying with a Kibana in production installed in a VM using the all-in-one deployment script and if provokes an unhandled promise, the process exists. But the server restarts due to the The service definition file for Wazuh dashboard has no the |
|
Apparently, the problem is that despite the environment variables being loaded, node js does not recognize them, the only way I found to make it work at the moment is: This is the log with this change: This is the log without this change: |
Description
We have discovered thanks to elastic/kibana#118536 that we have an unhandled promise when a request to the Wazuh API fails because of a connection error. This can make the process crash depending on the configuration of the node.config as stated here https://github.com/elastic/kibana/blob/main/config/node.options#L9
That configuration is disabled by default in opensearch fork.
Steps to reproduce
Expected Result
Actual Result
We need to fix all unhandled promises we find.
The text was updated successfully, but these errors were encountered: