Select node version failed on custom deployment

[chillyistkult]

So I have fought through endless bugs and problems to get to a point were the first deployment of our nodejs application works to see that it is broken again without even touching anything. No idea what is wrong this time, maybe you guys have an idea.

Command: bash deploy.sh
Handling YDM deployment!
Using start-up script dist/server from package.json.
Generated web.config.
Node.js versions available on the platform are: 0.6.20, 0.8.2, 0.8.19, 0.8.26, 0.8.27, 0.8.28, 0.10.5, 0.10.18, 0.10.21, 0.10.24, 0.10.26, 0.10.28, 0.10.29, 0.10.31, 0.10.32, 0.10.40, 0.12.0, 0.12.2, 0.12.3, 0.12.6, 4.0.0, 4.1.0, 4.1.2, 4.2.1, 4.2.2, 4.2.3, 4.2.4, 4.3.0, 4.3.2, 4.4.0, 4.4.1, 4.4.6, 4.4.7, 5.0.0, 5.1.1, 5.3.0, 5.4.0, 5.5.0, 5.6.0, 5.7.0, 5.7.1, 5.8.0, 5.9.1, 6.0.0, 6.1.0, 6.2.2, 6.3.0.
Selected node.js version 6.2.2. Use package.json file to choose a different version.
Selected npm version 3.9.5
ENOENT: no such file or directory, open 'D:\home\site\wwwroot\dist\iisnode.yml'
Updating iisnode.yml at D:\home\site\wwwroot\dist\iisnode.yml
An error has occurred during web site deployment.
select node version failed
ENOENT: no such file or directory, open 'D:\home\site\wwwroot\dist\iisnode.yml'\r\nC:\Program Files (x86)\SiteExtensions\Kudu\56.50706.2317\bin\Scripts\starter.cmd bash deploy.sh

And the deploy.sh

#!/bin/bash

# ----------------------
# KUDU Deployment Script
# Version: 1.0.6
# ----------------------

# Helpers
# -------

exitWithMessageOnError () {
  if [ ! $? -eq 0 ]; then
    echo "An error has occurred during web site deployment."
    echo $1
    exit 1
  fi
}

# Prerequisites
# -------------

# Verify node.js installed
hash node 2>/dev/null
exitWithMessageOnError "Missing node.js executable, please install node.js, if already installed make sure it can be reached from current environment."

# Setup
# -----

SCRIPT_DIR="${BASH_SOURCE[0]%\\*}"
SCRIPT_DIR="${SCRIPT_DIR%/*}"
ARTIFACTS=$SCRIPT_DIR/../artifacts
KUDU_SYNC_CMD=${KUDU_SYNC_CMD//\"}

if [[ ! -n "$DEPLOYMENT_SOURCE" ]]; then
  DEPLOYMENT_SOURCE=$SCRIPT_DIR
fi

if [[ ! -n "$NEXT_MANIFEST_PATH" ]]; then
  NEXT_MANIFEST_PATH=$ARTIFACTS/manifest

  if [[ ! -n "$PREVIOUS_MANIFEST_PATH" ]]; then
    PREVIOUS_MANIFEST_PATH=$NEXT_MANIFEST_PATH
  fi
fi

if [[ ! -n "$DEPLOYMENT_TARGET" ]]; then
  DEPLOYMENT_TARGET=$ARTIFACTS/wwwroot
else
  KUDU_SERVICE=true
fi

if [[ ! -n "$KUDU_SYNC_CMD" ]]; then
  # Install kudu sync
  echo Installing Kudu Sync
  npm install kudusync -g --silent
  exitWithMessageOnError "npm failed"

  if [[ ! -n "$KUDU_SERVICE" ]]; then
    # In case we are running locally this is the correct location of kuduSync
    KUDU_SYNC_CMD=kuduSync
  else
    # In case we are running on kudu service this is the correct location of kuduSync
    KUDU_SYNC_CMD=$APPDATA/npm/node_modules/kuduSync/bin/kuduSync
  fi
fi

# Node Helpers
# ------------

selectNodeVersion () {
  if [[ -n "$KUDU_SELECT_NODE_VERSION_CMD" ]]; then
    SELECT_NODE_VERSION="$KUDU_SELECT_NODE_VERSION_CMD \"$DEPLOYMENT_SOURCE\" \"$DEPLOYMENT_TARGET\" \"$DEPLOYMENT_TEMP\""
    eval $SELECT_NODE_VERSION
    exitWithMessageOnError "select node version failed"

    if [[ -e "$DEPLOYMENT_TEMP/__nodeVersion.tmp" ]]; then
      NODE_EXE=`cat "$DEPLOYMENT_TEMP/__nodeVersion.tmp"`
      exitWithMessageOnError "getting node version failed"
    fi

    if [[ -e "$DEPLOYMENT_TEMP/__npmVersion.tmp" ]]; then
      NPM_JS_PATH=`cat "$DEPLOYMENT_TEMP/__npmVersion.tmp"`
      exitWithMessageOnError "getting npm version failed"
    fi

    if [[ ! -n "$NODE_EXE" ]]; then
      NODE_EXE=node
    fi

    NPM_CMD="\"$NODE_EXE\" \"$NPM_JS_PATH\""
  else
    NPM_CMD=npm
    NODE_EXE=node
  fi
}

##################################################################################################################################
# Deployment
# ----------

echo Handling YDM deployment!

#1. Select node version
selectNodeVersion

#2. Install npm packages
if [ -e "$DEPLOYMENT_SOURCE/package.json" ]; then
  eval $NPM_CMD install --dev
  exitWithMessageOnError "npm failed"
fi

#3. Install bower packages
if [ -e "$DEPLOYMENT_SOURCE/bower.json" ]; then
  eval $NPM_CMD install bower
  exitWithMessageOnError "installing bower failed"
  ./node_modules/.bin/bower install
  exitWithMessageOnError "bower failed"
fi

#4. Run grunt
if [ -e "$DEPLOYMENT_SOURCE/Gruntfile.js" ]; then
  eval $NPM_CMD install grunt-cli
  exitWithMessageOnError "installing grunt failed"
  ./node_modules/.bin/grunt --no-color build:production
  exitWithMessageOnError "grunt failed"
fi

#5. KuduSync to Target
"$KUDU_SYNC_CMD" -v 500 -f "$DEPLOYMENT_SOURCE/dist" -t "$DEPLOYMENT_TARGET" -n "$NEXT_MANIFEST_PATH" -p "$PREVIOUS_MANIFEST_PATH" -i ".git;.hg;.deployment;deploy.sh"
exitWithMessageOnError "Kudu Sync to Target failed"

##################################################################################################################################

# Post deployment stub
if [[ -n "$POST_DEPLOYMENT_ACTION" ]]; then
  POST_DEPLOYMENT_ACTION=${POST_DEPLOYMENT_ACTION//\"}
  cd "${POST_DEPLOYMENT_ACTION_DIR%\\*}"
  "$POST_DEPLOYMENT_ACTION"
  exitWithMessageOnError "post deployment action failed"
fi

echo "Finished successfully."

[davidebbo]

Is the error random, or does it always happen? If always, can you share a test repo that we can use to repro the issue? Thanks!

[chillyistkult]

If you wish you can use the production repo (it's just a project for my bachelor thesis) https://github.com/chillyistkult/ydm. The deploy.sh is generated by the azure cli and only modified to run a grunt build:production task after bower install. As of now the error always happens but it worked in the past on a few commits. I haven't changed anything related to the deployment since then and it's also suspicious that Kudo fails to select a node version, because that looks like a rather basic task I can't really influence much.

The related Azure repository is http://ydm.azurewebsites.net/

[davidebbo]

I tried pushing your repo to a new site, and I did not see the error:

Handling YDM deployment!
Start script "dist/server" from package.json is not found.
Looking for app.js/server.js under site root.
Missing server.js/app.js files, web.config is not generated
Node.js versions available on the platform are: 0.6.20, 0.8.2, 0.8.19, 0.8.26, 0.8.27, 0.8.28, 0.10.5, 0.10.18, 0.10.21, 0.10.24, 0.10.26, 0.10.28, 0.10.29, 0.10.31, 0.10.32, 0.10.40, 0.12.0, 0.12.2, 0.12.3, 0.12.6, 4.0.0, 4.1.0, 4.1.2, 4.2.1, 4.2.2, 4.2.3, 4.2.4, 4.3.0, 4.3.2, 4.4.0, 4.4.1, 4.4.6, 4.4.7, 5.0.0, 5.1.1, 5.3.0, 5.4.0, 5.5.0, 5.6.0, 5.7.0, 5.7.1, 5.8.0, 5.9.1, 6.0.0, 6.1.0, 6.2.2, 6.3.0.
Selected node.js version 6.2.2. Use package.json file to choose a different version.
Selected npm version 3.9.5
Updating iisnode.yml at D:\home\site\wwwroot\iisnode.yml

What's interesting is that your error has "D:\home\site\wwwroot\dist\iisnode.yml", with the dist segment in there, while mine doesn't. Does that ring any bell about where it might be coming from?

[chillyistkult]

I think it happens after you push the repo a second time after the first succesful deployment. Don't ask my why, I hoped you could tell me. Kudo sync is configured to only copy the /dist directory which is fine and as I mentioned it also worked for me the first time on a fresh app environment. Then some error kicks in (like this one or Thread aborted error) and I revert the app environment because it doesn't look like the problems are app related.

So as I didn't changed anything in the generated deploy.sh (beside the grunt task) I am a little bit lost here.

[davidebbo]

I think I sort of see what's going on:

• We have logic in SelectNodeVersion.js (here) that looks at the start script in package.json and tried to make sense of it. In your case, you have "start": "node dist/server".
• The code then checks if it can find that script. In your case, the dist folder is not in the repo (i.e. in D:\home\site\repository), so in the initial deployment, we end up ignoring the start script. Hence the output (first deployment): Start script "dist/server" from package.json is not found.
• But then later in the deployment something creates dist/server.js in the repo.
• On the next deployment, it does find dist/server.js under D:\home\site\repository, so it chooses to make use of the start script. But because the dist folder never makes it to the wwwroot folder, it blows up.

So I'd say the root of the issue is that there is a bit of a clash between having "start": "node dist/server", and the fact that you don't actually have the dist folder is DEPLOYMENT_TARGET (based on how you run kudusync).

[chillyistkult]

What I don't understand is why is SelectNodeVersion.js trying to start my server.js before the build process is finished and the output /dist folder is copied to wwwroot? I mean even if I change the path in "start" to "node server" it will fail when it tries to boot up the script in D:\home\site\repository.

So as far as I understand Azure/Kudo actually tries to run the script described in package.json "start" before AND after the deployment process?

So I need a solution to either have the directory path of repository and wwwroot to my server.js the same (e.g. DEPLOYMENT_TARGET is not wwwroot but wwwroot/dist - which is awkward) or I somehow tell Azure to only boot up server.js after deployment.

Can I ask you what you would do in such a situation?

[davidebbo]

SelectNodeVersion does not try to run anything. It tries to parse out the start script so it can correctly generate iisnode.yml with nodeProcessCommandLine set to the right value.

But it may be that this logic needs to be adapted to cover scenarios like yours.

Is this a correct summary of your scenario:

• At build time, you create a dist folder under the repo root.
• You want the content of that folder (but not the folder itself) to end up in your wwwroot

[chillyistkult]

Yeah that's exactly how it should work. The dist folder contains a client and a server directory after the application is build in repo root (the grunt build:production task does that) , Kudo needs to sync the content of the dist folder to wwwroot (which is already working) and then I just need to run the app.js within my server directory to boot up the application.

So if I change "start": "node dist/server" in my package,json to "start": "node server"it should work.

[nwabdou85]

Hi @chillyistkult @davidebbo , I've similaire error , if you can help please,

here my log error
http://paste.ofcode.org/VVGYBc5yAJRRty5VYFH2Xs
http://paste.ofcode.org/7NTuffRHHTDKZgGqxMFw2a
http://paste.ofcode.org/hdRTHsfJQYWCzvN6nHCExn

Thank's

[chillyistkult]

@nwabdou85 after weeks of investigations, I moved my infrastructure to Amazon AWS.

Azure is just not made to use (or abuse) as a build instance at the moment. As there are no meaningful integrations with third party continuous integration providers like circleci or travis right now I would suggest save your time on that, because even if you resolve this error I can ensure that there are plenty of other problems you will run into (like sporadic timeouts during build and copy tasks).

[nwabdou85]

@chillyistkult thank's