script hangs after 400-jvm.sh (service tomcat9 status)? i.e. 500-nginx.sh is not run

[JonasEriksson]

Hi, I just run this on a fresh Ubuntu 20.04 LTS on ssdnodes with 32 GB RAM. The scripts fails to go on to step 500-nginx.sh?

`
● tomcat9.service - Apache Tomcat 9 Web Application Server
Loaded: loaded (/lib/systemd/system/tomcat9.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2021-06-10 13:00:25 UTC; 19ms ago
Docs: https://tomcat.apache.org/tomcat-9.0-doc/index.html
Process: 1535 ExecStartPre=/usr/libexec/tomcat9/tomcat-update-policy.sh (code=exited, status=0/SUCCESS)
Main PID: 1542 (java)
Tasks: 1 (limit: 38434)
Memory: 1.3M
CGroup: /system.slice/tomcat9.service
└─1542 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Djava.util.logging.config.file=/var/lib/tomcat9/conf/logging.properties -Djava.util.loggin>

Jun 10 13:00:25 ssd4.myserver.se systemd[1]: Starting Apache Tomcat 9 Web Application Server...
Jun 10 13:00:25 ssd4.myserver.se systemd[1]: Started Apache Tomcat 9 Web Application Server.
Jun 10 13:00:25 ssd4.myserver.se tomcat9[1542]: NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io>
~`

[pfreitag]

Thanks - I am trying to figure this out still. I've made some modifications to the script to try to get it working on Ubuntu 20.04 with Tomcat9. Let me know if you figure out anything, I'll update you as well if I figure it out.

[JonasEriksson]

Hi Pete, first of all, thanks so much for updating this! Manually runninng the remaining scripts works and I'll test it out now. My goal is to have a few personal sites with lucee on the same server so hope this will be a good fit. I suppose I need to launch a server with commandbox for each site but I will obviously try it out first (no need to answer here/now) :-)

[pfreitag]

@JonasEriksson I'm not sure why it failed and didn't run the rest of the scripts (based on what you pasted above), was there any output in the console that you still have?

With this approach you don't need to use commandbox for each site, you just setup the conf in nginx and include lucee.conf

[JonasEriksson]

Hi Pete, the output was what I pasted above in the opening issue text if thatr helps. But I just came across this which might be relevant: https://www.ringingliberty.com/2019/03/04/shell-scripting-running-service-status-command-and-exit/ quote: "By default in current versions of systemd, systemctl pipes the status output through a pager, by default less.

You can turn this behavior off by passing --no-pager in the command, in which case the output will just be dumped to standard out, and your terminal comes back immediately."

[pfreitag]

Ah thanks, does this work for you:

service tomcat9 status --no-pager

I'm not using systemctl because I'm using docker to test it, and it doesn't like systemd.

[JonasEriksson]

no, adding --no-pager to service ... does not work, but it works with this:

[JonasEriksson]

Hm, after re-running it and setting up two sites where two domain names point to the same server IP (and separate webroots: /web/mysite1/wwwroot/ and /web/mysite2/wwwroot/ I have the index.html working but not the index.cfm

Trying to access an index.cfm file it shows HTTP Status 500 – Internal Server Error Type Exception Report Message java.io.FileNotFoundException: /opt/lucee/config/server/lucee-server/patches/5.3.7.48.lco (Read-only file system) Description The server encountered an unexpected condition that prevented it from fulfilling the request. Exception javax.servlet.ServletException: java.io.FileNotFoundException: /opt/lucee/config/server/lucee-server/patches/5.3.7.48.lco (Read-only file system)

I tried switching the lucee version to 5.3.8.179-RC but the same msg comes up. I thought it was some permission issue ("read-only file system") but I am lacking the patches file (and patches directory). Will investigate further... (and probably has nothing to do with your script)

[pfreitag]

What does this output:

ls -lah /opt/lucee/
ls -lah /opt/lucee/config/server/
ls -lah /opt/lucee/config/server/lucee-server/
ls -lah /opt/lucee/config/server/lucee-server/patches/

[JonasEriksson]

After I downloaded the missing lucee lco file into patches, it went on but came up with this error msg: java.io.FileNotFoundException: /opt/lucee/config/server/lucee-server/bundles/org-apache-commons-net-3-3-0.jar (No such file or directory) here's the output to your question:

EDIT: I'll keep pulling down the missing files and see what happens :-)

[vitamindck]

My colleague, @HatBeardZero, and I have been working on this as well and have a solution that worked for us.

I apologize in advance for not submitting a proper pull request. I made a lot of additional changes for my own personal readability and am in a rush to get my weekend started with my family at the moment. I didn't want to delay a solution for anyone who was struggling with this, potentially over the weekend!

The main changes concern the mkdir commands and the .lco download. It took a surprisingly long time to figure out why the startup was failing despite suspecting directory permission issues the whole time.

scripts/300-tomcat.sh

#!/bin/bash
source ./out.fn

out "Installing Tomcat 9";
apt-get install tomcat9 openjdk-11-jdk-headless

out "Stopping Tomcat";
service tomcat9 stop

out "Configuring Tomcat";
mkdir backup
mkdir backup/etc
mkdir backup/etc/tomcat9
mkdir backup/etc/default
#backup default tomcat web.xml
cp /etc/tomcat9/web.xml backup/etc/tomcat9/web.xml-orig-backup
#copy our web.xml to tomcat directory
cp etc/tomcat9/web.xml /etc/tomcat9/

#backup default server.xml
cp /etc/tomcat9/server.xml backup/etc/tomcat9/server.xml-orig-backup
#copy our server.xml to tomcat dir
cp etc/tomcat9/server.xml /etc/tomcat9/

#backup default catalina.properties
cp /etc/tomcat9/catalina.properties backup/etc/tomcat9/catalina.properties-orig-backup
#copy our catalina properties
cp etc/tomcat9/catalina.properties /etc/tomcat9/

cp /etc/default/tomcat9 backup/etc/default/tomcat9

out "Installing mod_cfml Valve for Automatic Virtual Host Configuration";
if [ -f lib/mod_cfml-valve_v1.1.05.jar ]; then
  cp lib/mod_cfml-valve_v1.1.05.jar /opt/lucee/current/
else
  curl --location -o /opt/lucee/current/mod_cfml-valve_v1.1.05.jar https://raw.githubusercontent.com/utdream/mod_cfml/master/java/mod_cfml-valve_v1.1.05.jar
fi

MODCFML_JAR_SHA256="22c769ccead700006d53052707370c5361aabb9096473f92599708e614dad638"
if [[ $(sha256sum "/opt/lucee/current/mod_cfml-valve_v1.1.05.jar") =~ "$MODCFML_JAR_SHA256" ]]; then
    echo "Verified mod_cfml-valve_v1.1.05.jar SHA-256: $MODCFML_JAR_SHA256"
else
    echo "SHA-256 Checksum of mod_cfml-valve_v1.1.05.jar verification failed"
    exit 1
fi

if [ ! -f /opt/lucee/modcfml-shared-key.txt ]; then
  echo "Generating Random Shared Secret..."
  openssl rand -base64 42 >> /opt/lucee/modcfml-shared-key.txt
  #clean out any base64 chars that might cause a problem
  sed -i "s/[\/\+=]//g" /opt/lucee/modcfml-shared-key.txt
fi

shared_secret=`cat /opt/lucee/modcfml-shared-key.txt`

sed -i "s/SHARED-KEY-HERE/$shared_secret/g" /etc/tomcat9/server.xml

lco_url="https://cdn.lucee.org/$LUCEE_VERSION.lco"

out "Installing Lucee Core";
if [ ! -f /opt/lucee/config/server/lucee-server/patches/$LUCEE_VERSION.lco ]; then
  mkdir -p /opt/lucee/config/server/lucee-server/patches/
  curl --location -o /opt/lucee/config/server/lucee-server/patches/$LUCEE_VERSION.lco $lco_url
fi

out "Setting Permissions on Lucee Folders";
mkdir /var/lib/tomcat9/lucee-server
mkdir /opt/lucee/config/server/lucee-server/context
chown -R tomcat:tomcat /var/lib/tomcat9/lucee-server
chmod -R 750 /var/lib/tomcat9/lucee-server
chown -R tomcat:tomcat /opt/lucee
chmod -R 750 /opt/lucee

out "Setting JVM Max Heap Size to " $JVM_MAX_HEAP_SIZE

#sed -i "s/-Xmx128m/-Xmx$JVM_MAX_HEAP_SIZE/g" /etc/default/tomcat9
#-Dlucee.base.dir=/opt/lucee/config/server/
echo "JAVA_OPTS=\"\${JAVA_OPTS} -Xmx$JVM_MAX_HEAP_SIZE -Dlucee.base.dir=/opt/lucee/config/server/\"" >> /etc/default/tomcat9

echo "LUCEE_SERVER_DIR=\"/opt/lucee/config/server/\"" >> /etc/default/tomcat9
echo "LUCEE_BASE_DIR=\"/opt/lucee/config/server/\"" >> /etc/default/tomcat9
if [ ! -d "/etc/systemd/system/tomcat9.service.d" ] ; then mkdir /etc/systemd/system/tomcat9.service.d/; fi
echo "[Service]" > /etc/systemd/system/tomcat9.service.d/lucee.conf
echo "ReadWritePaths=/opt/lucee/" >> /etc/systemd/system/tomcat9.service.d/lucee.conf
echo "ReadWritePaths=/opt/lucee/config/" >> /etc/systemd/system/tomcat9.service.d/lucee.conf

#add if not in docker check
out "reloading systemctl daemon and sleeping 5 seconds";
systemctl daemon-reload && sleep 5

out "finished tomcat script";

out.fn (just for my convenience/readability)

#!/bin/bash
function out() {
  printf "\e[91;1;107m${0##*/}:\e[0m \e[1;32m ****** $* ****** \e[0m `date`\n";
}

[HatBeardZero]

Glad to contribute, @vitamindck !

[pfreitag]

Good catch @vitamindck @HatBeardZero - looks like the systemd config ReadWritePaths is the problem, makes sense! I'll update the scripts soon.

[JonasEriksson]

THANK YOU @pfreitag , @vitamindck & @HatBeardZero! I copied the above 300-script step and now it runs super-smooth :-) Finally a working modern (Ubuntu 20.04 LTS) Multi-Site Lucee install script :-)

[vitamindck]

@pfreitag the ReadWritePaths do impact the non-docker deployment--and yes, that was our first discovery. The rest of the updates were for the docker deployment since it does not use systemd.

[pfreitag]

Thanks @vitamindck - the only other difference I'm seeing is that you are using systemctl daemon-reload which when I run in docker I'm getting System has not been booted with systemd as init system (PID 1). Can't operate. Failed to connect to bus: Host is down

Just curious if I'm missing something, I did it working with systemd, so that's a win: https://github.com/foundeo/ubuntu-nginx-lucee/actions/runs/937090352 but having issues with docker still.

[vitamindck]

@pfreitag the systemd commands and service commands don't execute (successfully) inside docker. Based on the script, the $IN_DOCKER variable is set truthy to have the container run tomcat's startup script. Are you saying the docker-compose build and docker-compose up commands aren't working for you with the modifications?

[pfreitag]

@vitamindck yes, I just tried it again, still not working with docker-compose build --force-rm and docker-compose up still getting the file not found issue with the lco file.

[vitamindck]

@pfreitag hey, I apologize for losing track of this. The lco file should be downloaded by this update we made in the script I posted above

lco_url="https://cdn.lucee.org/$LUCEE_VERSION.lco"

out "Installing Lucee Core";
if [ ! -f /opt/lucee/config/server/lucee-server/patches/$LUCEE_VERSION.lco ]; then
  mkdir -p /opt/lucee/config/server/lucee-server/patches/
  curl --location -o /opt/lucee/config/server/lucee-server/patches/$LUCEE_VERSION.lco $lco_url
fi

was that included?