Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump v0.13.0 #159

Merged
merged 55 commits into from
Apr 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
53884dc
Merge pull request #144 from rtCamp/main
Xieyt Mar 28, 2024
6f067cb
fix: bake images
Xieyt Mar 28, 2024
d706a9e
Merge pull request #146 from rtCamp/cicd/fix-bake-images
Xieyt Mar 28, 2024
04a8646
fix: remove containers when creating services
Xieyt Mar 28, 2024
ba0616d
feat: ssl support
Xieyt Apr 8, 2024
48af111
Add ComposeProject class to manage Docker Compose services
Xieyt Apr 17, 2024
5768880
Update SSL types and exception names and update code to use ComposePr…
Xieyt Apr 17, 2024
0f0f78f
Refactor code and add new utils functions and add SubprocessOutput class
Xieyt Apr 17, 2024
340a70d
Add functionality for handling volumes and returning output in Python…
Xieyt Apr 17, 2024
34e1bf8
Add SSL certificate management classes and services
Xieyt Apr 17, 2024
45132bb
Add BenchConfig class for managing bench configuration.
Xieyt Apr 17, 2024
a3abc06
Add DatabaseManager class to handle db related tasks.
Xieyt Apr 17, 2024
80873de
Refactor and integrate SubprocessOutput
Xieyt Apr 17, 2024
f0578c0
Refactor code to utilize the newly created class derived from Compose…
Xieyt Apr 17, 2024
b370419
Add ssl command
Xieyt Apr 17, 2024
a69a14c
Refactor exception handling in main.py
Xieyt Apr 17, 2024
e122547
Update templates and Nginx configuration
Xieyt Apr 17, 2024
7d05dc5
fix: live logs prefix not showing
Xieyt Apr 17, 2024
a668422
Add migration for 0.13.0
Xieyt Apr 17, 2024
2e25089
bump v0.13.0
Xieyt Apr 17, 2024
9153cc0
Refactor: Decouple Adminer and MailHog from main stack; add separate …
Xieyt Apr 21, 2024
8feffba
remove redundant attributes saved to bench config.
Xieyt Apr 21, 2024
7efbf4c
refactor and import exception handling
Xieyt Apr 21, 2024
c9763c2
Refactor and update migrations.
Xieyt Apr 21, 2024
76eb1b4
feat: fm update command
Xieyt Apr 21, 2024
7416a23
fix nginx and frappe not stopping gracefully
Xieyt Apr 21, 2024
3f0090e
show default option for ssl in create command
Xieyt Apr 21, 2024
ce60a7e
fix: fm code user change not reflecting
Xieyt Apr 21, 2024
01cc3e4
cleanup minor cli commands options
Xieyt Apr 21, 2024
eb5c8b7
exclude non required config from export
Xieyt Apr 21, 2024
79f4a5e
update ssl command status message.
Xieyt Apr 21, 2024
feeb6cd
first remove nginx admin tools config
Xieyt Apr 21, 2024
ce747d3
Merge pull request #147 from rtCamp/fix/services-creation
Xieyt Apr 21, 2024
826ef07
fix: proper exit version in migratoin rollback
Xieyt Apr 22, 2024
75f5196
chore: improve status messages in migration
Xieyt Apr 22, 2024
d9f327c
fix: save version after first install
Xieyt Apr 22, 2024
fe70a61
fix: Clean install after services
Xieyt Apr 22, 2024
e7f92d6
fix: circular import error
Xieyt Apr 22, 2024
380aeeb
fix clean install of services
Xieyt Apr 22, 2024
51a3cc2
Add ssl dir in servies osx template
Xieyt Apr 22, 2024
6ea8956
fix: linking of issued certificates
Xieyt Apr 22, 2024
4dea978
fix: auto supervisord process kill
Xieyt Apr 23, 2024
c6c092e
feat: sync configuration changes to bench
Xieyt Apr 23, 2024
cc7e6aa
feat: --environment flag to update command
Xieyt Apr 23, 2024
45652ca
remove staging mode from letsencrypt service
Xieyt Apr 23, 2024
6833696
remove version from docker compose templates
Xieyt Apr 24, 2024
7958f45
fix: --all flag in ssl renew command
Xieyt Apr 25, 2024
64b67c9
Add `--developer-mode` flag to update command
Xieyt Apr 25, 2024
07b47fc
fix: help not showing
Xieyt Apr 25, 2024
f93e1a1
fix: remove certificate expiry date check
Xieyt Apr 25, 2024
f308d80
fix: restart global-nginx-proxy after certficate changes
Xieyt Apr 25, 2024
9eb236a
add config templates
Xieyt Apr 25, 2024
ac5480e
use cert utc when available
Xieyt Apr 25, 2024
c808690
fix(migration): v0.13.0 admin tools
Xieyt Apr 25, 2024
0446836
Merge pull request #155 from rtCamp/various_fixes
Xieyt Apr 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/bake-images.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ jobs:
- name: Checkout
uses: actions/checkout@v3
with:
ref: main
fetch-depth: 0

- name: Set up tag
Expand Down
5 changes: 4 additions & 1 deletion Docker/frappe/bench-dev-server
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
#!/bin/bash
trap "kill -- -$$" EXIT
cleanup (){
kill -s SIGTERM -- -$$
}
trap cleanup SIGQUIT SIGTERM
fuser -k 80/tcp
bench serve --port 80
5 changes: 4 additions & 1 deletion Docker/frappe/bench-dev-watch.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
#!/bin/bash
trap "kill -- -$$" EXIT
cleanup (){
kill -s SIGTERM -- -$$
}
trap cleanup SIGQUIT SIGTERM
bench watch
19 changes: 17 additions & 2 deletions Docker/frappe/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,16 @@ emer() {
echo "$1"
exit 1
}
cleanup() {
echo "Received signal, stopping..."
# Insert cleanup code here (e.g., stop services, clean temp files)
if [ -n "$running_script_pid" ]; then
kill -s SIGTERM "$running_script_pid"
fi
exit 0
}

trap cleanup SIGQUIT SIGTERM

[[ "${USERID:-}" ]] || emer "[ERROR] Please provide USERID environment variable."
[[ "${USERGROUP:-}" ]] || emer "[ERROR] Please provide USERGROUP environment variable."
Expand All @@ -31,10 +41,15 @@ if [[ ! -f "/workspace/.profile" ]]; then
fi

chown "$USERID":"$USERGROUP" /workspace /workspace/frappe-bench

ls -pA /workspace | xargs -I{} chown -R "$USERID":"$USERGROUP" /workspace/{} &

if [ "$#" -gt 0 ]; then
gosu "$USERID":"$USERGROUP" "/scripts/$@"
gosu "$USERID":"$USERGROUP" "/scripts/$@" &
running_script_pid=$!
else
gosu "${USERID}":"${USERGROUP}" /scripts/user-script.sh
gosu "${USERID}":"${USERGROUP}" /scripts/user-script.sh &
running_script_pid=$!
fi

wait $running_script_pid
2 changes: 2 additions & 0 deletions Docker/frappe/frappe-dev.conf
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ stdout_logfile=/workspace/frappe-bench/logs/web.dev.log
redirect_stderr=true
user=frappe
directory=/workspace/frappe-bench
stopasgroup=true
stopsignal=QUIT

[program:frappe-bench-frappe-watch]
command=/opt/user/bench-dev-watch.sh
Expand Down
57 changes: 40 additions & 17 deletions Docker/frappe/user-script.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,20 @@ source /prebake_info

set -e

cleanup() {
echo "Received signal, performing cleanup..."
# Add any necessary cleanup commands here

# Forward the signal to supervisord (if it's running)
if [ -n "$supervisord_pid" ]; then
kill -s SIGTERM "$supervisord_pid"
fi
exit 0
}

# Trap SIGQUIT, SIGTERM, SIGINT
trap cleanup SIGQUIT SIGTERM

emer() {
echo "$@"
exit 1
Expand All @@ -25,6 +39,10 @@ if [[ ! "${MARIADB_HOST:-}" ]]; then
MARIADB_HOST='global-db'
fi

if [[ "${DEBUG:-}" ]]; then
set -x
fi

if [[ ! "${MARIADB_ROOT_PASS:-}" ]]; then
MARIADB_ROOT_PASS='root'
fi
Expand All @@ -43,13 +61,10 @@ configure_common_site_config() {
update_common_site_config redis_cache "redis://${CONTAINER_NAME_PREFIX}-redis-cache:6379"
update_common_site_config redis_queue "redis://${CONTAINER_NAME_PREFIX}-redis-queue:6379"
update_common_site_config redis_socketio "redis://${CONTAINER_NAME_PREFIX}-redis-socketio:6379"
update_common_site_config mail_port 1025
update_common_site_config mail_server 'mailhog'
update_common_site_config disable_mail_smtp_authentication 1
update_common_site_config webserver_port "$WEB_PORT"
update_common_site_config developer_mode "$DEVELOPER_MODE"
update_common_site_config socketio_port "$REDIS_SOCKETIO_PORT"
update_common_site_config restart_supervisor_on_update 0
update_common_site_config developer_mode "$DEVELOPER_MODE"

# end_time=$(date +%s.%N)
# execution_time=$(awk "BEGIN {print $end_time - $start_time}")
Expand Down Expand Up @@ -90,7 +105,7 @@ if [[ ! -d "/workspace/frappe-bench/sites/$SITENAME" ]]; then
/scripts/divide-supervisor-conf.py config/supervisor.conf

echo "Environment: ${ENVIRONMENT}"
echo "Configuring frappe dev server"
echo "Configuring frappe server"
bench_serve_help_output=$($BENCH_COMMAND serve --help)
host_changed=$(echo "$bench_serve_help_output" | grep -c 'host' || true)

Expand All @@ -101,6 +116,10 @@ if [[ ! -d "/workspace/frappe-bench/sites/$SITENAME" ]]; then
awk -v a="$WEB_PORT" '{sub(/--port [[:digit:]]+/,"--port "a); print}' /opt/user/bench-dev-server >file.tmp && mv file.tmp /opt/user/bench-dev-server.sh
fi

if [[ "$DEVELOPER_MODE" == "true" ]]; then
bench setup requirements --dev
fi

chmod +x /opt/user/bench-dev-server.sh

$BENCH_COMMAND build &
Expand All @@ -119,7 +138,9 @@ if [[ ! -d "/workspace/frappe-bench/sites/$SITENAME" ]]; then
if [[ -n "$BENCH_START_OFF" ]]; then
tail -f /dev/null
else
supervisord -c /opt/user/supervisord.conf
supervisord -c /opt/user/supervisord.conf &
supervisord_pid=$!
wait $supervisord_pid
fi

else
Expand All @@ -140,6 +161,7 @@ else
$BENCH_COMMAND setup supervisor --skip-redis --skip-supervisord --yes --user "$USER"
/scripts/divide-supervisor-conf.py config/supervisor.conf

# Addresses the introduction of the --host flag in bench serve command for compatibility with Frappe version updates.
if [[ "$host_changed" -ge 1 ]]; then
awk -v a="$WEB_PORT" '{sub(/--port [[:digit:]]+/,"--host 0.0.0.0 --port "a); print}' /opt/user/bench-dev-server >file.tmp && mv file.tmp /opt/user/bench-dev-server.sh
else
Expand All @@ -148,23 +170,24 @@ else

chmod +x /opt/user/bench-dev-server.sh

# Addresses the introduction of the --host flag in bench serve command for compatibility with Frappe version updates.
if [[ "${ENVIRONMENT}" = "dev" ]]; then
cp /opt/user/frappe-dev.conf /opt/user/conf.d/frappe-dev.conf
else
if [[ -f '/opt/user/conf.d/frappe-bench-frappe-web.fm.supervisor.conf' ]]; then
# if [[ "${ENVIRONMENT}" = "dev" ]]; then
# cp /opt/user/frappe-dev.conf /opt/user/conf.d/frappe-dev.conf
# else
# if [[ -f '/workspace/frappe-bench/config/frappe-bench-frappe-web.fm.supervisor.conf' ]]; then

ln -sfn /workspace/frappe-bench/config/frappe-bench-frappe-web.fm.supervisor.conf /opt/user/conf.d/frappe-bench-frappe-web.fm.supervisor.conf
else
emer 'Not able to start the server. /opt/user/conf.d/frappe-bench-frappe-web.fm.supervisor.conf not found.'
fi
fi
# ln -sfn /workspace/frappe-bench/config/frappe-bench-frappe-web.fm.supervisor.conf /opt/user/conf.d/frappe-bench-frappe-web.fm.supervisor.conf
# else
# emer 'Not able to start the server. /workspace/frappe-bench/config/frappe-bench-frappe-web.fm.supervisor.conf not found.'
# fi
# fi

if [[ -n "$BENCH_START_OFF" ]]; then
tail -f /dev/null
else
echo "Starting supervisor.."
supervisord -c /opt/user/supervisord.conf
supervisord -c /opt/user/supervisord.conf &
supervisord_pid=$!
wait $supervisord_pid
fi

fi
4 changes: 2 additions & 2 deletions Docker/images-tag.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"frappe": "v0.12.0",
"nginx": "v0.10.0",
"frappe": "v0.13.0",
"nginx": "v0.13.0",
"mailhog": "v0.8.3"
}
1 change: 1 addition & 0 deletions Docker/nginx/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,6 @@ COPY --from=builder /dist/ /config/
COPY template.conf /config/
COPY ./entrypoint.sh /
RUN chmod +x /entrypoint.sh
RUN mkdir -p /etc/nginx/custom

ENTRYPOINT ["/bin/bash","/entrypoint.sh"]
16 changes: 11 additions & 5 deletions Docker/nginx/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
#!/usr/bin/bash
cleanup() {
echo "Received signal, performing cleanup..."
if [ -n "$nginx_pid" ]; then
kill -s SIGTERM "$nginx_pid"
fi
exit 0
}

# wait for all the programs to load first
echo "Waiting for mailhog adminer to start"
wait-for-it -t 120 mailhog:8025
wait-for-it -t 120 adminer:8080
trap cleanup SIGTERM

/config/jinja2 -D SITENAME="$SITENAME" /config/template.conf > /etc/nginx/conf.d/default.conf

nginx -g 'daemon off;'
nginx -g 'daemon off;' &
nginx_pid=$!
wait $nginx_pid
36 changes: 11 additions & 25 deletions Docker/nginx/template.conf
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,6 @@ upstream frappe-bench-frappe {
upstream frappe-bench-socketio-server {
server socketio:80 fail_timeout=120;
}
upstream mailhog {
server mailhog:8025 fail_timeout=120;
}
upstream adminer {
server adminer:8080 fail_timeout=120;
}
server {

listen 80;
Expand Down Expand Up @@ -80,25 +74,14 @@ server {
proxy_pass http://frappe-bench-frappe;
}

# mailhog
location ^~ /mailhog/ {

chunked_transfer_encoding on;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://mailhog/;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_http_version 1.1;
proxy_redirect off;
proxy_buffering off;
}

# adminer
location ^~ /adminer/ {
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
proxy_pass http://adminer/;
# certbot webroot
location ^~ /.well-known/acme-challenge/ {
auth_basic off;
auth_request off;
allow all;
root /usr/share/nginx/html;
try_files $uri =404;
break;
}

# error pages
Expand Down Expand Up @@ -145,4 +128,7 @@ server {
text/x-component
;
# text/html is always compressed by HttpGzipModule

# include custom config
include /etc/nginx/custom/*.conf;
}
23 changes: 17 additions & 6 deletions frappe_manager/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,17 @@
# TODO configure this using config
# sites_dir = Path().home() / __name__.split(".")[0]
CLI_DIR = Path.home() / "frappe"
CLI_METADATA_PATH = CLI_DIR / ".fm.toml"
CLI_FM_CONFIG_PATH = CLI_DIR / "fm_config.toml"
CLI_SITES_ARCHIVE = CLI_DIR / "archived"
CLI_LOG_DIRECTORY = CLI_DIR / 'logs'
CLI_SITES_DIRECTORY = CLI_DIR / 'sites'
CLI_BENCHES_DIRECTORY = CLI_DIR / 'sites'
CLI_SERVICES_DIRECTORY = CLI_DIR / 'services'

CLI_SERVICES_NGINX_PROXY_DIR = CLI_SERVICES_DIRECTORY / 'nginx-proxy'
CLI_SERVICES_NGINX_PROXY_SSL_DIR = CLI_SERVICES_NGINX_PROXY_DIR / 'ssl'

CLI_BENCH_CONFIG_FILE_NAME = 'bench_config.toml'
SSL_RENEW_BEFORE_DAYS = 30


DEFAULT_EXTENSIONS = [
Expand All @@ -25,8 +32,6 @@
class SiteServicesEnum(str, Enum):
frappe = "frappe"
nginx = "nginx"
mailhog = "mailhog"
adminer = "adminer"
mariadb = "mariadb"
redis_queue = "redis-queue"
redis_cache = "redis-cache"
Expand All @@ -36,6 +41,12 @@ class SiteServicesEnum(str, Enum):


STABLE_APP_BRANCH_MAPPING_LIST = {
"erpnext" :'version-15',
"hrms" :'version-15',
"frappe": 'version-15',
"erpnext": 'version-15',
"hrms": 'version-15',
}


class EnableDisableOptionsEnum(str, Enum):
enable = 'enable'
disable = 'disable'
Loading