Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker refresh #1001

Merged
merged 31 commits into from
May 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
5f2eac3
Docker files in local directory
jonocodes Apr 28, 2024
8719555
Add some TODO notes
jonocodes Apr 28, 2024
3330548
Working on db setup script
jonocodes May 3, 2024
6efb14d
User creating with sqlite working
jonocodes May 4, 2024
b6a2a84
Better error handling when creating user
jonocodes May 5, 2024
62c3a86
Healthcheck. Makefile. setup_system script
jonocodes May 6, 2024
572290a
entrypoint cleanup
jonocodes May 7, 2024
58fb25c
adding push and test to makefile
jonocodes May 7, 2024
4bbf969
Move run time data into local data dir.
jonocodes May 7, 2024
562cc53
fpm logging is now working
jonocodes May 7, 2024
8c39c10
fixed setup.php
jonocodes May 10, 2024
9fa3313
prevent existing account from causing crash
jonocodes May 10, 2024
613df9d
add setup to makefile
jonocodes May 10, 2024
ae86e2f
updating inline notes
jonocodes May 10, 2024
c3eb97e
fixed need for composer update
jonocodes May 10, 2024
fcff989
relocate functions import
jonocodes May 10, 2024
d4ae036
remove comment about debug logging
jonocodes May 10, 2024
19dab2d
update setup db
jonocodes May 10, 2024
8546f17
got sqlite working in container
jonocodes May 11, 2024
10aa2ea
installing suggested extensions
jonocodes May 12, 2024
a04a5e6
Setup image tagging
jonocodes May 15, 2024
0e6b6b4
clean up
jonocodes May 15, 2024
7c5afd7
create docker compose example
jonocodes May 15, 2024
0b95a55
create readme for dockerhub
jonocodes May 16, 2024
074a7c2
add tag note to readme
jonocodes May 16, 2024
36a18de
Fix test. Add readme upload
jonocodes May 18, 2024
c750373
env var notes
jonocodes May 19, 2024
53f855d
Fix attachment issue
jonocodes May 20, 2024
4f4ae38
Add env var descriptions to readme
jonocodes May 24, 2024
599132b
Splitting out command for pushing the readme
jonocodes May 24, 2024
2869597
fix broken TODO in readme
jonocodes May 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/data/
.git
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
/config/dynamic.php
/config/app.php
/site.js
/site.css
/site/
Expand All @@ -9,6 +8,8 @@
/tests/selenium/remote_creds.py
*.pyc
/.env
/.env.*
!.env.example
apigen4.sh
testuser.txt
website/docs
Expand All @@ -25,3 +26,4 @@ scripts/test.php
composer.phar
lib/hm3/users/
.env
/data/
35 changes: 35 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
.PHONY: docker-up
docker-up: ## start docker stack in foreground for development
docker compose -f docker-compose.dev.yaml up --build || true # --abort-on-container-exit

.PHONY: docker-push
.ONESHELL:
docker-push: ## build, tag, and push image to dockerhub. presumes you are logged in. run with a version like tag:1.2.3
@username=$$(docker info | sed '/Username:/!d;s/.* //')
@[ "$(tag)" = "" ] && (echo "Tag required. Example tag=1.2.3" ; exit 1)
@image=$${username}/cypht:$(tag)
@echo "Building image $${image}"
@docker buildx build . --platform linux/amd64 \
-t $${image} -f docker/Dockerfile --push
# TODO: build for arm architectures

.PHONY: dockerhub-push-readme
.ONESHELL:
dockerhub-push-readme: ## upload readme to dockerhub
@username=$$(docker info | sed '/Username:/!d;s/.* //')
@docker pushrm --file docker/DOCKERHUB-README.md $${username}/cypht
@echo docker pushrm --file docker/DOCKERHUB-README.md $${username}/cypht

.PHONY: setup
.ONESHELL:
setup: ## locally setup app and users. presumes env vars are set
set -e
echo "Installing dependencies"
composer install
echo "Creating tables and user"
./scripts/setup_database.php
echo "Creating directories and configs"
./scripts/setup_system.sh

help: ## get help
@grep -E '^[a-zA-Z_-]+:.*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
1 change: 0 additions & 1 deletion composer.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

44 changes: 44 additions & 0 deletions docker-compose.dev.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@

# this file should be used for development, not production

services:
db:
image: mariadb:10
ports:
- "3306:3306"
volumes:
- ./data/mysql:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=root_password
- MYSQL_DATABASE=cypht
- MYSQL_USER=cypht
- MYSQL_PASSWORD=cypht_password
cypht:
build:
context: .
dockerfile: ./docker/Dockerfile
args:
WITH_DEBUG: true
volumes:
- ./data/users:/var/lib/hm3/users
- ./data/attachments:/var/lib/hm3/attachments
- ./data/app_data:/var/lib/hm3/app_data
- ./data/sqlite:/var/lib/hm3/sqlite
# The following allow for some live code updates during development
- ./lib:/usr/local/share/cypht/lib
- ./modules:/usr/local/share/cypht/modules
ports:
- "80:80"
environment:
- AUTH_USERNAME=admin
- AUTH_PASSWORD=admin
- DB_CONNECTION_TYPE=host
- DB_DRIVER=mysql
- DB_HOST=db
- DB_NAME=cypht
- DB_USER=cypht
- DB_PASS=cypht_password
- SESSION_TYPE=DB
- USER_CONFIG_TYPE=DB
extra_hosts:
host.docker.internal: host-gateway # for xdebug
48 changes: 48 additions & 0 deletions docker/DOCKERHUB-README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Cypht

This is the official docker image of [Cypht](https://cypht.org/).

## Features of this image

* Alpine linux based image
* Bundled nginx and PHP 7 provides everything in one image
* Performs same install steps as found on [Cypht install page](https://cypht.org/install.html)
* All Cypht mods and configuration options can be set via environment variables
* Automatic database setup (if configured to use database)

It recommended that you choose a specific version number tag instead of using 'latest' since 'latest' may represent master which may not be stable.

## Example docker-compose

See example file here:
https://github.com/jonocodes/cypht/blob/docker-refresh/docker/docker-compose.yaml

* Starts a database container to be for user authentication
* Starts the Cypht container available on port 80 of the host with ...
* A local volume declared for persisting user settings across container reboots
* An initial user account for authentication
* Environment variables for accessing the database container

*NOTE: Please change usernames and passwords before using this docker-compose in your environment*

## Environment variables

See all the environment variables you can set here:
https://github.com/cypht-org/cypht/blob/master/.env.example

To see the meaning of what each variable see descriptions here:
https://github.com/cypht-org/cypht/blob/master/config/app.php


It is recommended that in production you instead make a copy of this file:
```
cp .env.example /etc/cypht-prod.env
```

Make changes to it and source it in to the docker-compose via 'env_file':
```yaml
env_file:
- /etc/cypht-prod.env
```

In order to avoid confusion, it is best to use only the env file and not set addition env vars in the docker compose file if possilbe.
62 changes: 62 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
FROM php:7.4.33-fpm-alpine

WORKDIR "/usr/local/share/cypht"

RUN set -e \
&& apk add --no-cache \
supervisor nginx composer sqlite freetype libpng libjpeg-turbo \
php-session php-fileinfo php-dom php-xml libxml2-dev php-xmlwriter php-tokenizer \
&& apk add --no-cache --virtual .build-deps \
ca-certificates \
libpng-dev libjpeg-turbo-dev freetype-dev \
&& docker-php-ext-configure gd --with-freetype=/usr/include/ --with-jpeg=/usr/include/ \
&& docker-php-ext-install gd pdo pdo_mysql \
&& curl -sSL https://github.com/mlocati/docker-php-extension-installer/releases/latest/download/install-php-extensions -o - | sh -s \
xdebug redis gnupg memcached \
&& composer self-update --2 \
&& apk del .build-deps \
&& ln -sf /dev/stdout /var/log/nginx/access.log \
&& ln -sf /dev/stderr /var/log/nginx/error.log \
&& ln -s /usr/local/etc/php/php.ini-production /usr/local/etc/php/php.ini

COPY <<EOF /tmp/xdebug.ini
[xdebug]
zend_extension=xdebug.so

xdebug.mode=debug
xdebug.client_host=host.docker.internal
EOF

COPY <<EOF /usr/local/etc/php/conf.d/cypht.ini
post_max_size = 60M
upload_max_filesize = 50M
# the following is needed for sqlite access
open_basedir = /var/lib/hm3/:/usr/local/share/cypht/:/tmp
EOF

COPY docker/nginx.conf /etc/nginx/nginx.conf
COPY docker/supervisord.conf /etc/supervisord.conf
COPY composer.* .

ARG WITH_DEBUG=false

RUN [ "$WITH_DEBUG" = "true" ] && mv /tmp/xdebug.ini /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini || true

RUN composer install

COPY config/ config/
COPY language/ language/
COPY lib/ lib/
COPY modules/ modules/
COPY scripts/ scripts/
COPY third_party/ third_party/
COPY index.php index.php

COPY docker/docker-entrypoint.sh docker/docker-entrypoint.sh
COPY .env.example .env

EXPOSE 80

HEALTHCHECK CMD curl --fail http://localhost || exit 1

ENTRYPOINT ["docker/docker-entrypoint.sh"]
32 changes: 32 additions & 0 deletions docker/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@

# this is a demo of using the production cypht image

services:
db:
image: mariadb:10
ports:
- "3306:3306"
volumes:
- ./data/mysql:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=root_password
- MYSQL_DATABASE=cypht
- MYSQL_USER=cypht
- MYSQL_PASSWORD=cypht_password
cypht:
image: jonocodes/cypht:2.0.1-docker-wip
ports:
- "80:80"
# env_file:
# - /etc/cypht-prod.env
environment:
- AUTH_USERNAME=admin
- AUTH_PASSWORD=admin
- DB_CONNECTION_TYPE=host
- DB_DRIVER=mysql
- DB_HOST=db
- DB_NAME=cypht
- DB_USER=cypht
- DB_PASS=cypht_password
- SESSION_TYPE=DB
- USER_CONFIG_TYPE=DB
37 changes: 37 additions & 0 deletions docker/docker-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#!/usr/bin/env sh

set -e

APP_DIR=/usr/local/share/cypht
cd ${APP_DIR}

# TODO: validate env var values here, perhaps in php or in Hm_Site_Config_File()

# TODO: source these defaults from an .env file or some other place?
USER_CONFIG_TYPE="${USER_CONFIG_TYPE:-file}"
USER_SETTINGS_DIR="${USER_SETTINGS_DIR:-/var/lib/hm3/users}"
ATTACHMENT_DIR="${ATTACHMENT_DIR:-/var/lib/hm3/attachments}"
APP_DATA_DIR="${APP_DATA_DIR:-/var/lib/hm3/app_data}"

# Wait for database to be ready then setup tables
./scripts/setup_database.php

# Setup filesystem and users
./scripts/setup_system.sh
kroky marked this conversation as resolved.
Show resolved Hide resolved

# Enable the program in the web-server

if [ "${USER_CONFIG_TYPE}" = "file" ]
then
chown www-data:www-data ${USER_SETTINGS_DIR}
fi

chown www-data:www-data ${ATTACHMENT_DIR}
chown -R www-data:www-data /var/lib/nginx
chown www-data:www-data ${APP_DATA_DIR}

rm -r /var/www
ln -s $(pwd)/site /var/www

# Start services
/usr/bin/supervisord -c /etc/supervisord.conf
40 changes: 40 additions & 0 deletions docker/nginx.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
worker_connections 768;
}
http {
sendfile off;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
gzip on;
gzip_disable "msie6";
server {
listen 80;
server_name localhost;
index index.php;
root /var/www;
client_max_body_size 60M;
location / {
try_files $uri /index.php$is_args$args;
}
location ~ \.php {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_index index.php;
fastcgi_pass 127.0.0.1:9000;
}
}
}
22 changes: 22 additions & 0 deletions docker/supervisord.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
[supervisord]
nodaemon=true
logfile=/var/log/supervisord.log
pidfile=/var/run/supervisord.pid

[program:nginx]
command=/usr/sbin/nginx -g "daemon off;"
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

[program:php-fpm]
command=php-fpm
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
1 change: 1 addition & 0 deletions lib/auth.php
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ public function create($user, $pass) {
$result = 0;
$res = Hm_DB::execute($this->dbh, 'select username from hm_user where username = ?', [$user]);
if (!empty($res)) {
print("user {$user} already exists\n");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm_Debug::add?
You should also check DEBUG_MODE constant - it is true in index.php but config_gen copies that to site/index.php (which is used as the main entry point of the web app) and turns it into false. Keep it to true in docker env if you want deubgging mode turned on or make it come from dotenv.

Copy link
Contributor Author

@jonocodes jonocodes May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont see where DEBUG_MODE is read in the environment. I only see it hard coded at the beginning of scripts. So I cant pass it in. I'm confused what DEBUG_MODE is for exactly, but I dont think it does what I'm going for. It looks like a way of enabling features and it does some log queuing.

I was looking for a way to log at different severity levels. I'm not sure of the php way to do that.

I am just trying to print a message to the console at the time it happens. So I think print, or error_log seem ok for here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, DEBUG_MODE is only hard-coded in scripts for now. It will be a good idea to extend that to come from the environment.
It is best to still use the debug class to queue these messages. Hm_Msgs is another one meant to be displayed to the user. Console scripts can dump the debug and/or msgs arrays at the end to the cli using print statements. error_log will send the message to the php error log which seldom appears on the screen for users (depends on several PHP settings to appear).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am genuinely confused with the logging setup and it has bitten me several times. There is a lot of good info being sent go Hm_Debug::add, but its hard to get the data out from there. I lost several hours trying to fix a sqlite bug that would have been minutes if those logs were simply displayed.

Its possible PHP logging is different because one of the output streams is meant to screen/browser output. I hear Laraval is a commonly used PHP framework, so I looked up their logging setup: https://laravel.com/docs/11.x/logging

which seems quite standard. It uses monolog which provides simple, immediate, customizable outputs.

Let me know if I am off base, or I can create a separate ticket for this and elaborate more.

Copy link
Member

@kroky kroky May 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm_Debug uses Hm_list which has get method to get the messages and show method to send them to the error log. PHP error logging might be complex and not everything logged, indeed, it has settings for error_reporting - what kind of errors to report, then displaying errors on screen or writing errors to a log file - thus third option also depends on your backend server setup. So, a lot of moving parts.

Integrating something like monolog is an option but note that cypht started with minimalism in mind while Laravel is enormous, so we still try to depend on fewer external libs. That being said, I think an ENV setting for DEBUG_MODE which is observed and makes sure debugging messages are written to the php error log at the end of a request + screen/cli if necessary seems like a better approach to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm moving this discussion to a separate issue #1027

$result = 1;
}
else {
Expand Down
2 changes: 2 additions & 0 deletions lib/framework.php
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
require APP_PATH.'lib/api.php';
require APP_PATH.'lib/webdav_formats.php';

require_once APP_PATH.'modules/core/functions.php';

/* load random bytes polyfill if needed */
if (!function_exists('random_bytes')) {
require VENDOR_PATH.'paragonie/random_compat/lib/random.php';
Expand Down
13 changes: 4 additions & 9 deletions scripts/create_account.php
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,12 @@

/* check config for db auth */
if ($config->get('auth_type') != 'DB') {
die("Incorrect usage\n\nThis script only works if DB auth is enabled in your site configuration\n\n");
print("Incorrect usage\n\nThis script only works if DB auth is enabled in your site configuration\n\n");
exit(1);
}

$auth = new Hm_Auth_DB($config);

if ($user && $pass) {
if ($auth->create($user, $pass) === 2) {
die("User '" . $user . "' created\n\n");
}
else {
print_r(Hm_Debug::get());
print_r(Hm_Msgs::get());
die("An error occured when creating user '" . $user . "'\n\n");
}
$auth->create($user, $pass);
}
Loading