Skip to content

Commit

Permalink
#4565 - Improve system-level backup documentation
Browse files Browse the repository at this point in the history
- Improve backup documentation
- Update system requirements description
- Update MariaDB and MySQL version in Docker compose files
- Replace mentions of mysql with mariadb where possible
  • Loading branch information
reckart committed Feb 27, 2024
1 parent 538eb1d commit a73fb44
Show file tree
Hide file tree
Showing 5 changed files with 71 additions and 41 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,6 @@ For production use of {product-name}, it is highly recommended to use a MariaDB
section, we briefly describe how to install a MariaDB server and how to prepare it for use with
the application.

NOTE: MariaDB is an open-source drop-in replacement for MySQL. As a matter of fact, most of the
commands and configuration files still use `mysql` in many places - so do not be confused if
you read a lot about MySQL below despite working with MariaDB.

== Prepare database

* Install MariaDB
Expand All @@ -47,7 +43,7 @@ Changing the character-set and
collation later can lead to serious trouble, so make sure you have a backup of your database.
In that case, you might also need to perform some additional migration steps. We do not
provide a database migration guide here, but if you search e.g. for `mariadb convert utf8 to
utf8mb4` (or `mysql` for that matter), you should find several.
utf8mb4`, you should find several.
=====
[%collapsible]
.Case-sensitive vs insensitive collation (`utf8mb4_bin` vs. `utf8mb4_unicode_ci`)
Expand All @@ -64,8 +60,8 @@ with a `*` in the following table).
+
[source,bash]
----
$ mysql -u root -p
mysql> SHOW VARIABLES WHERE Variable_name LIKE 'character\_set\_%' OR Variable_name IN ('collation_database', 'collation_server');
$ mariadb -u root -p
MariaDB> SHOW VARIABLES WHERE Variable_name LIKE 'character\_set\_%' OR Variable_name IN ('collation_database', 'collation_server');
+--------------------------+-------------+
| Variable_name | Value |
+--------------------------+-------------+
Expand Down Expand Up @@ -101,21 +97,21 @@ collation-server = utf8mb4_bin
+
[source,bash]
----
$ mysql -u root -p
$ mariadb -u root -p
----
* create a database
+
[source,mysql]
[source,mariadb]
----
mysql> CREATE DATABASE inception DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;
MariaDB> CREATE DATABASE inception DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;
----
* create a database user called `inception` with the password `t0t4llYSecreT` which is later used by the application to access the database (instructions for the `settings.properties` file in the <<sect_home_folder, Home Folder>> section).
+
[source,mysql]
[source,mariadb]
----
mysql> CREATE USER 'inception'@'localhost' IDENTIFIED BY 't0t4llYSecreT';
mysql> GRANT ALL PRIVILEGES ON inception.* TO 'inception'@'localhost';
mysql> FLUSH PRIVILEGES;
MariaDB> CREATE USER 'inception'@'localhost' IDENTIFIED BY 't0t4llYSecreT';
MariaDB> GRANT ALL PRIVILEGES ON inception.* TO 'inception'@'localhost';
MariaDB> FLUSH PRIVILEGES;
----

IMPORTANT: For production use, make sure you choose a different, secret, and secure password.
Expand All @@ -127,8 +123,8 @@ You can check these settings using the following command.

[source,bash]
----
$ mysql -u root -p
mysql> * SHOW VARIABLES WHERE Variable_name IN ('innodb_large_prefix', 'innodb_file_format', 'innodb_file_per_table', 'innodb_strict_mode', 'innodb_default_row_format');
$ mariadb -u root -p
MariaDB> * SHOW VARIABLES WHERE Variable_name IN ('innodb_large_prefix', 'innodb_file_format', 'innodb_file_per_table', 'innodb_strict_mode', 'innodb_default_row_format');
----

Depending on the result, you may have to add these settings to your MariaDB configuration files
Expand All @@ -154,7 +150,7 @@ This section explains some settings that can be added to the `database.url` in t
database.url=jdbc:mariadb://localhost:3306/inception?useSSL=false&serverTimezone=UTC&useUnicode=true&characterEncoding=UTF-8
----

To suppress the warning about non-SSL database connections with recent MySQL databases, append the
If use are using a non-SSL database connection append the
following setting to the `database.url`:

----
Expand All @@ -167,24 +163,17 @@ You might also need to add the following if the respective connection error occu
allowPublicKeyRetrieval=true
----

Recent MySQL drivers may refuse to work unless a database server timezone has been specified. The
easiest way to do this is to add the following setting to the `database.url`:
Connections to the database may be rejected by the database server unless a timezone is specified.
The easiest way to do this is to add the following setting to the `database.url`:

----
serverTimezone=UTC
----

If you plan to use 4 Byte UTF-8 encoding for project name and tagset/tag name, make sure both of the following settings for MySQL database are configured:
For proper Unicode support, ensure that the database server and database connection are configured correctly:

* in the `settings.properties` file, make sure that `database.url` includes
+
----
useUnicode=true&characterEncoding=UTF-8
----
+
NOTE: If you use older versions of MariaDB or MySQL (< 8), you may have to set the following property in the `settings.properties` file:
+
----
spring.datasource.hikari.connectionInitSql=SET NAMES utf8mb4 COLLATE utf8mb4_bin;
----
* change the `my.conf` MariaDB database configuration file if necessary to use `utf8mb4` as default character set and `utf8mb4_bin` for collation (see <<character-set-config, Preparing the database>>).
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ networks:

services:
db:
image: "mysql:8"
image: "mysql:8.3"
environment:
- MYSQL_RANDOM_ROOT_PASSWORD=yes
- MYSQL_DATABASE=inception
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ networks:

services:
db:
image: "mariadb:10.7"
image: "mariadb:10.11"
environment:
- MYSQL_RANDOM_ROOT_PASSWORD=yes
- MYSQL_DATABASE=inception
Expand Down Expand Up @@ -37,7 +37,6 @@ services:
- INCEPTION_DB_URL=jdbc:mariadb://db:3306/inception?useSSL=false&useUnicode=true&characterEncoding=UTF-8
- INCEPTION_DB_USERNAME=${DBUSER:-inception}
- INCEPTION_DB_PASSWORD=${DBPASSWORD:-inception}
- JAVA_OPTS=-Dspring.jpa.properties.hibernate.dialect.storage_engine=innodb
volumes:
- ${INCEPTION_HOME:-app-data}:/export
depends_on:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,48 @@
// See the License for the specific language governing permissions and
// limitations under the License.

= Backup your data
= System-level backup

* Make a copy of your {product-name} application home folder
* If you are using an external database like MariaDB, make a backup of your {product-name} database,
e.g. using the link:https://mariadb.com/kb/en/making-backups-with-mysqldump/[mysqldump] command.
The system-level backup procedure is directed mainly towards server administrators. Project managers can create a project-level backup from the project settings user interface.

{product-name} stores some data on the file system in its <<sect_home_folder,application home folder>> and other data in a database.

It is recommended to make backups while {product-name} is not running to ensure that the data in the backup is in a consistent state.

NOTE: Make sure that you can actually re-create a working installation by restoring your backup.
I.e. restore your database export into a fresh database and point a new {product-name} installation
at using your home folder copy and the restored database and then start the application and check
E.g. restore your database export into a fresh database and point a new {product-name} installation
at using your application home folder copy and the restored database and then start the application and check
that everything is there and working. A backup which cannot be successfully restored is worthless.


== File system backup

To create a backup your {product-name} <<sect_home_folder,application home folder>> you can just copy the folder to another location or create a ZIP archive from it.

This backup will contain amongst other things the actual annotated data.

To keep downtime low while you are preparing the backup, consider using a copy tool that supports incremental updates such as link:https://en.wikipedia.org/wiki/Rsync[rsync].

== Database backups

NOTE: If you are not using an external database, {product-name} is using an embedded database that stores its data in the <<sect_home_folder,application home folder>>. In this case, maintaining a backup of that folder is sufficient and you can skip this section. However, if you are using an external database like MariaDB, it is essential to also maintain backups of that database!

If you are using an external database like MariaDB, make a backup of your {product-name} database,
e.g. using the link:https://mariadb.com/kb/en/backup-and-restore-overview/[mariadb-dump] command.

This backup will contain infomation about projects, documents, users, and all other structured metadata maintained by {product-name}. It will **not** contain the actual annotated data which is stored on the file system!

Assuming you set up the database according to the instructions in this manual, you can use `mariadb-dump` to create a backup like this:

.Creating a database backup
[source,bash]
----
$ mariadb-dump inception > inception-backup.sql
----

Restoring the backup is similary. Please make sure that the database into which you restore the backup has been set up according to the instructions in this manual. This is particularly important if you restore the database to a new system.

.Restoring a database backup
[source,bash]
----
$ mariadb inception < inception-backup.sql
----
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,12 @@
[cols="2*"]
|===
| Browser
| Chrome or Safari
| Chrome or Safari (latest versions)
|===

[.small]
You should also be able to use {product-name} with other browsers such as Firefox, Brave, etc. However, those are less regularly tested by the developers. It is recommended to always use the latest version of any browser product you may be using to ensure best compatibility.

.Requirements to run the standalone version
[cols="2*"]
|===
Expand All @@ -33,6 +36,9 @@
| version 17 or higher
|===

[.small]
The examples in this guide are based on a recent Debian Linux. Most of them should apply quite directly to Debian-based distributions like e.g. Ubuntu. {product-name} will run on other distributions as well, but you may have to use different commands for managing users, installing software, etc.

.Requirements run the server version
[cols="2*"]
|===
Expand All @@ -43,14 +49,16 @@
| version 17 or higher

| MariaDB Server (or compatible)
| version 10.5 or higher
| MariaDB version 10.6 or higher +
MySQL version 8.0 or higher
|===

[.small]
You may be able to run {product-name} on older database server versions but it may require extra configuration that is not included in this documentation. You may consider referring to older versions of this administrators guide included with older versions of {product-name}.

.Requirements for a Docker-based deployment
[cols="2*"]
|===
| Docker
| version 24 or higher (arm64 or amd64)
|===

NOTE: link:https://mariadb.org[MariaDB] is an open source drop-in replacement for link:https://www.mysql.com[MySQL]. So generally, you can also use MySQL instead of MariaDB. Also, as far as we know, we do not use any fancy features of MariaDB 10.5 and did in fact until recently always use MySQL 5. So you can probably also use an older version of MariaDB or MySQL... but why should you? Think of all the unfixed bugs and consider also upgrading your database if you do a fresh installation of {product-name} - or use <<admin_guide.adoc#sect_docker,Docker>>.

0 comments on commit a73fb44

Please sign in to comment.