Skip to content

Commit

Permalink
[APPINT-1146] Fix database encoding configuration
Browse files Browse the repository at this point in the history
Sometimes the database is created with an character set different that
the one specified in `config/database.yml`.
This causes some issue when the database is created in non utf8 charset
(eg: latin1).

This commit adds an initializer that warn the user in case of mismatch
between the configured and actual encoding.
It also provides a migration generator to facilitate charset migration.
  • Loading branch information
ouranos committed Nov 2, 2018
1 parent 424a4a2 commit eb81358
Show file tree
Hide file tree
Showing 3 changed files with 75 additions and 0 deletions.
22 changes: 22 additions & 0 deletions config/initializers/db_encoding.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# frozen_string_literal: true

# Only applicable for MySQL
if ActiveRecord::Base.connection_config[:adapter] == 'mysql2'
# Get the configured encoding
configured_encoding = ActiveRecord::Base.connection_config[:encoding]

# Get the actual character set from the DB
database_charset = ActiveRecord::Base.connection.exec_query("SELECT @@character_set_database as charset").first['charset']

msg = <<-LOG
WARNING: The configured db encoding `#{configured_encoding}` is different from the actual one `#{database_charset}`!
This is likely to cause issues with special characters.
Please see https://maestrano.atlassian.net/wiki/x/rQ0nBg or run:
$ rails g connector:charset_migration
LOG

if configured_encoding != database_charset
Rails.logger.warn msg
warn msg
end
end
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# frozen_string_literal: true

require 'rails/generators/named_base'
require 'rails/generators/active_record/migration'

module Connector
module Generators
class CharsetMigrationGenerator < ::Rails::Generators::Base
include ActiveRecord::Generators::Migration

source_root File.expand_path('templates', __dir__)

def copy_migration
migration_template "migration.rb", "db/migrate/convert_tables_to_utf8.rb"
end
end
end
end
35 changes: 35 additions & 0 deletions lib/generators/connector/charset_migration/templates/migration.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# frozen_string_literal: true
class ConvertTablesToUtf8 < ActiveRecord::Migration
def change_encoding(encoding,collation)
# Allow for different adapter in different environment
unless ActiveRecord::Base.connection_config[:adapter] == 'mysql2'
say "Skipping conversion as non MySQL database (#{ActiveRecord::Base.connection_config[:adapter]})"
return
end
tables = connection.tables
dbname = ActiveRecord::Base.connection_config[:database]
say_with_time "Converting database to #{encoding}" do
execute <<-SQL
ALTER DATABASE `#{dbname}` CHARACTER SET #{encoding} COLLATE #{collation};
SQL
end
tables.each do |tablename|
say_with_time "Converting table `#{tablename}` to #{encoding}" do
execute <<-SQL
ALTER TABLE `#{dbname}`.`#{tablename}` CONVERT TO CHARACTER SET #{encoding} COLLATE #{collation};
SQL
end
end
end

def change
reversible do |dir|
dir.up do
change_encoding('utf8','utf8_general_ci')
end
dir.down do
change_encoding('latin1','latin1_swedish_ci')
end
end
end
end

0 comments on commit eb81358

Please sign in to comment.