Improved locale routing based on filenames or special front matter (#414

)

* Process filename-based locale content and multi-locale output generation

* Add tests for locale features

* Fix test descriptions

* Fix permalink feature tests

bridgetown-core/features/permalinks.feature

@@ -136,6 +136,7 @@ Feature: Fancy permalinks
     And I have a configuration file with:
       | key               | value                           |
       | permalink         | /:lang/:year/:month/:day/:slug/ |
+      | content_engine    | resource                        |
       | available_locales | [en, es]                        |
     When I run bridgetown build
     Then I should get a zero exit status
@@ -154,11 +155,12 @@ Feature: Fancy permalinks
     And I have a configuration file with:
       | key               | value                            |
       | permalink         | /:lang/:year/:month/:day/:title/ |
+      | content_engine    | resource                         |
       | available_locales | [en, es]                         |
     When I run bridgetown build
     Then I should get a zero exit status
     And the output directory should exist
-    And I should see "Impresionante!" in "output/es/2009/03/27/multi-lingual/index.html"
+    And I should see "Impresionante!" in "output/es/2009/03/27/custom-locale/index.html"
 
   Scenario: Don't use language permalink if locales aren't configured
     Given I have a _posts directory
@@ -170,39 +172,13 @@ Feature: Fancy permalinks
       Impresionante!
       """
     And I have a configuration file with:
-      | key       | value                           |
-      | permalink | /:lang/:year/:month/:day/:slug/ |
-    When I run bridgetown build
-    Then I should get a zero exit status
-    And the output directory should exist
-    And I should see "Impresionante!" in "output/2009/03/27/not-multi-lingual-es/index.html"
-
-  Scenario: Use custom permalink schema with multiple languages
-    Given I have a _posts directory
-    And I have an "_posts/2009-03-27-multi-lingual.en.md" file with content:
-      """
-      ---
-      title: English Locale
-      ---
-      Awesome! {{ site.locale }}
-      """
-    And I have an "_posts/2009-03-27-multi-lingual.es.md" file with content:
-      """
-      ---
-      title: Custom Locale
-      language: es
-      ---
-      Impresionante! {{ site.locale }}
-      """
-    And I have a configuration file with:
-      | key               | value                            |
-      | permalink         | /:lang/:year/:month/:day/:title/ |
-      | available_locales | [en, es]                         |
+      | key            | value                           |
+      | permalink      | /:lang/:year/:month/:day/:slug/ |
+      | content_engine | resource                        |
     When I run bridgetown build
     Then I should get a zero exit status
     And the output directory should exist
-    And I should see "Awesome! en" in "output/en/2009/03/27/multi-lingual/index.html"
-    And I should see "Impresionante! es" in "output/es/2009/03/27/multi-lingual/index.html"
+    And I should see "Impresionante!" in "output/2009/03/27/not-multi-lingual.es/index.html"
 
   Scenario: Use custom permalink schema with multiple languages and a default path
     Given I have a _posts directory
@@ -222,7 +198,8 @@ Feature: Fancy permalinks
       """
     And I have a configuration file with:
       | key               | value                              |
-      | permalink         | /:locale/:year/:month/:day/:title/ |
+      | permalink         | /:locale/:year/:month/:day/:slug/  |
+      | content_engine    | resource                           |
       | available_locales | [en, es]                           |
     When I run bridgetown build
     Then I should get a zero exit status
@@ -249,7 +226,8 @@ Feature: Fancy permalinks
       """
     And I have a configuration file with:
       | key               | value                            |
-      | collections       | {blogs: {output: true, permalink: "/:locale/:collection/:title/"}} |
+      | collections       | {blogs: {output: true, permalink: "/:locale/:collection/:slug/"}} |
+      | content_engine    | resource                         |
       | available_locales | [en, es]                         |
     When I run bridgetown build
     Then I should get a zero exit status
@@ -270,6 +248,7 @@ Feature: Fancy permalinks
     And I have a configuration file with:
       | key               | value                           |
       | permalink         | /:lang/:year/:month/:day/:slug/ |
+      | content_engine    | resource                        |
       | available_locales | [en, es]                        |
     When I run bridgetown build
     Then I should get a zero exit status

bridgetown-core/lib/bridgetown-core/collection.rb

@@ -304,8 +304,22 @@ def merge_environment_specific_metadata(data_contents)
     def read_resource(full_path)
       id = "repo://#{label}.collection/" + Addressable::URI.escape(
         Pathname(full_path).relative_path_from(Pathname(site.source)).to_s
-      )
-      resource = Bridgetown::Model::Base.find(id).to_resource.read!
+      ).gsub("#", "%23")
+      model = Bridgetown::Model::Base.find(id)
+
+      if model.attributes.key?(:locale) && model.locale.to_sym == :multi
+        site.config.available_locales.each do |locale|
+          model.locale = locale
+          add_model_resource model
+        end
+        return
+      end
+
+      add_model_resource model
+    end
+
+    def add_model_resource(model)
+      resource = model.to_resource.read!
       resources << resource if site.config.unpublished || resource.published?
     end
 

bridgetown-core/lib/bridgetown-core/configuration.rb

@@ -55,8 +55,8 @@ class Configuration < HashWithDotAccess::Hash
       "show_dir_listing"     => false,
 
       # Output Configuration
-      "available_locales"    => ["en"],
-      "default_locale"       => "en",
+      "available_locales"    => [:en],
+      "default_locale"       => :en,
       "permalink"            => nil, # default is set according to content engine
       "timezone"             => nil, # use the local timezone
 
@@ -100,6 +100,7 @@ class << self
       def from(user_config, starting_defaults = DEFAULTS)
         Utils.deep_merge_hashes(starting_defaults.deep_dup, Configuration[user_config])
           .merge_environment_specific_options!
+          .setup_locales
           .add_default_collections
           .add_default_excludes
           .check_include_exclude
@@ -244,6 +245,12 @@ def merge_environment_specific_options!
       self
     end
 
+    def setup_locales
+      self.default_locale = default_locale.to_sym
+      available_locales.map!(&:to_sym)
+      self
+    end
+
     def add_default_collections # rubocop:todo all
       # It defaults to `{}`, so this is only if someone sets it to null manually.
       return self if self[:collections].nil?
@@ -264,7 +271,7 @@ def add_default_collections # rubocop:todo all
         self[:permalink] = "pretty" if self[:permalink].blank?
         self[:collections][:pages] = {} unless self[:collections][:pages]
         self[:collections][:pages][:output] = true
-        self[:collections][:pages][:permalink] ||= "/:path/"
+        self[:collections][:pages][:permalink] ||= "/:locale/:path/"
 
         self[:collections][:data] = {} unless self[:collections][:data]
         self[:collections][:data][:output] = false

bridgetown-core/lib/bridgetown-core/resource/base.rb

@@ -295,13 +295,17 @@ def previous_resource
       private
 
       def ensure_default_data
+        determine_locale
+
         slug = if matches = relative_path.to_s.match(DATE_FILENAME_MATCHER) # rubocop:disable Lint/AssignmentInCondition
                  set_date_from_string(matches[1]) unless data.date
                  matches[2]
                else
                  basename_without_ext
                end
 
+        slug.chomp!(".#{data.locale}") if data.locale && slug.ends_with?(".#{data.locale}")
+
         data.slug ||= slug
         data.title ||= Bridgetown::Utils.titleize_slug(slug)
       end
@@ -334,6 +338,24 @@ def import_taxonomies_from_data
         end
       end
 
+      def determine_locale
+        unless data.locale
+          data.locale = locale_from_alt_data_or_filename.presence || site.config.default_locale
+        end
+
+        if data.locale_overrides&.is_a?(Hash) && data.locale_overrides&.key?(data.locale)
+          data.merge!(data.locale_overrides[data.locale])
+        end
+      end
+
+      # Look for alternative front matter or look at the filename pattern: slug.locale.ext
+      def locale_from_alt_data_or_filename
+        found_locale = data.language || data.lang || basename_without_ext.split(".")[1..-1].last
+        return unless found_locale && site.config.available_locales.include?(found_locale.to_sym)
+
+        found_locale
+      end
+
       def format_url(url)
         url.to_s.sub(%r{index\.html?$}, "").sub(%r{\.html?$}, "")
       end

bridgetown-core/lib/bridgetown-core/resource/permalink_processor.rb

@@ -66,13 +66,13 @@ def permalink_for_permalink_style(permalink_style)
 
         case permalink_style.to_sym
         when :pretty
-          "#{collection_prefix}/:categories/:year/:month/:day/:slug/"
+          "/:locale/#{collection_prefix}/:categories/:year/:month/:day/:slug/"
         when :pretty_ext, :date
-          "#{collection_prefix}/:categories/:year/:month/:day/:slug.*"
+          "/:locale/#{collection_prefix}/:categories/:year/:month/:day/:slug.*"
         when :simple
-          "#{collection_prefix}/:categories/:slug/"
+          "/:locale/#{collection_prefix}/:categories/:slug/"
         when :simple_ext
-          "#{collection_prefix}/:categories/:slug.*"
+          "/:locale/#{collection_prefix}/:categories/:slug.*"
         else
           permalink_style.to_s
         end
@@ -108,7 +108,13 @@ def add_base_path(permalink)
 
       # @param resource [Bridgetown::Resource::Base]
       register_placeholder :path, ->(resource) do
-        { raw_value: resource.relative_path_basename_without_prefix }
+        {
+          raw_value: resource.relative_path_basename_without_prefix.tap do |path|
+            if resource.data.locale && path.ends_with?(".#{resource.data.locale}")
+              path.chomp!(".#{resource.data.locale}")
+            end
+          end,
+        }
       end
 
       # @param resource [Bridgetown::Resource::Base]
@@ -128,8 +134,10 @@ def add_base_path(permalink)
 
       # @param resource [Bridgetown::Resource::Base]
       register_placeholder :locale, ->(resource) do
-        locale_data = resource.data.locale
-        resource.site.config.available_locales.include?(locale_data) ? locale_data : nil
+        next nil if resource.data.locale&.to_sym == resource.site.config.default_locale
+
+        locale_data = resource.data.locale&.to_sym
+        resource.site.config.available_locales.include?(locale_data) ? locale_data.to_s : nil
       end
       register_placeholder :lang, placeholder_processors[:locale]
 

bridgetown-core/lib/bridgetown-core/static_file.rb

@@ -21,10 +21,11 @@ def reset_cache
 
     # Initialize a new StaticFile.
     #
-    # site - The Site.
-    # base - The String path to the <source>.
-    # dir  - The String path between <source> and the file.
-    # name - The String filename of the file.
+    # @param site [Bridgetown::Site]
+    # @param base [String] path to the <source>.
+    # @param dir [String] path between <source> and the file.
+    # @param name [String] filename of the file.
+    # @param collection [Bridgetown::Collection] optional collection the file is attached to
     def initialize(site, base, dir, name, collection = nil) # rubocop:disable Metrics/ParameterLists
       @site = site
       @base = base
@@ -36,7 +37,7 @@ def initialize(site, base, dir, name, collection = nil) # rubocop:disable Metric
       @data = @site.frontmatter_defaults.all(relative_path, type).with_dot_access
       if site.uses_resource? && !data.permalink
         data.permalink = if collection && !collection.builtin?
-                           "/:collection/:path.*"
+                           collection.default_permalink.chomp("/").chomp(".*") + ".*"
                          else
                            "/:path.*"
                          end

bridgetown-core/test/helper.rb

@@ -149,6 +149,7 @@ def fixture_site(overrides = {})
 
   def resources_site(overrides = {})
     overrides["content_engine"] = "resource"
+    overrides["available_locales"] ||= %w[en fr]
     new_config = site_configuration(overrides)
     new_config.root_dir = resources_root_dir
     new_config.source = resources_root_dir("src")
@@ -231,6 +232,21 @@ class FakeLogger
   def <<(str); end
 end
 
+# stub
+module Bridgetown
+  module Paginate
+    class PaginationIndexer
+      def self.index_documents_by(pages_list, search_term)
+        # site.collections[@configured_collection].resources
+
+        pages_list.map do |resource|
+          [resource.data[search_term], nil]
+        end.to_h
+      end
+    end
+  end
+end
+
 module TestWEBrick
   module_function
 

bridgetown-core/test/resources/src/_pages/multi-page.md

@@ -0,0 +1,9 @@
+---
+title: "Multi-locale page"
+locale_overrides:
+  fr:
+    title: "Sur mesure"
+locale: multi
+---
+
+{% if site.locale == "en" %}English:{% elsif site.locale == "fr" %}French:{% endif %} {{ resource.data.title }}

bridgetown-core/test/resources/src/_pages/second-level-page.en.md

@@ -0,0 +1,7 @@
+~~~ruby
+{ title: "I'm a Second Level Page" }
+~~~
+
+That's **nice**.
+
+Locale: {{ resource.data.locale }}

bridgetown-core/test/resources/src/_pages/second-level-page.fr.md

@@ -0,0 +1,7 @@
+~~~ruby
+{ title: "I'm a Second Level Page in French" }
+~~~
+
+C'est **bien**.
+
+Locale: {{ resource.data.locale }}

bridgetown-core/test/test_locales.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+require "helper"
+
+class TestLocales < BridgetownUnitTest
+  context "similar pages in different locales" do
+    setup do
+      @site = resources_site
+      @site.process
+      # @type [Bridgetown::Resource::Base]
+      @english_resource = @site.collections.pages.resources.find do |page|
+        page.relative_path.to_s == "_pages/second-level-page.en.md"
+      end
+      @french_resource = @site.collections.pages.resources.find do |page|
+        page.relative_path.to_s == "_pages/second-level-page.fr.md"
+      end
+    end
+
+    should "have the correct permalink and locale in English" do
+      assert_equal "/second-level-page/", @english_resource.relative_url
+      assert_includes @english_resource.output, "<p>Locale: en</p>"
+    end
+
+    should "have the correct permalink and locale in French" do
+      assert_equal "/fr/second-level-page/", @french_resource.relative_url
+      assert_includes @french_resource.output, "<p>C’est <strong>bien</strong>.</p>\n\n<p>Locale: fr</p>"
+    end
+  end
+
+  context "one page which is generated into multiple locales" do
+    setup do
+      @site = resources_site
+      @site.process
+      # @type [Bridgetown::Resource::Base]
+      @resources = @site.collections.pages.resources.select do |page|
+        page.relative_path.to_s == "_pages/multi-page.md"
+      end
+      @english_resource = @resources.find { |page| page.data.locale == :en }
+      @french_resource = @resources.find { |page| page.data.locale == :fr }
+    end
+
+    should "have the correct permalink and locale in English" do
+      assert_equal "/multi-page/", @english_resource.relative_url
+      assert_includes @english_resource.output, "<p>English: Multi-locale page</p>"
+    end
+
+    should "have the correct permalink and locale in French" do
+      assert_equal "/fr/multi-page/", @french_resource.relative_url
+      assert_includes @french_resource.output, "<p>French: Sur mesure</p>"
+    end
+  end
+end