Allow dates to be parsed from filenames #3310

devjack · 2017-04-09T12:22:06Z

This has been partially discussed in #285.

This allows the date variable to be parsed from a filename with the aim to ease content migration. Date can be 'datestamped' in any part of a filename, with the example being 2017-01-31-mypost.md.

Filename fallback is enabled via the useFilenameDateAsFallback config value and can find custom date formats using filenameDateFallbackPattern (config) and parsed into the date param using filenameDateFallbackFormat (config).

devjack · 2017-04-09T12:40:25Z

First time golang and hugo contrib & I've had some trouble getting tests to pass locally too (both on master and my patch). Would appreciate any pointers on configuring/running/fixing these for the PR. Cheers!

CLAassistant · 2017-04-23T04:39:02Z

All committers have signed the CLA.

Parses filenames for a given date format and parses it into the date variable. Applied using the useFilenameDateAsFallback configuration option which defaults to false for backwards compatibility. Useful for content migration (e.g. from Jekyll et. al.) where date values are provided in filenames and not in frontmatter. Additional config values filenameDateFallbackPattern and filenameDateFallbackFormat allow custom date formats. Partially addresses functionality discussed in #285 fix tests

bep · 2017-04-23T09:09:49Z

Re. testing I would recommend using make:

I.e.

make sure you have govendor installed and the vendored lib synced: make vendor
Run all tests and checks: make check

I will comment on the PR itself in another comment.

bep · 2017-04-23T09:12:13Z

hugolib/page.go

@@ -1021,6 +1021,14 @@ func (p *Page) update(f interface{}) error {
 			p.Date = fi.ModTime()
 			p.Params["date"] = p.Date
 		}
+	} else if p.Date.IsZero() && p.s.Cfg.GetBool("useFilenameDateAsFallback") {


We have a little too many config variables as is, and I think we can drop this one and just check that filenameDateFallbackPattern is set. But other than this minor detail, this looks very good.

@bep thats a good point. With that in mind, perhaps one config value to provide the regex that defaults to nil? That way the parsing (regardless of format separator) can be configured but there's BC compat if Nil or not configured?

I'm not sure I follow the logic here. Are you saying we could reduce the number of configs to 1? How?

I'm thinking so - its balance between config and 'magic'.

If the regex and time format is always the same, you can infer it from named groups.

For example:

(?P<datestamp>(?P<stdYear>\d{4})\-(?P<stdZeroMonth>\d{2})-(?P<stdZeroDay>\d{2}))

The catch is that time format const's aren't public so there would need to be a copy of them in Hugo for mapping (probably just day/month/year with the 0 padded variants too).

So its either:
a) add 2x config params and let people choose format and pattern
b) add 1x config param and force people to use the same format as the pattern.

Sorry, I forgot this, as this will likely be copy-pasted from the documentation:

If the code to create a time format string from that regexp is not too complex, then 1 config is fine.

else just add two config vars.

Removed useFilenameDateAsFallback option and instead check to see if filenameDateFallbackPattern is set. As a result, filenameDateFallbackPattern now defaults to Nil. Added a reference example to the docs to ease copy/paste configuration for a popular YYYY-MM-DD filename format.

devjack · 2017-06-29T12:07:46Z

Cheers @bep,
Picked this one back up and patched (38a313f) as per your suggestion. My speculative automagic turned out to be too complex so settled on two config values with an updated sample in the docs.
Cheers!

maxandersen · 2017-07-31T23:08:19Z

I've made an updated patch that resolves the conflict on docs files, you can see it here: https://github.com/maxandersen/hugo/tree/devjack-filename-date-fallback

It seem to work great for me so I hope this can make it in as I really need this one for my migration of ~40 old blog posts using date in filenames.

maxandersen · 2017-07-31T23:10:29Z

btw. imo this should really just be enabled by default or at least have a default value for filenameDateFallbackPattern for iso dates - just so you don't have to put "complex" regular expressions into your config.

rdwatters · 2017-07-31T23:43:46Z

This is a great feature and will surely help with those migrating content from other SSGs 😃

That said, I wouldn't enable this by default with so many sites in the wild, IMHO.

maxandersen · 2017-07-31T23:49:22Z

That said, I wouldn't enable this by default with so many sites in the wild, IMHO.

true - but having a "useFilenameDateAsFallback = true" would then imo be better since I would argue in 99% of all cases the dateformat will be standard iso (yyyy-mm-dd).

One thing I noticed while using this patch is that it should also set the "slug" to remove the date from the path - at least as an option. so instead of /post/2017-08-01-my-new-blog as url it would say /post/my-new-blog.

maxandersen · 2017-08-01T00:03:17Z

i'll try get my patch updated to handle proper slug extraction too. will need some sleep first though ;)

maxandersen · 2017-08-01T00:08:08Z

also, just looked in jekyll and awestruct and they both basically assume <yyyy>-<mm>-<dd>-<slug> with some variance that can be better handled with logic rather than letting users set the regular expressions for matching. I'll get an updated patch that handles this as good as jekyll/awestruct seem to do it.

awestruct: https://github.com/awestruct/awestruct/blob/master/lib/awestruct/extensions/posts.rb#L42
jekyll: https://github.com/jekyll/jekyll/blob/master/lib/jekyll/document.rb#L479

maxandersen · 2017-08-01T07:17:38Z

hugolib/page_test.go

+	require.Len(t, s.RegularPages, 1)
+
+	p := s.RegularPages[0]
+	d, _ := time.Parse(time.RFC3339, "2017-01-31")


this always fails as RFC3339 requires timezone info.

lambrospetrou · 2017-11-15T22:13:30Z

What is the status of this? Is it going to be integrated, released?

bep · 2017-12-22T10:26:14Z

Sorry for the delay,

I haven't wanted to commit to this change (i.e. encode more information into the filename) until I saw the end of my current task (page bundles, a task that I have been revisiting for a pretty long time now).

This change may look innocent enough (and it probably is), but I have learnt some hard lessons in Hugo: Structural changes (i.e. stuff that makes people change their themes/content) is easy to add, but very hard to change/remove); even the smallest variable changes have had significant cost (.Now to now etc.).

These PRs are not forgotten and I understand them pretty well. But if you for some reason need to remove your forks, that is understandable.

maxandersen · 2017-12-25T22:53:11Z

But if you for some reason need to remove your forks, that is understandable.

don't need to remove them I just wish I could stop having to maintain separate fork :)

btw. this change shouldn't change any behavior - all past behavior is as before and if you dont want dates in title to affect anything you can disable the behavior.

But thanks for the update - hope it can get added soon :)

andriisoldatenko · 2018-02-17T08:32:43Z

@devjack are you going to continue or do you need any help?

devjack · 2018-02-20T22:16:56Z

@andriisoldatenko unfortunately I have no time at present to dedicate to this again in the immediate future. It’s open for anyone else that would like the feature etc.

This commit adds `enableFilenameDateAsFallback` which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. Fixes gohugoio#285 Closes gohugoio#3310

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: [frontmatter] dateFallbacks = ["filename"] Fixes gohugoio#285 Closes gohugoio#3310

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: [frontmatter] dateFallbacks = ["filename"] Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] dateFallbacks = ["filename"] ``` Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] defaultDate = ["filename"] ``` Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] defaultDate = ["filename"] ``` Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", "date"] ``` This commit also creates a testable unit from the date front matter handling. We should try to get `:git` as a keyword in the same config map, but that will have to wait for a later time. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", "date"] ``` This commit also creates a testable unit from the date front matter handling. We should try to get `:git` as a keyword in the same config map, but that will have to wait for a later time. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime"] lastMod = ["lastMod" ,":fileModTime", "date"] ``` The current config (what you get when doing nothing): ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`).. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", "date"] ``` This commit also creates a testable unit from the date front matter handling. We should try to get `:git` as a keyword in the same config map, but that will have to wait for a later time. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime"] lastMod = ["lastMod" ,":fileModTime", "date"] ``` The current config (what you get when doing nothing): ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", "date"] ``` This commit also creates a testable unit from the date front matter handling. We should try to get `:git` as a keyword in the same config map, but that will have to wait for a later time. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime"] lastMod = ["lastMod" ,":fileModTime", "date"] ``` The current config (what you get when doing nothing): ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). * The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", ":default"] ``` This commit is also a spring cleaning of how the different dates are configured in Hugo. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime", ":default"] lastMod = ["lastMod" ,":fileModTime", ":default"] ``` The current `:default` values for the different dates are ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` The above will now be the same as: ```toml [frontmatter] date = [":default"] lastmod = [":default"] publishDate = [":default"] expiryDate = [":default"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). * The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values. * The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit makes it possible to extract the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", ":default"] ``` This commit is also a spring cleaning of how the different dates are configured in Hugo. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime", ":default"] lastMod = ["lastMod" ,":fileModTime", ":default"] ``` The current `:default` values for the different dates are ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` The above will now be the same as: ```toml [frontmatter] date = [":default"] lastmod = [":default"] publishDate = [":default"] expiryDate = [":default"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). * The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values. * The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit makes it possible to extract the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", ":default"] ``` This commit is also a spring cleaning of how the different dates are configured in Hugo. Hugo will check for dates following the configuration from left to right, starting with `:filename` etc. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime", ":default"] lastMod = ["lastMod" ,":fileModTime", ":default"] ``` The current `:default` values for the different dates are ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` The above will now be the same as: ```toml [frontmatter] date = [":default"] lastmod = [":default"] publishDate = [":default"] expiryDate = [":default"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). * The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values. * The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit makes it possible to extract the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", ":default"] ``` This commit is also a spring cleaning of how the different dates are configured in Hugo. Hugo will check for dates following the configuration from left to right, starting with `:filename` etc. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime", ":default"] lastmod = ["lastmod" ,":fileModTime", ":default"] ``` The current `:default` values for the different dates are ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` The above will now be the same as: ```toml [frontmatter] date = [":default"] lastmod = [":default"] publishDate = [":default"] expiryDate = [":default"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). * The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values. * The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

github-actions · 2022-02-06T02:03:20Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

bep reviewed Apr 23, 2017

View reviewed changes

maxandersen reviewed Aug 1, 2017

View reviewed changes

maxandersen mentioned this pull request Aug 1, 2017

Better more complete fix for allowing filename to set date and slug #3762

Closed

bep modified the milestone: v0.35 Jan 22, 2018

This was referenced Jan 22, 2018

Extract date and permalink from filename #285

Closed

Design Proposal: defaultDate & defaultTitle #4341

Closed

bep added this to the v0.37 milestone Feb 20, 2018

bep mentioned this pull request Feb 21, 2018

Work In Progress: hugolib: Extract date and slug from filename #4436

Closed

bep modified the milestones: v0.37, v0.38 Feb 26, 2018

bep mentioned this pull request Mar 11, 2018

hugolib: Extract date and slug from filename #4494

Merged

bep closed this in #4494 Mar 11, 2018

dependencies bot mentioned this pull request Apr 2, 2018

Update https://github.com/gohugoio/hugo in repo_dependencies.yml from v0.37 to v0.38 matrixik/dobek.pro#14

Closed

github-actions bot locked as resolved and limited conversation to collaborators Feb 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow dates to be parsed from filenames #3310

Allow dates to be parsed from filenames #3310

devjack commented Apr 9, 2017

devjack commented Apr 9, 2017

CLAassistant commented Apr 23, 2017 •

edited

Loading

bep commented Apr 23, 2017 •

edited

Loading

bep Apr 23, 2017

devjack Apr 23, 2017

bep Apr 23, 2017

devjack Apr 23, 2017

bep Apr 26, 2017

devjack commented Jun 29, 2017

maxandersen commented Jul 31, 2017

maxandersen commented Jul 31, 2017

rdwatters commented Jul 31, 2017

maxandersen commented Jul 31, 2017

maxandersen commented Aug 1, 2017

maxandersen commented Aug 1, 2017 •

edited

Loading

maxandersen Aug 1, 2017

lambrospetrou commented Nov 15, 2017

bep commented Dec 22, 2017

maxandersen commented Dec 25, 2017

andriisoldatenko commented Feb 17, 2018

devjack commented Feb 20, 2018

github-actions bot commented Feb 6, 2022

Allow dates to be parsed from filenames #3310

Allow dates to be parsed from filenames #3310

Conversation

devjack commented Apr 9, 2017

devjack commented Apr 9, 2017

CLAassistant commented Apr 23, 2017 • edited Loading

bep commented Apr 23, 2017 • edited Loading

bep Apr 23, 2017

Choose a reason for hiding this comment

devjack Apr 23, 2017

Choose a reason for hiding this comment

bep Apr 23, 2017

Choose a reason for hiding this comment

devjack Apr 23, 2017

Choose a reason for hiding this comment

bep Apr 26, 2017

Choose a reason for hiding this comment

devjack commented Jun 29, 2017

maxandersen commented Jul 31, 2017

maxandersen commented Jul 31, 2017

rdwatters commented Jul 31, 2017

maxandersen commented Jul 31, 2017

maxandersen commented Aug 1, 2017

maxandersen commented Aug 1, 2017 • edited Loading

maxandersen Aug 1, 2017

Choose a reason for hiding this comment

lambrospetrou commented Nov 15, 2017

bep commented Dec 22, 2017

maxandersen commented Dec 25, 2017

andriisoldatenko commented Feb 17, 2018

devjack commented Feb 20, 2018

github-actions bot commented Feb 6, 2022

CLAassistant commented Apr 23, 2017 •

edited

Loading

bep commented Apr 23, 2017 •

edited

Loading

maxandersen commented Aug 1, 2017 •

edited

Loading