Skip to content

Commit

Permalink
feat(template): add site.robots support
Browse files Browse the repository at this point in the history
BREAKING CHANGE: Now you can define your custom `robots.txt` records in configuration file:

```yaml
robots:
  - "User-agent: *"
  - "Disallow: /ajax/"
  - "Disallow: /@async"
  - "Disallow: /log/"
```
  • Loading branch information
sparanoid committed Oct 31, 2015
1 parent dc290bb commit 0ddab1b
Show file tree
Hide file tree
Showing 4 changed files with 45 additions and 1 deletion.
14 changes: 14 additions & 0 deletions _app/_posts/note/2014-06-09-creating-themes.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,20 @@ Template:
{{ '{% endif ' }}%}
```

### `site.robots`

You can define your own records in `robots.txt`.

Config:

```yaml
robots:
- "User-agent: *"
- "Disallow: /ajax/"
- "Disallow: /@async"
- "Disallow: /log/"
```
## Theme Features
There're some features can be integrated into your theme.
Expand Down
5 changes: 4 additions & 1 deletion _app/robots.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
---
---

User-agent: *
{% for post in site.robots
%}{{ post | strip }}
{% endfor %}

Sitemap: {{ '/sitemap.xml' | prepend: site.base | prepend: site.url }}
17 changes: 17 additions & 0 deletions _config.init.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,13 @@ redcarpet:
- superscript
- tables

#
# Jekyll plugins
#

gems:
- jekyll-last-modified-at

#
# Almace Scaffolding settings
#
Expand Down Expand Up @@ -117,3 +124,13 @@ rss_excerpt: false
# Show generator credits
# Default: true
credits: true

# Custom records in robots.txt
# For example:
# robots:
# - "User-agent: *"
# - "Disallow: /ajax/"
# - "Disallow: /@async"
# - "Disallow: /log/"
# Default: "User-agent: *"
robots: "User-agent: *"
10 changes: 10 additions & 0 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -124,3 +124,13 @@ rss_excerpt: false
# Show generator credits
# Default: true
credits: true

# Items in robots.txt
# For example:
# robots:
# - "User-agent: *"
# - "Disallow: /ajax/"
# - "Disallow: /@async"
# - "Disallow: /log/"
# Default: "User-agent: *"
robots: "User-agent: *"

0 comments on commit 0ddab1b

Please sign in to comment.