Skip to content
This repository has been archived by the owner on Jan 29, 2020. It is now read-only.

Documentation build chain for gh-pages #279

Merged

Conversation

weierophinney
Copy link
Member

This pull-request provides a new documentation build chain to allow publishing to gh-pages for commits to master. It serves as a sandbox for how we can do the ZF component documentation going forward as well.

It implements the following:

  • Removes all usage of bookdown. Unfortunately, I found templating with it far too cumbersome, and it is missing features we could use (e.g. sidebar nav, top nav, etc.), but which would take some time to develop.
  • Implements a more thorough MkDocs build configuration, with fewer top-level items, more nesting, and specifying configuration features such as custom templates, extension directives to disable pygments highlighting, etc.
  • Reorganizes the documentation a fair bit to provide more natural groupings.

I implemented a custom MkDocs theme based on Cinder, which I have proposed at weierophinney/zf-mkdoc-theme. You can see it in action at https://zendframework.github.io/zend-expressive/

At this time, I still need to test the travis -> gh-pages workflow, and, for that reason, I'm marking it as a WIP. If it works as expected, I'll update the issue to indicate readiness.

@weierophinney
Copy link
Member Author

Awesome. I evidently branched from develop. sigh

Fortunately, the file changes shown are accurate; master and develop are in sync currently.

@weierophinney
Copy link
Member Author

Ping @ezimuel and @GeeH — curious to hear your opinions.

@geerteltink
Copy link
Member

Looks like its easy to use.

About the theme: You might want to add the previous and next links at the bottom of the page as well. Although the top navbar is static, on smaller devices the nav links are hidden behind the hamburger icon.

EDIT: And maybe add breadcrumbs, if mkdocs allows it, so it's easier to figure out at what section you are.

@akrabat
Copy link
Contributor

akrabat commented Jan 22, 2016

I like it, too.

@RalfEggert
Copy link
Contributor

I like it as well. 👍

But it seems that the search does not work. If it doesn't work it shouldn't be shown in the menu. And maybe its possible to add the ZF logo?

@froschdesign
Copy link
Member

@ALL
I think, this is no the right place to discuss the template. The topic should be the build chain.

@weierophinney
With Bookdown it is very easy to create a complete documentation with all components. ("Remote Content")
How does it work in MkDocs or is an own build script needed?

it is missing features we could use (e.g. sidebar nav, top nav, etc.), but which would take some time to develop.

I have a test script for a navigation. I will invest some work over the weekend and give you a feedback.

@weierophinney
Copy link
Member Author

@froschdesign The plan that @ezimuel and I have been discussing is to host docs for components on github pages, and to create a landing page for the organization. This vastly simplifies the problems of keeping the documentation up-to-date — and also means that the ability of merging remote documentation is no longer a requirement.

As such, I definitely think we can discuss the template. In particular, suggestions on navigation are very welcome, and I'll be looking into how to address the various suggestions already made.

@froschdesign
Copy link
Member

@weierophinney

The plan that @ezimuel and I have been discussing is to host docs for components on github pages, and to create a landing page for the organization.

That's good to know and a nice idea. 👍

In particular, suggestions on navigation are very welcome, and I'll be looking into how to address the various suggestions already made.

For suggestions and extensions (PRs) should we use your "zf-mkdoc-theme" repository? Or is there a plan to migrate the theme to the zendframework/documentation repository?

@sandrokeil
Copy link

@froschdesign It is possible to integrate your navigation script in bookdown?

Hosting on GitHub pages with Travis integration is really a good idea. 👍

@weierophinney
Copy link
Member Author

The latest commits to this update the build chain to use the --user flag when installing pip packages; this ensures permissions are correct. In order to use those packages, you also need to export $HOME/.local/bin. Finally, I decided to cache $HOME/.local between invocations, to speed up builds.

I've tested this at https://github.com/weierophinney/zf-docs-toolchain-experiment, and verified it works correctly.

At this point, I'll turn to some of the layout suggestions from the comments.

@weierophinney
Copy link
Member Author

@froschdesign

For suggestions and extensions (PRs) should we use your "zf-mkdoc-theme" repository? Or is there a plan to migrate the theme to the zendframework/documentation repository?

For now, we can use my zf-mkdoc-theme repo; I'll be moving this into the zendframework org and/or incorporating it as part of the zendframework/documentation repo eventually. I wanted to do it separately at first while experimenting with the approach; I think the results are proving worthwhile, and the effort needed to make things work has been minimal, which makes me happy.

@sandrokeil — The issues I had with bookdown were:

  • I was going to need to come up with a "theme" from scratch, as the only bookdown examples I've seen are the bookdown site and the Aura docs, none of which match my ideas for visual presentation.
  • I've not seen examples of topnav + sidenav in bookdown, and, in point of fact, there are open issues for these on the bookdown repo already. I started looking into it, but I felt the amount of effort required was going to be a distraction from getting the release ready.

MkDocs is mature, and we were already using it via a conversion layer (I wrote a script that converts bookdown.json to mkdocs.yml early on) to render on rtfd.org. It has a fairly rich ecosystem, which meant I was also able to find themes quickly that matched what I wanted for visual presentation. At this point, I've proven you can integrate it into a Travis-CI chain without much problem, too. As such, I say, "use the tool that gets the job done."

@sandrokeil
Copy link

use the tool that gets the job done.

@weierophinney Yes, that's it.

For those who looking for another extended bookdown theme, look at tobiju/bookdown-bootswatch-templates. But top and site nav are missing.

@weierophinney
Copy link
Member Author

Search is now working; it was an issue with the theme, which I've now corrected.

@weierophinney
Copy link
Member Author

Breadcrumbs and next/prev pager implemented. Breadcrumbs are not terribly useful, as we don't have "index" pages for sections yet; I'll see if I can figure out how to do that with mkdocs. Results are posted, though, so please review.

Next items on my list are:

  • move intro -> index pages (intro was a convention from bookdown, as index pages were always ToCs)
  • add zf logo in topnav, linking to zf site.
  • make initial landing page more of a "marketing" page, with links to most relevant doc pages.

@weierophinney
Copy link
Member Author

ZF logo is now used in the navbar and as the favicon. (ping @RalfEggert)

It looks like MkDocs shares the same issue/feature as bookdown in relation to index pages for subsections; essentially, they cannot be done. As such, the last remaining item (currently) is to update the landing page to make it more of a "why you should use Expressive" message.

@RalfEggert
Copy link
Contributor

👍

This is the documentation deployment toolchain for generating gh-pages
on-the-fly from the master branch.

The Travis-CI configuration now defines several environment variables for
specifying the repository, github username and email, and github token
(encrypted) to use when committing the files.

`doc/deploy.sh` performs the actual build process. It first initializes a bare
repository under `doc/html/`, adds the repository as a remote, fetches it, and
resets head to the gh-pages branch. It then builds the documentation, adds all
changes, and pushes them back to the gh-pages branch.
- instead of 2 top-level docs
@weierophinney weierophinney force-pushed the feature/doc-build-chain branch from c0277af to 31ad554 Compare January 25, 2016 17:48
@weierophinney
Copy link
Member Author

Rebased against master, which was my original plan all along.

@weierophinney weierophinney removed the WIP label Jan 25, 2016
@weierophinney
Copy link
Member Author

Okay, feature is complete. The latest iteration provides custom landing page content that is swapped into the landing page generated by mkdocs, and which is more "marketing" oriented (why use it? how do you use it? etc.), with links to relevant deeper pages of the documentation.

@weierophinney
Copy link
Member Author

Pinging @akrabat, @GeeH, and @RalfEggert for review!

@weierophinney
Copy link
Member Author

The only additional change I'm considering making, and it can be made after merge, is to push the doc/*.sh and doc/swap_index.php scripts into the documentation or zf-mkdoc-theme packages. They are re-usable already, which would allow us to download them when we're ready to deploy documentation only; they could even be cached between invocations.

The benefit to doing so is that if we want to make a change to the build process at any point, we can make it across every repository at once, instead of needing to push piecemeal.

@RalfEggert
Copy link
Contributor

@weierophinney

Looks really great and the intro page is a big improvement for the docs. Are you planning this for every ZF component?

Anyway, just a little, small detail. On my smart phone the hamburger icon is not shown in the portrait mode. It is only shown in the landscape mode. I am using an iPhone 5s.

Portrait Landscape
img_8585 img_8586

Maybe a front end guy knows why this happens?

@geerteltink
Copy link
Member

On my smart phone the hamburger icon is not shown in the portrait mode. It is only shown in the landscape mode. I am using an iPhone 5s.

That's because the expressive-skeleton screenshot is too big. Ideally you want to add a img-responsive class to it to make it responsive (auto-size). But if that's not possible, forcing all images to be responsive might work:

img {
    max-width: 100%;
    height: auto;
}

@RalfEggert
Copy link
Contributor

@xtreamwayz

I guess that might solve the issue...

@GeeH
Copy link

GeeH commented Jan 26, 2016

It works well, my only (massive) concern is that all the current repositories have bookdown.json - will I need to change this for every repo for this new tool?

@weierophinney
Copy link
Member Author

@GeeH I have a tool for that already (https://github.com/phly/bookdown2mkdocs), and we can do it individually as we prepare each component for gh-pages launch. (I was already using that tool in this repo to create a mkdocs.yml so we could use rtfd.org, and I use it on the SM, EM, and zend-code repos, too.)

@weierophinney
Copy link
Member Author

@RalfEggert Thanks for the feedback, and @xtreamwayz for the hint as to how to fix it; have the change pushed already.

@RalfEggert
Copy link
Contributor

@weierophinney

And I can confirm that it works now! 👍

@weierophinney
Copy link
Member Author

I've moved the deploy/build scripts into the zf-mkdoc-theme at this time, which allows us to re-use them across components. The latest commits have working Travis configuration for downloading, invoking, and caching the theme and build tools.

My plan at this point is:

  • to move the theme into the zendframework organization
  • to update the travis configuration to download the theme from the new repo
  • to squash commits in this repo to remove some of the noise (such as the changes to the build tools over time, particularly as they're now in a different repo)

At that point, I think I can go ahead and merge, based on the feedback on the pull request so far.

@akrabat
Copy link
Contributor

akrabat commented Jan 26, 2016

PR looks good to me.

We're already using rtfd.org, and validated that it works well. I've also
discovered a few things that make mkdocs more compelling:

- [superfences](http://facelessuser.github.io/pymdown-extensions/extensions/superfences/)
  fixes the nested code sample issues.
- bookdown doesn't currently support sidebar navs, and on looking at the
  templates, I suspect it'd be quite hard to implement.
- themes like [cinder](http://sourcefoundry.org/cinder/) are incredibly close to
  the look and feel I'd like to see for the documentation.

This patch removes the `bookdown.json` files, and pushes all structure into the
`mkdocs.yml` file. Additionally, it reorganizes the documentation to offer fewer
top-level items, which makes the navigation less cluttered, and removes
navbar artifacts I was observing when rendering with too many items.

It also removes the `Makefile`, as it's obsolete; no more bookdown to mkdocs
conversion is necessary.

Things still todo:

- Generate a decent landing page.
- Create ToC pages for each major section.
- Test the travis-ci -> gh-pages integration to ensure that we can install
  mkdocs and superfences.
- Promoted cookbook up a level
- Moved migration under the reference section
- Moved Reference after Cookbook
- Removed theme_dir from `mkdocs.yml`, as it depends on having downloaded a 3rd
  party theme.
- Added logic to deployment script to append theme_dir and markdown_extensions
  to the `mkdocs.yml`.
- Downloads our selected theme and unpack it.
- Added extension configuration for codehilite to disable pygment highlighting;
  instead, we'll use prismjs.
I've created a new repo to hold the custom theme, and tagged an initial release.
The deployment script now grabs it from there and extracts it into a directory.
Creates a new landing page for the documentation, and updates the "site name" to
simply "Expressive".
- Added site_url to configuration, for consumption in deploy script.
- Updated to 0.1.1 of zf-mkdoc-theme, to pick up fixes for search.
- Removed EXECUTE_DOC_CHECK travis directives; no longer relevant.
- Install mkdocs deps to user. Uses the `--user` flag to `pip install` to ensure
  pip installs local to the user, so that permissions will be correct. Also adds
  `$HOME/.local/bin` to the path, so that `mkdocs` can be executed once
  installed.
- Caches the `$HOME/.local` directory, so that subsequent builds do not need to
  install the deps again.
- Only install mkdocs and pymdown-extensions conditionally, for DEPLOY_DOCS
  builds.
- Updated GH_TOKEN, encrypted for correct repository.
Makes it simpler to build the docs on your own.
Fixes navbar collapse issues, and adds a pager to the top of content whenever
the navbar is collapsed.
This patch implements a landing page for the documentation. To do so, it
replaces the contents generated by mkdocs with bespoke contents that detail
marketing features and how to get started.

This patch also iterates through all generated HTML pages to add the
`class="img-responsive"` attribute to any images detected, ensuring that the
layout is consistent across all pages.
This commit removes the build/deploy scripts from the repository, and moves them
into the zf-mkdoc-theme repository, which includes them as of 0.2.0.

`.travis.yml` was updated to conditionally download and extract the latest
zf-mkdoc-theme archive, and to execute the build chain from it; it is also now
doing so from the repo under the zendframework organization.
@weierophinney weierophinney force-pushed the feature/doc-build-chain branch from b06b8e8 to 12d9a55 Compare January 26, 2016 22:30
@weierophinney weierophinney merged commit 12d9a55 into zendframework:master Jan 26, 2016
weierophinney added a commit that referenced this pull request Jan 26, 2016
weierophinney added a commit that referenced this pull request Jan 26, 2016
weierophinney added a commit that referenced this pull request Jan 26, 2016
@weierophinney weierophinney deleted the feature/doc-build-chain branch January 26, 2016 22:35
@weierophinney
Copy link
Member Author

And we have our first automated commit — f0b7285 !

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants