Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert stdlib to markdown. #14378

Merged
merged 32 commits into from
Dec 16, 2015
Merged

Conversation

MichaelHatherly
Copy link
Member

This completes the conversion of the stdlib docstrings from a mixture of markdown and rst to just markdown.

This PR consists of the following:

  • split helpdb.jl into several smaller files based on which module they're from
  • add footnote parsing and rendering: [^footnote] and [^footnote]: content
  • add rst simple table rendering
  • transform links when outputting rst if they're :func:, :ref:, etc. This is a temporary measure until the manual is also converted to markdown when we can switch to @StefanKarpinski's plan for refs.
  • adds inline tex using the scholarly markdown syntax of double backticks and fenced code blocks with language == "math"
  • Reformatting of all docstrings in Base to remove rst blocks, wrap text at ~92 chars
  • adds a macro to basedocs.jl to make the keyword docstrings defined there appear more uniform with @doc, ie. docstring above the object being documented
  • regenerate the stdlib docs and make some minor fixes to correct for incorrectly escaped \s

.. note:: syntax doesn't seem to have an equivalent in markdown so I've just made a basic change for now, a header called `notes``, I'll change to something else if anyone has a suggestion for formatting them differently.

make doctest fails currently with incorrect backtrace linenumbers, nothing related to this PR I believe.

The best commit to review these changes is probably the last one, 5edce89, most of the diff is just escaped backslash changes, which don't appear to affect the final docs in any way.

@ViralBShah ViralBShah added the docsystem The documentation building system label Dec 12, 2015
@StefanKarpinski
Copy link
Member

Did you do this automatically or by hand?

@MichaelHatherly
Copy link
Member Author

Did you do this automatically or by hand?

Automatic, for most of it, using vim macros. It was done in small steps though to hopefully avoid losing anything.

@StefanKarpinski
Copy link
Member

Great. Automatic is good since it's less error prone. Thanks for tackling this.

@tkelman
Copy link
Contributor

tkelman commented Dec 12, 2015

This is stellar work. The base reformatting commit is big and I'm still scrolling through that in git show , but this looks clean.

@MichaelHatherly
Copy link
Member Author

I've changed table rendering to "enclose" rst tables to avoid the trailing whitespace problems, renders the same as before though.

The extra whitespace caused by list indents is still included. As far as I can tell it's a simple one line change

-rstinline(io::IO, s::AbstractString) = print(io, rstescape(s))
+rstinline(io::IO, s::AbstractString) = print(io, replace(rstescape(s), r"[\s\t\n]+", " "))

to strip it out like it's done for term rendering. I can commit that change as well, but since that's kind of how rst would treat the extra spaces it didn't seem worth it.

@MichaelHatherly
Copy link
Member Author

Here's a RTDs build of this branch, https://julia-rst-conversion.readthedocs.org/en/mh-purge-rst/#stdlib.

The travis 64bit failure looks unrelated, I think.

@tkelman
Copy link
Contributor

tkelman commented Dec 13, 2015

Yeah, I don't know what's causing these but I don't think it's anything to do with your doc changes

fatal error during bootstrap:
(parse-error "read: invalid escape sequence")
ERROR: fatal error loading system image

@MichaelHatherly
Copy link
Member Author

Anything else I need to address here?

@tkelman
Copy link
Contributor

tkelman commented Dec 16, 2015

Don't think so, is it all good to go as far as you're concerned?

Only potential thing I'd want to double-check is whether there have been doc changes on master since this was opened. I think those would probably cause a conflict though?

@MichaelHatherly
Copy link
Member Author

I'll try rebasing to check for that.

Also remove rst block by adding special rst link printing.
'$WORK_DIR' was being treated as an interpolated `Symbol`. Surround
it in backticks to avoid this. Also add backticks to paths and enum
values in surrounding docs for consistency.
Backslashes in signatures of `writedlm`, `:`, `.\`, and `\` were
causing them to be skipped after converting the respective docstrings
to markdown from rst.

(This appears to fix the problem and not cause any regressions elsewhere,
but is a bit of a hack.)
The only significant changes here seem to be the loss of `.. note::`s.

Most other changes are extra `\ ` escapes at the end of inline code,
inline math, and links. Some code blocks have been indented by an extra
space, I've not managed to track down the cause of that, though they
should render the same anyway.

`make doctest` errors all look related to differences in backtrace line
numbers and aren't related to the changes in this commit.
@MichaelHatherly
Copy link
Member Author

Rebasing went fine, should I push the branch again and trigger a build?

The only change here that may be a problem is where anyone has used double backticks for inline code, which will now parse as inline latex, shouldn't cause actual breakages though.

@hayd
Copy link
Member

hayd commented Dec 16, 2015

💯 LGTM!

@tkelman
Copy link
Contributor

tkelman commented Dec 16, 2015

Did you re-run the automated steps that did all the work here post-rebasing? Just want to make sure we don't lose any doc changes that have been made over the last couple days. CI looks more or less idle right now.

@tkelman
Copy link
Contributor

tkelman commented Dec 16, 2015

where anyone has used double backticks for inline code

We should probably grep for double-but-not-triple backticks to see if there are any false positives there, but that can be done after merging this.

@MichaelHatherly
Copy link
Member Author

Did you re-run the automated steps that did all the work here post-rebasing?

I've rerun julia doc/genstdlib.jl, which should highlight any loses so long as they were added corectly to stdlib to begin with. There was no change, so that should be fine.

I've pushed the updated branch.

tkelman added a commit that referenced this pull request Dec 16, 2015
@tkelman tkelman merged commit 79b2418 into JuliaLang:master Dec 16, 2015
@MichaelHatherly
Copy link
Member Author

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docsystem The documentation building system
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants