Host a documentation sprint

[bensheldon]

Goal

Achieve a critical mass of of exemplary documentation that eases future documentation quality and maintenance.

Objectives and activities

• Tighten up the README and establish organizing principles. For example, covering:
  • Getting started
  • Configuration options or reference
  • Deep dives on sharp edges and optimization
• Improve code-level documentation:
  • get to 80% or more high quality code Level documentation coverage. (It’s at 40% right now). bundle exec yard stats --list-undoc
  • Establish principles for internal consumers (e.g. GoodJob developers) and application developer who may be composing elements in novel ways.
• get consistent terminology and validate that it makes sense with users/potential users (eg jobs vs tasks vs threads vs perform vs run vs executed)
• Ensure documentation is inclusive for all skill levels and people

Decisions

• GoodJob will use Yard for code-level documentation. This is more than Rails use of Rdoc. Yard is nice with Rubymine and potentially aids creation of Sorbet signatures.
  • Rubymine and metaprogrammed class attributes errata

Outside of sprint scope, parking lot:

• Set up CI tools for ensuring quality/integrity of documentation. For example:
  • Yardstick
  • https://alexjs.com

[bensheldon]

Example of documentation from Philip Hallstrom in Rails Link Slack:

https://github.com/instructure-bridge/scorm_engine/blob/master/lib/scorm_engine/api/endpoints/courses.rb

Challenges described:

Mostly just consistency. Terminology, phrasing, etc. I wrote it with the hopes that people wouldn’t ask me “what does this method do?” all the time.

[bensheldon]

Reviewing Yard's own documentation, some consistencies:

• ? predicates: "Tests if...", "whether..."

   # Tests if the object is defined as an alias of another method
   # @return [Boolean] whether the object is an alias
   def is_alias?
  

• Methods start with a verb, "Loads", "Saves", "Sanitizes"... though sometimes it's also "Load", "Save". Sometimes there is a qualifier e.g. "Temporarily loads..."
• @return and @param declarations are usually "the..."... and occassionally "a...". the statements are not capitalized. Arrays are described as "the list of..."
• @!attribute [r] only document the @return. I didn't see examples of [rw] in Yard, but other projects document the attribute, and not the @return value

Some examples:

• getter methods that return arrays

[Mr0grog]

The guidelines I wrote for the IPFS folks a couple years ago when consulting on their original docs site might be helpful or relevant here: https://github.com/ipfs/community/blob/master/DOCS_STYLEGUIDE.md#linguistic-rules

[bensheldon]

Suggested improvements to: Optimize queues, threads, and processes section of Readme

• needs explanation for why/when/how to have multiple processes
• technical question around max threads vs each pool group. be explicit that it is max-per-group, not a global max.
• explicitly say that a * can be used for any queue. Have a deeper explanation of the queue format. Minus is not clear (e.g. * - queue)
• Format examples with linebreaks
• Combined multi-process dyno explanation

[bensheldon]

• Get YARD to properly parse README with github flavored markdown

[Mr0grog]

Two working ways (see step 2 for where they differ) I could find to add custom markup, which it seems like you need to do. Neither seems lovely, but lsegal/yard#1157 strongly implies there’s no getting around the hardcoded list of available markdown parsers without monkeypatching in some form or another. These both use Kramdown + kramdown-parser-gfm, but there's also Commonmarker. Hopefully it’s clear here how you’d use it instead:

1. Add to gemspec:

   # good_job.gemspec
   spec.add_development_dependency "kramdown-parser-gfm"

2. Add a Ruby file to customize Yard. There are two ways to do this part:

   1. By adding a new markup provider (more verbose, but feels at least a little less like a hack):

      # docs/yard_support.rb (or wherever you want)
      require 'kramdown'
      require 'kramdown-parser-gfm'
      
      # Custom markup provider class that always renders Kramdown using GFM.
      class KramdownGfmDocument < Kramdown::Document
        def initialize(source, options = {})
          options[:input] = 'GFM' unless options.key?(:input)
          super(source, options)
        end
      end
      
      # Insert the new provider as the highest priority option for Markdown.
      # See:
      # - https://github.com/lsegal/yard/issues/1157
      # - https://github.com/lsegal/yard/issues/1017
      # - https://github.com/lsegal/yard/blob/main/lib/yard/templates/helpers/markup_helper.rb
      YARD::Templates::Helpers::MarkupHelper::MARKUP_PROVIDERS[:markdown].insert(
        0,
        { const: 'KramdownGfmDocument' }
      )

   2. By Monkeypatching YARD::Templates::Helpers::HtmlHelper (simpler, but less nice, I think):

      # docs/yard_support.rb (or wherever you want)
      require 'kramdown'
      require 'kramdown-parser-gfm'
      
      # Customize markdown rendering for GFM support.
      # See:
      # - https://github.com/lsegal/yard/issues/1157
      # - https://github.com/lsegal/yard/issues/1017
      # - https://github.com/lsegal/yard/blob/2b16190ced212bdb948fb47568645b29590bdbcd/lib/yard/templates/helpers/html_helper.rb#L74-L94
      module YARD
        module Templates
          module Helpers
            module HtmlHelper
              def html_markup_markdown(text)
                Kramdown::Document.new(text, input: 'GFM').to_html
              end
            end
          end
        end
      end

3. Set .yardopts to use the above customization file:

   # .yardopts
   --load docs/yard_support.rb

[bensheldon]

@Mr0grog thanks for digging into those. My preference is to add the provider instead of monkey-patching.

And for future me, this is necessary for Github Flavored Markdown rendering of the Readme by Yard, for example tables and fenced code blocks.

[bensheldon]

Method for defining custom YARD tags which could be useful when building a Rails engine in #50:

https://stackoverflow.com/questions/38744959/documentation-of-rails-controllers-using-yard

[bensheldon]

@Mr0grog I merged #111 (the big PR of code-level documentation changes that also includes #118)