Never compile this.

;; -*- no-byte-compile: t; -*-

Org-Mode Fundamentals

Configure Org-Mode core functionality to compile this system.

Sysop is likely to use this periodically.

Start EMACS with this command:

emacs --debug-init --no-init-file --no-splash --background-color white --foreground-color black --vertical-scroll-bars --eval '(switch-to-buffer "*Messages*")' --name FND-TEST --title FND-TEST --load ~/src/help/.org-mode-fundamentals.emacs.el &

(load-file "~/src/help/.org-mode-ecm.emacs.el")

Literate Programming

This system enables you to perform 3 Literate Document activities

Tangling
Evaluating
Weaving

Combined they provide a rich Literate Programming environment.

These activities are not performed interactively by the user. They are automatic operations that occur as a result of the configuration by the document itself.

The following is the guide for the default configuration of this system and how it behaves.

Key:

Columns

S
Source document modified?

T
Destination tangled-document modified?

W
Destination weaved-document modified?

C
Source-Block Evaluation occurred?

O
Org-Macro Expansion occurred?

Activity	S	T	W	C	O
Tangling	F	T	F	F	F
Evaluating	T	F	F	T	F
Weaving	F	F	T	F!	T

They are separate and distinct operations.

“Programming” is logically an activity that is the combination of these 3 activites. It is interactively performed by Sysop. It is not a distinct or isolated operation. Results of one activity exist here and serve as inputs to another activity.

Note about F!: Weaving Source-Block Evaluation occurred?^*
- Source block evaluation on export is disabled using header arguments: those source blocks will never be evaluated on weaving
- However the ability for them evaluate on weaving is enabled so that weaved source blocks can be replaced by their result value. This gives a kind of template system. More details here

Helper Functions

Help configure Org-Mode.

(defun help/set-org-babel-default-header-args (property value)
  "Easily set system header arguments in org mode.

PROPERTY is the system-wide value that you would like to modify.

VALUE is the new value you wish to store.

Attribution: URL `http://orgmode.org/manual/System_002dwide-header-arguments.html#System_002dwide-header-arguments'"
  (setq org-babel-default-header-args
        (cons (cons property value)
              (assq-delete-all property org-babel-default-header-args))))

(defun help/set-org-babel-default-inline-header-args (property value)
  "See `help/set-org-babel-default-header-args'; same but for inline header args."
  (setq org-babel-default-inline-header-args
        (cons (cons property value)
              (assq-delete-all property org-babel-default-inline-header-args))))

(defun help/set-org-babel-default-header-args:R (property value)
  "See `help/set-org-babel-default-header-args'; same but for R.

This is a copy and paste. Additional languages would warrant a refactor."
  (setq org-babel-default-header-args:R
        (cons (cons property value)
              (assq-delete-all property org-babel-default-header-args:R))))

(defun help/set-org-babel-default-header-args:ditaa (property value)
  "See `help/set-org-babel-default-header-args'; same but for ditaa.

This is a copy and paste. Additional languages would warrant a refactor."
  (setq org-babel-default-header-args:ditaa
        (cons (cons property value)
              (assq-delete-all property org-babel-default-header-args:ditaa))))

(defun help/set-org-babel-default-header-args:dot (property value)
  "See `help/set-org-babel-default-header-args'; same but for dot.

This is a copy and paste. Additional languages would warrant a refactor."
  (setq org-babel-default-header-args:dot
        (cons (cons property value)
              (assq-delete-all property org-babel-default-header-args:dot))))

(defun help/set-org-babel-default-header-args:plantuml (property value)
  "See `help/set-org-babel-default-header-args'; same but for plantuml.

This is a copy and paste. Additional languages would warrant a refactor."
  (setq org-babel-default-header-args:plantuml
        (cons (cons property value)
              (assq-delete-all property org-babel-default-header-args:plantuml))))

(defun help/org-toggle-macro-markers ()
  (interactive)
  (let ((old org-hide-macro-markers)
        (new (not org-hide-macro-markers)))
    (setq org-hide-macro-markers new)
    (message "Just changed org-hide-macro-markers from %s to %s" old new)
    (font-lock-mode)
    (font-lock-mode)))

(defun help/org-prp-hdln ()
  "Visit every Headline. If it doesn't have an ID property then add one and
  assign it a UUID. Attribution: URL
  `http://article.gmane.org/gmane.emacs.orgmode/99738'. It is OK to leave the
  colon separator in here because these are never used as Source-Blocks and
  the rest of the code expects the colon separator."
  (interactive)
  (save-excursion
    (goto-char (point-min))
    (dolist (p (nreverse
                (org-element-map (org-element-parse-buffer 'headline) 'headline
                  (lambda (headline) (org-element-property :begin headline)))))
      (goto-char p)
      (org-id-get-create))
    (save-buffer)))

(defun help/org-id-new ()
  "Re-purposing `org-id' hit a snag when colons were forbidden in Source-Block
  names. Adding support for a user-defined Org-Id separator would have fixed
  this but with no benefit to Org-Id. So this function removes the colon
  instead.
 "
  (interactive)
  (let* ((gend (org-id-new))
         (newid (replace-regexp-in-string ":" "_" gend)))
    newid))

(defun help/org-prp-src-blk ()
  "If it doesn't have a NAME property then add one and
   assign it a UUID. Attribution: URL `http://article.gmane.org/gmane.emacs.orgmode/99740'"
  (interactive)
  (help/org-2every-src-block
   #'(lambda (element)
       (if (not (org-element-property :name element))
           (let ((i (org-get-indentation)))
             (beginning-of-line)
             (save-excursion (insert "#+NAME: " (help/org-id-new) "\n"))
             (indent-to i)
             (forward-line 2))))))

(defconst help/org-special-pre "^\s*#[+]")

(defun help/org-2every-src-block (fn)
  "Visit every Source-Block and evaluate `FN'."
  (interactive)
  (save-excursion
    (goto-char (point-min))
    (let ((case-fold-search t))
      (while (re-search-forward (concat help/org-special-pre "BEGIN_SRC") nil t)
        (let ((element (org-element-at-point)))
          (when (eq (org-element-type element) 'src-block)
            (funcall fn element)))))
    (save-buffer)))

(defun help/org-babel-demarcate-block ()
  "Add a NAME property then assign it a UUID."
  (interactive)
  (org-babel-demarcate-block)
  (insert "#+name: " (help/org-id-new))
  (beginning-of-line)
  (insert "\n"))

Identity

A Headline’s primary key is ID. Use org-id to manage it.

(require 'org-id)

In Links: Never use ID or CUSTOM_ID; always use the file name and text to make it accessible outside of Emacs.

(setq org-id-link-to-org-use-id 'nil)

Make sure that ID is always unique, portable, and easy to maintain by

Using an acceptable prefix
- Memorable
  - So you can remember where you created it and when
  - So you can share it and let the recipient know (in theory useful)
  - So you can enable a non Emacs/Org-Mode user to work with the tangled code referencing it’s origin
- Valid
  - Must be both LaTeX label and XHTML identifier compliant
    - org-lint checks for this
Include the current login
Include the current domain
Use a UUID

(setq org-id-prefix (concat "org_" (user-real-login-name) "_" (help/get-timestamp-no-colons) "_" (system-name)))
(setq org-id-method 'uuid)

Tangling

ID and NAME are essential for successful LP using org-babel-tangle-jump-to-org.

(add-hook 'org-babel-pre-tangle-hook #'help/org-prp-hdln)
(add-hook 'org-babel-pre-tangle-hook #'help/org-prp-src-blk)

There is a way to disable property inheritance that speeds up tangling a lot. This is only for user-defined properties; not Org-Mode properties.

The problem is that you lose property inheritance which is unacceptable. Never, never allow that. Its inconsistent with how Org-Mode works.

(setq org-babel-use-quick-and-dirty-noweb-expansion nil)

Assume that tangled document always live within the same directory structure as their origin document.

(setq org-babel-tangle-use-relative-file-links t)

Post tangle actions
- Indentation
  - At first glance this is surprising! The author should be responsible for the indentation, right? Yes, that is right. But there is one exception: using :noweb-ref source block concatenation. It is powerful and elegant. But the source blocks are indented on their own line. It forces any reader format it to make any sense of it. That is a poor start to using the tangled files. So tangled files get indented.

(defun help/org-babel-post-tangle-hook-fn ()
  (interactive)
  (indent-region (point-min) (point-max) nil)
  (save-buffer))
(add-hook 'org-babel-post-tangle-hook #'help/org-babel-post-tangle-hook-fn)

comments

Toggle insertion of comments in tangled code files

Provide as much information as possible in the tangled artifact about the origin artifact.

(help/set-org-babel-default-header-args :comments "noweb")

mkdirp

Toggle creation of parent directories of target files during tangling

(help/set-org-babel-default-header-args :mkdirp "yes")

no-expand

Turn off variable assignment and noweb expansion during tangling

{{{lp-configure-each-sb}}}

noweb

Toggle expansion of noweb references

Expand noweb references in source-blocks before:

Activity	Expand
Tangling	T
Evaluating	T
Weaving	F

This embraces the notion that you are telling the right thing to the computer and the right thing to the human. By the time you get to exporting, you ought to refer to the generated document.

(help/set-org-babel-default-header-args :noweb "no-export")

noweb-ref

Specify block’s noweb reference resolution target

{{{lp-configure-each-sb}}}

noweb-sep

String used to separate noweb references

{{{lp-configure-each-sb}}}

padline

Control insertion of padding lines in tangled code files

org-babel-tangle-jump-to-org requires padded lines. This configuration could arguably appear in the “Programming” heading because it impacts operation. It lives here because it must occur as part of the Tangling activity so that it can be used in the Programming activity.
Often I go back and for on this one. Sometimes it is nicer to have less spaces in generated code when guests are viewing it. When no one else is reading it I love the spaces. Defaulting to what I like.

(help/set-org-babel-default-header-args :padline "yes")

session

Preserve the state of code evaluation

{{{lp-configure-each-sb}}}

For some situations, this may be the same for every source block for a particular language. R is a good example.

shebang

Make tangled files executable

{{{lp-configure-each-sb}}}

tangle

Toggle tangling and specify file name

(help/set-org-babel-default-header-args :tangle "no")

tangle-mode

Set permission of tangled files

{{{lp-configure-each-sb}}}

Evaluating

Org-Mode may use all of the listed languages.

(org-babel-do-load-languages
 'org-babel-load-languages
 '((emacs-lisp . t)
   (org . t)
   ;;
   (C . t)
   (R . t)
   (python . t)
   (sass . t)
   (scheme . t)
   (sql . t)
   (js . t)
   ;;
   (latex . t)
   ;;
   (makefile . t)
   (shell . t)
   ;;
   (ditaa . t)
   (dot . t)
   (plantuml . t)))

cache

Avoid re-evaluating unchanged code blocks

{{{lp-configure-each-sb}}}

Default no is correct for nearly every scenario.

colnames

Handle column names in tables

{{{lp-configure-each-sb}}}

dir

Specify the default (possibly remote) directory for code block execution

{{{lp-configure-each-sb}}}

epilogue

Text to append to code block body

See Prologue.

eval

Limit evaluation of specific code blocks

Never evaluate source-blocks or in-line-source-blocks on export.

(help/set-org-babel-default-header-args :eval "never-export")
(help/set-org-babel-default-inline-header-args :eval "never-export")

org-export-use-babel

How does this overlap with the :eval header arg? Are they the same or different? What is the point? For a while I thought I understood the difference and how it worked. Later when I ran into a problem with my exports I realized that I didn’t understand the difference!

I thought that I had configured inline source blocks to

Have their results replaced on each export
Only include their results, excluding their source code
Allow execution of source blocks interactively, never on export

It is all documented here Literate Programming.

Instead of that, when I exported, the results weren’t replaced and the source code was included: exactly the opposite of what I had wanted to happen. Ouch!

Source blocks include a header arg :eval that controls evaluation of source blocks. I’d configured them all (both normal source blocks and inline source blocks) with the setting “never-export”. Never-export makes it so that you can evaluate source blocks when you are editing the document but they can never be evaluated during export. That is why #3 worked correctly. But I will still stuck with #1-#2.

Long story short after reviewing what I was thought every setting regarding evaluating and exportation I ended up on org-export-use-babel. It seemed silly to read it’s documentation again because I’d read it so many times that I though I knew it inside and out: it controls whether or not code blocks can be evaluated on export. I’d set it to true though, to be totally sure that the system worked as I had expected. Now two places disabled evaluation on export: header args and this variable. It was here though that my understanding had a major mistake!

org-export-use-babel— answers two questions (controls two features) with one answer:

Is code evaluated on export?
Are header args obeyed?

The key is the second part: the header args must be obeyed to make replace work. My problem was that I never noticed that this variable controls both execution and header args use. The latter, somehow I totally missed that. So no matter how I configured the header-args, those results could never be replaced because the header-args are totally ignored. Wow, I was so happy to discover this.

In the end the configuration was super simple: set org-export-use-babel to true, make sure the desired source blocks were set to :never-export, and the inline source blocks were setup to replace.

(setq org-export-use-babel t)

file

Specify a path for file output

{{{lp-configure-each-sb}}}

file-desc

Specify a description for file results

{{{lp-configure-each-sb}}}

file-ext

Specify an extension for file output

{{{lp-configure-each-sb}}}

hlines

Handle horizontal lines in tables

{{{lp-configure-each-sb}}}

output-dir

Specify a directory to write file output to

{{{lp-configure-each-sb}}}

One example is a System where all intermediate results are stored to individual files.

post

Post processing of code block results

{{{lp-configure-each-sb}}}

prologue

Text to prepend to code block body

{{{lp-configure-each-sb}}}

For some situations, this may be the same for every source block for a particular language. The user manual described gnuplot, which often shows up on the list and the solution is to reset the session.

Another example, say that you’ve got a bunch of R Source-Blocks and you want to be able to rearrange them as you please. You want to be sure that there are no dependencies between them on bindings created in the workspace. Set prologue to rm(list = ls()).

Epilgue works hand-in-hand with this.

results

Specify the type of results and how they will be collected and handled

Ways to configure :results: src_emacs-lisp{(apply ‘* (-keep ‘cdr ‘((Collection . 2) (Type . 4) (Format . 7) (Handling . 4))))} {{{results(224)}}}.

This system stores the results of evaluation in the source document. It believes that the results are critical to the research.

Keep the document as close to being executable as possible; make it very visible when it is not.

Collection
- value: Functions have a single result. So do Source-Blocks.
Type
- scalar
  - Functions always return a single result
  - Evidence demonstrates that I use this or output most of the time and I want to configure this to work right for Literate Programming by default because it feels better.
- WAS
  - Because in theory returning a collection was flexible (see below). In practice I never ever used this.
  - table:
    - Tables are the best type because
      - Dimensions make them human-readable in text.
      - Work with Babel LP.
      - Appear as lists to programming languages.
      - Weaves well.
      - Inline Source-Blocks disallow tables so use scalars instead.
Format
- drawer: Enable results replacement
Handling
- replace: Replace them each time you evaluate the block.

(defconst help/org-sb-results-cfg "value scalar drawer replace")
(help/set-org-babel-default-header-args :results help/org-sb-results-cfg)

Their format will show that they are results. Inline source blocks automatically get formatted as verbatim. For some reason, this only needs to be configured as replace to work unlike normal source blocks. Copying the configuration from normal source blocks here breaks the replacement functionality.

(defconst help/org-isb-results-cfg "replace")
(help/set-org-babel-default-inline-header-args :results help/org-isb-results-cfg)

rownames

Handle row names in tables

{{{lp-configure-each-sb}}}

sep

Delimiter for writing tabular results outside Org

{{{lp-configure-each-sb}}}

var

Pass arguments to code blocks

The most revealing of the power of Org-Mode’s LP offering
Values-by-reference
- Table
- List
- Source-Block without and with parameters
- Literal-Block
Idexable variable values
Emacs Lisp evaluation of variables

Weaving

Help the reader make sense of the document by displaying it’s internal properties.

(setq org-export-with-properties t)

Stop your flow to monitor the export for errors
- <2016-01-19 Tue> Expect it to start weaves for all weavers asynchronously. Does not do so; main thread is blocked until weaves complete.

(setq org-export-in-background nil)

Make sure that exported files are Unicode UTF-8.

(setq org-export-coding-system 'utf-8)

Line breaks are for humans typing them, not for publishing.

When publishing to ASCII, set this property in the file.

(setq org-export-preserve-breaks nil)

When exporting anything, do not insert the exported content into the kill ring.

(setq org-export-copy-to-kill-ring nil)

By default I never want a table of contents generated. It is so easy to enable it with a property, it will be fine to turn it off.

(setq org-export-with-toc nil)

On export, maintain the literal spacing as found in the source block. Obviously this is important for make-files. It is really important everywhere because anything else would violate the law of least surprise.

(setq org-src-preserve-indentation t)

Maximize flexibility for weaving operations during export.

(setq org-export-allow-bind-keywords t)

Disable element caching because it might break weaves via this thread.

(setq org-element-use-cache nil)

exports

Export code and/or results

Always share source blocks and their results. Whether or not to generate a result for a particular source block is configured per-block. If you don’t want to share a result for a source block then disable storage of results on that block.

(help/set-org-babel-default-header-args :exports "both")

Use inline Source-Blocks to provide values read as part of the document. Don’t show their source code. Allows inline Source-Blocks to function as rich macros when combined with org-sbe.

(help/set-org-babel-default-inline-header-args :exports "results")

wrap

Mark source block evaluation results

Inline-Source-Blocks are recognizable by their verbatim font. They do not interrupt the flow. Source-Blocks are their own entities. They stand out. Their results need to be visibly noticeably different for the reader by making them EXAMPLE special blocks.

(help/set-org-babel-default-header-args :wrap "EXAMPLE")

Diagramming languages require RESULTS output for exporting.

(help/set-org-babel-default-header-args:ditaa :wrap "RESULTS")
(help/set-org-babel-default-header-args:dot :wrap "RESULTS")
(help/set-org-babel-default-header-args:plantuml :wrap "RESULTS")

Files

Org-Mode_Fundamentals.org

Latest commit

History

Org-Mode_Fundamentals.org

File metadata and controls

Org-Mode Fundamentals

Literate Programming

Helper Functions

Identity

Tangling

comments

mkdirp

no-expand

noweb

noweb-ref

noweb-sep

padline

session

shebang

tangle

tangle-mode

Evaluating

cache

colnames

dir

epilogue

eval

file

file-desc

file-ext

hlines

output-dir

post

prologue

results

rownames

sep

var

Weaving

exports

wrap