Integrering med den digitala sångboken. (#4)

* Standardiserade titlarna.

* Removed logs

* Moved .sty files to main dir.

* Update readme.

* α is now up to spec.

* Beta is now up to spec.

* Kapitelnamn är up-to-spec.

* Gamma is up-to-spec.

* Delta titles up to spec.

* Delta is now up-to-spec

* Song titles up-to-spec.

* Authors & epsilon now up-to-spec.

* Added added compilation workflow.

* Fixed a compilation error.

* Zeta - Iota up to spec.

* Others (not omega) up to spec.

* Parser prototype.

* Added json parser workflow.

* Parser RegEx corrections.

* Updated parser.

* Minor parser improvements.

* Updated parser.

* Update readme.

* Fixed a messed-up regex bug.

* Updated readme.

* Fixed a bug causing tex not to compile.

* Updated workflows

* Added lint for text size adjustment within lyrics

* \small is now (almost) only used below chapter title.

* Added png generator for latex build testing.

* Added png comparator to workflow.

* Updated image comparison workflow.

* Updated compile workflow.

* Now uses TEXINPUTS env variable for .sty reference.

* Fixed a sidspaltHack bug (hopefully)

* Fixed layout bugs.

* Spacing fixes.

* Minor changes.

* Fixed a chapter title spacing bug.

* Update readme.

* Updated img.sh res.

* Set image compare workflow res to 90.

* Marginaler är inte längre hårdkodade (ej 0, μ, σ)

* Eta is now equivalent.

* Alfa & Kappa are now complete

* Minor changes.

* Beta complete (ish).

* Most chapters complete.

* Added PNG diff workflow for orignal PDF:s.

* Update readme.

* Minor fixes.

* Layout fixes.

* Failed parser overhaul.

* Now uses pylatexenc for parsing.

* Added parser "tests" (manual"

* Lots of small error fixes.

* Updated parser.

* More minor changes.

* More changes.

* More minor changes.

* Even more small changes.

* Parsing complete.

* Minor layout fixes.

* Workflow fix

* Moved some stuff.

* Added sigma 18 & 19 to JSON.

* Added hard-coded misc. texts.

* Fixed an injection ordering bug.

* Minor fixes.

* Cleanup

* Fixed table of contents

.github/assets/Readme.txt

@@ -0,0 +1,3 @@
+original.90.txz - compiled on Ubuntu WSL from the original pdf:s.
+original.90c.txz - compiled using GitHub workflows from original tex-files.
+original.png.txz - same as original.90.tex, except with 45dpi instead of 90.

.github/scripts/compile.sh

@@ -0,0 +1,22 @@
+#!/usr/bin/env bash
+
+export TEXINPUTS=::$(pwd)
+
+# We remove aux files to get a clean compilation.
+rm **/*.aux
+
+# This script compiles all .tex-files in the main directory (that is, not in Äldre/)
+for dir in *\ -\ */
+do
+    echo -e "\e[1;36mBearbetar mapp:\e[0m $dir";
+    cd "$dir";
+
+    for file in *.tex
+    do
+        echo -e " - \e[35mBearbetar fil:\e[0m $dir$file"
+        # First run generates the aux file, then used by the second run.
+        pdflatex -halt-on-error "$file" ||  { echo -e "\e[31mFel vid bearbetning av:\e[0m $dir/$file" ; exit 1; }
+        pdflatex -halt-on-error "$file" ||  { echo -e "\e[31mFel vid bearbetning av:\e[0m $dir/$file" ; exit 1; }
+    done;
+    cd ..;
+done

.github/scripts/img.sh

@@ -0,0 +1,22 @@
+#!/usr/bin/env bash
+
+# Pre-cleanup
+#echo -e "\e[1;36mRensar upp\e[0m...";
+rm **/*.pdf-*.png
+
+# This script compiles all .pdf-files in the main directory to png (that is, not in Äldre/)
+for dir in *\ -\ */
+do
+    echo -e "\e[1;36mBearbetar mapp:\e[0m $dir";
+    cd "$dir";
+
+    for file in *.pdf
+    do
+        echo -e " - \e[35mBearbetar fil:\e[0m $dir$file"
+        pdftoppm $file $file -png -r 90
+    done;
+    cd ..;
+done
+#echo -e "\e[1;36mKomprimerar\e[0m till png.txz";
+#echo -e "\e[1;36mKomprimerar\e[0m till png.txz";
+#tar -cJvf png.txz **/*.pdf-*.png

.github/scripts/img_cmp_pdf.sh

@@ -0,0 +1,50 @@
+#!/usr/bin/env bash
+
+echo -e "\e[1;36mExtraherar\e[0m originalbilder...";
+mkdir -p cmp/
+echo -e "Jämför med original\e[1;36mPDF\e[0mer.";
+tar -xJ -C cmp/ -f .github/assets/original.90.txz
+
+
+# Remove old diff files.
+rm **/*.pdf-*diff.png
+
+AE_TOT=0
+FILE_COUNT=0
+NF_FILE_COUNT=0
+NE_FILE_COUNT=0
+
+for file in *\ -\ */*.pdf-*.png
+do
+    if [ ! -f "cmp/$file" ]; then
+        echo -e "  \e[31mFEL:\e[0m Antalet png-filer skiljer sig från originalet."
+        NF_FILE_COUNT=$((NF_FILE_COUNT + 1))
+        continue
+    fi
+
+    AE=$(compare -metric AE "$file" "cmp/$file" "$file.diff.png" &> /dev/stdout)
+    AE_TOT=$((AE_TOT + AE))
+    FILE_COUNT=$((FILE_COUNT + 1))
+
+    if [[ "$AE" -gt 0 ]]; then
+        NE_FILE_COUNT=$((NE_FILE_COUNT + 1))
+        echo -e "\e[35mFil:\e[0m $file "
+        if [[ "$AE" -gt 20000 ]]; then
+            echo -e "  Absolut fel: \e[31m$AE\e[0m"
+        else
+            echo "  Absolut fel: $AE"
+        fi
+    else
+        rm "$file.diff.png"
+    fi
+
+done;
+
+echo "Files checked: $FILE_COUNT"
+echo "Files not found in original: $NF_FILE_COUNT"
+echo "Files not matching: $NE_FILE_COUNT"
+echo "Total Absolute Error: $AE_TOT"
+
+if [[ "$NF_FILE_COUNT" -gt 1 ]]; then
+    exit 1
+fi

.github/scripts/img_cmp_tex.sh

@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+
+echo -e "\e[1;36mExtraherar\e[0m originalbilder...";
+mkdir -p cmp/
+echo -e "Jämför med ursprungliga \e[1;36mTeX\e[0mfiler.";
+tar -xJ -C cmp/ -f .github/assets/original.90c.txz
+
+# Remove old diff files.
+rm **/*.pdf-*diff.png
+
+AE_TOT=0
+FILE_COUNT=0
+NF_FILE_COUNT=0
+NE_FILE_COUNT=0
+
+for file in *\ -\ */*.pdf-*.png
+do
+    if [ ! -f "cmp/$file" ]; then
+        echo -e "  \e[31mFEL:\e[0m Antalet png-filer skiljer sig från originalet."
+        NF_FILE_COUNT=$((NF_FILE_COUNT + 1))
+        continue
+    fi
+
+    AE=$(compare -metric AE "$file" "cmp/$file" "$file.diff.png" &> /dev/stdout)
+    AE_TOT=$((AE_TOT + AE))
+    FILE_COUNT=$((FILE_COUNT + 1))
+
+    if [[ "$AE" -gt 0 ]]; then
+        NE_FILE_COUNT=$((NE_FILE_COUNT + 1))
+        echo -e "\e[35mFil:\e[0m $file "
+        if [[ "$AE" -gt 20000 ]]; then
+            echo -e "  Absolut fel: \e[31m$AE\e[0m"
+        else
+            echo "  Absolut fel: $AE"
+        fi
+    else
+        rm "$file.diff.png"
+    fi
+
+done;
+
+echo "Files checked: $FILE_COUNT"
+echo "Files not found in original: $NF_FILE_COUNT"
+echo "Files not matching: $NE_FILE_COUNT"
+echo "Total Absolute Error: $AE_TOT"
+
+if [[ "$NF_FILE_COUNT" -gt 1 ]]; then
+    exit 1
+fi

.github/workflows/compile.yml

@@ -0,0 +1,99 @@
+name: Build LaTeX documents
+
+on:
+  push:
+    paths:
+      - '**.tex'
+      - '**.sty'
+      - '**.cls'
+      - '**/compile.yml'
+  workflow_dispatch:
+
+jobs:
+  build-latex:
+    runs-on: ubuntu-latest
+    name: Compile LaTeX to PDF
+    steps:
+      - name: Set up Git repository
+        uses: actions/checkout@v2
+      - name: Install texlive
+        run: sudo apt-get update && sudo apt-get install -y texlive texlive-binaries texlive-fonts-recommended texlive-lang-european texlive-latex-base texlive-latex-extra
+      - name: Compile LaTeX
+        run: .github/scripts/compile.sh
+      - name: Archive pdf:s
+        uses: actions/upload-artifact@v2
+        with:
+          name: pdfs
+          path: |
+            ./*/*.pdf
+      - name: Archive texlive logs
+        uses: actions/upload-artifact@v2
+        with:
+          name: texlive-logs
+          path: |
+            ./*/*.log
+
+  # We build a bunch of low-res png:s in order to compare them to the original ones.
+  generate-pngs:
+    runs-on: ubuntu-latest
+    needs: build-latex
+    name: Generate PNGs from PDFs
+    steps:
+      - name: Set up Git repository
+        uses: actions/checkout@v2
+      - name: Install poppler-utils
+        run: sudo apt-get install -y poppler-utils
+      - name: Fetch generated pdf:s
+        uses: actions/download-artifact@v2
+        with:
+          name: pdfs
+      - name: Generate png:s from pdf:s
+        run: ./.github/scripts/img.sh
+      - name: Archive png:s
+        uses: actions/upload-artifact@v2
+        with:
+          name: pngs
+          path: |
+            **/*.pdf-*.png
+
+  # compare-pngs-tex:
+  #   runs-on: ubuntu-latest
+  #   needs: generate-pngs 
+  #   name: Compare PNGs to PNGs from original TeX-files
+  #   steps:
+  #     - name: Set up Git repository
+  #       uses: actions/checkout@v2
+  #     - name: Fetch generated png:s
+  #       uses: actions/download-artifact@v2
+  #       with:
+  #         name: pngs
+  #     - name: Compare png:s to original png:s
+  #       run: ./.github/scripts/img_cmp_tex.sh
+  #     - name: Archive diff png:s
+  #       uses: actions/upload-artifact@v2
+  #       with:
+  #         name: diff-pngs-tex
+  #         path: |
+  #           **/*.diff.png
+
+  # compare-pngs-pdf:
+  #   runs-on: ubuntu-latest
+  #   needs: generate-pngs
+  #   name: Compare PNGs to PNGs from original PDF-files
+  #   steps:
+  #     - name: Set up Git repository
+  #       uses: actions/checkout@v2
+  #     - name: Fetch generated png:s
+  #       uses: actions/download-artifact@v2
+  #       with:
+  #         name: pngs
+  #     - name: Compare png:s to original png:s
+  #       run: ./.github/scripts/img_cmp_pdf.sh
+  #       env:
+  #         USE_ORIGINAL_PDFS: true
+  #     - name: Archive diff png:s
+  #       uses: actions/upload-artifact@v2
+  #       with:
+  #         name: diff-pngs-pdf
+  #         path: |
+  #           **/*.diff.png

.github/workflows/json-parse.yml

@@ -0,0 +1,31 @@
+# This is a basic workflow to help you get started with Actions
+
+name: Parse into JSON
+
+# Controls when the workflow will run
+on:
+  push:
+    paths:
+      - '**.tex'
+      - '**.py'
+      - '**/json-parse.yml'
+
+jobs:
+  parse-to-json:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v2
+      - name: Install pylatexenc
+        run: sudo pip3 install pylatexenc
+      - name: Run JSON converter (experimental)
+        run: python3 main.py
+        working-directory: parser
+      - name: Archive production artifacts
+        uses: actions/upload-artifact@v2
+        with:
+          name: out.json
+          path: parser/out.json
+      - name: Compare output to target
+        run: python3 compare.py
+        working-directory: parser

.gitignore

@@ -0,0 +1,12 @@
+.vscode
+*.log
+*.aux
+*.pyc
+*.synctex.gz
+
+__pycache__
+out.json
+
+*.pdf-*.png
+*.diff.png
+cmp