Reworking cleopatra presentation

1 files changed, 344 insertions, 0 deletions

diff --git a/site/cleopatra/Bootstrap.org b/site/cleopatra/Bootstrap.org
new file mode 100644
index 0000000..a783f8f
--- /dev/null
+++ b/site/cleopatra/Bootstrap.org
@@ -0,0 +1,344 @@
+#+BEGIN_EXPORT html
+<h1>Bootstrapping an Extensible Toolchain</h1>
+#+END_EXPORT
+
+A literate program is a particular type of software program where code is not
+directly written in source files, but rather in text document as code
+snippets. In some sense, literate programming allows for writing in the same
+place both the software program and its technical documentation.
+
+That being said, *~cleopatra~* is a toolchain to build a website before being a
+literate program, and one of its objective is to be /part of this very website
+it is used to generate/. To acheive this, *~cleopatra~* has been written as a
+collection of org files which can be either “tangled” using [[https://orgmode.org/worg/org-contrib/babel/][Babel]] or “exported”
+as a HTML document. Tangling here refers to extracted marked code blocks into
+files.
+
+The page you are currently reading is *~cleopatra~* entry point. Its primilarly
+purpose is to introduce two Makefiles: ~Makefile~ and ~bootstrap.mk~.
+
+#+TOC: headlines 2
+
+* The Root of Generation
+
+~Makefile~ serves two purposes: it initiates a few global variables, and it
+provides a rule to generate ~bootstrap.mk~.  At this point, some readers may
+wonder /why/ we need ~Makefile~ in this context, and the motivation behind this
+choice is really reminescent of a boot sequence. The rationale is that we need a
+“starting point” for *~cleopatra~*. The toolchain cannot live solely inside
+org-files, otherwise there would not have any code to execute the first time we
+tried to generate the website. We need an initial Makefile, one that has little
+chance to change, so that we can almost consider it read-only. Contrary to the
+other Makefiles that we will generate, this one will not be deleted by ~make
+clean~.
+
+This is similar to your computer: it requires a firmware to boot, whose purpose
+—in a nutshell— is to find and load an operating system.
+
+Modifying the content of ~Makefile~ in this document /will/ modify
+~Makefile~. This means one can easily put *~cleopatra~* into an inconsistent
+state, which would prevent further generation. This is why the generated
+~Makefile~ should be versioned, so that you can restore it using ~git~ if you
+made a mistake when you modified it.
+
+For readers interested in using *~cleopatra~* for their own websites, this
+documents tries to highlight the potential modifications they would have to
+make.
+
+** Global Constants and Variables
+
+First, ~Makefile~ defines several global “constants” (although as far as I know
+~make~ does not support true constant values, it is expected further generation
+process will not modify them).
+
+In a nutshell,
+
+- ~ROOT~ ::
+  Tell Emacs where the root of your website sources is, so that tangled output
+  filenames can be given relative to it rather than the org files.  So for
+  instance, the ~BLOCK_SRC~ headers for ~Makefile~ looks like
+
+  #+BEGIN_SRC org
+  #+BEGIN_SRC makefile :tangle Makefile :noweb tangle
+  #+END_SRC
+
+  instead of, /e.g./,
+
+  #+BEGIN_SRC org
+  #+BEGIN_SRC makefile :tangle ../../../Makefile :noweb tangle
+  #+END_SRC
+
+- ~CLEODIR~ ::
+  Tell *~cleopatra~* where its sources live. If you place it inside the ~site/~
+  directory (as it is intended), and you enable the use of ~org~ files to author
+  your contents, then *~cleopatra~* documents will be part of your website. If
+  you don’t want that, just move the directory outside the ~site/~ directory,
+  and update the ~CLEODIR~ variable accordingly.
+
+For this website, these constants are defined as follows.
+
+#+BEGIN_SRC makefile :tangle Makefile :noweb tangle
+ROOT := $(shell pwd)
+CLEODIR := site/cleopatra
+#+END_SRC
+
+We then introduce a variable that “generation” components will populate with
+their output files (using ~+=~).
+
+- ~GENFILES~ ::
+  List *~cleopatra~* Makefiles and scripts tangled throughout the generation
+  process (with the notable exception of ~Makefile~ itself).
+- ~GENSASS~ ::
+  List auxiliary ~sass~ files which can later be imported by the main ~sass~
+  files (see [[./Theme.org][“Theming and Templating”]]).
+- ~CONTENTS~ ::
+  List generated files which are part of the target website, and acts as inputs
+  for ~soupault~.
+
+#+BEGIN_SRC makefile :tangle Makefile :exports none
+GENFILES :=
+CONTENTS :=
+GENSASS :=
+#+END_SRC
+
+** Easy Tangling of Org Documents
+
+We provide the necessary bits to easily tangle Org documents.
+
+The configuration of Babel is done using an emacs lisp script called
+~tangle-org.el~ whose status is similar to ~Makefile~. It is part of the
+bootstrap process, and therefore lives “outside” of *~cleopatra~* (it is not
+deleted with ~make clean~ for instance).  However, it is overwritten. If you try
+to modify it and find that *~cleopatra~* does not work properly, you should
+restore it using ~git~.
+
+#+BEGIN_SRC emacs-lisp :tangle scripts/tangle-org.el
+(require 'org)
+(cd (getenv "ROOT"))
+(setq org-confirm-babel-evaluate nil)
+(setq org-src-preserve-indentation t)
+(org-babel-do-load-languages
+ 'org-babel-load-languages
+ '((shell . t)))
+(org-babel-tangle)
+#+END_SRC
+
+We define variables that ensure that the ~ROOT~ environment variable is set and
+~tangle-org.el~ is loaded when using Emacs. You can modify ~EMACS~ to use a
+custom Emacs that you build yourself if you so desire.
+
+#+BEGIN_SRC makefile :tangle Makefile :noweb tangle
+EMACSBIN := emacs
+EMACS := ROOT="${ROOT}" ${EMACSBIN}
+TANGLE := --batch --load="${ROOT}/scripts/tangle-org.el" 2>> build.log
+#+END_SRC
+
+** Bootstrapping
+
+The core purpose of ~Makefile~ remains *(1)* to bootstrap the generation process
+by generating and loading ~bootstrap.mk~, and *(2)* to enforce the ~build~ rules
+hopefully defined by the latter is called.
+
+For *(2)*, we introduce a ~default~ rule with ~build~ as a
+dependency.
+
+#+BEGIN_SRC makefile :tangle Makefile :noweb tangle
+default: init-log build
+
+init-log:
+	@echo "==============[CLEOPATRA BUILD LOG]==============" \
+	    > build.log
+
+.PHONY: init-log default build
+#+END_SRC
+
+For *(1)*, we rely on a particular behavior of ~make~ regarding the ~include~
+directive. If an operand of ~include~ does not yet exists, ~make~ will search
+for a rule to generate it.
+
+Basically, we are looking for recipes of the following form:
+
+#+BEGIN_SRC makefile :noweb yes
+<<extends(MK="${MK}", MF="${MF}", IN="${IN}", GF="${GF}", GS="${GS}")>>
+#+END_SRC
+
+where
+
+- ~${IN}~ is the input Org document
+- ~${MK}~ lists the tangled Makefiles (typically one, but it could be more)
+- ~${GF}~ lists the tangled scripts
+- ~${GS}~ lists the tangled SASS scripts
+
+~&:~ is used in place of ~:~ to separate the target from its dependencies in
+this rule to tell to ~make~ that the runned commands will generate all these
+files.
+
+Writing these rules manually —has yours truly had to do in the early days of his
+website— has proven to be error-prone.
+
+One desirable feature for *~cleopatra~* would be to generate them automatically,
+by looking for relevant ~:tangle~ directives inside the input Org document. The
+challenge lies in the “relevant” part: the risk exists that we have false
+posivite. However and as a first steps towards a fully automated solution, we
+can leverage the evaluation features of Babel here.
+
+Here is a bash script which, given the proper variables, would generate the
+expected Makefile rule.
+
+#+NAME: extends
+#+BEGIN_SRC bash :var MK="" :var IN="" :var GF="" :var GS="" :results output
+cat <<EOF
+GENFILES += ${MK} ${GF}
+GENSASS += ${GS}
+
+include ${MK}
+
+${MK} ${GF} ${GS} \\
+  &: \${CLEODIR}/${IN}
+	@echo "  tangle  \$<"
+	@\${EMACS} $< \${TANGLE}
+EOF
+#+END_SRC
+
+The previous source block is given a name (=extends=), and an explicit lists of
+variables (~IN~, ~MK~, ~GF~, and ~GS~). Thanks to the [[https://orgmode.org/worg/org-tutorials/org-latex-export.html][noweb syntax of Babel]], we
+can insert the result of the evaluation of =extends= inside another source block
+when the latter is tangled.
+
+The twist is, we derive the rule to tangle ~bootstrap.mk~ using
+=extends=. The syntax is the following:
+
+#+BEGIN_SRC verbatim
+<<extends(IN="Bootstrap.org", MK="bootstrap.mk", GF="scripts/update-gitignore.sh")>>
+#+END_SRC
+
+For purpose of illustrations, here is the snippet generated by Babel from the
+previous source block.
+
+#+BEGIN_SRC makefile :tangle Makefile :noweb yes
+<<extends(IN="Bootstrap.org", MK="bootstrap.mk", GF="scripts/update-gitignore.sh")>>
+#+END_SRC
+
+Beware that, as a consequence, modifying code block of =extends= is as
+“dangerous” as modifying ~Makefile~ itself. Keep that in mind if you start
+hacking *~cleopatra~*!
+
+From now on, the bootstrap process is completed: further generation processes
+will fully be defined using literate programming, with no special treatment for
+its output. For instance, you may not want to use ~soupault~? You can! Just
+modify ~bootstrap.mk~ accordingly.
+
+* Generation Processes
+
+Thanks to =extends=, *~cleopatra~* is easily extensible. In this section, we
+enumerate the generation processes that are currently used to generate the
+website you are reading.
+
+** Authoring Contents
+
+The fact that *~cleopatra~* is a literate program which gradually generates
+itself was not intended: it is a consequence of my desire to be able to easily
+use whatever format I so desire for writing my contents, and Org documents in
+particular.
+
+In the present website, contents can be written in the following format:
+
+- HTML Files ::
+  This requires no particular set-up, since HTML is the /lingua franca/ of
+  ~soupault~.
+- Regular Coq files ::
+  Coq is a system which allows to write machine-checked proofs, and it comes
+  with a source “prettifier” called ~coqdoc~.
+  [[./Contents/Coq.org][Learn more about the generation process for Coq files​]]
+- Org documents ::
+  Emacs comes with a powerful editing mode called [[https://orgmode.org/][Org mode]], and Org documents
+  are really pleasant to work with.
+  [[./Contents/Org.org][Learn more about the generation process for Org documents]]
+
+If you want *~cleopatra~* to support more input formats, you have to
+
+1. Create an Org document which, once tangled, provides a dedicated makefile
+2. Edit this file (~Bootstrap.org~) here, and use =extends= to make sure it
+   is actually tangled when necessary
+
+#+BEGIN_SRC makefile :tangle bootstrap.mk :noweb tangle :exports none
+<<extends(MK="coq.mk", IN="Contents/Coq.org", GS="site/style/coq.sass")>>
+<<extends(MK="org.mk", IN="Contents/Org.org", GF="scripts/export-org.el emacs.d", GS="site/style/org.sass")>>
+#+END_SRC
+
+** Postprocessing HTML using ~soupault~
+
+The drawback of using different input formats and generators (~coqdoc~, Org,
+etc.) is the heterogeneity of the outputted HTML. This is why *~cleopatra~*
+started using ~soupault~. You can read more about [[./Soupault.org][how the ~soupault~
+configuration of the present website in the dedicated document]].
+
+#+BEGIN_SRC makefile :tangle bootstrap.mk :noweb tangle :exports none
+<<extends(IN="Soupault.org", GF="soupault.conf")>>
+#+END_SRC
+
+Since ~soupault.conf~ is an input for ~soupault~, we explicitely add it to the
+~CONTENTS~ variables.
+
+#+BEGIN_SRC makefile :tangle bootstrap.mk
+CONTENTS += soupault.conf
+#+END_SRC
+
+** Theming and Templating
+
+The last missing piece is the appearance of the website. By default, ~soupault~
+assumes there exists a template available (~templates/main.html~). You can read
+more about [[./Theme.org][the structure of this template and how its companion CSS file is
+generated in the appropriate document]].
+
+#+BEGIN_SRC makefile :tangle bootstrap.mk :noweb tangle :exports none
+<<extends(MK="theme.mk", IN="Theme.org", GF="templates/main.html", GS="site/style/main.sass")>>
+#+END_SRC
+
+** Wrapping-up
+
+#+BEGIN_SRC makefile :tangle bootstrap.mk
+build : ${CONTENTS}
+	@echo "     run  soupault"
+	@soupault
+	@echo "  update  .gitignore"
+	@scripts/update-gitignore.sh ${CONTENTS} ${GENFILES} ${GENSASS}
+#+END_SRC
+
+#+BEGIN_SRC bash :tangle scripts/update-gitignore.sh :tangle-mode (identity #o755)
+#!/bin/bash
+
+BEGIN_MARKER="# begin generated files"
+END_MARKER="# begin generated files"
+
+# remove the previous list of generated files to ignore
+sed -i -e "/${BEGIN_MARKER}/,/${END_MARKER}/d" .gitignore
+# remove trailing empty lines
+sed -i -e :a -e '/^\n*$/{$d;N;};/\n$/ba' .gitignore
+
+# output the list of files to ignore
+echo "" >> .gitignore
+echo ${BEGIN_MARKER} >> .gitignore
+for f in $@; do
+    echo "${f}" >> .gitignore
+done
+echo ${END_MARKER} >> .gitignore
+#+END_SRC
+
+#+BEGIN_SRC makefile :tangle bootstrap.mk
+serve :
+	@echo "   start  a python server"
+	@cd build; python -m http.server 2>/dev/null
+
+clean :
+	@echo "  remove  generated files"
+	@rm -rf ${CONTENTS} ${GENFILES} build/
+
+force : clean build
+
+.PHONY : serve clean force build
+#+END_SRC
+
+# Local Variables:
+# org-src-preserve-indentation: t
+# End: