diff options
Diffstat (limited to 'site/cleopatra/Bootstrap.org')
| -rw-r--r-- | site/cleopatra/Bootstrap.org | 396 |
1 files changed, 0 insertions, 396 deletions
diff --git a/site/cleopatra/Bootstrap.org b/site/cleopatra/Bootstrap.org deleted file mode 100644 index 0a2f821..0000000 --- a/site/cleopatra/Bootstrap.org +++ /dev/null @@ -1,396 +0,0 @@ -#+BEGIN_EXPORT html -<h1>Bootstrapping an Extensible Toolchain</h1> -#+END_EXPORT - -A literate program is a particular type of software program where code is not -directly written in source files, but rather in text document as code -snippets. In some sense, literate programming allows for writing in the same -place both the software program and its technical documentation. - -That being said, *~cleopatra~* is a toolchain to build a website before being a -literate program, and one of its objective is to be /part of this very website -it is used to generate/. To acheive this, *~cleopatra~* has been written as a -collection of org files which can be either “tangled” using -[[https://orgmode.org/worg/org-contrib/babel/][Babel]] or “exported” as a HTML -document. Tangling here refers to extracted marked code blocks into files. - -The page you are currently reading is *~cleopatra~* entry point. Its primilarly -purpose is to introduce two Makefiles: ~Makefile~ and ~bootstrap.mk~. - -#+TOC: headlines 2 - -* The Root of Generation - -~Makefile~ serves two purposes: it initiates a few global variables, and it -provides a rule to generate ~bootstrap.mk~. At this point, some readers may -wonder /why/ we need ~Makefile~ in this context, and the motivation behind this -choice is really reminescent of a boot sequence. The rationale is that we need a -“starting point” for *~cleopatra~*. The toolchain cannot live solely inside -org-files, otherwise there would not have any code to execute the first time we -tried to generate the website. We need an initial Makefile, one that has little -chance to change, so that we can almost consider it read-only. Contrary to the -other Makefiles that we will generate, this one will not be deleted by ~make -clean~. - -This is similar to your computer: it requires a firmware to boot, whose purpose -—in a nutshell— is to find and load an operating system. - -Modifying the content of ~Makefile~ in this document /will/ modify -~Makefile~. This means one can easily put *~cleopatra~* into an inconsistent -state, which would prevent further generation. This is why the generated -~Makefile~ should be versioned, so that you can restore it using ~git~ if you -made a mistake when you modified it. - -For readers interested in using *~cleopatra~* for their own websites, this -documents tries to highlight the potential modifications they would have to -make. - -** Global Constants and Variables - -First, ~Makefile~ defines several global “constants” (although as far as I know -~make~ does not support true constant values, it is expected further generation -process will not modify them). - -In a nutshell, - -- ~ROOT~ :: - Tell Emacs where the root of your website sources is, so that tangled output - filenames can be given relative to it rather than the org files. So for - instance, the ~BLOCK_SRC~ tangle parameter for ~Makefile~ looks like ~:tangle - Makefile~, instead of ~:tangle ../../Makefile~. -- ~CLEODIR~ :: - Tell *~cleopatra~* where its sources live. If you place it inside the ~site/~ - directory (as it is intended), and you enable the use of ~org~ files to author - your contents, then *~cleopatra~* documents will be part of your website. If - you don’t want that, just move the directory outside the ~site/~ directory, - and update the ~CLEODIR~ variable accordingly. - -For this website, these constants are defined as follows. - -#+BEGIN_SRC makefile :tangle Makefile :noweb no-export -ROOT := $(shell pwd) -CLEODIR := site/cleopatra -#+END_SRC - -We then introduce two variables to list the output of the generation processes, -with two purposes in mind: keeping the ~.gitignore~ up-to-date automatically, -and providing rules to remove them. - -- ~ARTIFACTS~ :: - Short-term artifacts which can be removed frequently without too much - hassle. They will be removed by ~make clean~. -- ~CONFIGURE~ :: - Long-term artifacts whose generation can be time consuming. They will only be - removed by ~make cleanall~. - -#+BEGIN_SRC makefile :tangle Makefile -ARTIFACTS := build.log -CONFIGURE := -#+END_SRC - -Generation processes shall declare new build outputs using the ~+=~ assignement -operators. Using another operator will likely provent an underisable result. - -** Easy Tangling of Org Documents - -*~cleopatra~* is a literate program implemented with Org mode, an Emacs major -editing mode. We provide the necessary bits to easily tangle Org documents. - -The configuration of Babel is done using an emacs lisp script called -~tangle-org.el~ whose status is similar to ~Makefile~. It is part of the -bootstrap process, and therefore lives “outside” of *~cleopatra~* (it is not -deleted with ~make clean~ for instance). However, it is overwritten. If you try -to modify it and find that *~cleopatra~* does not work properly, you should -restore it using ~git~. - -#+BEGIN_SRC emacs-lisp :tangle scripts/tangle-org.el -(require 'org) -(cd (getenv "ROOT")) -(setq org-confirm-babel-evaluate nil) -(setq org-src-preserve-indentation t) -(add-to-list 'org-babel-default-header-args - '(:mkdirp . "yes")) -(org-babel-do-load-languages - 'org-babel-load-languages - '((shell . t))) -(org-babel-tangle) -#+END_SRC - -We define variables that ensure that the ~ROOT~ environment variable is set and -~tangle-org.el~ is loaded when using Emacs. - -#+BEGIN_SRC makefile :tangle Makefile -EMACSBIN := emacs -EMACS := ROOT="${ROOT}" ${EMACSBIN} -TANGLE := --batch \ - --load="${ROOT}/scripts/tangle-org.el" \ - 2>> build.log -#+END_SRC - -Finally, we introduce a -[[https://www.gnu.org/software/make/manual/html_node/Canned-Recipes.html#Canned-Recipes][canned -recipe]] to seamlessly tangle a given file. - -#+BEGIN_SRC makefile :tangle Makefile -define emacs-tangle = -echo " tangle $<" -${EMACS} $< ${TANGLE} -endef -#+END_SRC - -** Bootstrapping - -The core purpose of ~Makefile~ remains to bootstrap the chain of generation -processes. This chain is divided into three stages: ~prebuild~, ~build~, and -~postbuild~. - -This translates as follows in ~Makefile~. - -#+BEGIN_SRC makefile :tangle Makefile -default : postbuild ignore - -init : - @rm -f build.log - -prebuild : init - -build : prebuild - -postbuild : build - -.PHONY : init prebuild build postbuild ignore -#+END_SRC - -A *generation process* in *~cleopatra~* is a Makefile which provides rules for -these three stages, along with the utilities used by these rules. More -precisely, a generation process ~proc~ is defined in ~proc.mk~. The rules of -~proc.mk~ for each stage are expected to be prefixed by ~proc-~, /e.g./, -~proc-prebuild~ for the ~prebuild~ stage. - -Eventually, the following dependencies are expected between within the chain of -generation processes. - -#+BEGIN_SRC makefile -prebuild : proc-prebuild -build : proc-build -postbuild : proc-postbuild - -proc-build : proc-prebuild -proc-postbuild : proc build -#+END_SRC - -Because *~cleopatra~* is a literate program, generation processes are defined in -Org documents –which may contains additional utilities like scripts or -templates—, and therefore need to be tangled prior to be effectively -useful. *~cleopatra~ relies on a particular behavior of ~make~ regarding the -~include~ directive. If there exists a rule to generate a Makefile used as an -operand of ~include~, ~make~ will use this rule to update (if necessary) said -Makefile before actually including it. - -Therefore, rules of the following form achieve our ambition of extensibility. - -#+BEGIN_SRC makefile :noweb yes -<<extends(PROC="${PROC}", IN="${IN}", AUX="${AUX}")>> -#+END_SRC - -where - -- ~${IN}~ is the Org document which contains the generation process code -- ~${PROC}~ is the name of the generation process -- ~${AUX}~ lists the utilities of the generation process tangled from ~${IN}~ - with ~${PROC}.mk~ - -We use ~&:~ is used in place of ~:~ to separate the target from its dependencies -in the “tangle rule.” This tells ~make~ that the recipe of this rule generates -all these files. - -Writing these rules manually —has yours truly had to do in the early days of his -website— has proven to be error-prone. - -One desirable feature for *~cleopatra~* would be to generate them automatically, -by looking for relevant ~:tangle~ directives inside the input Org document. The -challenge lies in the “relevant” part: the risk exists that we have false -posivite. However and as a first steps towards a fully automated solution, we -can leverage the evaluation features of Babel here. - -Here is a bash script which, given the proper variables, would generate the -expected Makefile rule. - -#+NAME: extends -#+BEGIN_SRC bash :var PROC="" :var AUX="" :var IN="" :results output -cat <<EOF -include ${PROC}.mk - -prebuild : ${PROC}-prebuild -build : ${PROC}-build -postbuild : ${PROC}-postbuild - -${PROC}-prebuild : ${PROC}.mk ${AUX} -${PROC}-build : ${PROC}-prebuild -${PROC}-postbuild : ${PROC}-build - -${PROC}.mk ${AUX} &:\\ - \${CLEODIR}/${IN} - @\$(emacs-tangle) - -CONFIGURE += ${PROC}.mk ${AUX} - -.PHONY : ${PROC}-prebuild \\ - ${PROC}-build \\ - ${PROC}-postbuild -EOF -#+END_SRC - -The previous source block is given a name (=extends=), and an explicit lists of -variables (~IN~, ~PROC~, and ~AUX~). Thanks to the -[[https://orgmode.org/worg/org-tutorials/org-latex-export.html][noweb syntax of -Babel]], we can insert the result of the evaluation of =extends= inside another -source block when the latter is tangled. - -We derive the rule to tangle ~bootstrap.mk~ using =extends=, which gives us the -following Makefile snippet. - -#+BEGIN_SRC makefile :tangle Makefile :noweb yes -<<extends(IN="Bootstrap.org", PROC="bootstrap", AUX="scripts/update-gitignore.sh")>> -#+END_SRC - -Beware that, as a consequence, modifying code block of =extends= is as -“dangerous” as modifying ~Makefile~ itself. Keep that in mind if you start -hacking *~cleopatra~*! - -Additional customizations of *~cleopatra~* will be parth ~bootstrap.mk~, rather -than ~Makefile~. - -* Generation Processes - -Using the =extends= noweb reference, *~cleopatra~* is easily extensible. In -this section, we first detail the structure of a typical generation process. -Then, we construct ~bootstrap.mk~ by enumerating the generation processes that -are currently used to generate the website you are reading. - -** Getting Started - -#+BEGIN_TODO -1. Defining ~proc-prebuild~, ~proc-build~, and ~proc-postbuild~ -2. Declaring dependencies between stages of generation processes -3. Declaring build outputs (see ~ARTIFACTS~ and ~CONFIGURE~) -#+END_TODO - -** Active Generation Processes - -*** Theming and Templating - -The [[./Theme.org][~theme~]] generation process controls the general appearance -of the website. More precisely, it introduces the main template used by -~soupault~ (~main/templates.html~), and the main SASS sheet used by this -template. - -If a generation process produces a set of styles within a specific SASS files, -the current approach is - -1. To make this file a dependency of ~theme-build~ -2. To modify ~style/main.sass~ in ~theme~ - to import this file - -#+BEGIN_TODO -Eventually, the second step will be automated, but in the meantime -this customization is mandatory. -#+END_TODO - -#+BEGIN_SRC makefile :tangle bootstrap.mk :noweb yes :exports none -<<extends(IN="Theme.org", PROC="theme", AUX="templates/main.html site/style/main.sass")>> -#+END_SRC - -*** Configuring Soupault - -The [[./Soupault.org][~soupault~]] generation configures and run ~soupault~, in -order to generate a static website. - -If a generation process ~proc~ produces files that will eventually be integrated to -your website, its ~proc-build~ recipe needs to be executed /before/ the -~soupault-build~ recipe. This can be enforced by making the dependency explicit -to ~make~, /i.e./, - -#+BEGIN_SRC makefile -soupault-build : proc-build -#+END_SRC - -#+BEGIN_TODO -Eventually, generation processes shall be allowed to produce specific ~soupault~ -widgets to be integrated into ~soupault.conf~. -#+END_TODO - -#+BEGIN_SRC makefile :tangle bootstrap.mk :noweb yes :exports none -<<extends(IN="Soupault.org", PROC="soupault", AUX="soupault.conf plugins/urls-rewriting.lua plugins/external-urls.lua site/style/plugins.sass scripts/history.sh templates/history.html package.json scripts/katex.js katex.mk")>> -#+END_SRC - -*** Authoring Contents - -The fact that *~cleopatra~* is a literate program which gradually generates -itself was not intended: it is a consequence of my desire to be able to easily -use whatever format I so desire for writing my contents, and Org documents in -particular. - -In the present website, contents can be written in the following format: - -- HTML Files :: - This requires no particular set-up, since HTML is the /lingua franca/ of - ~soupault~. -- Regular Coq files :: - Coq is a system which allows to write machine-checked proofs, and it comes - with a source “prettifier” called ~coqdoc~. [[./Contents/Coq.org][Learn more - about the generation process for Coq files]] -- Org documents :: - Emacs comes with a powerful editing mode called [[https://orgmode.org/][Org - mode]], and Org documents are really pleasant to work with. - [[./Contents/Org.org][Learn more about the generation process for Org - documents]] - -#+BEGIN_SRC makefile :tangle bootstrap.mk :noweb yes :exports none -<<extends(IN="Contents/Coq.org", PROC="coq", AUX="site/style/coq.sass")>> -<<extends(IN="Contents/Org.org", PROC="org", AUX="scripts/packages.el scripts/export-org.el site/style/org.sass")>> -#+END_SRC - -** Wrapping-up - -#+BEGIN_TODO -~clean~ and ~cleanall~ should probably follow a similar approach than the build -stages. -#+END_TODO - -#+BEGIN_SRC bash :tangle scripts/update-gitignore.sh :shebang "#+/bin/bash" - -BEGIN_MARKER="# begin generated files" -END_MARKER="# begin generated files" - -# remove the previous list of generated files to ignore -sed -i -e "/${BEGIN_MARKER}/,/${END_MARKER}/d" .gitignore -# remove trailing empty lines -sed -i -e :a -e '/^\n*$/{$d;N;};/\n$/ba' .gitignore - -# output the list of files to ignore -echo "" >> .gitignore -echo ${BEGIN_MARKER} >> .gitignore -for f in $@; do - echo "${f}" >> .gitignore -done -echo ${END_MARKER} >> .gitignore -#+END_SRC - -#+BEGIN_SRC makefile :tangle bootstrap.mk -ignore : - @echo " update gitignore" - @scripts/update-gitignore.sh \ - ${ARTIFACTS} \ - ${CONFIGURE} - -clean : - @rm -rf ${ARTIFACTS} - -cleanall : clean - @rm -rf ${CONFIGURE} -#+END_SRC - -# Local Variables: -# org-src-preserve-indentation: t -# End: |