Emacs as an Operating System

Please note that in the present document, “operating system” does not refer to the full implementation of software that controls the hardware, boots the system, and runs tasks tasks for the user, but instead as the last part.

Introduction

In my early years of immersion into computing I had the advantage of having access to a friend that was really good at it and ended up doing a PhD. in bioinformatics. At the time I barely had a first-try of installing Ubuntu on a desktop computer that happened to be shared with other members of my family (failing miserably and loosing during the process all of the data therein saved). Talking to this knowledgeable friend as well as reading about good editors for using R (that was not as popular at the time in my university as it is today), I heard about this weird editor with an African ruminant in his logo, called Emacs. The very first thing I heard was that it was a very good R editor that it was worth giving a try, probably from some online free course or such; and the second came from my friend, who described the editor as basically “an operating system that was also useful as a coding editor, but so hard to learn that it was basically pointless to even give it a try!”. I decided that I wanted to give it a try, just to perceive that I could not even understand its most basic commands, and that I could not even create or open a file. I dropped immediately and ended up using the built-in R’s script editor, concluding that the editor was basically beyond my reach and that I did not want to try harder.

I continued learning, next focusing on GNU/Linux and learning R along the way, writing code in RStudio and concluding that I found was just everything I needed for code production. As the time passed (several years, actually), I started perceiving some weird behaviors on RStudio’s side (specially concerning graphics) that made me look for another code editor. Then I tried sublime, and sticked to it for a while just to detect some other nuances
but nothing so serious as to make me look for some other editor. At the time I was starting my PhD. and had some “free” time, so that I started learning other “useful” tools such as LaTeX. I then decided to try again Emacs, as I was more familiar to coding and computing in general. This time I just sticked to it, learning quickly the small details that made its learning curve somewhat steep in the beginning but that were funny to learn; actually, the very same thing happened with LaTeX, so that with time it become my main editor. As a side note I did not touch again a computer running with Windows in years, so that I even forgot where to find things and such. GNU/Linux with Emacs was just wonderland I was happy, and had no reason to change something about. However, what if I had to interact with a machine running Windows?

I have to use such machines sometimes out of both my lab and home, where all machines run GNU/Linux. I found myself also using Windows machines when doing an internship abroad in a lab that did not have GNU/Linux available and had the urgency to compile a LaTeX document, and that time had to ask my sister to do it for me at home. What if I could just bring with me a mini-laptop with all my beloved GNU/Linux tools ready to use so that situations like that could not happen again? Something like a tablet with laptop capabilities, eventually capable of running code, compiling stuff and still being portable in the sense that I could bring it with me all the time. I already knew that USB sticks were useful as booting devices for computer administration or just for installing GNU/Linux, so in theory a small device such as a USB stick could behave as a full-featured system capable of turning a sad machine suffering with Windows into a powerful computing beast using GNU/Linux as its operating system. Would it be possible of having something similar for emergency situations?

I recalled the conversation with my old friend on Emacs and after some thoughts concluded that it was already a sub-operating system in my computers, where I spent most of the time when logged in, but my experience was mostly restricted to GNU/Linux, and I had no clue about whether it ran on Windows and how. Luckily I discovered that Emacs’ install process was just to unzip the directory into the desired location, and it happened to run also from a USB. It was possible to bring the Windows version of Emacs in the USB stick and even to install things on it! Then an extension seemed natural: Provided that a program can be executed from the USB without the need for any component outside the device, was it possible to make Emacs to interact with such programs? To what extent? Could I use portable versions of things such as a LaTeX distribution, R, git, and pandoc inside a USB that I could execute in any Windows computer? The short answer is: Yes (up to now), and below I describe step by step how I managed to do it. Nearly half of the process was carried out in a virtual machine with Windows 8.1 using virtualbox on GNU/Linux, and the other on a Windows machine with Windows 10. There was no perceivable difference in execution. Please note that I “name” files in the code blocks with an underline consisting of lines of equal signs (====) so that it is clear that the text below should be located in the file name.

Setting the scene

The first thing we need to do is to format the USB stick with a file system compatible with Windows. The USB stick was an old one with a couple years of use and 16 Gb of capacity. For this case I used the NTFS filesystem (following this source) and a single partition. whether you format it on Windows or GNU/Linux the result should be the same, or at least I could not find any difference as used both approaches. As a result you should have a device completely blank and mountable on Windows machines. Now we need to get some programs and decide how will we organize the files in the device.

Cluttering can easily happen when installing a bunch of programs into any device in the same directory, so that a special directory called miniOS (after mini-operating-system) was created for this purpose. There we will put all the programs that compose our “operating system.” Next thing is to download the first and most important program -Emacs-, and to set some things up on it. Please note that I will refer to the USB initial point as \ or \\ (or /, or even //) throughout the text. The last important directory will be user, where Emacs will install stuff that usually goes in .emacs.d because Windows doesn’t like dots in file and directory names.

Installing emacs and setting things up for real portability

Emacs can be found here for any platform, we will need to download the Windows version and then unzip the content into miniOS. I chose version 25 for this.

Some notes on portability

There is an issue when running Emacs on a USB stick: Every time you plug it into a machine the device can potentially be named differently, despite most of the time its name is E:\. Otherwise, our “root” and other paths can not be used if we explicitly call them E and they happen to be mounted as F or whatever. Inspired in this thread on StackExchange I found that creating a DOS script that calls emacs with a specific init file can overcome this issue, since Emacs reads the location from where it was called at startup and seizing on this we can create “relative” paths to our different directories, specially to those bearing the executable files.

Starting up Emacs

There is an special directory in the emacs directory that was unzipped: dir. It contains the executable files for running emacs under several situations. Keep it in mind when preparing the scripts.

First, create the script that sets the basic HOME and PATH variables (don’t know what do these weird guys are? take a look here), and then call Emacs from the binary that we should be using, called runemacs.exe. Why this file if there are other executables in the same directory, one of them even called just emacs.exe? See the reasons here. Our startup script will be called Emacs.cmd, and we will double-click it every time we want to open Emacs. Its content will be:

\Emacs.cmd
==========
set "HOME=%~dp0"
set PATH="%PATH%;\miniOS\emacs\bin"
miniOS\emacs\bin\runemacs.exe   -q   --load \miniOS\emacs\init.el %*

The first two lines set the HOME environment variable and then use it in order to set the PATH variable that Emacs will remember them as the starting execution point, and where to find programs for execution, respectively. The third line runs Emacs with the -q option that starts Emacs faster (with the cost of not running some startup code, see here again). The first line is where magic happens: It sets the path to the USB whatever the name of the device: E, F, G, whatever. This script makes out Emacs instance fully portable. Please note that each directory put in the PATH must be separated by a semicolon (;).

Next thing to do is to create an init script so that Emacs Lisp code is run at startup. Important code that should always be run at startup goes here:

\miniOS\emacs\init.el
=====================
(setenv "HOME"
        (expand-file-name (concat invocation-directory "../../..")))
(setq user-emacs-directory
      (expand-file-name (concat (getenv "HOME") "user/")))
(setq abbrev-file-name
      (locate-user-emacs-file "abbrev_defs"))
(setq auto-save-list-file-prefix
      (locate-user-emacs-file "auto-save-list/.saves-"))
(cd "~")
(load "~/_emacs")

The third important file is the so-called .emacs file (or dot-emacs), that we will call instead _emacs and place in \ so that Windows does not complain about having dots in the file name:

\_emacs
=======
;; see if .emacs is loading
(message ".emacs loading")

This small elisp code tells us that Emacs is loading appropriately the dot-emacs file, and therefore we should see the text “.emacs loading” in the minibuffer. Next we will need to tell Emacs to use also the MELPA repository in order to install packages with the following code in _emacs:

\_emacs
=======
;; setting up melpa
(require 'package)
(let* ((no-ssl (and (memq system-type '(windows-nt ms-dos))
                    (not (gnutls-available-p))))
       (proto (if no-ssl "http" "https")))
  ;; Comment/uncomment these two lines to enable/disable MELPA and MELPA Stable as desired
  (add-to-list 'package-archives (cons "melpa" (concat proto "://melpa.org/packages/")) t)
  ;;(add-to-list 'package-archives (cons "melpa-stable" (concat proto "://stable.melpa.org/packages/")) t)
  (when ( setwd('e:/')
> 

Installing git

Git has a portable version called “thumbdrive edition” under the windows release that will install everything in the specified location. Just download it into \miniOS and let it install everything in the default folder called PortableGit.

Interacting with git: The magit package

We will use magit in order to interact with git and install it with M-x package-install and then magit. After a lot of times trying I could not make magit to find the executables and ended up modifying the magit-git-executable variable in _emacs:

\_emacs
==========
;; set magit git executable from portable git
(setq magit-git-executable "\\miniOS\\PortableGit\\mingw64\\bin\\git")

We can test that everything works properly by cloning any repo from github, and it simultaneously will tell us if we can access the internet from Emacs: M-x magit-clone and then type the repo address: http://github.com/user/repository.git. Please note that sometimes secure protocols can give problems, so maybe it’s better to type http and not https.

Installing LaTeX

The recommended approach at installing a LaTeX distribution was found to be MiKTeX. This distribution has a portable version that we will use for our case. Please note that rmarkdown will have problems interacting with MiKTeX (see here) and you should choose the “Install missing packages” option as “Yes” during install. After installing it into \miniOS\miktex-portable, we will modify the PATH in Emacs.cmd:

\Emacs.cmd
==========
set PATH="%PATH%;\miniOS\emacs\bin;\miniOS\R-3.4.4\bin\x64;\miniOS\miktex-portable\texmfs\install\miktex\bin"

I also did it in init.el because Emacs was struggling with the windows backslashes and just found the executables when using unix slashes:

\miniOS\emacs\init.el
=====================
(setenv "PATH" (concat
		(getenv "PATH")
		";"
		"/miniOS/miktex-portable/texmfs/install/miktex/bin"))

I actually got the inspiration when seeing that MiKTeX itself uses both kinds of slash in order to avoid problems depending on the platform.

Interacting with LaTeX: The AUCTeX package

The AUCTeX package is another marvel from Emacs that offers a full-featured system for compiling LaTeX documents from inside Emacs. This is the responsible of allowing us to use Emacs as ourur LaTeX platform provided that it finds the executables for compilation of such documents that we can used for authoring papers, theses, and even presentations. We can install it with M-x package-install and then auctex. if the PATH was properly set, AUCTeX should not have problems finding the LaTeX compilers (e.g., pdflatex or bibtex). We can test that everything is in place creating a test LaTeX file and compiling it with C-c C-c and choosing LaTeX from the list of compilers:

\documentclass{article}
\begin{document}
Hello world!
\end{document}

If everything is OK we would compile our first LaTeX document to pdf in the USB stick.

Some games

Sudoku

Install the sudoku package as already indicated for other Emacs packages.

Go

We can use the GNU Go engine for such wonderful game. Install gnugo as we did with sudoku, and also install the program from here in miniOS. We will need to add it to the path both in Emacs.cmd and init.el as well as to set the variable gnugo-program to point to the path in _emacs. Maybe just one of these three will suffice, but so far I wanted to guarantee it:

\Emacs.cmd
==========
set PATH="%PATH%;\miniOS\emacs\bin;\miniOS\R-3.4.4\bin\x64;\miniOS\miktex-portable\texmfs\install\miktex\bin;\miniOS\gnugo-3.8"
\miniOS\emacs\init.el
=====================
(setenv "PATH" (concat
		(getenv "PATH")
		";"
		"/miniOS/miktex-portable/texmfs/install/miktex/bin"
                ";"
		"/miniOS/gnugo-3.8"))
\_emacs
=======
;; set gnugo executable explicit path
(setq gnugo-program "\\miniOS\\gnugo-3.8\\gnugo")

Portable Pandoc

Pandoc is a program for converting text files between a wide array of formats, and is the battelhorse of packages such as knitr and rmarkdown, so if we want to author markdown and/or rmarkdown files we must guarantee that it can be used from the USB. We can install it from here. Please note that we want the .zip compressed file that acts as a portable version and not the .msi file.

Then we need to add pandoc to the PATH in init.el as already mentioned:

\miniOS\emacs\init.el
=======
(setenv "PATH" (concat
		(getenv "PATH")
		";"
		"/miniOS/miktex-portable/texmfs/install/miktex/bin"
                ";"
		"/miniOS/gnugo-3.8"
		";"
		"/miniOS/pandoc-2.4-windows-x86_64"))

Please remember the warning when discussing MiKTeX: Pick the auto-install option during install of the latter package. If you try to run the render function of rmarkdown you will have problems.

Authoring markdown

There is an Emacs package that I like to use for markdown, called markdown-mode. You can install it as already described for other packages. Another package is polymode, also available as already described. Finally, you may want to take a look at rmarkdown-goodies, my own Emacs package for extending markdown specifically for providing some functionality that I was missing from other packages more general in scope (such as markdown-mode). I plan to describe such package in more detail in the future (as well as to submit it to melpa), but for the time being you can clone it from Emacs with magit-clone and htttp://github.com/gaballench/rmarkdown-goodies.git. As a comment, I prepared the present document with these three packages directly from Emacs.

The final chunk of software we will need in order to author markdown documents from Emacs is the rmarkdown R package. Open R with M-x R and then install it from CRAN with:

install.packages("rmarkdown", repos = "http://cran.us.r-project.org")

Please note the explicit use of the repos argument in install.packages. Since we are using http instead of https, we try to avoid problems with this protocol, and also, we avoid the X emergent window asking us to specify a repository; it may not work from within Emacs!

Assorted additions

There is a number of functionalities I like in Emacs, and all of them can be specified in _emacs. First install the packages mc and undo-tree as usual and then add the following to the _emacs file:

\_emacs
=======
;; set save abbrevs to silent
(setq save-abbrevs 'silent)        ;; save abbrevs when files are saved

;; deal with temporary files
;; store all backup and autosave files in the tmp dir
(setq temporary-file-directory "\\miniOS\\tmp")
;(setq backup-directory-alist
;      `((".*" . ,temporary-file-directory)))
;(setq auto-save-file-name-transforms
;      `((".*" ,temporary-file-directory t)))

;; use count-words instead of count-words-region as it works on buffer
;; if no region is selected
(global-set-key (kbd "M-=") 'count-words)

;; multiple cursors
;; first install mc with `package-install' and also install libgnutls
;; see the following https://emacs.stackexchange.com/questions/27202/how-do-i-install-gnutls-for-emacs-25-1-on-windows
;; http://gnu.c3sl.ufpr.br/ftp/emacs/windows/emacs-25/emacs25-x86_64-deps.zip
;; extract in e:/miniOS/emacs so that binaries are properly included in its directory (i.e., emacs/bin)
(global-set-key (kbd "C-c m c") 'mc/edit-lines)

;;; undo tree mode
;;turn on everywhere
(global-undo-tree-mode 1)
;; make ctrl-z undo
(global-set-key (kbd "C-z") 'undo)
;; make ctrl-Z redo
(defalias 'redo 'undo-tree-redo)
(global-set-key (kbd "C-S-z") 'redo)

;; maximized at startup
(add-to-list 'initial-frame-alist '(fullscreen . maximized))

Please note that in order to use the multiple cursors you will need to install the emacs25 dependencies by downloading the file and unzipping it into \miniOS\emacs so that binaries are properly included in its directory (i.e., emacs\bin).

Closing summary of scripts

All of the important scripts are transcribed below for further reference. The system runs adequately and I hope to prepare a zipped file for sharing the current state of the USB portable software. Please drop me a note if you want it before I release it publicly.

\Emacs.cmd
==========
set "HOME=%~dp0"
set PATH="%PATH%;\miniOS\emacs\bin;\miniOS\R-3.4.4\bin\x64;\miniOS\miktex-portable\texmfs\install\miktex\bin;\miniOS\gnugo-3.8"
miniOS\emacs\bin\runemacs.exe   -q   --load \miniOS\emacs\init.el %*

\_emacs
=======
;; see if .emacs is loading
(message ".emacs loading")

;; setting up melpa
(require 'package)
(let* ((no-ssl (and (memq system-type '(windows-nt ms-dos))
                    (not (gnutls-available-p))))
       (proto (if no-ssl "http" "https")))
  ;; Comment/uncomment these two lines to enable/disable MELPA and MELPA Stable as desired
  (add-to-list 'package-archives (cons "melpa" (concat proto "://melpa.org/packages/")) t)
  ;;(add-to-list 'package-archives (cons "melpa-stable" (concat proto "://stable.melpa.org/packages/")) t)
  (when (< emacs-major-version 24)
    ;; For important compatibility libraries like cl-lib
    (add-to-list 'package-archives (cons "gnu" (concat proto "://elpa.gnu.org/packages/")))))
(package-initialize)

;; set save abbrevs to silent
(setq save-abbrevs 'silent)        ;; save abbrevs when files are saved

;; set magit git executable from portable git
(setq magit-git-executable "\\miniOS\\PortableGit\\mingw64\\bin\\git")

;; deal with temporary files
;; store all backup and autosave files in the tmp dir
(setq temporary-file-directory "\\miniOS\\tmp")
;(setq backup-directory-alist
;      `((".*" . ,temporary-file-directory)))
;(setq auto-save-file-name-transforms
;      `((".*" ,temporary-file-directory t)))

;; use count-words instead of count-words-region as it works on buffer
;; if no region is selected
(global-set-key (kbd "M-=") 'count-words)

;; multiple cursors
;; first install mc with `package-install' and also install libgnutls
;; see the following https://emacs.stackexchange.com/questions/27202/how-do-i-install-gnutls-for-emacs-25-1-on-windows
;; http://gnu.c3sl.ufpr.br/ftp/emacs/windows/emacs-25/emacs25-x86_64-deps.zip
;; extract in e:/miniOS/emacs so that binaries are properly included in its directory (i.e., emacs/bin)
(global-set-key (kbd "C-c m c") 'mc/edit-lines)

;;; undo tree mode
;;turn on everywhere
(global-undo-tree-mode 1)
;; make ctrl-z undo
(global-set-key (kbd "C-z") 'undo)
;; make ctrl-Z redo
(defalias 'redo 'undo-tree-redo)
(global-set-key (kbd "C-S-z") 'redo)

;; maximized at startup
(add-to-list 'initial-frame-alist '(fullscreen . maximized))

;; set gnugo executable explicit path
(setq gnugo-program "\\miniOS\\gnugo-3.8\\gnugo")

\miniOS\emacs\init.el
=====================
(setenv "HOME"
        (expand-file-name (concat invocation-directory "../../..")))
(setq user-emacs-directory
      (expand-file-name (concat (getenv "HOME") "user/")))
(setq abbrev-file-name
      (locate-user-emacs-file "abbrev_defs"))
(setq auto-save-list-file-prefix
      (locate-user-emacs-file "auto-save-list/.saves-"))
(cd "~")
(setenv "PATH" (concat
		(getenv "PATH")
		";"
		"/miniOS/miktex-portable/texmfs/install/miktex/bin"
                ";"
		"/miniOS/gnugo-3.8"
		";"
		"/miniOS/pandoc-2.4-windows-x86_64"))
(load "~/_emacs")
Advertisements

Author: gaballench

Ictiólogo con interés en informática y música, simpatizante de iniciativas open source.

2 thoughts on “Emacs as an Operating System”

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s