quickies

To install additional ones, use asdf, e.g.

(require 'asdf)
(require 'asdf-install)
(asdf-install:install 'cl-ppcre)

will install Perl-style regular expression support. Then, when you restart SBCL, you'll need to load it with

(asdf:operate 'asdf:load-op :cl-ppcre)

Look at dpans2texi.el.

We need to create a small startup script, say clj. The simplest one is something like:

#! /usr/bin/env bash
java -cp /usr/local/share/clojure/clojure.jar clojure.main

The problem is that we won't benefit from command-line history, backward search, etc. For that to be effective, we need to use JLine (rlwrap doesn't work on OS X, due to the lacking Readline library). Then, we just have to modify the above script with:

java -cp /.../jline-0.9.94.jar:/.../clojure.jar jline.ConsoleRunner clojure.main

where ... denotes actual path to related files. Alternatively, this could be setup using CLASSPATH, or by placing the jar file in the system-wide java extensions directory, e.g. /Library/Java/Extensions or /System/Library/Java/Extensions. The latter doesn't seem to be working as jline would be loaded by the extension classloader which is unaware of class path variable. Startup tips are described in http://clojure.org/getting_started, and on http://www.stackoverflow.com.

It is recommended to install clojure.contrib which is a set of additional features (like common mathematical operations) that are not present in the base package. To install it, just grab a binary (jar) distribution and make it available for Java. For that purpose, we can use a custom CLASSPATH, e.g. in my .profile:

CLASSPATH=/usr/local/share/jars

or add an extra argument to -cp when invoking clojure.jar from Java.

To check that everything works well, we can use a simple script test.clj, like this:

(ns test
  (:use clojure.contrib.math
	clojure.test))
(deftest foo
  (is (= 4 (sqrt 16))))
(run-tests)

that can be run under Clojure REPL. If it is ok, we should see something like:

Ran 1 tests containing 1 assertions.
    0 failures, 0 errors.
    {:type :summary, :test 1, :pass 1, :fail 0, :error 0}

Here is another example:

(ns test
  (:use clojure.contrib.pprint))
(pprint (for [x (range 10)] (range x)))

is described at http://bit.ly/eda9IS. It is possible to add as many libraries as we want (e.g., incanter).

environment, we can use slime. A specific port for Aquamacs is available for download.

how much Java memory, cpu and threads Clojure actually uses. h/t http://www.fatvat.co.uk/2009/05/jvisualvm-and-clojure.html

L-99 – Ninety-Nine Lisp Problems. There is also functional-koans on Github.

(doc histogram) brings a short description of the function with optional parameters (no pager though).

Volkmann's website, http://java.ociweb.com/mark/clojure.

M-1 C-x $; to revert, C-x $ (See set-selective-display)

try:

(when (featurep 'aquamacs) ... )

On OS X, this just has to be negated to address built-in emacs.

all shortcuts for current buffer.

is the possibility of creating temp buffer on the fly. E.g., C-x b (aka switch-to-buffer) allows to switch to a temporary buffer.

C-x 1 = delete-other-windows, C-x 2 = split-window-vertically, C-x 3 = split-window-horizontally, C-x + = balance-windows, C-x o = other-window, C-x C-b = list-buffers. Use M-x describe-bindings to view all the Buffer-menu key bindings.

e.g. equations, figures, etc. (With reftex-mode enabled, it is C-=.)

Under OS X, an easy replacement of ispell is done with aspell. You need to populate /usr/local/lib/aspell-0.60/ with compiled dictionnaries. To check which dictionnaries are available on your system, type aspell dicts. A useful replacement of ispell-mode is the interactive flyspell-mode. To use it with LaTeX, just add in your .emacs file:

(add-hook ‘LaTeX-mode-hook ‘flyspell-mode)

just use C-u nn pp, where nn is the number of times character pp as to be repeated. For example, C-u 78 - put a pseudo line of length 78; it is useful to choose this length as the default fill column width. To check which one is currently defined, type fill-column in the scratch buffer and evaluate this expression (C-x C-e). Common values are less than 80, e.g. 68, 70, or 78.

is that R comment (#) are automagically tabbed to the right (40th column by default). To get ride of this, you can:

• use ## instead, which yields comments aligned to the current level of indentation;
• use ### instead, which results in comments aligned to the beginning of the line.

See http://ess.r-project.org/Manual/ess.html#Indenting. Another option is to put (setq ess-fancy-comments nil) in your .emacs file.

Just M-q (fill-paragraph) on the region of interest. Another option is to use newcomment.el which provides comment-auto-fill-only-comments.

(require 'newcomment)
(setq comment-auto-fill-only-comments 1)
(setq-default auto-fill-function 'do-auto-fill)

h/t http://stackoverflow.com/questions/2214199.

They help mastering buffers and provide enhanced completion.

It provides package.el which acts as a package manager for Emacs based on a repository of pre-packed Emacs Lisp code. http://tromey.com/elpa/

upon pressing the Enter key or after C-M-\, use two or three '#'. E.g.,

# this is a comment that will be right-justified
## This one will not move

just type C-h k followed by the combination.

http://jblevins.org/projects/markdown-mode/

C-c C-o (inside function body) generate help template

use tabbar or remap F5 (not used on Air OS X), e.g.:

(global-set-key (kbd "<f5>")   'tabbar-forward-tab)
(global-set-key (kbd "C-<f5>") 'tabbar-backward-tab)

just use M-x browse-url-of-buffer

just C-x C-f and type /sudo::/path/file

C-@ set mark
C-x C-x exchange point and mark
C-x n n narrow down to the specified region (between mark and point)
C-x n w revert to full buffer editing
M-< go to the top of the buffer
M-> go to the end of the buffer
C-l center buffer
C-v scroll one page forward
M-v scroll one page backward

you might prefer M-x show-paren-mode

just C-u C-x = to get its name.

use a: it won't open a new buffer for each directory you visit.

M-x occur

C-c C-z will switch to the end of ess inferior buffer.

in Emacs term or ansi-term when using zsh. (See also http://stackoverflow.com/a/7437783/420055 for git prompt and unicode characters).

Info dir are up to date, C-h v Info-directory-list, and use install-info to update the Info dir (linking or moving an Info file is not enough). E.g., we can put mu4e.info in /usr/local/share/info and use

$ sudo install-info /usr/local/share/info/mu4e.info /usr/local/share/info/dir

lot of useful tricks to know about how Mathematica actually handle some of its calculus. Available at http://reference.wolfram.com/mathematica/tutorial/ (SomeNotesOnInternalImplementation.html)

$ defaults write com.apple.dock no-glass -boolean YES
$ killall Dock

To revert, use -boolean NO.

(default to Bash): .bashrc, .bash_profile, and .profile. The first two are for non-interactive and interactive shell, respectively. The latter one is loaded when no .bash_profile exists, but its existence will supercedes .profile. The .bash_profile seems to be the recommended one (at least, according to Apple support). Some authors suggest to fill the .bash_profile with something like

if [ -f ~/.bashrc ]; then
    source ~/.bashrc
fi

so that everything goes into .bashrc. There are also two system-wide configuration files that are located under /etc: /etc/profile and /etc/bashrc.

Instead, libedit is used a replacement, which might cause some troubles with applications relying on console. If necessary, it can be installed with a few steps:

$ curl -O ftp://ftp.cwru.edu/pub/bash/readline-6.1.tar.gz
$ tar xvf readline-6.1.tar.gz && cd readline-6.1
$ ./configure && make
$ sudo make install

For Python, it is not really a problem as we can simply do

$ sudo easy_install readline

and use the excellent ipython or bpython interactive shells.

in (at least) two ways. First, we can just set an alias to ls, e.g. alias ls'ls -G'= (=ls --color=auto= don't under OS X, but we can use export LS_OPTIONS'–color=auto'= instead). Another option is to use CLI colors, that is:

export CLICOLOR=1
export LSCOLORS=dxfxcxdxbxegedabagacad

The order matters: DIR, SYM_LINK, SOCKET, PIPE, EXE, BLOCK_SP, CHAR_SP, EXE_SUID, EXE_GUID, DIR_STICKY, DIR_WOSTICKY. Color codes read as follows: a = black, b = red, c = green, d = brown, e = blue, f = magenta, g = cyan, h = light gray, x = default.

in shell scripts, e.g. ${file%.*}. More generally, ${varname%pattern} allows to remove pattern at the end of a variable. h/t Jake Hofman

and !! gives the arguments of the previously-executed command.

use comm (or grep -xF) to check for lines common to two files. h/t Jake Hofman

we often find root (user) and wheel or admin (group). In fact, wheel is the system administrator group in BSD, while root is the system administrator user. It is common to add sudo permissions to users in the wheel group. The choice of wheel comes from the BSD UNIX group at UC Berkeley; "big wheel" means an influential or important person.

has:attachment pdf after:2011/01/01 before:2011/01/20
has:attachment (*.ppt || *.pptx || *.pps)
label:Label1 | label:Label2 (case insensitive)
label:Label1 filename:pdf
from:John OR from:Joe cc: bcc:
subject:"match this exactly"
from:John (keyword1 OR keyword2) (remove OR to match both keys)
keyword1 -keyword2 (has key1 but not key2)
in:anywhere (Spam and Trash excluded)
is:starred

No solution for filtering large attachment, though. But see http://www.findbigmail.com or http://www.searchgmailbysize.com.

just try to add a custom function in an awk script. E.g., from the gawk manual

function changeit(array, ind, nvalue)
{
    array[ind] = nvalue
}

BEGIN {
    a[1] = 1; a[2] = 2; a[3] = 3
    changeit(a, 2, "two")
    printf "a[1] = %s, a[2] = %s, a[3] = %s\n",
	a[1], a[2], a[3]
}

and test it:

$ awk -f test_awk.awk

use ls like this:

$ ls -l -t /usr/bin | tail

The -c switch allows to view file sorted by modification date instead.

e.g.:

$ convert -delay 20 -loop 0 fig*.png spin_plot.gif

let's try to generate an SWF file thanks to the swftools:

$ png2swf -r 2 fig*.png -o afig.swf

in Sytem Preferences > Keyboard > Services. Useful for counting words in an abstract.

$ sort file1 file2 | uniq -d h/t

h/t @mathieuen, 25 Mar

to list all installed packages (with Leopard installer). Useful command include pkgutil --forget org.r-project.R.Leopard.fw.pkg when you want to install two concurrent versions of R.

select them and Shift-Command Del.

which allows to insert hard line breaks in a whole file according to some fixed line width (but respecting word breaks).

and order by percentage used:

$ df -Ph | column -t | sort -n -k5

h/t climagic

$ ls -ltra | tail -n 30

h/t climagic

$ find /dir -type f -a -mtime +30 -delete

h/t climagic

$ ls -l | sed 's/^/  /'

h/t climagic

then ping !$:s/user@// will ping the IP itself, removing the user@ part. h/t climagic

but the home Library folder is not visible anymore. To bring it back in the Finder, just run

$ chflags nohidden ~/Library

$ grep "da$" /usr/share/dict/words

h/t climagic

$ fmt -ut -w 80 filename.txt

h/t climagic

$ find . -mtime +365  | while read -r file ; do dirname "$file" ; done | uniq

h/t climagic

$ C-z bg ; wait %1 ; echo "done" | mail -s "done" you@example.com

h/t climagic

$ find / -type f -size +1G | xargs ls -lh > listofbigfiles.txt

h/t rmounce

but for Libraries it is often harder to tell what's been installed by a pkg. While events are logged in in /var/log/install.log, we won't find useful information there.

Solution 1 (easy):

• Run the pkg installer
• Look into "Show Files" just after installer has been launched

Solution 2 (harder):

• In /Library/Receipts there is everything we need, including an InstallHistory.plist where all packages are registered.
• Otherwise, you can just look at

$ lsbom /Library/Receipts/package_name/Contents/Archive.bom

to get the list of files (with relative paths) and Contents/Info.plist to get the root directory (IFPkgFlagDefaultLocation).

we can use

$ mdfind -name "proqol scoring"

just issue at the terminal $ purge. See also: http://apple.stackexchange.com/q/67031.

command-line. E.g., http://apple.stackexchange.com/a/18665.

and type Ctrl-Cmd-d to get its definition from the built-in dictionnary.

there's no TRANSPOSE() command, but we can use the following:

=INDEX(Table 1::$A$1:$A$10,COLUMN(),ROW())

allows to extract any metadata from a given file, including EXIF and geolocalisation information from an image file.

for Fasta files (https://www.biostars.org/p/225913/):

from itertools import groupby

def transcribe(sequence):
    return sequence.replace('T', 'U')

def translate_rna(s):
    codon2aa = {"AAA":"K", "AAC":"N", "AAG":"K", "AAU":"N",
		"ACA":"T", "ACC":"T", "ACG":"T", "ACU":"T",
		"AGA":"R", "AGC":"S", "AGG":"R", "AGU":"S",
		"AUA":"I", "AUC":"I", "AUG":"M", "AUU":"I",

		"CAA":"Q", "CAC":"H", "CAG":"Q", "CAU":"H",
		"CCA":"P", "CCC":"P", "CCG":"P", "CCU":"P",
		"CGA":"R", "CGC":"R", "CGG":"R", "CGU":"R",
		"CUA":"L", "CUC":"L", "CUG":"L", "CUU":"L",

		"GAA":"E", "GAC":"D", "GAG":"E", "GAU":"D",
		"GCA":"A", "GCC":"A", "GCG":"A", "GCU":"A",
		"GGA":"G", "GGC":"G", "GGG":"G", "GGU":"G",
		"GUA":"V", "GUC":"V", "GUG":"V", "GUU":"V",

		"UAA":"_", "UAC":"Y", "UAG":"_", "UAU":"T",
		"UCA":"S", "UCC":"S", "UCG":"S", "UCU":"S",
		"UGA":"_", "UGC":"C", "UGG":"W", "UGU":"C",
		"UUA":"L", "UUC":"F", "UUG":"L", "UUU":"F"}

    l = [codon2aa.get(s[n:n+3], 'X') for n in range(0, len(s), 3)]
    return "".join(l)


with open('dna.fasta') as h:
    faiter = (x[1] for x in groupby(h, lambda l: l[0] == ">"))
    for header in faiter:
	header = next(header)[1:].strip()
	seq = "".join(s.strip() for s in next(faiter))
	print(header)
	print(translate_rna(transcribe(seq)))

>>> x = 5
True
>>> 10 < x < 20
False

http://stackoverflow.com/questions/101268/hidden-features-of-python

that can be solved with few lines: http://www.pythonchallenge.com

year,month,day = time.strftime('%Y-%m-%d',time.localtime()).split('-')

h/t neilkod

Nice trick when using enumerate:

next(i for i,v in enumerate(l) if is_odd(v))

https://stackoverflow.com/a/1701404

match only finds a pattern at the beginning of the string; if you need to scan all the string, just use search or regex.match("(.*?*)pattern(.*?*)") eventually.

See also this SO post:

import random
import re
import string
import time

LENGTH = 10
LIST_SIZE = 1000000

def generate_word():
    word = [random.choice(string.ascii_lowercase) for _ in range(LENGTH)]
    word = ''.join(word)
    return word

wordlist = [generate_word() for _ in range(LIST_SIZE)]

start = time.time()
[re.search('python', word) for word in wordlist]
print('search:', time.time() - start)

start = time.time()
[re.match('(.*?)python(.*?)', word) for word in wordlist]
print('match:', time.time() - start)

X <- replicate(2, rnorm(100))
y <- X[,1]+X[,2]+rnorm(100)
df <- data.frame(y=y, X=X)

Here is a one-line solution:

df <- transform(X <- as.data.frame(replicate(2, rnorm(100))), y = V1+V2+rnorm(100))

This is also a nice way to generate two uncorrelated predictors, while allowing a strong association between the outcome and each of them.

is capture.output(). From the on-line help, it can even be combined to enscript like so:

ps <- pipe("enscript -o tempout.ps","w")
capture.output(example(glm), file=ps)
close(ps)

R> png("grv.png"); plot(replicate(2, rnorm(100))); dev.off()
$ convert grv.png \( +clone -background black -shadow \
    55x15+0+5 \) +swap -background none -layers merge  +repage grv2.png

when no return values is expected, e.g. for an animation

r_ply(10, plot(runif(50)))

better than which (colnames(d)="a")= or grep("^a$", colnames(df)) use match("a", names(d)) With huge data.frame, it's even better to use the fastmatch package (use fmatch instead of match). h/t Matthew Dowle, http://bit.ly/yFeDTT

we can use pkgutil to look for available versions on a Mac system:

$ pkgutil --packages / | grep org.r-project

Mac versions usually include Leopard.fw.* in the installaed receipts. Then, we just have to tell the system to forget about a previous version, using e.g.,

$ sudo pkgutil --forget org.r-project.R.Leopard.fw.pkga

page(d, method"print")= to view a long data frame d through a pager.

https://stackoverflow.com/q/41135630 On the use of the #:when clause in for/list form.

selection scheme for use with common PostScript fonts. Example of a nice title typesetted in Times New Roman (h/t tex.stackexchange.com, 8768) :

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{mathptmx}

\begin{document}
\noindent\fontsize{2cm}{2.2cm}\selectfont
Geometric\\ Invariant Theory
\end{document}

a given typeface.

where any symbol can be drawn by hand.

\usepackage[pdftex]{graphicx}
\usepackage{epstopdf}

h/t Jake Hofman

\, = thin space; \; = medium space; \ = space; \quad = quad space; \! = negative space (useful with multiple integrands).

use {}; i.e., {}_nC_k. h/t Jake Hofman

into high resolution pixmap graphics for inclusion in text processors or presentations. http://www.nought.de/tex2im.html

try The LATEX2ε Sources (texdoc source2e).

just run

$ ls `kpsewhich --var-value=TEXMFDIST`/doc/latex

h/t tex.stackexchange.com, 4358

http://refcards.com/docs/silvermanj/tex/tex-refcard-a4.pdf

say tex_ttf.tex is:

\font\1="Inconsolata"\1 hello
\bye

Compile with:

$ xetex -no-pdf tex_ttf
$ xdvipdfmx -vv tex_ttf

of a (La)TeX document by using the atbegshi package.

may be found at the following address: http://nitens.org/taraborelli/latex (Dario Taraborelli).

Hoefler, Gentium, Fontin Sans, Myriad Pro, Minion Pro, Gill Sans, Inconsolota.

T<span style"position: relative; top: .2em">E</span>X=. A better solution is to directly embed relevant code into a CSS stylesheet, like proposed by Edward O’Connor's on his blog: http://edward.oconnor.cx/2007/08/tex-poshlet.

(instead of [1,2,3,4,5]), but here is a nice solution from http://tex.stackexchange.com (Credit: Martin Scharrer):

\usepackage{pgffor}
\makeatletter
\newcommand*\rcite[1]{%
\def\@gtempa{}%
\foreach \n in {#1} {%
\edef\@tempa{,bibitem\n}%
\expandafter\g@addto@macro\expandafter\@gtempa
\expandafter{\@tempa}%
}%
\edef\@gtempa{{\expandafter\@gobble\@gtempa}}%
\expandafter\cite\@gtempa
}
\makeatother

Then, we can use things like

\rcite{3,...,10}      % 3-10
\rcite{2,5,6,...,10}  % 2, 5-10
\rcite{3,5,...,10}    % 3, 5, 7, 9

\let\oldhref\href
\renewcommand{\href}[2]{\footnote{\oldhref{#1}{#2}}}

h/t http://tex.stackexchange.com/questions/12774

See this SO thread, or just use C-x 8 SPC.