jemnotes

2009-06-19

hush login

Sick of messages like this when logging in?

Last login: Thu May 21 23:31:00 2009

Simply

touch .hushlogin

and they’ll be all gone.

2009-06-11

filename modifiers in vim

I found it difficult to find help on the various ways of modifying the filename in a vim script. Turns out that :help filename-modifiers uses the correct keyword.

2009-05-27

test-driven development

I wrote a few notes on test-driven development.

2009-05-25

rails / single table inheritance

A good page about single table inheritance.

2009-05-22

python / line_profiler

cProfile and friends work well for profiling larger programs where things are split into functions. For more detailed investigation, however, you need something like line_profiler, where you can see output like this (trimmed):

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
...
202      1741        12946      7.4      0.0      for k in zip(K.I, K.J):
203      1740       222659    128.0      0.9          K[k] = randn()
...

Once installed, decorate any function worthy of analysis with @profile, run kernprof.py -l <scriptname.py> and then view results with python -m line_profiler <scriptname>.lprof. Works well.

2009-05-21

python

I wrote down below about enumerate, which lets you access (index, item) pairs within a loop. I guess I thought it was a good idea, because I was about to write it here again. If you need both the index and item from a list, use enumerate:

for index, item in enumerate(items):
    pass

I haven’t used it in the intervening two months, but oh well.

2009-05-18

rails + .gitignore

A nice page explaining how to set up .gitignore rules for rails apps.

2009-05-07

terminal on mac

For some reason, Terminal.app appeared to be completely ignoring my request to disable the audible bell. This worked perfectly instead: xset b off.

vim

Reminder: use ctrl+t and ctrl+d to increase/decrease the indent while in insert mode.

2009-05-05

vim

In smartindent mode, vim removes the indent if # is the first character on the line. This is annoying. I've tried various solutions that should work but don't. Here's one that should, and does:

:inoremap # a#<Left><BS><Right>

(Courtesy of this vim tip, but beware; it contains some false information as well.) Use ^v (ctrl+v) to enter the special sequences easily.

2009-05-01

python

This code will fail:

l = []
def app():
    l += (1,)

app()

It will fail with an UnboundLocalError. According to this website, this is because ‘If a value is assigned to a variable in a function body, the variable will be local, even if there is a global variable with the same name, and this global variable has been used before the assignment.’

This is quite different to the behaviour if you don’t try and perform an assignment, and so is a bit confusing. Of course, there are few cases where global variables are the correct tool, but sometimes they can be handy.

2009-04-14

py.test

Could py.test be used for test-driven development in Python?

nice tour of git

This is a better summary of git than I’ve seen before. I intend to write my own guide, sometime — mostly for my own learning — and will probably use some of this material to remind me what to write about.

2009-03-31

ruby

Ruby has an unless statement equivalent to if not. Nice. And the RailsTips blog has some good suggestions on usage.

google trends

Compare the popularity of two search terms over time at Google Trends.

2009-03-31

git

Start using a remote server to backup / distribute a local app:

# On the server, to avoid having copies of local files.
git --bare init

# On the client.
git remote add origin ssh://hostname/path/to/repo
git push origin master

2009-03-30

fluid

A brilliant way of creating a ‘Site Specific Browser’ so that your dock contains a particular web app / site as a separate process. Makes a web app feel even more like it belongs as a fully-fledged app.

git

Undo the last git commit while leaving the tree as is. (Perhaps your commit message was wrong, or you forgot to add certain files.)

git reset --soft HEAD^

2009-03-28

git

Set up a global ignore file for git using

git config --global core.excludesfile ~/.gitignore

jQuery

The first javascript libraries I used were prototype and scriptaculous. However, I just started using jQuery, which is insanely great. The (css-like) selectors they use, for example, are so much fun: consider

$('li:visible:not(:has(img.checkmark.checked))').hide('slow')

This slowly hides and removes any visible list items that have no child elements that are images in the checkmark (tick for commonwealth readers) and checked (ticked) classes.

In other words, this slowly fades any checked off items that haven’t yet been hidden. So clean!

2009-03-27

python

A few interesting things about Python that I had forgotten come from here. I’ve modified the examples.

First, you can use an instance’s namespace to give an interesting way to print strings. Sometimes this will be better:

# Basic version.
print 'My name is %s and I am %d.' % (p.name, p.age)
# Advanced version.
print 'My name is %(name)s and I am %(age)d.' % p.__dict__

There is also the enumerate function, which returns (index, item) pairs. For example,

for (index, item) in enumerate(items):
    print index, item

Dictionaries have a setdefault method. Consider these alternative pieces of code:

# Naive piece of code.
for (cat, amount) in data:
    if cat not in d:
        d[cat] = 0
    d[cat] += amount

# Simpler piece of code.
for (cat, amount) in data:
    d.setdefault(cat, 0)
    d[cat] += amount

# A cunning third option: setdefault also gets.
for (cat, amount) in data:
    d[cat] = d.setdefault(cat, 0) + amount

But, better than all of these optons is the new defaultdict in Python 2.5.

from collections import defaultdict
d = defaultdict(generator_function)

# Example like those above.
d = defaultdict(lambda: 0)

Finally, generator expressions: say we want to sum a large list that we generate on the fly (ie, we don’t care about the list):

sum(i**2 for i in xrange(1e5))

(I didn’t find much difference in evaluation time, though.)

2009-03-26

slicehost

Slicehost clearly know what they’re doing, with a simple, clean website, and nice tools — including a web browser based console for emergency situations.

2009-03-17

assertions in python

The brief version first:

# fine.
assert expr1, expr2

# WRONG!
assert(expr1, expr2)

Be very careful with the syntax of assertions in Python! If you get the syntax wrong, the assertions will just always hold. This is particularly disastrous, as assertions are usually used to expose hidden bugs.

The exact syntax that you should use is assert expr1, expr2. This tests expr1 in the boolean sense, and raises an exception with value str(expr2) if expr1 is False. You can also just use assert expr1, which won’t give a special message. Note the total lack of brackets. If you insert parentheses, though, as in assert(thing1, thing2), what you are really doing is calling assert (thing1, thing2). That is, an assertion with expr1 being (thing1, thing2)! This will never evaluate to False, so you will never get an error.

2009-03-16

git

Show the history of a particular file tracked by git, using

git log -p filename

2009-03-14

git

A nice list of some of git’s functionality.

2009-03-09

git

To immediately view the old version of a file tracked by git, use, for example,

git show HEAD^:filename

2009-03-03

creating a group on mac

To create a new group on mac with a specific gid, then add a user, type

dseditgroup -o create -i 499 groupname
dseditgroup -o edit -a username groupname

2009-02-26

interesting bio

Bio of Jeff Bezos, Amazon’s CEO.

2009-02-24

nicely designed website, about fruit

Nicely designed fruit website. Especially check out the charts which show fruit availability. Very clear.

2009-02-11

quick python code for a fifo dictionary

This dictionary stores a limited number of entries, removing the oldest entries when it gets full. This could be used for a cache. Create a max-50-entry dictionary with f = FifoDict(50), for example.

class FifoDict(object):
    def __init__(self, maxentries=100):
        self.lastentries = []
        self.mydict = {}
        self.maxentries = maxentries

    def __getitem__(self, k):
        return self.mydict[k]

    def __setitem__(self, k, val):
        if len(self.lastentries) >= self.maxentries:
            del self.mydict[self.lastentries.pop()]
        self.mydict[k] = val
        self.lastentries.insert(0, k)

    def __str__(self):
        return str(self.mydict)

    def __repr__(self):
        return repr(self.mydict)

2009-02-10

nice web hosting website

(mt) Media Temple have a very nicely designed website (though quite small font sizes).

2009-01-31

xdvi

The letter k toggles keep mode in xdvi; with keep on, the horizontal and vertical page offset remains the same, even when switching pages.

2009-01-30

batch rename files in bash

Let’s say we want to remove the prefix one from a series of files. Use this:

for FILE in one* ; do mv $FILE `echo $FILE | sed 's/one//'` ; done

or use `rename`

Thanks to the reader who pointed out that, in Debian GNU Linux (for example), you can also use rename, as in

rename 's/^one//' one*

Much cleaner, if rename is available.

2009-01-09

adobe color scheme picker

kuler.

2009-01-08

guide to git

A nice guide to git.

2008-12-06

logging javascript error messages

In Safari (and possibly other browsers), you can log a small message to the debug console with the javascript command console.log(message). This is helpful for debugging.

2008-11-30

interesting blog

amix.dk. Has some code segments, which might be interesting to look at if nothing else. (2009-02-26, appears to be broken.)

2008-11-18

readline on mac

This page pointed out a bizarre but perfect way of fixing readline on mac. Apparently the GNU license doesn’t play nice for Apple so it doesn’t work by default. I did this:

sudo easy_install ipython
sudo easy_install -f http://ipython.scipy.org/dist/ readline

And suddenly readline worked in ipython on mac.

interesting design

Very odd, but also quite interesting industrial design. Reading this made me realise that sorting these entries by date is silly. I’ll fix this sometime.

2008-11-06

stack size

I was shown an optimization solver that ran fine with certain parameter sizes, yet started segfaulting with larger problem sizes. It turned out that a desired single large, amorphous array could not be allocated within the available stack size limit. Increasing the stack size limit via ulimit -s 50000 (for example) fixed the problem.

You can view a shell’s resource limits using, for example,

monolith> ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) 6144
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 256
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 266
virtual memory          (kbytes, -v) unlimited

coffee

Peregrine espresso in Washington, DC (map). The best espresso I’ve had in the US.

2008-10-19

git configuration

Useful page with git configuration details.

2008-10-05

nice airline booking website

travelocity done right: ITA software.

vpn on mac

After adding a VPN on mac, you might find it doesn’t really work. Add a new (ethernet, say) interface with network settings, and pick an address on the VPN’s subnet. Then you’ll be able to use services on that server by explicitly referring to that IP.

Sadly, mac’s ‘seemless user interface’ doesn’t seem to carry over to leopard server…

2008-09-08

python

(Only) somewhat ok page about re-raising exceptions.

2008-09-03

git

Set up a new tracked repository.

git remote add remotename username@server:~/repo/

2008-09-02

passwd

Expire a user’s password, requiring them to change it immediately upon their next login:

passwd -e username

free fonts

Nice looking free fonts: exljbris and Gentium.

samba over ssh on mac

Leopard irritatingly prevents you from creating an ssh tunnel to another computer and then connecting a samba share to your localhost. Get around this by first creating an alias for your localhost interface:

sudo ifconfig lo0 alias 127.0.0.2 up

Then, forward the ports via ssh, eg with

sudo ssh -N -L 127.0.0.2:139:localhost:139 -L 127.0.0.2:445:localhost:445 user@server.com

Finally, connect using Finder to smb://127.0.0.2/sharename.

2008-09-01

vim

Paste last executed command in vim: ":p.

vim

Nice way of having two different kinds of syntax highlighting within one file, with vim.

2008-08-25

openvpn

Great setup guide for openvpn. Goes well with Tunnelblick.

2008-08-21

valgrind

Check for common memory errors in C with valgrind. Also profile cache misses and branch mispredictions.

Sample usage:

# Check memory accesses.
valgrind --tool=memcheck ./prog

# View cache and branch profiling information.
valgrind --tool=cachegrind ./prog

For more detailed output from valgrind, add the -g tag to your gcc compilation flags to add debugging information to the binary.

2008-08-16

psfrag

The key component of psfrag is the position specification. Here is the relevant syntax:

\psfrag{tag}[position]{Replacement text}

The position is up to two letters; one from {t, b, B, c} (top, bottom, baseline, center) and one from {l, r, c} (left, right, center). Center is assumed if either is omitted.

mac

Infinite labs: excellent little applications for mac, including PlugSuit and Afloat.

vim

Nice color scheme for vim: ir_black.

2008-08-14

vim

Disable highlighting of spurious _ and ^ symbols in LaTeX files as errors (say, when a macro is used to start equation environments).

autocmd Filetype tex syntax clear texOnlyMath

2008-08-13

make

Use multiple processes (and hence cores) with make by using make -j.

ssh

Disconnect quickly, even when ^d is not allowed, with ~..

python

A quick way of finding out where a Python package lives:

>>> import pylab
>>> print pylab
<module 'pylab' from '/usr/lib/python2.5/site-packages/pylab.pyc'>

2008-08-11

python

Run a Python script through a profiler, then interpret the output. I don’t think it’s worth the bother going through iPython; I couldn’t get %run -p to work properly.

At the shell:

> python -m cProfile -o outprof.tmp examples/bb30.py

Then, in iPython:

>>> import pstats
>>> p = pstats.Stats('outprof.tmp')
>>> p.strip_dirs().sort_stats('cumulative').print_stats(50)

bash

A reminder of how tee works comes from hunterdavis.com.

./prog | tee >(gzip > logfile.tgz)

ipe on mac

From this blog, where there is an installer for ipe for mac, it seems that to convert back from a pdf to an eps the following is required.

gs -dBATCH -dNOPAUSE -dNOPROMPT -sDEVICE=epswrite -sOutputFile=new.eps old.pdf

find and sed on mac

find and sed operate slightly differently on a mac. Here’s some code that replaces 4.3 with 4.4 in all Makefiles, recursively.

find . -name 'Makefile' -exec sed -i -e 's/4\.3/4.4/g' {} \;

2008-08-06

vim

The numbered register "0 contains the text from the most recent yank command, while the numbered register "1 contains the text from the most recent delete or change command. If the delete or change was ‘small’, though, (ie, less than one line), the small delete register "- is used instead.

This means that if you yank some text to move it elsewhere, then rearrange some stuff (including deleting, say, newlines), you can still easily retrieve the text that you yanked with "0p.

2008-08-05

latex

I made some aliases for use when writing mbox{LaTeX} documents. Add these (perhaps unwieldly) lines to your .bashrc:

# Next command on a single line.
alias lr='latex -file-line-error-style -interaction=nonstopmode $LR.tex >
  /tmp/latexoutput || cat /tmp/latexoutput ;
  grep -e "\(LaTeX Warning\)\|\(Overfull\)\|\(Underfull\)" /tmp/latexoutput |
  grep -v "There were undefined references";
  killall -SIGUSR1 xdvi.bin 2> /dev/null'

alias xdj='xdvi -paper us -expert -keep $LR.dvi 2>/dev/null >/dev/null&;'

Note: depending on your system, you made need to replace xdvi.bin with xdvi-xaw3d.bin.

Then, when you are preparing a document, you can do this:

> LR=myfilename # set filename. The suffix .tex is added automatically.
> lr            # compiles your latex document.
> xdj           # displays your latex document.

This sounds pretty ordinary, but these aliases take advantage of these things:

lr will only show the output if there was a problem. This avoids polluting your screen.
lr will never stop with an error while running and leave you punching keys in the hopes of getting back to the command line.
will tell you the line number of any errors.
lr will automatically kick xdvi after compiling the document to show your changes. No need to click on xdvi or wait for it to notice the file changed. I really like the way xdvi permits such updates without flickering.

2008-08-04

gcc

To see the assembly instructions generated for a piece of C code, use this:

gcc -c -g -Wa,-ahl src.c

gcc

Find out the macros, and values, predefined by gcc (includes useful things like __INT_MAX__).

gcc -dM -E - <; /dev/null

2008-08-02

google chart api

At http://code.google.com/apis/chart/. Nice charts, like this one.

line chart

You generate these by going to a specially written url such as http://chart.apis.google.com/chart?cht=lc&chs=200x125&chd=t:50.0,5.0,4.0,93.0,17.0,18.0,13.0,15.0 for the chart above. It looks like it would be a bit of work to translate from a particular application to this api, though.

2008-07-31

matlab

Sampling from a discrete distribution, where p is a vector of probabilities:

min(find(rand() < cumsum(p)))

2008-07-29

skype

Woah, typing s/search/repl/ in a Skype conversation actually performs a replacement for both parties on your last message (only). You can also remove the last messages with s///. It turns out you can also do the same by right clicking.

matlab

Have a look at this bizarre Matlab behaviour:

>> 1 / [1 2 3]'
ans =
         0         0    0.3333

Apparently, this is how they wanted it. Excusing the use of backslash as if it were mathematics, we have the equivalence:

leading to the bizarre conclusion that in Matlab,

Of course, the syntax that I really wanted was 1 ./ [1 2 3]'. This is fully consistent with Matlab’s elementwise operations, but would it be so bad if 1 / [1 2 3]' just did not work, especially rather than silently giving you something totally weird but the same shape?

find

I’ve often tried to use find, but have never really got over its syntax. It seems that it’s one of those powerful tools that’s not necessarily obvious to use at first. Here’s one way to use it:

find . -name '*.sw*' -print

prints filename which have .sw in them (say, leftover vim swapfiles).

You can combine this with grepping:

find . -name "*.bib" -exec echo {}; grep "tribute to" '{}'

looks for the words ‘tribute to’ in files with the extension .bib.

2008-07-22

python & vim

Nice page describing enhancements vim has for Python. Will try the omni-complete from this page soon.

vim

Exceedingly useful for cutting and pasting from the clipboard in vim without indentation problems: :put +. (More info.) Depending on your system, :put * might be what you’re after instead.

gcc

Version 4.3 of gcc, apart from significant improvements, has this nice feature:

> gcc -c -Q -Os --help=optimizers
The following options control optimizations:
  -falign-jumps                         [disabled]
  -falign-labels                        [disabled]
  -falign-loops                         [enabled]
# ---8<---
  -fwhole-program                       [disabled]
  -fwrapv                               [disabled]

2008-07-21

C

A few code snippets for timing in C. First, tic() and toc() functions. These work like Matlab: tic() starts the timer, and toc() stops it and gives formatted output. There’s also a tocq() function which just returns the number of elapsed seconds.

#include <time.h>
static clock_t tic_timestart;

void tic(void) {
    tic_timestart = clock();
}

float toc(void) {
    clock_t tic_timestop;
    tic_timestop = clock();
    printf("time: %8.2f.\n", (float)(tic_timestop - tic_timestart) / CLOCKS_PER_SEC);
    return (float)(tic_timestop - tic_timestart) / CLOCKS_PER_SEC;
}

float tocq(void) {
    clock_t tic_timestop;
    tic_timestop = clock();
    return (float)(tic_timestop - tic_timestart) / CLOCKS_PER_SEC;
}

A bit of code for counting flops: first, a statistics structure, then some defines.

/* CFDIV    divides             1/a[3]
 * CFADD    adds                c[12] + d[12]
 * CFMUL    multiplies          b[4] * h[7]
 * CFASN    assigns             A[5] = 0
 * CFSTO    stores              A[5] = (...)
 * CFTFR    transfers           A[5] = b[7]
 * CFNTF    negated transfers   A[5] = -b[7]
 * CFCOM    compares            c[5] < b[7]
 */
typedef struct statistics_t *statistics;
typedef struct statistics_t {
    long div;
    long add;
    long mul;
    long asn;
    long sto;
    long tfr;
    long ntf;
    long com;
} Statistics;

#define CFDIV(i) if (countflops) stats->div += i
#define CFADD(i) if (countflops) stats->add += i
#define CFMUL(i) if (countflops) stats->mul += i
#define CFASN(i) if (countflops) stats->asn += i
#define CFSTO(i) if (countflops) stats->sto += i
#define CFTFR(i) if (countflops) stats->tfr += i
#define CFNTF(i) if (countflops) stats->ntf += i
#define CFCOM(i) if (countflops) stats->com += i

jemnotes

2009-06-19

hush login

2009-06-11

filename modifiers in vim

2009-05-27

test-driven development

2009-05-25

rails / single table inheritance

2009-05-22

python / line_profiler

2009-05-21

python

2009-05-18

rails + .gitignore

2009-05-07

terminal on mac

vim

2009-05-05

vim

2009-05-01

python

2009-04-14

py.test

nice tour of git

2009-03-31

ruby

google trends

2009-03-31

git

2009-03-30

fluid

git

2009-03-28

git

jQuery

2009-03-27

python

2009-03-26

slicehost

2009-03-17

assertions in python

2009-03-16

git

2009-03-14

git

2009-03-09

git

2009-03-03

creating a group on mac

2009-02-26

interesting bio

2009-02-24

nicely designed website, about fruit

2009-02-11

quick python code for a fifo dictionary

2009-02-10

nice web hosting website

2009-01-31

xdvi

2009-01-30

batch rename files in bash

or use rename

2009-01-09

adobe color scheme picker

2009-01-08

guide to git

2008-12-06

logging javascript error messages

2008-11-30

interesting blog

2008-11-18

readline on mac

interesting design

2008-11-06

stack size

coffee

2008-10-19

git configuration

2008-10-05

or use `rename`