ENIGMA CURRY

Wrangling my finances

I've gotten better at thinking about my finances over the years but I've never had anything better than a mental budget. I've never actually written anything down on paper. I've got a brand new job so I figure I've got a clean slate. Starting right now, I pledge to myself to keep a balanced checkbook at all times and to do monthly reviews of my finances looking for things to cut out of my budget.

Fiscal Conscience: Ha! How many times have you said that?
Me: Dozens of times!
Fiscal Conscience: And did you ever actually do it?
Me: No, but this time it will be different!
Fiscal Conscience: You're going to have do to better than that!
Me: Fine, I'll show you.

GnuCash is a wonderful piece of open-source accounting software. I have used it myself, dozens of times. But as it happens I inevitably stop using it. Not because the software is flawed per se, it does everything a good accounting application should do:

• Use Double-Entry accounting
• Support multiple currencies
• OFX import from online banking sites
• etc

The only thing they get wrong is the fundamental design choice they made when developing the application: they made it a GUI. Sure, GUIs are great, but in my opinion GUIs should be interfaces to a service oriented application rather than the application itself. Unless I'm sitting at my desk at home I can't use my application. I don't consider remote desktops a reasonable response to this problem due to the huge network latency issues as well as firewall problems.

I want to be able to record my transactions from anywhere. At home at my desk, at work at my desk, on my N800 when traveling, or on my phone when running around. With an application fundamentally written as a GUI this is essentially an insurmountable problem. An application written as a service can do all of these things quite easily with minimal amounts of programming.

Enter Ledger.

Ledger does not keep track of your accounts. You keep track of your accounts in a plain text file and Ledger helps you understand them better. I can update my checkbook register from any text editor, anywhere I am. Most often this is Emacs through SSH on my home computer. Because of the triviality of the file format (being plain text) I can write a simple application that takes text messages from my phone and adds them to the register automatically.

But what about all my automatic payments?

I have most of my monthly payments automatically debited from my account. This is nice since I don't have to spend the time submitting payments to X number of companies each month, but is a nightmare when it comes time to balancing my checkbook. What I have done in the past is to just import an OFX file from my bank directly into GnuCash. This works great until you realize that something is wrong with the import and your checkbook is no longer balancing correctly. For this reason alone I prefer to record each transaction I make by hand. My money is my responsibility after all.

So, I wanted to be instantly notified of any debit on my checking account as soon as it happens. This gives me the following:

• A text message to my phone describing the time, place and amount of the transaction
• The total balance of my account
• A way to constantly keep an eye on how I'm spending my money without the hassle of (remembering to) signing into my bank account
• A reminder to balance my checkbook. (tells me when my actual checking balance has been out of sync with ledger for over 48 hours.)

The central peice to this process is getting the transaction data from my bank account. My bank offers online banking but it does not offer an easy way to download my transactions. Sure it supports Quicken and MS Money exports, but its all protected behind a password protected javascript-enabled website. I wanted to use the wonderful mechanize library for python. However, it doesn't support javascript.

Enter Selenium RC.

Selenium RC is an application that can remote control a real web browser and submit and receive data. I run selenium which launches Firefox in a headless X-server (Xvfb) and continuously refreshes my online banking site and parses out my account activity. This is probably a bit memory intensive to always keep a browser open for this specific purpose. I may want to experiment with python-spidermonkey in the future and go back to using mechanize, but I think there's a lot of glue missing in that solution whereas Selenium RC works perfectly right now!

No code to share at the moment, it's all a giant hack. If a similar setup appeals to someone out there, let me know and I'll think about releasing something.

Shortened URLs in Emacs using is.gd (like tinyurl)

I've casually been teaching myself emacs lisp lately. Today I wrote a utility that shortens long urls within regions using the http://is.gd URL shortening service. There's plenty of existing code out there that is more lisp like, but this is supposed to be a learning experience for me so I did it myself. I like python and so I used python for most of the heavy lifting.

I created a directory to hold all of my emacs specific python functions: ~/.emacs.d/ryan-pymacs-extensions

I wrote the following python function, shorten_url.py in that directory:

!/usr/bin/env python
# -*- coding: utf-8 -*-

__author__ = "Ryan McGuire (ryan@enigmacurry.com)"
__date__   = "Mon Sep 15 12:27:14 2008"

import doctest
import urllib2
import re

def shorten_with_is_gd(url):
    """Shorten a URL with is.gd

    >>> shorten_with_is_gd('http://www.enigmacurry.com')
    'http://is.gd/FFP'

    """
    u = urllib2.urlopen("http://is.gd/api.php?longurl="+url)
    return u.read()

def shorten_in_text(text):
    """Shorten all the urls found inside some text

    >>> shorten_in_text('Hi from http://www.enigmacurry.com')
    'Hi from http://is.gd/FFP'
    
    """
    replacements = {} #URL -> is.gd URL
    #Only check for urls that start with "http://" for now
    for m in re.finditer("http://[^ \n\r]*", text):
        try:
            replacements[m.group()] = shorten_with_is_gd(m.group())
        except:
            replacements[m.group()] = m.group()
    for url,replacement in replacements.items():
        text = text.replace(url, replacement)
    return text

if __name__ == '__main__':
    doctest.testmod(verbose=True)

and the following lisp makes "M-x shorten-url" do the rest of the replacement work:

;add ~/.emacs.d/ryan-python-extensions to python path
(pymacs-exec "import sys, os")
(pymacs-exec "sys.path.append(os.path.join(os.path.expanduser('~'),'.emacs.d','ryan-pymacs-extensions'))")

;;Shorten URLs with is.gd
(pymacs-exec "import shorten_url")
(defun shorten-url (start end)
  (interactive "r")
  (let ((region (buffer-substring start end)))
    (let ((rt (pymacs-eval (format "shorten_url.shorten_in_text('''%s''')" region))))
      (kill-region start end)
      (insert rt)
      )
  ))

Updated my Free Talk Live log

I forget if I ever mentioned this on here or not -- I regularly (though not frequently) call Free Talk Live to express some of my opinions on various issues. I like to record these calls for posterity mostly so that I can, like with this blog, see where I have grown and evolved over the years.

Today I called in about Intellectual Property.

Check out my Free Talk Live log

I need to step up the frequency of both blog posts and calls to talk shows. Both are great fun and make me feel like I've accomplished something.

Importing foreign (non-iCal) calendars into Google Calendar

Google Calendar is highly useful. I use it to keep track of all my appointments and due dates and I get helpful reminders when dates approach via email and SMS. Not only that, but it allows me to collaborate with other people's calendars as well, and they don't even have to use Google Calendar because Google supports the industry standard iCalendar format. Things are great.

Unfortunately, iCalendar format is pretty new, and not everyone is using it.

Because Google Calendar is so useful, it is annoying when you find a calendar that is not in iCal format. Two of note that I want to follow are

• New Hampshire Underground
• Free Keene

Both of these calendars are running Simple Machines Forum, version 1.x which does not support iCalendar format (presumably they will in 2.0).

So I wrote an exporter: Download SMF iCal exporter

The exporter scrapes the calendar page on an SMF enabled site and dumps out an iCal compatible file.

Usage: smf_ical_converter.py -u http://yourforum.com/index.php -o cal.ics

Download a SMF 1.x forum calendar and dump in iCal format

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -u URL, --url=URL     URL of forum (up to and including /index.php)
  -n Name, --name=Name  Name of the Forum / Calendar (name goes in .ics file)
  -i EXPR, --ignore-re=EXPR
                        Ignore any event containing this regular expression
                        (specify as many -i as you want)
  -o File, --output=File
                        iCal filename to write
  -v, --verbose         Be verbose about process
  --months-backward=NUM
                        Number of months to go backward
  --months-forward=NUM  Number of months to go forward

I have this tool running in a cron job to keep up to date with the two above mentioned calendars. You can import these URLs directly into your Google Calendar:

• NHUnderground.com iCal calendar
(Includes just events, I use "-i 'Birthdays:'" for this one)
• FreeKeene.com iCal calendar

Update 09/16/08: I uploaded version 2 of this application. It has a better README and it now supports user specific date ranges.

Emacs IRC (ERC) with Noticeable Notifications

I use ERC for all my IRC chatting. I finally got fed up with not noticing someones message because I didn't have emacs focused. So I spent my evening concocting a more noticeable messaging system through Pymacs and libnotify.

Half way though implementing this, wouldn't you know it, I found ErcPageMe which does exactly what I wanted. I figured I was learning quite a bit and I continued writing my own version. I expanded on their code and (at least for me) made some improvements. So kudos go to whoever wrote ErcPageMe :)

The following code will pop up a message on your gnome desktop alerting you whenever you receive a personal message or when someone mentions your nickname in a channel. It also avoids notification for the same user in the same channel if they triggered a message within the last 30 seconds.

Here is my lisp and embedded python/pymacs code:

(defun notify-desktop (title message &optional duration &optional icon)
  "Pop up a message on the desktop with an optional duration (forever otherwise)"
  (pymacs-exec "import pynotify")
  (pymacs-exec "pynotify.init('Emacs')")
  (if icon 
      (pymacs-exec (format "msg = pynotify.Notification('%s','%s','%s')"
                           title message icon))
    (pymacs-exec (format "msg = pynotify.Notification('%s','%s')" title message))
    ) 
  (if duration 
      (pymacs-exec (format "msg.set_timeout(%s)" duration))
    )
  (pymacs-exec "msg.show()")
  )

;; Notify me when someone wants to talk to me.
;; Heavily based off of ErcPageMe on emacswiki.org, with some improvements.
;; I wanted to learn and I used my own notification system with pymacs
;; Delay is on a per user, per channel basis now.
(defvar erc-page-nick-alist nil
  "Alist of 'nickname|target' and last time they triggered a notification"
  )
(defun erc-notify-allowed (nick target &optional delay)
  "Return true if a certain nick has waited long enough to notify"
  (unless delay (setq delay 30))
  (let ((cur-time (time-to-seconds (current-time)))
        (cur-assoc (assoc (format "%s|%s" nick target) erc-page-nick-alist))
        (last-time))
    (if cur-assoc
        (progn
          (setq last-time (cdr cur-assoc))
          (setcdr cur-assoc cur-time)
          (> (abs (- cur-time last-time)) delay))
      (push (cons (format "%s|%s" nick target) cur-time) erc-page-nick-alist)
      t)
    )
  )
(defun erc-notify-PRIVMSG (proc parsed)
  (let ((nick (car (erc-parse-user (erc-response.sender parsed))))
	(target (car (erc-response.command-args parsed)))
	(msg (erc-response.contents parsed)))
    ;;Handle true private/direct messages (non channel)
    (when (and (not (erc-is-message-ctcp-and-not-action-p msg))
               (erc-current-nick-p target)
	       (erc-notify-allowed nick target)
	       )
      ;Do actual notification
      (ding)
      (notify-desktop (format "%s - %s" nick
                              (format-time-string "%b %d %I:%M %p"))
                      msg 0 "gnome-emacs")
      )
    ;;Handle channel messages when my nick is mentioned
    (when (and (not (erc-is-message-ctcp-and-not-action-p msg))
               (string-match (erc-current-nick) msg)
               (erc-notify-allowed nick target)
	       )
      ;Do actual notification
      (ding)
      (notify-desktop (format "%s - %s" target
                              (format-time-string "%b %d %I:%M %p"))
                      (format "%s: %s" nick msg) 0 "gnome-emacs")
      )
    )
      
  )

(add-hook 'erc-server-PRIVMSG-functions 'erc-notify-PRIVMSG)