ENIGMA CURRY

Shortened URLs in Emacs using is.gd (like tinyurl)

I've casually been teaching myself emacs lisp lately. Today I wrote a utility that shortens long urls within regions using the http://is.gd URL shortening service. There's plenty of existing code out there that is more lisp like, but this is supposed to be a learning experience for me so I did it myself. I like python and so I used python for most of the heavy lifting.

I created a directory to hold all of my emacs specific python functions: ~/.emacs.d/ryan-pymacs-extensions

I wrote the following python function, shorten_url.py in that directory:

!/usr/bin/env python
# -*- coding: utf-8 -*-

__author__ = "Ryan McGuire (ryan@enigmacurry.com)"
__date__   = "Mon Sep 15 12:27:14 2008"

import doctest
import urllib2
import re

def shorten_with_is_gd(url):
    """Shorten a URL with is.gd

    >>> shorten_with_is_gd('http://www.enigmacurry.com')
    'http://is.gd/FFP'

    """
    u = urllib2.urlopen("http://is.gd/api.php?longurl="+url)
    return u.read()

def shorten_in_text(text):
    """Shorten all the urls found inside some text

    >>> shorten_in_text('Hi from http://www.enigmacurry.com')
    'Hi from http://is.gd/FFP'

    """
    replacements = {} #URL -> is.gd URL
    #Only check for urls that start with "http://" for now
    for m in re.finditer("http://[^ \n\r]*", text):
        try:
            replacements[m.group()] = shorten_with_is_gd(m.group())
        except:
            replacements[m.group()] = m.group()
    for url,replacement in replacements.items():
        text = text.replace(url, replacement)
    return text

if __name__ == '__main__':
    doctest.testmod(verbose=True)

and the following lisp makes "M-x shorten-url" do the rest of the replacement work:

;add ~/.emacs.d/ryan-python-extensions to python path
(pymacs-exec "import sys, os")
(pymacs-exec "sys.path.append(os.path.join(os.path.expanduser('~'),'.emacs.d','ryan-pymacs-extensions'))")

;;Shorten URLs with is.gd
(pymacs-exec "import shorten_url")
(defun shorten-url (start end)
  (interactive "r")
  (let ((region (buffer-substring start end)))
    (let ((rt (pymacs-eval (format "shorten_url.shorten_in_text('''%s''')" region))))
      (kill-region start end)
      (insert rt)
      )
  ))

Importing foreign (non-iCal) calendars into Google Calendar

Google Calendar is highly useful. I use it to keep track of all my appointments and due dates and I get helpful reminders when dates approach via email and SMS. Not only that, but it allows me to collaborate with other people's calendars as well, and they don't even have to use Google Calendar because Google supports the industry standard iCalendar format. Things are great.

Unfortunately, iCalendar format is pretty new, and not everyone is using it.

Because Google Calendar is so useful, it is annoying when you find a calendar that is not in iCal format. Two of note that I want to follow are

• New Hampshire Underground
• Free Keene

Both of these calendars are running Simple Machines Forum, version 1.x which does not support iCalendar format (presumably they will in 2.0).

So I wrote an exporter: Download SMF iCal exporter

The exporter scrapes the calendar page on an SMF enabled site and dumps out an iCal compatible file.

Usage: smf_ical_converter.py -u http://yourforum.com/index.php -o cal.ics

Download a SMF 1.x forum calendar and dump in iCal format

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -u URL, --url=URL     URL of forum (up to and including /index.php)
  -n Name, --name=Name  Name of the Forum / Calendar (name goes in .ics file)
  -i EXPR, --ignore-re=EXPR
                        Ignore any event containing this regular expression
                        (specify as many -i as you want)
  -o File, --output=File
                        iCal filename to write
  -v, --verbose         Be verbose about process
  --months-backward=NUM
                        Number of months to go backward
  --months-forward=NUM  Number of months to go forward

I have this tool running in a cron job to keep up to date with the two above mentioned calendars. You can import these URLs directly into your Google Calendar:

• NHUnderground.com iCal calendar

(Includes just events, I use "-i 'Birthdays:'" for this one)

• FreeKeene.com iCal calendar

Update 09/16/08: I uploaded version 2 of this application. It has a better README and it now supports user specific date ranges.

Emacs IRC (ERC) with Noticeable Notifications

I use ERC for all my IRC chatting. I finally got fed up with not noticing someones message because I didn't have emacs focused. So I spent my evening concocting a more noticeable messaging system through Pymacs and libnotify.

Half way though implementing this, wouldn't you know it, I found ErcPageMe which does exactly what I wanted. I figured I was learning quite a bit and I continued writing my own version. I expanded on their code and (at least for me) made some improvements. So kudos go to whoever wrote ErcPageMe

The following code will pop up a message on your gnome desktop alerting you whenever you receive a personal message or when someone mentions your nickname in a channel. It also avoids notification for the same user in the same channel if they triggered a message within the last 30 seconds.

Here is my lisp and embedded python/pymacs code:

(defun notify-desktop (title message &optional duration &optional icon)
  "Pop up a message on the desktop with an optional duration (forever otherwise)"
  (pymacs-exec "import pynotify")
  (pymacs-exec "pynotify.init('Emacs')")
  (if icon
      (pymacs-exec (format "msg = pynotify.Notification('%s','%s','%s')"
                           title message icon))
    (pymacs-exec (format "msg = pynotify.Notification('%s','%s')" title message))
    )
  (if duration
      (pymacs-exec (format "msg.set_timeout(%s)" duration))
    )
  (pymacs-exec "msg.show()")
  )

;; Notify me when someone wants to talk to me.
;; Heavily based off of ErcPageMe on emacswiki.org, with some improvements.
;; I wanted to learn and I used my own notification system with pymacs
;; Delay is on a per user, per channel basis now.
(defvar erc-page-nick-alist nil
  "Alist of 'nickname|target' and last time they triggered a notification"
  )
(defun erc-notify-allowed (nick target &optional delay)
  "Return true if a certain nick has waited long enough to notify"
  (unless delay (setq delay 30))
  (let ((cur-time (time-to-seconds (current-time)))
        (cur-assoc (assoc (format "%s|%s" nick target) erc-page-nick-alist))
        (last-time))
    (if cur-assoc
        (progn
          (setq last-time (cdr cur-assoc))
          (setcdr cur-assoc cur-time)
          (> (abs (- cur-time last-time)) delay))
      (push (cons (format "%s|%s" nick target) cur-time) erc-page-nick-alist)
      t)
    )
  )
(defun erc-notify-PRIVMSG (proc parsed)
  (let ((nick (car (erc-parse-user (erc-response.sender parsed))))
	(target (car (erc-response.command-args parsed)))
	(msg (erc-response.contents parsed)))
    ;;Handle true private/direct messages (non channel)
    (when (and (not (erc-is-message-ctcp-and-not-action-p msg))
               (erc-current-nick-p target)
	       (erc-notify-allowed nick target)
	       )
      ;Do actual notification
      (ding)
      (notify-desktop (format "%s - %s" nick
                              (format-time-string "%b %d %I:%M %p"))
                      msg 0 "gnome-emacs")
      )
    ;;Handle channel messages when my nick is mentioned
    (when (and (not (erc-is-message-ctcp-and-not-action-p msg))
               (string-match (erc-current-nick) msg)
               (erc-notify-allowed nick target)
	       )
      ;Do actual notification
      (ding)
      (notify-desktop (format "%s - %s" target
                              (format-time-string "%b %d %I:%M %p"))
                      (format "%s: %s" nick msg) 0 "gnome-emacs")
      )
    )

  )

(add-hook 'erc-server-PRIVMSG-functions 'erc-notify-PRIVMSG)

cycle_xrandr.py : dual and single displays on Ubuntu

Linux power management has gotten good recently. My last two laptops (HP dv9000 and Macbook Pro) have supported hardware suspend mode out of the box. I'm impressed!

Being that I hardly ever turn off the laptop (I just suspend now) I'm left with an annoyance: I use two displays at work and one everywhere else. In Ubuntu, I was easily able to setup dual and single modes such that when I boot up, Xorg detects how many displays are connected appropriately. But now that I don't turn off the machine, I want to be able to just plug in another monitor and go. I don't want to have to reboot the computer. I don't even want to have to log out. I want to plug the monitor in, leave all my apps running and in the state they are in.

Xrandr does this and it works great. But what if I'm in dual display mode but I only actually have one monitor connected? Can I get to an open terminal to use xrandr? Easily? Probably not.

I wrote the following python script to take care of this. It detects all of the modes that xrandr knows about and cycles to the next size listed. I then bind this script to a custom keyboard shortcut so I don't have to type or even see anything on the screen:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

__author__ = "Ryan McGuire (ryan@enigmacurry.com)"
__date__   = "Thu Jul 10 15:27:18 2008"

"""Cycle through all screen resolutions detected by xrandr"""

from subprocess import Popen, PIPE
import re

size_re = re.compile("^   ([0-9]*x[0-9]*)\W*[0-9]*\.[0-9]*(\*)?")

def list_sizes():
    """List all sizes detected by xrandr,
    ordered by the next resolution to cycle to"""
    p1 = Popen(['xrandr'], stdout=PIPE)
    sizes = []
    current_size_index = 0
    for line in  p1.communicate()[0].split("\n"):
        m = size_re.match(line)
        if m:
            sizes.append(m.group(1))
            if m.group(2) == "*":
                current_size_index = len(sizes) - 1
    return sizes[current_size_index+1:] + sizes[:current_size_index+1]

if __name__ == '__main__':
    Popen(['xrandr','-s',list_sizes()[0]])

Emacs as a powerful Python IDE

Last night at the Python user group I gave a short demo on using Emacs as a Python editor/IDE. My macbook pro refused to display on the projector so I thought my demo was going to be a 'no go'. Thankfully, sontek allowed me to use his Linux laptop. I hurriedly copied over my emacs environment, installed a few packages and was able to present after all. I think the demo went fairly well even though I think it was a bit hurried and I forgot to cover a few things, I think I was pretty nervous at the same time because of the fact that the mac didn't work and got me flustered. Oh well, I think people enjoyed it.

My Emacs Environment

Below are the Emacs features most applicable to Python development:

• Rope and Ropemacs
  Rope is a general (non-emacs specific) Python IDE library. It has awesome support for multiple refactoring methods and code introspection. Inside Emacs, this gives us:
  • Full (working!!) code-completion support of modules, classes, methods etc. (M-/ and M-?)
  • Instant documentation for element under the cursor (C-c d)
  • Jump to module/class/method definition of element under the cursor (C-c g). This works for any Python code it finds in your PYTHONPATH, including things from the stdlib.
  • Refactoring of code (like rename — C-c r r)
  • List all occurences of of a name in your entire project
  • and More.
• YASnippet
  YASnippet is a snippet tool like TextMate. You can expand user defined keywords into whole blocks of predefined code. This is especially useful for the usual boilerplate that would go into a python file like

  #!/usr/bin/env python
  and
  if __name__ == '__main__':

  Granted, Python doesn't require much boilerplate, and therefore this package is much more suited to languages like Java, but I bring it up because I think its cool and if you get into the habit of using it, then a few keystrokes saved here and there will add up over time.

• Subversion support with psvn.el
  Psvn is a comprehensive subversion client for Emacs. It integrates well with ediff mode so you can use it to check changes between versions. It does all of the other boring subversion stuff well too.
• Ido-mode for buffer switching and file opening.
  Emacs, to the uninitiated, can be confusing because by default there is only one view into a single file at a time. How does one get to another file? Instead of cluttering the interface with GUIness and making the user click somewhere (and thereby forcing the user to waste their time by moving their hand off of the keyboard), Emacs gives powerful ways to switch between files. Ido-mode is one of these useful ways — it makes a list of open files starting with the most frequently visted files and widdles this list down as you type part of the filename. You can have dozens of files open and only be a few keystrokes away from any one of them.

A lot of people, for whatever reason, don't feel that Emacs is an IDE at all. I don't personally care what you define it as — the fact remains — Emacs is a powerful Python environment and despite being over 32 years old has proven to be just as modern as any IDE today, and remains THE most configurable editor (operating system?) ever.

I've tarred up my Emacs environment for general consumption. Instructions:

• Install Pymacs
• Install Rope and Ropemacs
• BTW, those three packages should be the only packages other than Emacs you'll need. Everything else is self contained.
• Extract the tarball to your home directory. This creates a directory called ryan-emacs-env.
• Rename "ryan-emacs-env" to ".emacs.d"
• Symlink my dot-emacs file to your .emacs. Run "ln -s .emacs.d/dot-emacs .emacs".
• If you also want to do Java development run "tar xfvz jde-2.3.5.1.tar.gz". I leave it tarred because you don't need to pollute your environment if you're not going to use Java. (Also for whatever reason, jde doesn't like to be stuck in my subversion repository so I just leave it tarred up and untar on every machine I check it out on.)

Extra tips:

• Put your .emacs.d directory under version control. Never rely on your distros emacs packages, install all future elisp files yourself in your .emacs.d file and commit to your repository often. This way you've got an environment that is easily transportable and synchronizable across multiple machines. This is the major reason why my emacs environment was so fast to trasnfer from my macbook pro to sontek's laptop during the demo.
• Speaking of sontek, he brought up an excellent point in #utahpython the other day, he's not going to be using my emacs environment except for reference, instead he's starting with a clean slate. This is by far the best and most prudent thing to do. My emacs environment is a culmination of several years of plugging in and deleting various packages and writing various snippets of elisp. Your needs are always going to be different than mine and you are also going to be better off by educating yourself along the way by creating your own.

Some more fun Emacs evangelism:

• a fascinating screencast of Emacs in the role of Ruby editor (many tips here apply to Python too)
• Emacs blogs I read:
  • Emacs life
  • Life is too short for bad code
  • Minor Emacs Wizardry
  • M-x all-things-emacs
  • Platypope.org
  • Know of any other good ones? Let me know.

New Slacker Screenlet released

Check it out: I updated my Slacker Screenlet today.

New in this version:

• Bug fixed where the screenlet-manager would respawn infinitely
• Integrated LastSlacker support

You can read more about the original version in my previous post.

LastSlacker is a way to scrobble what you listen to on Last.FM. This lets you share a log of what you listen to with friends. I've been meaning to add this feature for a long time now, until today it's been in a sorta working state for months! It feels good to have finally gotten this project pretty much finished.

Automated Time Lapse Photography in Python

Last night at the Utah Python User Group meeting, John Harrison had an awesome game he has made called Marshie Attacks. It combines PyGame, PySight, a projector and lasers. You point the laser at Marshie and the isight camera detects if the laser is on target, and if so, he blows up. Very cool stuff indeed.

His game depends on the Macintosh PySight library for video capture, but is otherwise cross platform. I decided to take a stab at getting images off my webcam in linux. Using gstreamer, it wasn't hard at all.

I don't have a projector, so I can't make a full Linux port Marshie Attacks, yet. This first stab is just to show how to grab images from a v4l2 device in Linux. This program takes time lapse photos, and saves them to disk with a configurable frequency.

Requirements include:
* Python (2.5)
* gstreamer 0.10
* Python Imaging Library

Download the code

Ron Paul on the Dollar, Gold, and Oil

Everyone that drives a car knows that gasoline has gotten a lot more expensive over the last few years. But very few people seem to correctly understand why this is. Ron Paul, in his latest debate appearance, laid it out quite plainly, but it takes keen and willing ears to fully appreciate it:

The value of oil hasn't changed at all — instead, the value of our dollar has tanked. The value of oil, relative to gold, has sat unchanged since 2000 (and even well before that). This means that if you got your paycheck in gold instead of dollars you'd still be paying the same price for gas as you were a decade ago.

The gas prices are not the fault of greedy capitalists. No gas company is "gouging" you. On the contrary, seemingly high gas prices are not due to any free market force at all, but rather due to the fascist cooperation of a complacent, apathetic, congress and the Federal Reserve. Our government has an insatiable appetite to print money out of thin air to support a massive military industrial complex as well as a socialist, redistributionist, welfare state. This monstrously inflates the money supply — robbing the value of your dollar.

I decided to prove this for myself, that the golden price of oil is relatively stable. I downloaded historical gold spot prices, as well as historical crude oil prices. Using a bit of Python and matplotlib, I produced the following graph:

I'm not an economist, so I was fairly pleased when I saw that what I came up with correlated fairly well with the graph that the Wall Street Journal published. This graph shows that oil costs 3.5 times as many dollars as it did in 2000, and yet the oil price in gold has barely changed at all in the past eight years.

Everyone should be putting their money into an account with interest of some kind, and not just letting their money sit around uninvested. But at the rate that the dollar is losing its value, even "high" interest investments aren't paying out faster than the current rate of inflation (and as the above graph shows, that inflation is a lot more than the 3% the government would tell you). If we simply legalized alternative, market based currencies (as opposed to raiding and plundering them), we could have much larger gains on our investments as well as not losing any value on any money left uninvested.

This issue has been the core of Dr. Paul's career since the 1960's. It has taken the American people a very long time to wake up to this issue, and so it is an immense credit to his character that he has shown an undying vigilance, these many years, to the pursuit of liberty. I too, have hope for America.

(If anyone is interested, here is the python code (as well as the data files) used for the graph. )

Slacker Screenlet

Having an absurdly large music collection can sometimes have it's downsides. Slacker is a nice way to listen to music when you can't decide what to listen to. Like with Pandora or Last.fm, you can create your own radio station by telling the player what kind of music you like and the player will automatically choose music to play for you. With Slacker, you can customize how much or how little new music you want to allow into the mix, which is the main reason I prefer it over Pandora.

On Windows, Slacker has created a nice standalone player. On Linux or Mac you're stuck with the web player. The nice thing about a standalone player is it won't interfere with my habit of closing my browser (thus killing my music). I decided I needed to find a way to separate the player from my main browser.

Screenlets are little, graphical, single-purpose eye-candy (written in Python :)). They can do things like show you the time or the current weather forecast, CPU load, the word of the day etc. Compiz allows these applications to run in their own layer such that they can appear and disappear quickly with the press of a button. They are quick and easy to get to, but stay out of your way when not needed.

I decided a screenlet for Slacker would be ideal, so I wrote one.

You can download my Slacker Screenlet for yourself.

Also see the following:

• Screenlets.org page
• Gnome-look page

Recovering an Ncftp password

A coworker today asked me for the password to an FTP account we use. I didn't have any idea what it was because I just have it stored in an ncftp bookmark. I looked in my ~/.ncftp/bookmarks file and sure enough it was in there but it was encoded.

For future reference, I just wanted to document here that the password is simply base64 encoded. So it's pretty easy to decode using a bit of python:


>>> import base64
>>> base64.decodestring("TheEncodedPassword")