ENIGMA CURRY

Distributing Jython Apps in a Single JAR file

I've been writing a lot of one-off type applications at work lately, which is always kind of a joy for me because these sorts of applications don't need to be maintained or supported in any way, which means I can write them however I want.

So I write them in Python :)

Jython allows me to interface with all the plethora of Java libraries that we use at work, and it lets me program in a language that not only I can tolerate, but one that I respect and love.

But even though these are one-off applications, they still need to be easy to use, and in some cases I won't even be the one running the application. I want these applications to just work damnit.

So, packaging my Jython application into a single executable jar file that contains all of the third party dependancies inside is my goal. I want to send the user the jar file, have them double click on it, and have it immediately start running. It can't get a whole lot easier than that.

The Jython wiki has a page about doing something along these lines. The recipe there called the Jar Method works quite well. The one drawback that it has is that all of the Java dependancies need to be exploded into the main jar root, which when you're dealing with dozens of jar dependancies, it can start to get tedious, messy, and in some cases will even violate the license of a particular library.

One-Jar is a special class loader that can load a jar file that is inside of another jar file, something that the regular class loader from Sun is incapable of doing. Using One-Jar lets my application reside inside of a jar file and contain all my dependancies as seperate jar files inside the main jar file.

I've created a sample project that shows how I normally create a new Jython project hosted inside a single jar file with One-Jar. You'll need the following tools to check out the project:

• A Java SDK (tested with OpenJDK 1.6)
• Apache Ant (tested with version 1.7)
• Git (to checkout the project)

Check out the project like so:

 git clone git://github.com/EnigmaCurry/Single-JAR-Jython-Example.git

Build the project:

cd single-jar-jython-example

ant

Run the example by double clicking it or via the command line:

 java -jar JythonExcelExample.jar

This is just a demonstration app, it doesn't do a whole lot, it outputs an excel file in the current directory listing some computer parts. The point of the application is to show how Jython can integrate with existing Java third-party libraries (in this case Apache POI.)

Instructions for basing your own application on this example are contained inside the README.txt file.

Jython Shell Server -- Adding a REPL to any Java app

I love programming in Python. I get paid to write Java though. Due to Java's verboseness, and lack of a REPL, this can be very frustrating for me.

In Python, the usual way I explore a new library or mock up a new idea is to immediately start coding in Python's interactive interpreter (or REPL). This is often times more efficient than reading (let alone finding) the documentation for the library. I can quickly see if something is going to work before I code inside my larger application.

Java doesn't have an interactive interpreter.. but Jython does! However, setting up Jython, especially interfacing with an already large Java application, can be difficult. One such difficult situation I deal with at work is in Weblogic. With Weblogic, I deploy my application directly to a running Weblogic server, and I never get to see a console in this process, so how am I ever going to run a Jython interactive interpreter?

This morning I got bored and wrote up a quick solution: JythonShellServer. JythonShellServer embeds into any Java application and starts a Telnet server that serves up Jython interactive shells. You can push any Java object that you want to manipulate into Jython's local environment. Run "telnet localhost 7000" and you can use Python code to explore your entire application's running environment.

JythonShellServer works, but I only just wrote it this morning, so consider it alpha quality at the moment. Check out the project page on github.

AutoComplete.el : Python Code Completion in Emacs

A friend of mine and I like to show off to each other little Emacs tips and tricks we learn. Today, he introduced to me the very cool AutoComplete.el package. AutoComplete.el is intriguing to me because, unlike ropemacs mode which I've blogged about before, the completions AutoComplete.el provides are inline with your code in a dropdown box instead of in a seperate window (windows in Emacs are what most people call frames). However, AutoComplete is only a completion framework, it doesn't know anything about Python. Instead, it allows the user to plug into it, feeding it whatever sorts of intelligent completion you want, including Rope.

Setup

The two most important completions I want to integrate into it are Rope and yasnippet. AutoComplete can handle both of them nicely. You'll need to install the very latest development version (as of December '08) of Rope, Ropemacs and Ropemode:

sudo apt-get install mercurial
mkdir /tmp/rope && cd /tmp/rope
hg clone http://bitbucket.org/agr/rope
hg clone http://bitbucket.org/agr/ropemacs
hg clone http://bitbucket.org/agr/ropemode
sudo easy_install rope
ln -s ../ropemode/ropemode ropemacs/
sudo easy_install ropemacs

You'll also need to install Pymacs and Yasnippet if you haven't already:

mkdir -p ~/.emacs.d/vendor && cd ~/.emacs.d/vendor
wget http://pymacs.progiciels-bpi.ca/archives/Pymacs-0.23.tar.gz
tar xfv Pymacs-0.23.tar.gz
cd Pymacs-0.23
make
sudo easy_install .
cd ~/.emacs.d/vendor
wget http://yasnippet.googlecode.com/files/yasnippet-0.5.9.tar.bz2
tar xfv yasnippet-0.5.9.tar.bz2
cd ~/.emacs.d
ln -s vendor/yasnippet-0.5.9/snippets/ .

Make sure Pymacs and Yasnippet get into your load path, in your .emacs:

(add-to-list 'load-path "~/.emacs.d/vendor")
(progn (cd "~/.emacs.d/vendor")
       (normal-top-level-add-subdirs-to-load-path))

Install AutoComplete.el 0.1.0 (0.2.0 is now out, and is not compatible with this post, I'll try and update later):

cd ~/.emacs.d/vendor
wget http://www.emacswiki.org/emacs/?action=browse;id=auto-complete.el;raw=1;revision=5

Now to add some more elisp to your .emacs somewhere:

(require 'python)
(require 'auto-complete)
(require 'yasnippet)

(autoload 'python-mode "python-mode" "Python Mode." t)
(add-to-list 'auto-mode-alist '("\\.py\\'" . python-mode))
(add-to-list 'interpreter-mode-alist '("python" . python-mode))

;; Initialize Pymacs                                                                                           
(autoload 'pymacs-apply "pymacs")
(autoload 'pymacs-call "pymacs")
(autoload 'pymacs-eval "pymacs" nil t)
(autoload 'pymacs-exec "pymacs" nil t)
(autoload 'pymacs-load "pymacs" nil t)
;; Initialize Rope                                                                                             
(pymacs-load "ropemacs" "rope-")
(setq ropemacs-enable-autoimport t)

;; Initialize Yasnippet                                                                                        
;Don't map TAB to yasnippet                                                                                    
;In fact, set it to something we'll never use because                                                          
;we'll only ever trigger it indirectly.                                                                        
(setq yas/trigger-key (kbd "C-c <kp-multiply>"))
(yas/initialize)
(yas/load-directory "~/.emacs.d/snippets")



;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;                                         
;;; Auto-completion                                                                                            
;;;  Integrates:                                                                                               
;;;   1) Rope                                                                                                  
;;;   2) Yasnippet                                                                                             
;;;   all with AutoComplete.el                                                                                 
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;                                         
(defun prefix-list-elements (list prefix)
  (let (value)
    (nreverse
     (dolist (element list value)
      (setq value (cons (format "%s%s" prefix element) value))))))
(defvar ac-source-rope
  '((candidates
     . (lambda ()
         (prefix-list-elements (rope-completions) ac-target))))
  "Source for Rope")
(defun ac-python-find ()
  "Python `ac-find-function'."
  (require 'thingatpt)
  (let ((symbol (car-safe (bounds-of-thing-at-point 'symbol))))
    (if (null symbol)
        (if (string= "." (buffer-substring (- (point) 1) (point)))
            (point)
          nil)
      symbol)))
(defun ac-python-candidate ()
  "Python `ac-candidates-function'"
  (let (candidates)
    (dolist (source ac-sources)
      (if (symbolp source)
          (setq source (symbol-value source)))
      (let* ((ac-limit (or (cdr-safe (assq 'limit source)) ac-limit))
             (requires (cdr-safe (assq 'requires source)))
             cand)
        (if (or (null requires)
                (>= (length ac-target) requires))
            (setq cand
                  (delq nil
                        (mapcar (lambda (candidate)
                                  (propertize candidate 'source source))
                                (funcall (cdr (assq 'candidates source)))))))
        (if (and (> ac-limit 1)
                 (> (length cand) ac-limit))
            (setcdr (nthcdr (1- ac-limit) cand) nil))
        (setq candidates (append candidates cand))))
    (delete-dups candidates)))
(add-hook 'python-mode-hook
          (lambda ()
                 (auto-complete-mode 1)
                 (set (make-local-variable 'ac-sources)
                      (append ac-sources '(ac-source-rope) '(ac-source-yasnippet)))
                 (set (make-local-variable 'ac-find-function) 'ac-python-find)
                 (set (make-local-variable 'ac-candidate-function) 'ac-python-candidate)
                 (set (make-local-variable 'ac-auto-start) nil)))

;;Ryan's python specific tab completion                                                                        
(defun ryan-python-tab ()
  ; Try the following:                                                                                         
  ; 1) Do a yasnippet expansion                                                                                
  ; 2) Do a Rope code completion                                                                               
  ; 3) Do an indent                                                                                            
  (interactive)
  (if (eql (ac-start) 0)
      (indent-for-tab-command)))

(defadvice ac-start (before advice-turn-on-auto-start activate)
  (set (make-local-variable 'ac-auto-start) t))
(defadvice ac-cleanup (after advice-turn-off-auto-start activate)
  (set (make-local-variable 'ac-auto-start) nil))

(define-key python-mode-map "\t" 'ryan-python-tab)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;                                         
;;; End Auto Completion                                                                                        
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

Github

These changes are also applied to my Emacs environment on Github. I'm continuously trying to improve my emacs setup, so if you're reading this a few months/years after this post was made, you may want to check there for improvements.

Update Jan 30 2009: I made some modifications to the tab complete order, a regular indent is applied first before autocompletion. Autocompletion is also not applied unless you are at the end of a word. This is useful when you press Tab at the beginning of a line to indent/outdent.

Usage

Once you got everything setup, usage is real easy:

1. Open up a python (.py) file
2. Press TAB when you want to use code completion or to insert a snippet.

The first time you attempt to use code completion you'll be prompted to enter the root of your project directory.

Future

It wasn't too long ago when Python code completion inside Emacs was just a pipedream. So now that we have it, let's make some more pipedreams: I'd like to popup some nice contextual help for method arguments as well as python documentation for the current method/class being completed.

The possibilities are pretty much endless. I love Emacs.

256 colors on the Linux terminal

I've been using Linux as my main OS for well over a decade now. I can hardly believe I didn't pick up on this tip before.

On your linux desktop, open up your favorite terminal emulator. Enter this command:

tput colors

What number do you see? 8?

8 stinking colors. That's what my terminal is capable of displaying. It's 2009 folks, and my terminal can only display 8 stinking colors.

Load up emacs in the terminal (emacs -nw) and see for yourselves, M-x list-colors-display shows you all of the different colors your terminal can display.

Pitiful.

Ok, so how to improve this? On ubuntu:

sudo apt-get install ncurses-term

and stick the following in your ~/.bashrc and/or ~/.bash_profile:

export TERM=xterm-256color

Ahhhhh...

My Emacs config on Github

The most popular post on this blog is Emacs as a powerful Python IDE. I get quite a few emails regarding that post and I realized the other day that I've done a lot of customizations to my .emacs, and subsequently that post is starting to get a little dated. Time is a curious creature, it seems like yesterday that I wrote that post, but it's been almost 9 months now.

I've been learning and playing around with git recently, so I've decided to post my entire .emacs config on github.

Here are the latest instructions for emulating my Emacs environment on Ubuntu 8.10:

• Enable the universe repository
• sudo apt-get update
• sudo apt-get install emacs-snapshot git-core automake libgconf2-dev texinfo python-setuptools
• cd ~
• git clone git://github.com/EnigmaCurry/emacs .emacs.d
• cd ~/.emacs.d
• git submodule init
• git submodule update
• cd ~/.emacs.d/vendor
• ./vendor-compile.sh
• sudo easy_install "http://downloads.sourceforge.net/rope/rope-0.9.1.tar.gz?modtime=1225268769&big_mirror=0"
• sudo easy_install "http://downloads.sourceforge.net/rope/ropemacs-0.6.tar.gz?modtime=1223039342&big_mirror=0"

I've done some reorganization of the directory structure:

1. There actually is no ~/.emacs file anymore, having realized that Emacs looks for ~/.emacs.d/init.el. Convenient!
2. All third party packages are now installed in ~/.emacs.d/vendor. This was a great tip I saw in the Emacs Peepcode screencast. (it's $9, but worth it).
3. I've taken the suggestion that greg made in the comments to use submodules for some of the vendor packages. The vendor-compile.sh script does the compiling of those packages.
4. All private code goes in ~/.emacs.private. This lets me keep passwords and such outside of the main git repository and allows me to publish my config more easily. Before I did this I had to spend time sanitizing the code before each release.

I have appreciated your many encouraging Emails and comments. :)