Saturday, March 21, 2009

Stone Tools and Scala Development - Part II Building

Maven2+Scala-Maven-Plugin+ScalaTest+Emacs

In this post I'll fly through how I use Maven in combo with Emacs to build Scala programs.

But first, a couple of additional notes on Part 1, where I covered the basics of how I use Emacs to edit Scala source files.

My one only commenter (thanks Normen) seemed to find the blog useful but skipped the ctags setup. I highly recommend that anyone doing a trial of Emacs for Scala development do the ctags setup, build your tag file and then Alt-. over a type in your source. You just can't code without this.

This function and Alt-/ like completion are often used as reasons why people pick the heavier IDE's over Emacs even though they like the raw editing power of Emacs. They often don't know one can achieve decent support for these features in Emacs as well. Admittedly the IDEs are semantic in analysis where Emacs is syntactic. Ctags, completion and yas-snippets get you close to these capabilities and you can flat out edit code as well.

Second, I posted a small patch to the scala-tools list for a very annoying indentation issue which was just applied in the Scala repository. So either grab the patch from Nabble scala-tools or pull the recent Emacs mode code from subversion. Note the first patch post was incorrect and a correct patch was posted later in the chain.

I customized my colors a bit, and for me editing Scala code looks like this. The lower left corner shows the results of compiling my project from inside Emacs via Maven2 and the Scala Maven plugin.

Maven in and of itself is a big topic and I am not a Maven maven by any means. So I'm only covering the specifics of using Maven + Scala + Emacs.







The Setup

First we want to run Maven inside of Emacs to take advantage of the using Emacs to move from compiler errors to the location of the error in the source code. A google search located a small but nice bit of elisp to run maven from inside Emacs. I modified it somewhat for Scala purposes. Here it is in its entirety.
;; Simple but effective Maven2 support.

(require 'compile)

(defvar mvn-command-history nil
"Maven command history variable")

(defvar scala-compile-error-regex
'("^\\[WARNING\\] \\([a-zA-Z0-9/-]+[.]scala\\):\\([0-9]+\\): error:" 1 2 nil 2 nil))

(defun mvn-compile-buffer-name (mode)
"*Maven Compile*")

(set 'scala-compile-error-regex '("^\\[WARNING\\] \\([a-zA-Z0-9/-]+[.]scala\\):\\([0-9]+\\): error:" 1 2 nil 2 nil))

(set 'compilation-buffer-name-function 'mvn-compile-buffer-name)
(set 'compilation-error-regexp-alist (list scala-compile-error-regex))

(set 'compilation-mode-font-lock-keywords
'(("^\\[ERROR\\] BUILD FAILURE"
(0 compilation-error-face))
("^\\[WARNING\\]"
(0 compilation-warning-face))
("^\\(\\[INFO\\]\\)\\([^\n]*\\)"
(1 compilation-info-face)
(2 compilation-line-face))
("^\\[ERROR\\]"
(0 compilation-error-face))
("\\(Failures\\): [1-9][0-9]*,"
(0 compilation-error-face))
("\\(Failures\\): 0"
(0 compilation-info-face))
("\\(Errors\\): [1-9][0-9]*,"
(0 compilation-error-face))
("\\(Errors\\): 0"
(0 compilation-info-face))
("\\(Skipped\\): [1-9][0-9]*,"
(0 compilation-error-face))
("\\(Skipped\\): 0"
(0 compilation-info-face))
("Tests run: [0-9]+"
(0 compilation-warning-face))
("T E S T S"
(0 compilation-warning-face))
("Compilation finished at [^\n]+"
(0 compilation-info-face))
("^Compilation \\(exited abnormally\\|interrupt\\|killed\\|terminated\\|segmentation fault\\)\\(?:.*with code \\([0-9]+\\)\\)?.*"
(0 '(face nil message nil help-echo nil mouse-face nil) t)
(1 compilation-error-face)
(2 compilation-error-face nil t))))

(defun mvn-at-root (path)
"Determine if the given path is root."
(equal path (mvn-parent-path path)))

(defun mvn-parent-path (path)
"The parent path for the given path."
(file-truename (concat path "/..")))

(defun mvn-pom-at-path-p (path)
"Does a pom.xml exist in the given path."
(file-exists-p (concat path "/pom.xml")))

(defun mvn-find-path-to-pom ()
"Move up the directory tree for the current buffer until root or a pom.xml is found."
(let ((fn (buffer-file-name)))
(let ((path (file-name-directory fn)))
(while (and (not (mvn-pom-at-path-p path))
(not (mvn-at-root path)))
(setf path (file-truename (mvn-parent-path path))))
path)))

(defun mvn-master (&optional args)
(interactive)
"Determine the potential master pom by checking if there is a pom in a pom's parent directory. nil if no parent pom else the path to the parent."
(let ((pom-path (mvn-find-path-to-pom)))
(let ((parent-pom-path (file-truename (concat pom-path "/.."))))
(if (not (mvn-pom-at-path-p parent-pom-path))
(message "No master pom found.")
(compile (read-from-minibuffer "Command: "
(concat "mvn -o -f " parent-pom-path "/pom.xml clean install")
nil nil 'mvn-command-history))))))

(defun mvn (cmd)
"Runs maven in the current project. Starting at the directoy where the file being vsisited resides, a search is
made for pom.xml recsurively. A maven command is made from the first directory where the pom.xml file is found is then displayed
in the minibuffer. The command can be edited as needed and then executed. Errors are navigate to as in any other compile mode"
(interactive "n0-clean 1-install 2-install,bundle:install 3-compile 4-test : ")
(let ((path (mvn-find-path-to-pom)))
(let ((goal (case cmd
((0) "clean")
((1) "install")
((2) "install bundle:install")
((3) "compile")
((4) "test"))))
(if (not (file-exists-p (concat path "/pom.xml")))
(message "No pom.xml found")
(compile (read-from-minibuffer "Command: "
(concat "mvn -o -f " path (concat "/pom.xml " goal))
nil nil 'mvn-command-history))))))

;; String pattern for locating errors in maven output. This assumes a Windows drive letter at the beginning
;;(add-to-list
;; 'compilation-error-regexp-alist
;; '("^\\([a-zA-Z]:.*\\):\\[\\([0-9]+\\),\\([0-9]+\\)\\]" 1 2 3))


First, the color coding stuff is sort of nice. It colorizes compile and unit test errors to red and passed tests to green, informational output to yellow/orange and that sort of thing. It also defines regular expressions which identify scala compiler errors and extraction of the file and line numbers for Emacs' compile.el.

Cut and paste the above in a file mvn.el, put it in your site-lisp directory and modify your .emacs file as follows. Here I repeat my Scala-mode setup as well to show where we bind the F6 key to a function which runs Maven from Emacs.
;;;;;;;;;;;;
;; MAVEN 2
;;;;;;;;;;;;

(let ((path "/usr/local/share/emacs/site-lisp/mvn"))
(setq load-path (cons path load-path))
(load "mvn.el"))

;;;;;;;;;;;;
;; SCALA
;;;;;;;;;;;

(let ((path "/usr/local/share/emacs/site-lisp/scala"))
(setq load-path (cons path load-path))
(load "scala-mode-auto.el"))

(defun scala-turnoff-indent-tabs-mode ()
(setq indent-tabs-mode nil))

;; scala mode hooks
(add-hook 'scala-mode-hook 'scala-turnoff-indent-tabs-mode)

;; Maven bound to
(add-hook 'scala-mode-hook
'(lambda ()
(define-key scala-mode-map '[f6] 'mvn)))


Using mvn.el

As I've indicated, this post is not about using Maven2 and the Scala plugin which is covered quite well in a number of places. See the scala-tools site on how to setup a Maven+Scala project. My focus is using a Maven+Scala from within Emacs and that is done easily enough.

So at this point I am assuming that you have a working pom.xml, and in the standard Maven file layout have one or more Scala source files and if you issue a "mvn install" command from the command line your code builds just fine. To achieve this level of enlightenment I again refer you to the scala-tools site. In addition, I'm assuming you've installed the mvn.el file and setup your .emacs as discussed.

So now lets switch from running Maven from a command line to within Emacs.

Fire up Emacs and open a Scala source file. You have to be in active buffer which contains a Scala source file. Press F6. You should see a small list of Maven goals in the interactive buffer. Pick a number and hit return. And now you have a small Maven command. You can change it by editing it or just hit return to run it.


A *Maven-Compile* buffer should appear within which scrolling Maven output is happening. A typical run for me (notice the complete lack of errors :) ) looks like this.

OK I know what you're thinking. Pretty colors, but what's the big deal. Well first the mvn.el script uses Emacs' compile.el so we can leverage pre-existing functionality to navigate to the compile errors.

Click on an error and you jump to it the source file. Alternatively you can move the cursor over the error msg and hit return. ctrl-x ` navigates to the next error. All this and more is available and explained in detail here.



mvn.el Insight

A couple of notes on mvn.el. Its not very complex and gains much of its power from Emacs compile.el for error navigation.

So what is really happening when the F6 key is pressed? Well first the directory of the Scala source file is determined. Then we look to see if a pom.xml is present, if not we recursively move up the directory tree until one is found or report an error if one is not found. Using the compile command found in compile.el, which does all the heavy lifting, Maven is launched with the located pom.xml and the selected goal.

For "advanced" Maven users who have a top level pom.xml with sub-projects or modules with their own pom.xml there is an interactive command which move up the directory tree and finds the top most pom.xml. alt-x mvn-master

So there it is. The basics of using using Maven+ScalaPlugin+Emacs for Scala program development.

Next post. Scala + OSGi + Maven + Felix + Emacs


Note: I added the "pick a number" thing awhile back, didn't particularly care for it, and never got around to removing it, and got sort of used to it. I just F6 3 RTN without much thinking. But the mvn.el is easily modified. See mvn-master for the old way.

Tuesday, February 17, 2009

Stone Tools and Scala Development - Part I Editing

I see great potential in Scala, the programming language.

Scala is a good thing. Scala deserves to win, it deserves to supplant Java in enterprise IT because it is better. If their exists some law of nature dictating that the universe must maintain itself in good kilter and balance every "worse is better" thing with a better thing then Scala will succeed.

Currently the language and compiler exceeds any IDEs capability to properly host it. And a huge proportion of developers have never experienced anything other then the all mothering Eclipse, Netbeans, or Microsoft IDE. Without the skeletal support of the IDE they are rendered near impotent, become jelly like, puddle and quiver back and forth producing nary a line of code.

Thankfully there are Scala plugins for Netbeans, Intellij and Eclipse. Yet there exists no more gnashing teeth on the Scala mailing list then concerning the relative immaturity of these plugins.

Amazingly these days, so interlinked is the concept of computer language and its IDE(s) many no longer see them as less then monolithic. Scala, the language, sucks because the IDE support sucks.

Fortunately, frankly surprisingly, one sees continued steady effort and work invested in improving these plugins. Surprisingly because it is a near thankless job and borders on pure drudgery. We salute you.

Until these tools mature, I claim it is possible to eek out a development existence, even a highly productive one using nothing more then the stone tools of a compiler, editor, make-like system, unix utilities and the command line. Some even view this as the superior, even ultimate IDE.

Here is how I roll with my stone tools for Scala development.

First off Linux. All of details below assume a Linux system.

Editing Scala Code With Emacs.


I know, we all heard it a 1000 times about those emacs gods who transcend the very laws of nature through advanced enlightenment with emacs which simply escapes us low karmic souls.

It's true, if you invest in emacs long enough it starts to pay off like a rigged slot machine. Put you have to feed it a lot of quarters first. I'm far from an emacs guru, but I've reached a point down the 7-fold path where I can't live without it and marvel how I ever did.

Get the latest emacs. Emacs was moribund for years but has experience a renaissance recently. The vastly improved font handling is worth it alone. It is once again better then XEmacs IMHO. Get good ol' emacs, I build it from cvs and it's easy. Standard configure, make, dance and if I remember the README gives an option to bypass configure with the proper Make incantation.

Here is my base .emacs stuff.

(global-font-lock-mode t)

(windmove-default-keybindings)

(setq use-file-dialog nil)

;; "y or n" instead of "yes or no"
(fset 'yes-or-no-p 'y-or-n-p)

;; Highlight regions and add special behaviors to regions.
;; "C-h d transient" for more info
(setq transient-mark-mode t)

;; Display line and column numbers
(setq line-number-mode t)
(setq column-number-mode t)

(global-set-key "\C-c\C-w" 'backward-kill-word)

;; Remove unnecessary gui stuff
(if (fboundp 'scroll-bar-mode) (scroll-bar-mode -1))
(if (fboundp 'tool-bar-mode) (tool-bar-mode -1))
(if (fboundp 'menu-bar-mode) (menu-bar-mode 1))

(blink-cursor-mode 0) ;; no blink

(set-background-color "black")
(set-face-background 'default "black")
(set-face-background 'region "black")
(set-face-foreground 'default "white")
(set-face-foreground 'region "gray60")
(set-foreground-color "white")
(set-cursor-color "red")

This will give you a clean, black background based emacs with maximum screen space, but leaving the main menu, which I found on someones (thank you) emacs blog.

Next install the Scala folks provided scala-mode for emacs. They did a nice job.

svn co http://lampsvn.epfl.ch/svn-repos/scala/scala-tool-support/trunk/src/emacs

Follow the readme and install. The yasnippit stuff is not bad, but I don't use it extensively, but give it a whirl and decide yourself.

For the record here is what my .emacs file looks like after I installed the provided scala-mode and yasnippit for scala.

;;;;;;;;;;;;
;; SCALA
;;;;;;;;;;;

(let ((path "/usr/local/share/emacs/site-lisp/scala"))
(setq load-path (cons path load-path))
(load "scala-mode-auto.el"))

(defun scala-turnoff-indent-tabs-mode ()
(setq indent-tabs-mode nil))

;; scala mode hooks
(add-hook 'scala-mode-hook 'scala-turnoff-indent-tabs-mode)

;; Maven bound to
(add-hook 'scala-mode-hook
'(lambda ()
(define-key scala-mode-map '[f6] 'mvn)))

;;;;;;;;;;;;;
;; YASNIPPIT
;;;;;;;;;;;;;

(add-to-list 'load-path "/usr/local/share/emacs/site-lisp/yasnippet-0.5.7")
(require 'yasnippet)
(yas/initialize)
(yas/load-directory "/usr/local/share/emacs/site-lisp/yasnippet-0.5.7")

More on that Maven key binding later.

Now we can edit Scala code effectively. There are some indentation issues and I've been able to improve it a bit but I still have fix a few things which I broke in the process before it is worthy.

With Yasnippit and alt-/ completion its not too bad. The next piece is exuberant ctags so you can alt-. and jump directly to that class, object or trait definition.

First there is ctags and then there is ctags. Emacs comes with a program called ctags. ctags will index your source code symbols so you can hop around your source code files.

There is a variation of ctags called exuberant ctags found at http://ctags.sourceforge.net/ and you need to install it. Again it should be quick and easy if you've ever done the ol' Linux Make dance.

Be careful, when you make install emacs as it reinstalls its version of ctags so you need to make install exhuberant ctags. Just make sure however you go about it, exhuberant ctags is what is executed in your PATH and not the standard emacs ctags.

In your home directory create a .ctags file with the following.

--langdef=Scala
--langmap=Scala:.scala
--regex-Scala=/^[^\*\/]*class[ \t]*([a-zA-Z0-9_]+)/\1/c,classes/
--regex-Scala=/^[^\*\/]*object[ \t]*([a-zA-Z0-9_]+)/\1/o,objects/
--regex-scala=/^[^\*\/]*trait[ \t]*([a-zA-Z0-9_]+)/\1/t,traits/
--regex-Scala=/^[^\*\/]*case[ \t]*class[ \t]*([a-zA-Z0-9_]+)/\1/m,case-classes/
--regex-Scala=/^[^\*\/]*abstract[ \t]*class[ \t]*([a-zA-Z0-9_]+)/\1/a,abstract-classes/
#--regex-Scala=/^[^\*\/]*def[ \t]*([a-zA-Z0-9_]+)[ \t]*.*[:=]/\1/f,functions/
#--regex-Scala=/^[^\*\/]*val[ \t]*([a-zA-Z0-9_]+)[ \t]*[:=]/\1/V,values/
#--regex-Scala=/^[^\*\/]*var[ \t]*([a-zA-Z0-9_]+)[ \t]*[:=]/\1/v,variables/
#--regex-Scala=/^[^\*\/]*type[ \t]*([a-zA-Z0-9_]+)[ \t]*[\[<>=]/\1/T,types/

I just like to index classes, traits, objects and case-classes. Hmmm just noticed I should add case objects as well.

Now when editing a scala file in emacs, under the emacs Scala menu item, Features you can build and use tag files which index key symbols in your source code.

Explore your Scala editing environment. No two people use emacs in the same fashion.
  • alt-x rgrep in emacs to ad-hoc search your scala files.
  • use alt-/ for simple minded completion.
  • Hit the F1 key to pop up your speed bar window. Note at the bottom you can click and switch to view, buffers, files etc in the speed bar.
  • alt-x erc and join #scala in your emacs for instant Scala support. :)
  • Are you using ctr-x o to move between buffers? There are tons of better ways. Open up a several scala buffer windows ctr-x 2 or 3, now hold down shift and use your arrow keys to move around.
  • Notice that there are 5-6 Tag search options under the Scala Menu, try them all.
  • While editing a scala file, put your cursor over a type definition (x: MyType, ...) and do alt-.
  • Toggle yas/snippit, alt-x yas/minor-mode. In your scala source file type 'def ' or 'class ' and hit the tab key.
  • Run the scala interpreter in emacs its right there in the Scala menu.

Next post I'll address setting up and building your Scala project using Maven from inside emacs.