Usually I’d turn to a “scripting” language like Python, Groovy, or back in the day, Perl for this type of thing.

Today such a need came up, and I decided to try tackling it with Scala since it has many of the features that make the above dynamic languages good for this:

• first-class support for map and list data structures
• an interactive shell
• minimal overhead to write a program, compile, and run it
• support for functional programming
• good regular expression support

The problem:
A small performance test program ran a large number of tests, and measured the elapsed time for each execution. The program output a line like “request completed in 10451 msecs” for each test. I needed to parse the output, collect the elapsed time measurements, and get some basic statistics on them; simple average, minimum, and maximum.

I used a Scala 2.8 snapshot, and fleshed out the code using the Scala interactive shell. First, define a value with the raw output to be processed:

scala> val rawData = """request completed in 10288 msecs
     | request completed in 10321 msecs
     | request completed in 10347 msecs
     | request completed in 10451 msecs
     | request completed in 10953 msecs
     | request completed in 11122 msecs
... hundreds of lines ...
     | request completed in 11672 msecs"""

The above uses Scala’s support for multi-line string literals.

The next thing I needed to do was parse the above output, using a regular expression to extract just the milliseconds. There’s several ways to create a regular expression in Scala. This is the one I like:

val ReqCompletedRE = """\s*request completed in (\d+) msecs"""r

There’s a bit of magic in how the above string literal actually ends up becoming a regular expression. There’s an implicit conversion in the Scala Predef object which turns a Java String into a RichString. RichString provides a ‘r’ method that returns a regular expression object. The members of the Predef object are automatically imported into every Scala module, so the Scala compiler will attempt to apply any conversions it finds in Predef when trying to resolve the ‘r’ method. So the above expression is creating a RichString from a String via an implicit conversion, then calling the ‘r’ method on it, which returns the regular expression.

To apply the regular expression to a line of the output and to extract the milliseconds, we can use an expression like:

scala> val ReqCompletedRE(msecs) = " request completed in 10451 msecs"
msecs: String = 10451

msecs gets bound to the first group in the regular expression (the part that matches (\d+)). This takes place via the Scala extractors feature – the scala regular expression class defines an extractor which extracts the grouping results.

The next step is to iterate over the lines of the output, extract the milliseconds, and turn the results into a list.

scala> val msecsVals = rawData.lines.map { line => val ReqCompletedRE(msecs) = line; Integer.parseInt(msecs);} toList
res11: List[Int] = List(10288, 10321, 10347, 10451, 10953, 11122, ..., 11672)

The above code is using the RichString lines, Iterator.map method, along with Scala closures.

Finally, to get the simple statistics:

scala> (msecsVals.sum / msecsVals.length, msecsVals.min, msecsVals.max)
res21: (Int, Int, Int) = (10736,10288,11672)

Putting the whole script together:

val ReqCompletedRE = """\s*request completed in (\d+) msecs"""r
val msecsVals = rawData.lines.map { line => val ReqCompletedRE(msecs) = line; Integer.parseInt(msecs);} toList
(msecsVals.sum / msecsVals.length, msecsVals.min, msecsVals.max)

We’ve seen several dynamically typed languages gain widespread adoption in the last eight years or so… Python, Ruby, Groovy, and you can go back to Javascript, Perl, PHP, etc..  What’s the last statically typed language that’s gained a wide following?  C#?  I’ve pretty much come to assume that any new language that was highly expressive, concise, and good for DSLs, would be a dynamic language.  Dynamic languages do have their drawbacks..  performance is usually not up to par with statically typed languages (although it rarely makes a difference), and the lack of type information at compile type make it very difficult for IDEs to provide features such as autocomplete, symbol lookup, and early error detection.  So if languages such as Scala manage to deliver a highly development-efficient language while retaining static typing, that’s all the better in my eyes.

Here’s what I did to get a minimal project set up in Maven and IntelliJ IDEA:

Create a shell of a project using a Maven archetype:

mvn org.apache.maven.plugins:maven-archetype-plugin:1.0-alpha-7:create \
-DarchetypeGroupId=org.scala-tools.archetypes \
-DarchetypeArtifactId=scala-archetype-simple \
-DarchetypeVersion=1.1 \
-DremoteRepositories=http://scala-tools.org/repo-releases \
-DgroupId=com.rps -DartifactId=scalatest

This created a project with a single Scala class, test, and a Maven POM containing the minimal dependencies and plugins.

The version of Scala generated by the archetype isn’t the most current. Just update the version in the POM:

<scala.version>2.6.1</scala.version>

to 2.7.6, or whatever is the version you want.

I then imported the project into IntelliJ IDEA using the excellent IDEA Maven integration.  I had to add the Scala nature to the project in IDEA myself, but otherwise, everything is working..  Editing, unit tests, debugging, etc.

The IntelliJ IDEA editor support for autocomplete seems to work pretty well, but it doesn’t seem to do much detection of errors as you’re editing yet.