Wednesday, December 28, 2011

Convert java.util.Properties to a Clojure Map

As I previously mentioned, a lot of the work I do involves Clojure & Java interop. This work includes the occasional case of working with a java.util.Properties object from within Clojure. Working with a Properties object isn't a huge deal, but while in Clojure I prefer to use destructuring and the various functions (e.g. update-in, assoc, dissoc, etc) that are designed to work with Clojure maps.

The following example shows how easy it is to convert a Properties object to a Clojure map.
user=> (def prop-obj (doto (java.util.Properties.) (.putAll {"a" 1 "b" 2})))
user=> prop-obj
#<Properties {b=2, a=1}>

user=> (reduce (fn [x [y z]] (assoc x y z)) {} prop-obj)
{"a" 1, "b" 2}
That's fairly easy, but you'll quickly want your keys to be keywords if you plan on destructuring. You can drop in a quick call to keyword to convert the keys; however, you'll probably also want to dasherize the keys to allow for easy destructuring using keys and idiomatic names.
user=> (defn dash-match [[ _ g1 g2]]      
          (str g1 "-" g2))

user=> (defn dasherize [k]
          (-> k
            (clojure.string/replace #"([A-Z]+)([A-Z][a-z])" dash-match)
            (clojure.string/replace #"([a-z\d])([A-Z])" dash-match)
            (clojure.string/lower-case)))#'user/dash-match

user=> (def prop-obj (doto (java.util.Properties.) (.putAll {"FirstName" "Mike" "LastName" "Green"})))
user=> (reduce (fn [x [y z]] (assoc x y z)) {} prop-obj)                                              
{"LastName" "Green", "FirstName" "Mike"}
user=> (reduce (fn [x [y z]] (assoc x (-> y dasherize keyword) z)) {} prop-obj)
{:last-name "Green", :first-name "Mike"}
That looks good, but you might also find yourself working with a properties file that uses dots or dashes to group similar data. For example, you might find the following entry in your properties file (and the resulting Properties object).
person.name=Mike Green
person.age=26
person.sex=male
Loading this into a map works fine; however, it would be nice if the resulting map was nested (to keep common data together, and for easier access using get-in, update-in, etc).

The following code splits on dots and nests the values appropriately.
user=> (def prop-obj (doto (java.util.Properties.) (.putAll {"person.name" "Mike Green" "person.age" "26" "person.sex" "male"})))        
user=> prop-obj                                                                                                                  
#<Properties {person.name=Mike Green, person.age=26, person.sex=male}>
user=> (reduce (fn [x [y z]] (assoc-in x (-> y dasherize (clojure.string/split #"\.")) z)) {} prop-obj)
{"person" {"sex" "male", "age" "26", "name" "Mike Green"}}

user=> (-> (reduce (fn [x [y z]] (assoc-in x (-> y dasherize (clojure.string/split #"\.")) z)) {} prop-obj) clojure.walk/keywordize-keys)
{:person {:sex "male", :age "26", :name "Mike Green"}}
So, not too complicated, but much more complicated than what we started with. It turns out you'll likely want to do other things like parse integers, parse booleans, require keys, and add defaults when a key=value pair isn't specified.

We could go through the effort here of providing all that functionality, but instead I've created a small library that provides all of the above mentioned features: propertea

If you'd like to see the implementation of the above features, you'll just need to look in propertea.core. If you want examples of all of the features of propertea, checkout the tests in github.

Clojure & Java Interop

About a year ago I got a phone call asking if I wanted to join another team at DRW. The team supports a (primarily) Java application, but the performance requirements would also allow it to be written in a higher level language. I'd been writing Clojure (basically) full-time at that point - so my response was simple: I'd love to join, but I'm going to want to do future development using Clojure.

A year later we still have plenty of Java, but the vast majority of the new code I add is Clojure. One of the big reasons I'm able to use Clojure so freely is the seamless interop with Java.

Execute Clojure from Java
Calling Clojure from Java is as simple as loading the .clj file and invoking a method from that file. I used the same example years ago, but I'll inline it here for simplicity.
; interop/core.clj
(ns interop.core)

(defn print-string [arg]
  (println arg))

// Java calling code
RT.loadResourceScript("interop/core.clj");
RT.var("interop.core", "print-string").invoke("hello world");
note: examples from this blog entry are available in this git repo. The commit with the code from the previous example is available here and I'm running the example from the command line with:
lein jar && java -cp "interop-1.0.0.jar:lib/*" interop.Example
Execute Java from Clojure
At this point we have Java executing some Clojure code, and we also have Clojure using an object that was created in Java. Even though we're in Clojure we can easily call methods on any Java object.
(ns interop.core)

(defn print-string [arg]
  (println arg "is" (.length arg) "characters long"))
commit

The above code (using the length method of a String instance) produces the following output.
hello world is 11 characters long
Calling a Java method and passing in additional arguments is also easy in Clojure.
(ns interop.core)

(defn print-string [arg]
  (println (.replace arg "hello" "goodbye")))
commit

The above code produces the following output.
goodbye world
There are a few other things to know about calling Java from Clojure. The following examples show how to call static methods, use enums, and use inner classes.
(ns interop.core)

(defn print-string [arg]
  ;;; calling a static method
  (println (String/valueOf true))

  ;;; using an enum
  (println (java.util.concurrent.TimeUnit/SECONDS))

  ;;; using a Java nested (inner) class. Note, in Clojure you
  ;;; use a $ instead of a .
  (println (java.util.AbstractMap$SimpleEntry. "key" "val")))
commit

And, the output:
true
#< SECONDS>
#<SimpleEntry key=val>
Create Java objects in Clojure
When working with Clojure you'll likely want to interact with existing Java objects, but you'll probably also want to create new instances of Java objects. You might have noticed the dot at the end of Abstract$SimpleEntry. in the previous example - that's how you instruct Clojure to create an instance of a Java object. The following example shows the dot notation for calling a constructor of the String class.
(ns interop.core)

(defn print-string [arg]
  (println (String. arg)))
commit

At this point our output is back to the original output.
hello world
When creating Java objects it's often beneficial to know which Java interfaces the Clojure data structures implement. The following examples demonstrate how you can create Java objects while passing Clojure datastructures (and functions) as constructor arguments.
(ns interop.core)

(defn print-string [arg]
  ;;; pass a Clojure vector where Java expects a java.util.Collection
  (println (java.util.HashSet. ["1" "2"]))

  ;;; pass a Clojure map where Java expects a java.util.Map
  (println (java.util.LinkedHashMap. {1 "1" 2 "2"}))

  ;;; pass a Clojure function where Java expects a Runnable
  (println (Thread. (fn [] (println "clojure fns are runnables (and callables)")))))
commit

The output shows the constructed Java objects.
#<HashSet [2, 1]>
#<LinkedHashMap {1=1, 2=2}>
#<Thread Thread[Thread-1,5,main]>
Calling constructors in Clojure is very easy, but that's not always an option when creating a Java object. At times you will likely need to create an instance of a Java interface. Clojure provides both proxy and reify for creating instances of Java interfaces. The following example demonstrates the syntax for using either proxy or reify.
(ns interop.core)

(defn proxy-coll []
  (proxy [java.util.Collection] []
    (add [o]
         (println o)
         true)))

(defn reify-coll []
  (reify java.util.Collection
    (add [this o]
         (println o)
         (println this)
         true)))

(defn main []
  (.add (proxy-coll) "this string is printed on proxied.add")
  (.add (reify-coll) "this string is printed on reified.add"))
commit

note, I also changed Example.java (the details are available in the above linked commit). The syntax for proxy and reify are fairly similar, and both offer additional options that are worth looking into. The primary differences between these two simple examples are:
  • The proxy implementation requires an empty vector where we could specify constructor arguments (if this were an abstract class instead of an interface).
  • The arg list for all methods of reify will specify the reified instance as the first argument. In our example the Collection.add method only takes one argument, but in our reify we also get the instance of the collection.
You might have also noticed that both implementations of add have "true" at the end - in our example we're hard-coding the return value of add to always return true. The following output is the result of running the current example code.
this string is printed on proxied.add
this string is printed on reified.add
#<core$reify_coll$reify__11 interop.core$reify_coll$reify__11@556917ee>
It's worth reading the docs to determine whether you want proxy or reify; however, if you don't see a clear choice I would opt for reify.

Returning objects from Clojure to Java
Our current Example.java returns something from the call to invoke on the clojure.lang.Var that is returned from RT.var("interop.core", "main"), but we're ignoring it so we have no idea what's returned.* Let's change the code and return something on purpose.
// interop/Example.java
package interop;

import clojure.lang.RT;

public class Example {
    public static void main(String[] args) throws Exception {
        RT.loadResourceScript("interop/core.clj");
        System.out.println(RT.var("interop.core", "main").invoke());
    }
}

; interop/core.clj
(ns interop.core)

(defn main []
  {:a "1" :b "2"})
Running our changes produces the following output.
{:a "1", :b "2"}
commit

At this point we are back in Java land after making a quick trip to Clojure to get a value. Returning most objects will be pretty straightforward; however, at some point you may want to return a Clojure function. This turns out to be fairly easy as well, since Clojure functions are instances of the IFn interface. The following code demonstrates how to return a Clojure function and call it from within Java.
// interop/Example.java
package interop;

import clojure.lang.RT;

public class Example {
    public static void main(String[] args) throws Exception {
        RT.loadResourceScript("interop/core.clj");
        clojure.lang.IFn f = (clojure.lang.IFn) RT.var("interop.core", "main").invoke();
        f.invoke("hello world");
    }
}

// interop/core.clj
(ns interop.core)

(defn main [] println)
commit

The above example returns the println function from interop.core/main and then invokes the println function from within Java. I only chose to pass one argument to invoke; however, the IFn.invoke method has various overrides to allow you to pass several arguments. The above code works, but it can be simplified to the following example.
package interop;

import clojure.lang.RT;

public class Example {
    public static void main(String[] args) throws Exception {
        clojure.lang.IFn f = (clojure.lang.IFn) RT.var("clojure.core", "println");
        f.invoke("hello world");
    }
}
commit

It seems like a fitting end that our final output is the same as our original output.
hello world
*actually, it's the last thing that's returned, or "true" for this specific case.