Saturday, February 18, 2012

GWT and its lack of reflection support

When you start to develop applications using Google Web Toolkit, you relatively quickly learn that its support of Java standard library is... lacking... to say the least. Among the things that, to me personally, were most lacking is the support of Java's reflection API. This becomes very apparent when you try to dynamically instantiate an object knowing only its class name - there simply is no support for Class.newInstance() call in GWT. Similarly, you don't have a generic way to retrieve a given field's value knowing only the field's name (stored in a string), you need to hardcode the appropriate getter call in your application.

Now, you may think, why would I need to access such low-level and, arguably, non-OO functionality? As it turns out, there are multiple cases where it comes extremely helpful. A good example of this is a web app that provides a palette of objects and allows you to arbitrarily instantiate and modify each object's properties. Once again, you could hardcode each object's properties in the GUI, or even use a generic Map to store them, but neither of these two is (in my opinion) a good solution; the former needs you to modify your GUI code each time a new object type is added to the palette, the latter is just an invitation for bugs to creep into your application. And if only GWT supported the reflection interface, you could dynamically instantiate the objects and, also dynamically, discover, query and set each of their JavaBean-esque properties. But, again, GWT does not support it.

So, is it a hopeless case?

Generators to the rescue

As you certainly already guessed, the case is all but hopeless - why else would I write a blog article about it? Yes, Google Web Toolkit does not support reflection, but it does support the next best thing - generators. These little gems are executed during your application's compilation phase in order to dynamically generate some code that you can then use in your GWT application. If you have not used them in the past, the above explanation is probably as clear as mud to you. Fear not, we are about to explore what the generators really are! Let us start with a very simple generator example. Our battle plan is as such:
  1. Create a very simple GWT project containing just one label
  2. Create an interface containing one no-args method returning a string
  3. Use a generator to provide this interface's implementation
  4. Populate the label with the string returned by our interface's implementation
You may think that such an application is completely worthless - after all, it is much simpler to just implement the interface on our own instead of using fancy and complicated concepts such as a generator. However, even this primitive thing has a practical use and that is - yes, you guessed it, internationalization. You could (and GWT does) use generators to dynamically wire all the translation strings (read from, for example, a property file) to your interface. Neat, huh? But, lets not get ahead of ourselves. Following the battle plan, we start with a web page:
<!doctype html>
<html>
  <head>
    <meta http-equiv="content-type" 
        content="text/html; charset=UTF-8">
    <title>Hello Generators</title>
    <script type="text/javascript" language="javascript" 
        src="test/test.nocache.js"></script>
  </head>

  <body>
    <h1>Hello Generators</h1>
 <div id="output"></div>
  </body>
</html>
and its companion GWT entry point:
public class Test implements EntryPoint {
 public void onModuleLoad() {
  final Label outputLabel = new Label();
  
  Generated generated = ...INVOKE GENERATOR MAGIC HERE...
  
  outputLabel.setText(generated.sayHello());
  RootPanel.get("output").add(outputLabel);
 }
}
and the interface:
public interface Generated {
 public String sayHello();
}
All of this is very simple and, if you have a basic GWT knowledge, should come as no surprise. Now, all that is left out is to fill in the blank in the Test class that will invoke our generator and provide a meaningful implementation. So, how can we do it?

Writing our first generator

Writing and using a generator is a three-step process. First, you need to create the generator class itself - and this, of course, is usually the most complicated and time-consuming step. Second, you modify your module's *.gwt.xml file and add rules that decide when the generator should be used (that is, for what classes should it be invoked). Finally, you put a call to GWT.create(...) in your code - and this call will actually invoke your generator and return the generated class's instance as a result. That's it.

So, to sum it up, this is what we need to do:

  1. Create the generator class
  2. Create the rule in Test.gwt.xml
  3. Invoke our generator in the Test class
To make the example clearer, we will start with point 3 and work our way up the list.

Invoking our generator

Invoking our generator is very simple. We just modify the Test class like this:
 ...
 Generated generated = GWT.create(Generated.class);
 ...
That's all there is to it. But what does GWT do behind the scenes of this call? We will get to it in a moment, after we create a rule for our generator.

A very important thing to mention here is that, when making a call to GWT.create(), you need to provide a class literal (that is, you need to write SomeClass.class) as the argument to the method. You cannot use a variable holding a Class object here! This is because GWT must know what the passed value is in compile-time, so it cannot wait until run-time before determining the value passed - that would be too late for the generator to run.

This is especially deceptive because, when testing your application in development mode, everything may appear to be working correctly even if a class literal is not used. However, as soon as you try to actually compile your application for production mode, you will receive an error message.

Mapping our generator

Next step in our quest for generators is to modify Test.gwt.xml file and put a mapping to our generator class there. This mapping serves as a bridge between the GWT.create(...) call and the generator class that we write. What it does is basically tell GWT "If GWT.create(...) is called with class literal X, invoke our generator Y to generate a custom class code, instantiate an object of that class and return it as a result of the call":

In our case, the mapping looks like this:

 <generate-with class="com.test.rebind.ExampleGenerator">
  <when-type-is class="com.test.client.Generated" />
 </generate-with>
This, as I have mentioned before, tells GWT to invoke an instance of ExampleGenerator when GWT.create(Generated.class) is called.

Please note that our generator is not part of the client code (that is, it does not reside in the *.client subpackage) - this is very important! Otherwise, you could receive compile error when generating your GWT code, because generators use the part of GWT API that is not compilable into JavaScript. Of course, if you never invoke your generator directly from the client code, GWT compiler will not attempt to generate JavaScript for it, but it is still good practice not to put non-client code inside the *.client subpackage.

Implementing our generator

Now, with point 2 and 3 out of the way, all that remains is to actually implement our generator. To implement our generator, we need to:

  1. Create a new class that extends com.google.gwt.core.ext.Generator class
  2. Implement the only method there, called generate()
  3. Optionally, the method can (but does not have to!) create source code for our class
  4. The method must return a fully-qualified class name.
  5. GWT will attempt to instantiate an object of the returned class and return it as the result of GWT.create() call

Of note here is step 3. We can use it to provide (generated "on the fly") source code for a custom class, which we later return as the result of the generate() method. This is our gateway to providing custom code - one that we will use in a moment. Please note, though, that this step is not mandatory - if we want, we can implement our class the old-fashioned way and use the generator simply to decide which implementation to use. In our case, however, it is not enough.

Lets start the work on our generator, then! The result of following steps 1 and 2 can be seen below:

public class ExampleGenerator extends Generator {
 @Override
 public String generate(TreeLogger logger, 
   GeneratorContext context,
   String typeName) 
       throws UnableToCompleteException {
  return null;
 }
}

Not much here, so far. We still need to write code that will generate our custom class and return its name. Before we start, though, lets take a brief look at the arguments provided by GWT to our method. There are 3 of them:

  • logger is, as I am sure you have already guessed, an instance of a logger that can be used to output any debug, info and error messages that you want to print during the class generation process
  • context is a very important argument that can be used to retrieve some helper objects (among them the object used to generate our class).
  • typeName is a string literal corresponding to the class object passed to GWT.create call

Okay, this is all great to know, but how do we implement our custom class? This is how:

public class ExampleGenerator extends Generator {
 @Override
 public String generate(TreeLogger logger, 
   GeneratorContext context,
   String typeName) 
       throws UnableToCompleteException {
  
  PrintWriter pw = context.tryCreate(logger, 
   "com.test.client", "GeneratedImpl");
  if (pw != null) {
   pw.println("package com.test.client;");
   pw.println();
   pw.println("public class GeneratedImpl implements Generated {");
   pw.println("    public String sayHello() { return \"Hello, Generators!\"; }");
   pw.println("}");
   context.commit(logger, pw);
  }
  return "com.test.client.GeneratedImpl";
 }
}

Quite some things happening in this short snipped of code. Lets go through it line by line:

  1. In line 8, we retrieve an instance of PrintWriter. Contrary to what you may think, there is not much special to this object - you use it as you would use any other PrintWriter instance (though no, you cannot substitute any random PrintWriter instance for it, because line 16 would throw an error). What is important, though, is that the call to tryCreate will return null on every invocation other than the first one - so that you only generate a given class once, and then it is reused.
  2. In line 10, we check if the returned PrintWriter is null and only attempt to generate the code if it is not.
  3. In lines 11 through 15, we hand-craft the generated class's code. Yes, unfortunately you have to do it this way (though there are some additional utility objects that help to automatically generate the necessary imports and extend other classes/interfaces - for simplicity, though, we choose not to use them here), so you cannot depend on your IDE to automatically highlight syntax errors. On the other hand, though, you have total freedom on how the final class will look like - you can inspect all types to your heart's content, you can read any external files (like property files) before deciding what to put in your final code and so on - there is a lot of power at your fingertips here, and with it comes responsibility to be careful that your generated code is a correct Java code. Fear not, though, because if it is not - your application's compilation will fail fast with an appropriate error message.
  4. In line 16, we commit our custom code and it is here where it is actually compiled and turned into a Java class. If there is a syntax error, this is where an appropriate exception will be thrown and the error printed into the passed logger object.
  5. Finally, in line 18, we return the fully-qualified name of our generated class

Whew! Finally, we are done. Our generator class is ready, it is mapped appropriately and we have a call to GWT.create() in our code, which will make sure to invoke our generator. All that is left is to run our application.

It works! The correct message is displayed, which means that our generator really is invoked and called.

Enhancing our generator

Okay, but this is really a lot of work to get what we could get with just a few lines of regular Java code. Why should we bother with the generators? To show why, let us modify our example a little: instead of returning a hard-coded string, we will now read it from a companion properties file. This is something that we cannot do from normal GWT code!

First, let us create a simple properties file: com/test/rebind/example.properties

text=Hello, Properties!

Next, let us modify our generator class as follows

(...)
if (pw != null) {
 Properties prop = new Properties();
 try {
  InputStream in = getClass().getResourceAsStream("example.properties");
  prop.load(in);
  in.close();
 } catch (Exception ex) {
  logger.log(Type.ERROR, "Error reading properties file", ex);
  throw new UnableToCompleteException();
 }
 String value = prop.getProperty("text").replaceAll("\\\"", "\\\\\"");
 
 pw.println("package com.test.client;");
 pw.println();
 pw.println("public class GeneratedImpl implements Generated {");
 pw.println("    public String sayHello() { return \"" + value + "\"; }");
 pw.println("}");
 context.commit(logger, pw);
}
(...)

There is some boilerplate code here, needed to load the properties file contents, but it should be pretty basic to every Java coder. For the sake of completeness, though, let us again go through it line by line:

  • In lines 3 through 7, we instantiate a new Properties object and load its contents from a resource present in the same package as the generator class
  • In lines 9 and 10, we handle any potential IO errors. Note the use of the TreeLogger to neatly log the incoming exception, followed by throwing of UnableToCompleteException (which does not receive any exception details or message)
  • In line 12, we retrieve our property. Note the presence of a replaceAll call, which makes sure all the quotation marks (") in the property are properly escaped (\") when put in the final generated code
  • Lines 14 through 19 are almost the same as before - the only difference is the usage of value variable in line 17 instead of a hard-coded string

As you can see, once we have set our generator, it is relatively trivial to add real dynamism to the code we generate - and that includes things that are unavailable in the client-side GWT code, like reading external configuration files and - yes! - using Java reflection. Let us run our modified example and see that...

...it works as expected. We are able to read a properties file and dynamically generate code based on it. Great!

Reflecting on GWT

By now, I hope, you have a solid grasp on how generators work and can think of at least some examples where they may come handy. That said, we have not yet touched on how can we use them for instantiating a class dynamically based on its classname passed in a string. Let us do it now!

Before we begin, though, let us think for a moment how are we going to do it. After all, we need some way to pass a string parameter to our generator, so that it knows which class to instantiate - right? Wrong! As I have mentioned before, you need to pass a class literal as the only parameter to GWT.create - there is simply no way to pass a parameter to our generator class. Is it a hopeless case, then?

Thankfully, no. In order to solve the problem, we just need to make one step backwards when looking at it - what we want to do is not to use the generator to instantiate our class; what it will do, instead, is instantiate a factory object which we then use to create the final class. This additional level of indirection solves all our problems - we use the full power of generator to generate a factory object that knows how to instantiate every class we would ever want to instantiate, and then use that object (which, being a normal Java object, can have a string passed to it) to instantiate the final class. Simple.

A careful reader may now raise his or her brow at the "every class we would ever want to instantiate" part. What exactly does it mean, you may think - how can this factory object know beforehand which classes we will want to instantiate? Well, there are two options. The first one (pretty nonviable, you may admit) is for it to know how to create literally every class that is out there. The second, more reasonable one is for us to mark the classes we may want to instantiate. We can mark them using a marker interface or (what we will do in this example) using a marker annotation. Then we simply scan all the available classes for those correctly annotated and only instruct the factory object how to instantiate these classes.

To make it all a little clearer for those of you who are still confused, here is a little diagram illustrating how will all these things interact:

The type oracle, that you can see in this diagram, is one of the helper classes available to generators. It is similar to the standard reflection API, but contains a richer set of methods that allow, among other things, precisely what we want - to quickly scan all the available classes and find the ones annotated with our custom annotation, something that cannot be easily done using just the vanilla reflection.

Before we move to our generator code, let us modify the previous example. This is the modified Test entry point class:

public class Test implements EntryPoint {
 public void onModuleLoad() {
  final Label outputLabel = new Label();
  
  GeneratedFactory factory = GWT.create(GeneratedFactory.class);
  Generated generated = 
   factory.create("com.test.client.GeneratedClass2");
  outputLabel.setText(generated.sayHello());
  RootPanel.get("output").add(outputLabel);
 }
}

GeneratedClass1 and GeneratedClass2 are simple implementations of the Generated interface:

@MayCreate
public class GeneratedClass1 implements Generated {
 @Override
 public String sayHello() {
  return "I am class #1";
 }
}

@MayCreate
public class GeneratedClass2 implements Generated {
 @Override
 public String sayHello() {
  return "I am the second class";
 }
}

As you can see, both of them have been annotated with a custom @MayCreate annotation. This is (pretty unsurprising) definition of said annotation:

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
public @interface MayCreate {
}
We use an annotation as a marker here for demonstration purposes, but truth be told - in this very example, we already implement a "marker" interface (Generated), so we could as well use the interface instead of the annotation altogether.

Since we are no longer generating subclasses of Generated, but instead a subclass of GeneratedFactory, we need to adjust our Test.gwt.xml file accordingly:

 <generate-with class="com.test.rebind.ExampleGenerator">
  <when-type-is class="com.test.client.GeneratedFactory" />
 </generate-with>

The last piece of our puzzle (aside from the generator itself) is the base GeneratedFactory class. Yes, this time it is a class, not an interface; since writing generated code is pretty error-prone, we should always try to limit it as much as possible. Using a base class, we can put all the static functionality in that class and provide only the dynamic part in the implementation returned by the generator. So, in our case, the base class looks like this:

public abstract class GeneratedFactory {
 private Map<String, FactoryMethod>: map 
  = new HashMap<String, FactoryMethod>();
 
 public Generated create(String type) {
  FactoryMethod m = map.get(type);
  if (m != null)
   return m.create();
  else
   throw new RuntimeException("Unknown type: " + type);
 }
 
 protected void register(String type, FactoryMethod factory) {
  map.put(type, factory);
 }

 protected interface FactoryMethod {
  public Generated create();
 }
}

Few comments to how we plan to use the above class:

  1. For each class we want the factory to instantiate, we will create an implementation of the FactoryMethod interface.
  2. We will put said FactoryMethod implementations in a map, together with each class's name
  3. When the factory is asked to instantiate a class, it retrieves its corresponding FactoryMethod from the map and delegates the actual instantiation to that
  4. We will use a helper register() method in our Factory implementation
  5. We mark the base GeneratedFactory class as abstract (even though it has no abstract methods) to prevent accidental direct creation - the only way anyone is allowed to create an instance of it is via GWT.create() call

Finally, we are ready to implement the generator itself! Perhaps disappointingly, though, the generator's code is relatively short - only slightly longer than the GeneratedFactory code:

public class ExampleGenerator extends Generator {
 @Override
 public String generate(TreeLogger logger, GeneratorContext context,
   String typeName) throws UnableToCompleteException {
  
  PrintWriter pw = context.tryCreate(logger, "com.test.client", "GeneratedFactoryImpl");
  if (pw != null) {
   pw.println("package com.test.client;");
   pw.println();
   pw.println("public class GeneratedFactoryImpl extends GeneratedFactory {");
   pw.println("    public GeneratedFactoryImpl() {");
   
   TypeOracle oracle = context.getTypeOracle();
   for (JClassType type : oracle.getTypes()) {
    if (type.getAnnotation(MayCreate.class) != null) {
     String name = type.getQualifiedSourceName();
     pw.println("register(\"" + name + "\", ");
     pw.println("    new FactoryMethod() {");
     pw.println("        public Generated create() {");
     pw.println("            return new " + name + "();");
     pw.println("        }");
     pw.println("    });");
    }
   }
   pw.println("    }");
   pw.println("}");
   context.commit(logger, pw);
  }
  return "com.test.client.GeneratedFactoryImpl";
 }
}
If we wanted to go with the marker interface approach, we would need to change line 15 of the generator. Instead of the annotation check, we would put a call to type.isAssignableTo instead.

Even though it is short, however, there is a lot of stuff going on there. Traditionally, let us analyze it line by line:

  • Lines 1 through 7 are nothing we haven't see before
  • Lines 8 through 10 define our implementation of GeneratedFactory and make it extend the latter
  • Line 11 marks the beginning of the class's constructor
  • Line 13 shows how we can retrieve an instance of the aforementioned TypeOracle
  • Line 14 loops through all the types known to the TypeOracle
  • Line 15 filters out all the types not annotated with MayCreate annotation
  • Lines 16 through 22 call the register() method as described before, passing an anonymous class implementing the FactoryMethod interface
  • Most importantly, line 20 performs the actual instantiation of a given class
  • The rest of the code, again, is nothing new

That's it. We are done! Let us run the application and check the results...

This confirms that our application works as expected! But just to be sure, change this line in the Test class:

Generated generated = 
 factory.create("com.test.client.GeneratedClass2");

to this:

Generated generated = 
 factory.create("com.test.client.GeneratedClass1");

When we run it, we get:

This confirms that our factory implementation really does know how to instantiate both the annotated classes based on a dynamic string (you don't have to believe me, simply change the application so that it contains a text field where the user can enter a class name and call the factory method after a button is pressed).

Summary

We learned how to utilize the GWT generators through a simple example. Then we have shown their real power by dynamically generating a class based on an external properties file, and finally we learned about TypeOracle and utilized it in a somewhat complicated example to dynamically generate a factory class able to create other objects at will.

This is a lot, but we can go even further. We could utilize the power of TypeOracle to further modify both our factory class and its corresponding generator to not only allow creation of new objects, but also listing, reading and writing of each of the objects' properties. The potential of it is endless: from the (mentioned at the beginning of this post) dynamic property editor to an entire automatically generated grid displaying a list of custom objects, with each object's property being a separate column - the sky is the limit!

Have you found this tutorial helpful? Do you see something that could be improved here? Or maybe you have found another interesting use of GWT generators? In either case, do not hesitate and drop a comment!

Saturday, January 21, 2012

Comet in a socket: supporting the non-supporters

This is part 3 in WebSocket series. Read part 1 and part 2 here.

In the previous part of the series we have covered all the historical basis of WebSocket protocol and learned how to use it in order to implement a very basic webchat service. Unfortunately, the WebSockets protocol is still only a draft and a novelty, and as such it is not yet consistently supported across all the major browser platforms. Have no fear, though, as there are ways to easily add the necessary support to all the major browsers - though usually not with a pure JavaScript solution.

Before we cover it, though, lets see how bad the situation really is.

Tuesday, January 17, 2012

Daily dig: interfaces and JDBC

It is funny how fast we sometimes jump to conclusions or try to use the solutions we know without even thinking if they are right for a given problem. Just yesterday I had an opportunity to witness such a thing (and, moreover, fall victim to it myself) when this seemingly innocent question has been asked: "In C++ we split the class code between its declaration (.h) and definition (.cpp). Can/should we do something like this in Java?".

Sunday, January 15, 2012

Android: First Encounters (newbie guide)

Yesterday I have finally found some time to start learning native Android development. I must admit that the official Android documentation is very good and the learning curve so far was pretty low. That being said, I am not a total newbie when it comes to mobile development - after all, I've done it in J2ME, WebOS and even on Android (but using PhoneGap, admittedly). Still, a new framework is a new framework and when it comes to learning, Android is definitely user-friendly.

Friday, January 13, 2012

Comet in a socket: implementing web chat in Jetty

This is part 2 in WebSocket series. Read part 1 here. If you are impatient, you can find a very brief WebSocket implementation cheat sheet at the end of this post.

In one of my previous posts, I have briefly described the history of real-time communication techniques used in web pages. As much as it is good to know, however, it will not help you much in implementing a working, modern solution to the problem. So, this time we are going to do just that - implement a simple website chat using HTML5 WebSockets and a Jetty server.

Thursday, January 12, 2012

Daily dig: RESTful versioning

Even digging through mud can sometimes lead to a hidden treasure. Now, I don't want to imply that StackOverflow is an equivalent of mud (though the fact is, most of the questions asked there are uninspired, to say the least), nor that this question related to REST APIs versioning is treasure - it isn't. In fact, it was so terse that it might have been overlooked easily.

Luckily, this wasn't the case. This answer, although over 2 years old now, is still an interesting read for those interested in RESTful services. I agree with the author - older API / resource versions should be encoded as part of the URI. However, in most cases, the URIs containing version number should not be permanent and should be phased out after some time, preferably redirecting clients to the current version (unless the resource is intentionally versioned, like for example a Wiki page). Also, there should exist a "canonical" permalink versionless URI for resources, that should always serve the most recent version

Obvious? Perhaps, but as can be seen in various answers posted to that question, the opinion is divided between people preferring to keep version as part of the URI and those advocating to keep it as part of HTTP headers (and one answer stating that you should never, ever provide access to previous versions of your APIs / resources - but let's not fool ourselves, this is not realistic in the real world). Ask an IT guy a yes/no question and you will get three conflicting answers, eh?

Wednesday, January 11, 2012

Comet in a socket: let's talk real-time

I must admit, writing web applications which provide real-time functionality is a personal favorite of mine. It is still a novel thing among the mostly-static web pages - although the novelty is fading fast with more and more web pages providing real-time or pseudo real-time updates (like Twitter). It is also, to some degree, using (or abusing, as some would say) the HTTP protocol in a way it was not intended to. In short, a challenge! Don't all programmers like a little challenge now and then?

Unfortunately (or rather, fortunately) it is becoming less and less of a challenge every day, with new web technologies entering common use. Although there does not yet exist an agreed-on standard on how such a real-time communication should be done while still only using the HTTP protocol, the WebSockets proposal is gaining traction, with some major browsers already implementing it in their newest versions. And for those that don't - there is always Flash as a fallback.

But let us not get ahead of ourselves. Before we move on to WebSockets, it would be beneficial to quickly cover the other methods used for real-time communication in the past.