Kohei Nozaki's blog

The State Pattern

2022-08-11T04:21:15+00:00

What is the State Pattern?

It’s one of the GoF Design Patterns which helps to organize code that handles multiple states. If you have a code base where there are many boolean fields or an enum field used to determine the state, similar if/switch branches and unclear state transitions, a giant class where there is all of the stuff in messy code causing a maintainability issue, applying the State Pattern might be worth considering.

The State Pattern consists of the following participants:

The Context class provides the clients of your code with the public API. All of the other participants are implementation details which the clients of your class don’t have to know about.

The Context class holds a reference to an object which implements the State interface. The Context class forwards some of the API calls to the state object. The State interface defines part of the API whose behavior has to be changed depending on the state.

The ConcreteState classes implement the State interface. Each implementation exists for each possible state of your application. It handles the API calls forwarded from the Context class appropriately according to the requirement of the state it corresponds to. A ConcreteState class can trigger a state transition.

Case study: the SMTP protocol

As an example application for the State Pattern, let’s think about an SMTP server. SMTP is a protocol used for email transport. Usually email client software like Mozilla Thunderbird supports this protocol. When such software sends an email, it connects to an SMTP server and talks to the server using this protocol to send an email.

This is a typical flow of an SMTP communication session (taken from Wikipedia):

S: 220 smtp.example.com ESMTP Postfix
C: HELO relay.example.com
S: 250 smtp.example.com, I am glad to meet you
C: MAIL FROM:<bob@example.com>
S: 250 Ok
C: RCPT TO:<alice@example.com>
S: 250 Ok
C: RCPT TO:<theboss@example.com>
S: 250 Ok
C: DATA
S: 354 End data with <CR><LF>.<CR><LF>
C: From: "Bob Example" <bob@example.com>
C: To: Alice Example <alice@example.com>
C: Cc: theboss@example.com
C: Date: Tue, 15 Jan 2008 16:02:43 -0500
C: Subject: Test message
C:
C: Hello Alice.
C: This is a test message with 5 header fields and 4 lines in the message body.
C: Your friend,
C: Bob
C: .
S: 250 Ok: queued as 12345
C: QUIT
S: 221 Bye
{The server closes the connection}

The lines beginning with "S:" are sent by the server and the ones beginning with "C: " are sent by the client.

Typically, after establishing a TCP connection, the server sends a short message which indicates it is ready to accept a command from the client. The client first sends an HELO message with its hostname, then begins an email transaction with specifying the email address of the sender of the email with a MAIL FROM command. After that, the client sends the email addresses of the recipients of the email with RCPT TO commands. Then finally, the client sends a DATA command, sends the content of the email and finishes it with a line which contains only a period. If everything goes fine, the server responds with an OK message which means the email is accepted by the server and the email will be delivered to the recipients specified.

What we can see from here is that in this protocol a client has to send necessary information to the server in a specific order. The server reacts differently for each state of the communication process. For example, the client must provide a sender identification first. Otherwise, any other command from the client doesn’t get processed and the server returns an error. It means that the server remembers the current communication state at any given moment.

So, what are those states? A good way to find that out is drawing a state diagram. It will look like the following:

There are 4 states in the communication process. The first one is the idle state where the client has to start a session with an HELO command. Then it proceeds to the initial state, where the client has to provide the email address of the sender of the email with a MAIL FROM command. Then we proceed to the transaction started state where the client provides the recipients of the email with an RCPT TO command. The client might stay in this state until it finishes sending all of the recipients and that is why there is this transition which goes back to the same state. After that, finally it proceeds to the data transfer state with a DATA command where the client sends the body of the email and finishes the transaction with a line which contains only a period. Then it gets back to the initial state and if the client wants to send another email, it can start over from there.

Implementing an SMTP server with the State Pattern

In order to implement such a communication process which consists of multiple stages or states, applying the State Pattern can be a good way to keep the implementation clean and organized. Otherwise, we might end up having a giant class where there are a lot of mutable states and if/switch conditionals which are highly likely to be a maintenance problem.

Let’s look at one possible design based on the State Pattern which can handle the SMTP protocol:

The SMTPSessionHandler class corresponds to the Context class in the previous class diagram. An instance of this class exists for each SMTP conversation session. There is a public method called handleCommand() which receives a command from the client and returns the response for the command. This method has to handle the commands from the client appropriately depending on the current state. In order to achieve that, we have those 4 concrete state classes that correspond to the states in the state diagram given earlier, and what this method does is basically just forwarding the method calls to the currentState object.

In this design, state transitions are done by each concrete state class. For that purpose, the SMTPSessionHandler class has the setCurrentState() method. And when the SMTPSessionHandler class forwards the method calls to the currentState object, it also passes its own reference as an additional parameter. It allows each concrete state class to call the setCurrentState() method.

Let’s look at the source code of each participant. The SMTPSessionHandler class just holds the currentState object and forwards any handleCommand() calls to the currentState object with a reference to this object. It also has the setCurrentState() method for the concrete state classes.

public class SMTPSessionHandler {
    private State currentState = new IdleState();

    public String handleCommand(String command) {
        return currentState.handleCommand(command, this);
    }

    void setCurrentState(State newState) {
        this.currentState = newState;
    }
}

The SMTPSessionHandler class calls the underlying current state object through this State interface, which has 4 concrete implementations.

interface State {
    String handleCommand(String command, SMTPSessionHandler context);
}

The IdleState class corresponds to the very first state when a connection is established with the client. The only command it accepts is the HELO command. When it receives the HELO command, it makes a state transition by calling the setCurrentState() method of the context object with a new instance of the InitialState class and returns a greeting to the client. Otherwise, it returns an error.

class IdleState implements State {
    @Override
    public String handleCommand(String command, SMTPSessionHandler context) {
        if (command.startsWith("HELO")) {
            context.setCurrentState(new InitialState());
            return "250 smtp.example.com, I am glad to meet you";
        } else {
            return "500 5.5.1 Invalid command";
        }
    }
}

When the state transition in the IdleState class happens, the InitialState class takes over. The only command it accepts is the MAIL FROM command. When it happens, it extracts the email address which the client has sent and makes another state transition with a new instance of the TransactionStartedState class. When it creates the instance, it passes the email address it has extracted in order to let the subsequent process use it. Otherwise it returns an error.

class InitialState implements State {

    private static final Pattern PATTERN_FOR_EXTRACTING_EMAIL =
            Pattern.compile("MAIL FROM:<([^>]+)>");

    @Override
    public String handleCommand(String command, SMTPSessionHandler context) {
        Matcher matcher = PATTERN_FOR_EXTRACTING_EMAIL.matcher(command);
        if (matcher.find()) {
            String from = matcher.group(1);
            context.setCurrentState(new TransactionStartedState(from));
            return "250 Ok";
        } else {
            return "500 5.5.1 Invalid command";
        }
    }
}

The TransactionStartedState class is where we receive the destinations of the email. After specifying at least one destination, we can proceed to the next state but if there is none, it returns an error. The client has to send the destinations with the RCPT TO command. When it receives the RCPT TO command, it extracts the email address and keeps it in the List object. After sending at least one destination, the client can proceed to the next state with the DATA command. At this point, we have the address of the sender and the destinations and those are passed as the parameters of the constructor of the DataTransferState class.

class TransactionStartedState implements State {
    private static final Pattern PATTERN_FOR_EXTRACTING_EMAIL =
            Pattern.compile("RCPT TO:<([^>]+)>");
    private final String from;
    private final List<String> destinations = new ArrayList<>();

    TransactionStartedState(String from) {
        this.from = from;
    }

    @Override
    public String handleCommand(String command, SMTPSessionHandler context) {
        if (command.equals("DATA")) {
            if (destinations.isEmpty()) {
                return "500 5.5.1 Invalid command";
            } else {
                context.setCurrentState(new DataTransferState(from, destinations));
                return "354 End data with <CR><LF>.<CR><LF>";
            }
        }

        Matcher matcher = PATTERN_FOR_EXTRACTING_EMAIL.matcher(command);
        if (matcher.find()) {
            String to = matcher.group(1);
            destinations.add(to);
            return "250 Ok";
        } else {
            return "500 5.5.1 Invalid command";
        }
    }
}

The DataTransferState class corresponds to the final state of this communication process. What it does is just accumulating the body of the email in a StringBuilder object until it receives one single period which means the end of the body. After that, it will trigger the email delivery process which involves a DNS lookup, relaying the message to another SMTP server using the pieces of the data we received from the client during the process and making a state transition to the initial state. If the client wants to send another email, the client can start all over from there.

class DataTransferState implements State {
    private final String from;
    private final List<String> destinations;
    private final StringBuilder body = new StringBuilder();

    static DeliverySystem deliverySystem = (from, destinations, body) -> {
        // looks up MX records, connects to external SMTP servers and relays the message
    };

    DataTransferState(String from, List<String> destinations) {
        this.from = from;
        this.destinations = destinations;
    }

    @Override
    public String handleCommand(String command, SMTPSessionHandler context) {
        if (command.equals(".")) {
            deliverySystem.deliver(from, destinations, body.toString());
            context.setCurrentState(new InitialState());
            return "250 Ok: the email has been delivered";
        } else {
            body.append(command);
            body.append('\n');
            return null;
        }
    }
}

Conclusion: benefits of the State Pattern

That’s all of the implementation. The responsibility of each concrete state class is clear and it helps to make each class concise and short. The state transitions are also clear because when they happen, they are done in a clear way, which is the setCurrentState() method in our example. There are no cryptic boolean flags that maintain states in an unclear way. It makes the code readable, easy to maintain and extend.

For example, when you need to add a new state, what you need to do is write a new class which corresponds to the new state and add state transitions to the relevant existing state classes that are likely to be much simpler than a giant single class maintaining all of the states. It will reduce the risk of degradation.

In my experience, there are not so many use cases where this pattern is a great match, but it greatly improves the maintainability if it matches the use case and is applied appropriately.

Synchronize access to shared mutable data

2020-12-17T09:00:34+00:00

In Java, an update on mutable data made by one thread may not be visible by other threads unless an appropriate synchronization is applied. Such unsynchronized mutable data can be the cause of a nasty bug which is difficult to spot, reproduce and fix.

Why do we need a synchronization?

Let’s say we have a class called Counter which holds mutable data:

class Counter {
    int count = 0;
    int getCount() { return count; }
    int increment() { return ++count; }
}

There is an int field named count. There are also a simple getter and a method which increments the counter. The increment method can change the value in the count field which means that this class holds mutable data.

Now let’s see what happens when an instance of the Counter class is shared across 2 threads without any synchronization. We have this client code of the Counter class:

class ThisNeverFinishesOnMyLaptop {
    public static void main(String[] args) throws InterruptedException {
        Counter counter = new Counter();
        Thread backgroundThread = new Thread(() -> {
            while (true) { // wait until the counter gets incremented
                if (counter.getCount() != 0) break;
            }
            System.out.println("Finished");
        });
        backgroundThread.start();
        TimeUnit.SECONDS.sleep(1);
        counter.increment();
    }
}

First we create an instance of the Counter class then we launch a background thread which starts a while loop and waits until the counter gets incremented. Then the main thread sleeps for 1 second and calls the increment method. This increment() call is supposed to make the background thread get out of the while loop. So what this code is supposed to do is to finish after 1 second with having the "Finished" message printed into the console.

But in some environment, this code never even finishes which means the while loop becomes an infinite loop. The problem here is that due to lack of synchronization, the background thread fails to see the new value which the main thread set. In Java, without synchronization, it’s not guaranteed that updates that have been made by one thread will be visible by other threads. Another thread might see the update at some point or might not see at all like in this example on my laptop. The behavior might vary depending on the environment where this code runs in and it is totally unpredictable.

How do we do a synchronization?

Then how can we make it work properly? One way to do that is the following:

class NowItFinishesOnMyLaptop {
    public static void main(String[] args) throws InterruptedException {
        Counter counter = new Counter();
        Object lock = new Object();
        Thread backgroundThread = new Thread(() -> {
            while (true) { // wait until the counter gets incremented
                synchronized (lock) {
                    if (counter.getCount() != 0) break;
                }
            }
            System.out.println("Finished");
        });
        backgroundThread.start();
        TimeUnit.SECONDS.sleep(1);
        synchronized (lock) {
            counter.increment();
        }
    }
}

Now we create an object named lock and when we read or write data in the counter object, we do it in a synchronized block with the lock object. This guarantees that any update which is done in a synchronized block will be seen by any other code inside a synchronized block that uses the same lock object.

There is also another way to fix this. We can add the volatile keyword to the count field:

class VolatileCounter {
    volatile int count = 0;
    int getCount() { return count; }
    int increment() { return ++count; }
}

And we can replace the use of the Counter class in the ThisNeverFinishesOnMyLaptop class by the VolatileCounter class. The volatile keyword guarantees that updates made by one thread will be visible to other threads.

class ThisAlsoFinishesOnMyLaptop {
    public static void main(String[] args) throws InterruptedException {
        VolatileCounter counter = new VolatileCounter();
        Thread backgroundThread = new Thread(() -> {
            while (true) { // wait until the counter gets incremented
                if (counter.getCount() != 0) break;
            }
            System.out.println("Finished");
        });
        backgroundThread.start();
        TimeUnit.SECONDS.sleep(1);
        counter.increment();
    }
}

But there are some cases where the volatile keyword is not enough. Let’s think about the following code:

class IncrementByMultipleThreads {
    public static void main(String[] args) {
        VolatileCounter counter = new VolatileCounter();
        Set<Integer> ints = Collections.synchronizedSet(new HashSet<>());
        Runnable incrementer = () -> {
            while (true) {
                int increment = counter.increment();
                boolean added = ints.add(increment);
                if (!added) System.out.println("duplicate number detected");
            }
        };
        Thread t1 = new Thread(incrementer), t2 = new Thread(incrementer), t3 = new Thread(incrementer);
        t1.start(); t2.start(); t3.start();
    }
}

First we create a volatile counter object and a Set of integers. We use a synchronized wrapper for the set in order to make it work properly in a concurrent usecase like this. And create a Runnable object which launches an infinite loop. In the loop, it increments the counter object and puts the return value of the increment method into the set. The set will be keeping all of the numbers the counter returned. And when it fails to add the number to the set, which means the same number was already added to the set, it prints that "duplicate number detected" message. And we launch 3 threads that execute the same Runnable object.

It prints out a lot of the "duplicate number detected" messages immediately when I run it on my laptop. Which means the counter object has returned duplicate numbers. The reason behind this behavior is that the incrementation done by ++count is not atomic. What it does is that first it reads the number stored in the count field, adds 1 to it and stores the result into the count field and those are not atomically executed. And since we have 3 threads that are executing the increment method concurrently, there are chances that some of those threads read, calculate and return the same value.

If we want the counter to return unique values for all of the threads, volatile is not sufficient. We can use a synchronized block to make it happen instead:

class IncrementByMultipleThreadsWithLock {
    public static void main(String[] args) {
        Counter counter = new Counter();
        Object lock = new Object();
        Set<Integer> ints = Collections.synchronizedSet(new HashSet<>());
        Runnable incrementer = () -> {
            while (true) {
                int increment;
                synchronized (lock) {
                    increment = counter.increment();
                }
                boolean added = ints.add(increment);
                if (!added) System.out.println("duplicate number detected");
            }
        };
        Thread t1 = new Thread(incrementer), t2 = new Thread(incrementer), t3 = new Thread(incrementer);
        t1.start(); t2.start(); t3.start();
    }
}

We have introduced the lock object again, and put the increment() call inside the synchronized block. Since we have the synchronized block, we don’t need the counter to be volatile anymore, therefore we can simply use the plain Counter class instead.

Now we get unique numbers from the counter and it doesn’t show the "duplicate number detected" message anymore at least until the overflow of the counter field. In addition to the guarantee about the memory synchronization of mutable data, the synchronized block also guarantees that only one thread can execute the code placed inside the block at any given time. It effectively prevents the race condition where the counter returns duplicate numbers.

Which classes in the JDK require synchronization?

So we have seen that we need to use an appropriate way of synchronization for mutable data when it’s shared across multiple threads. There are many classes in the Java standard library that contain mutable data or require synchronization if they are shared across multiple threads. It includes some of collection classes (HashMap, ArrayList etc.) and the Format class and its subclasses (e.g. SimpleDateFormat). To find out if a class requires synchronization, the first thing you can do is to consult the documentation of the class.

If we fail to apply necessary synchronization, the consequences can be horrible. I once saw code which uses a HashMap shared across multiple threads without a synchronization. The HashMap was used as some kind of cache and there are multiple threads that update the HashMap and it could happen simultaneously. It eventually broke the underlying data structure of the HashMap and triggered an infinite loop under heavy load. Due to the fact that this kind of problem rarely shows up, it passed all of the QA processes we had and unfortunately ended up appearing in the production environment. I will never forget this incident because I was the one to fix the issue in midnight. There is a great article on the web which explains how an unsynchronized HashMap triggers an infinite loop: https://mailinator.blogspot.com/2009/06/beautiful-race-condition.html

I also sometimes see a SimpleDateFormat object stored in a static field and shared across multiple threads. This is also dangerous. It might seem to be working fine for most of the time but occasionally it produces a wrong result or throws a mysterious exception under heavy load.

How do we do synchronization for an unsynchronized class in the JDK?

Whenever we encounter a situation where we need to use some class which requires a synchronization in concurrent usecases, the first thing we can do is to look for a thread-safe equivalent.

As for collection classes, there are quite a few thread-safe classes which cover almost all of the interfaces in the collection framework (e.g. ConcurrentHashMap, CopyOnWriteArrayList). If one of those suits your usecase, simply using one of them is the safest and easiest way to apply the synchronization. But one thing we need to keep in mind is that just using thread-safe classes doesn’t always make your code work properly in concurrent usecases. For example, sometimes we need to make certain operations atomic like we did in one of the examples earlier. In such cases, we still need to use some kind of mutual execution exclusion mechanism like we did with the synchronization blocks.

As for SimpleDateFormat, if the version of your Java is 8 or greater, please check the DateTimeFormatter class, which is immutable and thread-safe.

Also one important thing to remember is that just putting volatile doesn’t make unsynchronized objects thread-safe. For example, when you have a HashMap field shared across multiple threads and you put volatile into the declaration of the HashMap field, it doesn’t guarantee that the HashMap will work property in concurrent use cases.

Conclusion

We have discussed why we need to synchronize mutable data, how we can do that and what happens when we fail to do that in concurrent usecases. The consequences can be horrible and a nasty thing about the failure of applying an appropriate synchronization is that everything looks fine for most of the time and the problem shows up only occasionally and sometimes it’s even hard to reproduce. So I recommend paying extra attention when you have to deal with mutable data in concurrent usecases.

Another good way to reduce this kind of risk is minimizing mutability in the code. Immutable data is always safe to share across threads which means we don’t have to even think about it if no mutable data is shared. One good starting point is that whenever you write a class, I recommend trying making the fields in your class final as much as possible. It will make your class one step closer to immutable.

There are a lot of more things to learn for writing safe concurrent applications in Java. If you are interested, I recommend checking this book out: https://jcip.net

The Singleton Pattern in Java

2020-09-22T03:47:08+00:00

In this entry, I’ll introduce some common ways to apply the Singleton Pattern in Java and a problem that can harm maintainability of your code base and a solution for the problem.

What is it?

The Singleton Pattern is a commonly used idiom to create and maintain objects called singletons. A singleton is the only instance of a class.

Common use cases are an object which keeps the user’s preferences in a desktop application, cache or a resource pool. One example would be a TCP connection pool where there are multiple active connections to a backend service like a database server or a web service. Typically there should be only one pool object which maintains all of the connections. Having multiple separate pool objects in one application doesn’t make much sense in most cases because it will make the use of the pool less effective.

As an example, let’s imagine that there is the following interface which allows us to fetch the weather data from some external web service:

interface WeatherFetcher {
    String fetchWeather(String cityCode) throws IOException;
    int fetchTemperature(String cityCode) throws IOException;
}

There are a couple of methods to fetch the weather (atmospheric conditions) and the temperature of a city. Since it involves network communcation, there can be an IOException.

Now imagine that you are working on implementing a feature which uses this interface. You are told that the performance requirement for this feature is strict, therefore you have decided to maintain a pool of active TCP connections to the external web service in order to keep the latency low. Applying the Singleton Pattern here sounds like a good idea because it will make sure that you will have only one pool object in your application.

Class with a private constructor

Let’s see how we can implement this with the Singleton Pattern. One simple implementation would be something like this:

public class SimpleSingleton implements WeatherFetcher {
    private static final SimpleSingleton INSTANCE = new SimpleSingleton();
    public static WeatherFetcher getInstance() { return INSTANCE; }
    private SimpleSingleton() { System.out.println("Populating the connection pool..."); }

    @Override
    public int fetchTemperature(String cityCode) {
        // gets an idle connection from the pool, sends a request
        return 71; // please assume it's from a response
    }

    @Override
    public String fetchWeather(String cityCode) {
        // gets an idle connection from the pool, sends a request
        return "Sunny"; // please assume it's from a response
    }

    // client code
    public static void main(String[] args) throws IOException {
        WeatherFetcher singleton = SimpleSingleton.getInstance();
        System.out.printf("Weather in New York: %s, %dF\n",
                singleton.fetchWeather("NewYork"), singleton.fetchTemperature("NewYork"));
    }
}

The class has a static final field called INSTANCE, where the only instance of this class is kept. There is a simple getter for that.

This class has a constructor populating the pool. Let’s imagine that it opens a lot of connections to an external web service and makes those ready to use. The constructor is private, which prevents creation of another instance of the class. Without this, any other class can create an instance of this class, which means that there is no guarantee that there is only one instance of this class. It also prevents inheritance because there is no constructor a subclass can call.

In order to get the singleton, the client needs to call the static method getInstance() first. After that, the client can fetch the weather data via the methods defined in the WeatherFetcher interface.

Class with a static holder class for lazy initialization

The SimpleSingleton class works fine, but there is one common requirement you might encounter. Populating the connection pool can be expensive and there might be a situation where the feature we just implemented will be used by only a very small number of your users, and you want to make sure that the connection pool gets populated only when it’s really needed.

We cannot guarantee that with the SimpleSingleton class. Let’s add a System.out.println() call to the main method and see what is happening there:

public static void main(String[] args) throws IOException {
    System.out.println("Doing some other stuff - I don't need the singleton yet"); // added
    WeatherFetcher singleton = SimpleSingleton.getInstance();
    System.out.printf("Weather in New York: %s, %dF\n",
            singleton.fetchWeather("NewYork"), singleton.fetchTemperature("NewYork"));
}

The main method yields the following output:

Populating the connection pool...
Doing some other stuff - I don't need the singleton yet
Weather in New York: Sunny, 71F

As you can see, the pool has got initialized even though it’s not needed yet; the constructor gets executed before the getInstance() method is called because the constructor invocation is written in the static initializer of the class. It can lead to a situation where unnecessary resource consumption is imposed on the user even though the user might not need the feature. We can work around this with having a simple private static holder class like the following:

public class LazySingleton implements WeatherFetcher {
    private static class SingletonHolder {
        static final LazySingleton INSTANCE = new LazySingleton();
    }
    public static WeatherFetcher getInstance() { return SingletonHolder.INSTANCE; }
    private LazySingleton() { System.out.println("Populating the connection pool..."); }
    ...
    // client code
    public static void main(String[] args) throws IOException {
        System.out.println("Doing some other stuff - I don't need the singleton yet");
        WeatherFetcher singleton = LazySingleton.getInstance();
        System.out.printf("Weather in New York: %s, %dF\n",
                singleton.fetchWeather("NewYork"), singleton.fetchTemperature("NewYork"));
    }
}

The LazySingleton singleton instance is kept in the static field in the static inner class called SingletonHolder and the getInstance() method returns that. Executing the main method will show the following output, as expected:

Doing some other stuff - I don't need the singleton yet
Populating the connection pool...
Weather in New York: Sunny, 71F

A problem that can harm maintainability: getInstance() everywhere

Let’s imagine that now you need to implement some code for generating a report about the weather of some cities with the singleton we just wrote. It might be implemented like this:

public class WeatherReporter {
    String generateWeatherReport(List<String> cityCodeList) {
        StringBuilder sb = new StringBuilder("=== Weather Report ===\n");
        for (String cityCode : cityCodeList) {
            try {
                String weather = LazySingleton.getInstance().fetchWeather(cityCode);
                int temperature = LazySingleton.getInstance().fetchTemperature(cityCode);
                sb.append(String.format("%s: %s, %dF\n", cityCode, weather, temperature));
            } catch (IOException e) {
                sb.append(String.format("%s: Failed to fetch data\n", cityCode));
            }
        }
        return sb.toString();
    }

    public static void main(String[] args) {
        WeatherReporter reporter = new WeatherReporter();
        System.out.println(reporter.generateWeatherReport(
                Arrays.asList("NewYork", "Montreal", "Tokyo")));
    }
}

It receives a List of city code which we need to create the report for. Then it constructs the header, iterates the list of city code and fetches the weather and the temperature for each city and builds the String which represents the report. If there is an I/O problem, we just mention that it failed to fetch the data for the city.

Executing the main method should produce the following output:

Populating the connection pool...
=== Weather Report ===
NewYork: Sunny, 71F
Montreal: Cloudy, 68F
Tokyo: Rainy, 69F

That’s a fairly complicated piece of code. Therefore, having some unit tests for the WeatherReporter class would be nice, but unfortunately it’s almost impossible because the current implementation is hard-coded to directly call the static getInstance() method to get the WeatherFetcher object, which means it always sends requests to the real external web service.

We cannot make a reliable assertion if it depends on the output of the real external service that way. One option would be running a fake of the external service, but it would require a lot of effort. And it would be nice if we could write a test case for IOException handling, but reproducing IOException in a real environment is also too much work. And the connection pool population might take a lot of time. The LazySingleton class might be hard-coded to create thousands of TCP connections and in that case it might take a few tens of seconds to populate. Waiting for such a long time with every unit test execution doesn’t sound reasonable.

So, the problem here is that we have business logic tightly coupled to an external data source. Having that kind of a method call in the middle of business logic makes writing unit tests very difficult or almost impossible because in order to make that kind of thing work, in many cases it requires some sort of specific setup or configuration and oftentimes that is too much work.

One thing we can try to work around this problem is replacing the static getInstance() method call by a functional interface. In our case, we can do something like this:

public class ImprovedWeatherReporter {

    private final Supplier<? extends WeatherFetcher> weatherFetcherSupplier;

    public ImprovedWeatherReporter(Supplier<? extends WeatherFetcher> weatherFetcherSupplier) {
        this.weatherFetcherSupplier = weatherFetcherSupplier;
    }

    String generateWeatherReport(List<String> cityCodeList) {
        StringBuilder sb = new StringBuilder("=== Weather Report ===\n");
        for (String cityCode : cityCodeList) {
            try {
                String weather = weatherFetcherSupplier.get().fetchWeather(cityCode);
                int temperature = weatherFetcherSupplier.get().fetchTemperature(cityCode);
                sb.append(String.format("%s: %s, %dF\n", cityCode, weather, temperature));
            } catch (IOException e) {
                sb.append(String.format("%s: Failed to fetch data\n", cityCode));
            }
        }
        return sb.toString();
    }

    public static void main(String[] args) {
        ImprovedWeatherReporter reporter = new ImprovedWeatherReporter(LazySingleton::getInstance);
        System.out.println(reporter.generateWeatherReport(
                Arrays.asList("NewYork", "Montreal", "Tokyo")));
    }
}

The ImprovedWeatherReporter class is not tightly coupled to the getInstance() method anymore. Instead, now it has a field for a Supplier object which returns a WeatherFetcher object. The client of this class can inject the Supplier via its constructor. In our case, we can inject LazySingleton.getInstance() for the production use.

That seems to be not much change, but it is a great improvement from the perspective of unit testing. Now we can inject any Supplier returning a WeatherFeather object, and it enables us to write reliable, fast and stable unit tests like the following:

@ExtendWith(MockitoExtension.class)
class ImprovedWeatherReporterTest {

    @Mock
    WeatherFetcher weatherFetcher;
    ImprovedWeatherReporter sut;

    @BeforeEach
    void setUp() {
        sut = new ImprovedWeatherReporter(() -> weatherFetcher);
    }

    @Test
    void producesReports() throws IOException {
        when(weatherFetcher.fetchTemperature("NewYork")).thenReturn(71);
        when(weatherFetcher.fetchWeather("NewYork")).thenReturn("Sunny");
        when(weatherFetcher.fetchTemperature("Montreal")).thenReturn(68);
        when(weatherFetcher.fetchWeather("Montreal")).thenReturn("Cloudy");

        String actual = sut.generateWeatherReport(Arrays.asList("NewYork", "Montreal"));

        assertThat(actual).isEqualTo("=== Weather Report ===\n" +
                "NewYork: Sunny, 71F\n" +
                "Montreal: Cloudy, 68F\n");
    }

    @Test
    void exception() throws IOException {
        when(weatherFetcher.fetchTemperature("Tokyo")).thenThrow(new IOException("catch this"));

        String actual = sut.generateWeatherReport(Collections.singletonList("Tokyo"));

        assertThat(actual).isEqualTo("=== Weather Report ===\n" +
                "Tokyo: Failed to fetch data\n");
    }
}

First, we create a mock of WeatherFetcher and inject a Supplier which returns the mock. With that, we can have complete control of the WeatherFetcher object the ImprovedWeatherReporter class relies on. Now we can specify what the WeatherFetcher object returns for what parameter with the mock framework you use. It will even enable us to test a case where the IOException is thrown, which was impossible with the tightly-coupled version of the WeatherReporter class. Just having a simple abstraction layer between your business logic and an external service makes writing unit tests much easier.

Another solution you might want to check is using a dependency injection framework to maintain singletons. You can easily make an object singleton and let the framework inject the singleton to the objects the framework maintains. It will reduce a lot of boilerplates, and some of the code I introduced here will be unnecessary. Especially if your code base has a dependency injection framework already, I recommend checking the documentation of your framework. For Google Guice, check this out: https://github.com/google/guice/wiki/Scopes

Conclusion

We’ve seen a couple of common idioms that can be used to apply the Singleton Pattern in Java, and discussed a common maintainability issue that is sometimes caused by applying the pattern. When you see code where there are a lot of getInstance() calls in the middle of business logic, it might be good to take some time to see if you would be able to run your business logic in isolation for unit tests. If your code was unit test friendly, your code would be likely to have wider coverage of tests and it would make future maintenance easier and the code base more stable.

Favor composition over inheritance

2020-08-14T07:52:30+00:00

In this entry, I’ll discuss problems of inheritance, which is often overused in software written in an object oriented programming language, and how we can do better with composition, which is usually a much better alternative to inheritance.

Inheritance can make your code fragile

Let’s think about some piece of software used by some cafe. It contains the following classes:

There is an interface called Beverage, which can return the price and description of a beverage. Most probably there are Coffee or Tea classes that implement the interface. And there is a class called Order, where you can add Beverage objects to it in order to calculate the grand total on the bill for a customer of the cafe. The code of these looks like this:

interface Beverage {
    BigDecimal price();
    String description();
}

class Order {

    private static final BigDecimal TAX_RATE = new BigDecimal("0.1");
    private BigDecimal subTotal = BigDecimal.ZERO;

    void add(Beverage beverage) {
        subTotal = subTotal.add(beverage.price());
    }

    void addAll(Collection<? extends Beverage> beverages) {
        for (Beverage beverage : beverages)
            subTotal = subTotal.add(beverage.price());
    }

    BigDecimal grandTotal() {
        BigDecimal tax = subTotal.multiply(TAX_RATE);
        return subTotal.add(tax);
    }
}

Now, let’s consider the following scenario: the owner of the cafe wants to start a campaign to boost their sales. The idea of the campaign is that if a customer orders more than 2 beverages at the same time, he/she will get 20% discount from the grand total. In order to implement this requirement, a programmer comes up with the CampaignOrder class which extends the Order class. It looks like the following:

class CampaignOrder extends Order {

    private static final BigDecimal DISCOUNT_RATE = new BigDecimal("0.2");
    private int numberOfBeverages;

    @Override
    void add(Beverage beverage) {
        super.add(beverage);
        numberOfBeverages++;
    }

    @Override
    void addAll(Collection<? extends Beverage> beverages) {
        super.addAll(beverages);
        numberOfBeverages += beverages.size();
    }

    @Override
    BigDecimal grandTotal() {
        BigDecimal grandTotal = super.grandTotal();
        if (numberOfBeverages > 2) {
            BigDecimal discount = grandTotal.multiply(DISCOUNT_RATE);
            grandTotal = grandTotal.subtract(discount);
        }
        return grandTotal;
    }
}

It captures add() and addAll() method calls, forwards them to its superclass and keeps track of the number of the beverages which are added in the numberOfBeverages variable. And it also captures grandTotal() method calls, forwards them to its superclass and applies the 20% discount to the grand total the superclass calculated depending on the numberOfBeverages variable.

It might look reasonable as it reuses the Order class effectively so that it won’t introduce any duplicate code. But this approach can lead to an unforseen issue due to the fact that the CampaignOrder class relies on a hidden behavior of the Order class, which is that the add() method and the addAll() method work independently. Consider that at some point some other programmer has done some quick refactoring in the addAll() method:

class Order {
    ...
    void add(Beverage beverage) {
        subTotal = subTotal.add(beverage.price());
    }

    void addAll(Collection<? extends Beverage> beverages) {
        for (Beverage beverage : beverages)
            // Someone has done refactoring. Original code was:
            // subTotal = subTotal.add(beverage.price());
            // Now:
            add(beverage);
    }
    ...
}

It hasn’t broken anything in terms of the functionality of the Order class but unfortunately it has just broken the CampaignOrder class. Remember the implementation of the CampaignOrder class which captures both of the add() and the addAll() method calls and counts the number of the beverage objects it receives. Due to the fact that now the Order class calls the add() method from the addAll() method, whenever the addAll() method of the CampaignOrder class gets called, the number of the beverage objects gets counted twice in the CampaignOrder class. In other words, now the following test case fails:

@ExtendWith(MockitoExtension.class)
class CampaignOrderTest {

    @Mock
    Beverage coffee, tea;
    CampaignOrder sut = new CampaignOrder();

    @Test
    void addAll() {
        when(coffee.price()).thenReturn(new BigDecimal("2.0"));
        when(tea.price()).thenReturn(new BigDecimal("3.0"));

        sut.addAll(Arrays.asList(coffee, tea));

        // It fails after the refactoring. Now grandTotal() returns 4.4
        // The discount is applied unexpectedly since the logic which counts beverages is broken
        // Now it's counted as 4, which is over the threshold of the discount
        assertThat(sut.grandTotal()).isEqualByComparingTo("5.5");
    }
}

The person who has done this refactoring should not be blamed. In fact this person removed a duplicate piece of code, which is a good thing, and it’s not easy to catch such an error. The real problem here is the wrong use of inheritance, which is writing a subclass that relies on an implementation detail of its super class. That introduces fragility to the codebase. Hence using inheritance this way should be avoided.

And also there can be another problematic case where a new method has been added to the Order class. If the new method can be used for adding a beverage, we need to make sure that the CampaignOrder class captures method calls to the new method, but chances are we would not even notice that there was a subclass which we might have to change.

What could have been done instead of inheritance then? The most obvious approach is using composition instead. Let’s rework the class hierarchy and make it like this:

The implementation:

interface Order {
    void add(Beverage beverage);
    void addAll(Collection<? extends Beverage> beverages);
    BigDecimal grandTotal();
}

class RegularOrder implements Order {

    private static final BigDecimal TAX_RATE = new BigDecimal("0.1");
    private BigDecimal subTotal = BigDecimal.ZERO;

    @Override
    public void add(Beverage beverage) {
        subTotal = subTotal.add(beverage.price());
    }

    @Override
    public void addAll(Collection<? extends Beverage> beverages) {
        for (Beverage beverage : beverages)
            subTotal = subTotal.add(beverage.price());
    }

    @Override
    public BigDecimal grandTotal() {
        BigDecimal tax = subTotal.multiply(TAX_RATE);
        return subTotal.add(tax);
    }
}

class CampaignOrder implements Order {

    private static final BigDecimal DISCOUNT_RATE = new BigDecimal("0.2");
    private int numberOfBeverages;

    private final Order delegate;

    CampaignOrder() {
        this(new RegularOrder());
    }

    private CampaignOrder(Order delegate) {
        this.delegate = delegate;
    }

    @Override
    public void add(Beverage beverage) {
        delegate.add(beverage);
        numberOfBeverages++;
    }

    @Override
    public void addAll(Collection<? extends Beverage> beverages) {
        delegate.addAll(beverages);
        numberOfBeverages += beverages.size();
    }

    @Override
    public BigDecimal grandTotal() {
        BigDecimal grandTotal = delegate.grandTotal();
        if (numberOfBeverages > 2) {
            BigDecimal discount = grandTotal.multiply(DISCOUNT_RATE);
            grandTotal = grandTotal.subtract(discount);
        }
        return grandTotal;
    }
}

Now the Order class has become an interface which has 2 implementations. One is the RegularOrder class, which was formerly the Order class, and the other one is the CampaignOrder class. An instance of the CampaignOrder class has a reference to a RegularOrder instance in order to reuse its functionality, but the CampaignOrder class doesn’t have any superclass anymore. With the new design, no refactoring of the RegularCampaign class can break the CampaignOrder class as long as the RegularOrder class keeps the contract of the public methods.

One important difference from the inheritance approach is that now the CampaignOrder class no longer relies on any of the implementation details of the RegularOrder class. What it relies on now is the behavior of the public methods of the RegularOrder class, which are all defined in the Order interface.

The good thing about sticking with this approach is that as long as a class keeps its functionality on the public interface level the same, it can change its internal structure without you worrying about breaking something accidentally. It greatly reduces chances of unforseen breakage. Therefore, it will make your codebase more stable. Also, with having the interface, when a new method has been added to the interface, it will make the compilation of its implementors fail since the implementation is missing. With that, unlike with the inheritance solution, we can notice that we have to add the missing implementation to its implementors immediately.

Inheritance is inflexible

Now let’s think about some details of the Beverage interface in the codebase. It has an implemention class called Coffee. The customer can add a condiment such as milk, whip or sugar into it if they want to. For some reason, the original programmer implemented this requirement using inheritance:

interface Beverage {
    BigDecimal price();
    String description();
}

class Coffee implements Beverage {
    @Override public BigDecimal price() { return new BigDecimal("1.99"); }
    @Override public String description() { return "Coffee"; }
}

class CoffeeWithMilk extends Coffee {
    @Override public BigDecimal price() { return super.price().add(new BigDecimal("0.10")); }
    @Override public String description() { return super.description() + ", Milk"; }
}

class CoffeeWithWhip extends Coffee {
    @Override public BigDecimal price() { return super.price().add(new BigDecimal("0.15")); }
    @Override public String description() { return super.description() + ", Whip"; }
}

class CoffeeWithSugar extends Coffee {
    @Override public BigDecimal price() { super.price().add(new BigDecimal("0.05")); }
    @Override public String description() { return super.description() + ", Sugar"; }
}

Now we’ve got a new requirement to implement, which is that a customer should be able to add multiple condiments into coffee. Let’s see if we can do that with inheritance. It would look like the following:

We ended up creating a lot of subclasses for all of the combinations. That will do for the time being but think about the future expansion. We will need to create many subclasses everytime we introduce a new condiment. And what if we want to reuse code which is responsible for a condiment for another beverage class, say, a Tea class? It’s not clear if we can do that in a reasonable way with this approach.

Let’s see if we could do better with composition. It would look like the following:

The implementation (the Beverage interface and the Coffee class are the same as the ones we have seen before):

class MilkWrapper implements Beverage {
    private final Beverage delegate;

    MilkWrapper(Beverage delegate) { this.delegate = delegate; }

    @Override public BigDecimal price() { return delegate.price().add(new BigDecimal("0.10")); }
    @Override public String description() { return delegate.description() + ", Milk"; }
}

class WhipWrapper implements Beverage {
    private final Beverage delegate;

    WhipWrapper(Beverage delegate) { this.delegate = delegate; }

    @Override public BigDecimal price() { return delegate.price().add(new BigDecimal("0.15")); }
    @Override public String description() { return delegate.description() + ", Whip"; }
}

class SugarWrapper implements Beverage {
    private final Beverage delegate;

    SugarWrapper(Beverage delegate) { this.delegate = delegate; }

    @Override public BigDecimal price() { return delegate.price().add(new BigDecimal("0.05")); }
    @Override public String description() { return delegate.description() + ", Sugar"; }
}

In the new implementation, the classes responsible for condiments hold a reference to a Beverage object instead of extending the Coffee class. They can still reuse the code of the Coffee class but through the Beverage interface. Instead of creating an instance of a subclass of the Coffee class, you need to provide a Coffee instance to the constructor of one of the wrapper classes. Let’s do it like this:

@Test
void coffeeWithMilk() {
    Beverage coffeeWithMilk = new MilkWrapper(new Coffee());
    assertThat(coffeeWithMilk.description()).isEqualTo("Coffee, Milk");
    assertThat(coffeeWithMilk.price()).isEqualByComparingTo("2.09");
}

@Test
void coffeeWithWhip() {
    Beverage coffeeWithWhip = new WhipWrapper(new Coffee());
    assertThat(coffeeWithWhip.description()).isEqualTo("Coffee, Whip");
    assertThat(coffeeWithWhip.price()).isEqualByComparingTo("2.14");
}

@Test
void coffeeWithSugar() {
    Beverage coffeeWithSugar = new SugarWrapper(new Coffee());
    assertThat(coffeeWithSugar.description()).isEqualTo("Coffee, Sugar");
    assertThat(coffeeWithSugar.price()).isEqualByComparingTo("2.04");
}

Now, let’s see how we can implement the new requirement about multiple condiments with the new design. In fact, no change is needed in the Coffee class and the other classes in the diagram. We can do that like this:

@Test
void coffeeWithMilkAndWhip() {
    Beverage coffee = new Coffee();
    coffee = new MilkWrapper(coffee);
    coffee = new WhipWrapper(coffee);

    assertThat(coffee.description()).isEqualTo("Coffee, Milk, Whip");
    assertThat(coffee.price()).isEqualByComparingTo("2.24");
}

@Test
void coffeeWithMilkAndSugar() {
    Beverage coffee = new Coffee();
    coffee = new MilkWrapper(coffee);
    coffee = new SugarWrapper(coffee);

    assertThat(coffee.description()).isEqualTo("Coffee, Milk, Sugar");
    assertThat(coffee.price()).isEqualByComparingTo("2.14");
}

@Test
void coffeeWithMilkAndWhipAndSugar() {
    Beverage coffee = new Coffee();
    coffee = new MilkWrapper(coffee);
    coffee = new WhipWrapper(coffee);
    coffee = new SugarWrapper(coffee);

    assertThat(coffee.description()).isEqualTo("Coffee, Milk, Whip, Sugar");
    assertThat(coffee.price()).isEqualByComparingTo("2.29");
}

First, we create an instance of the Coffee class and then wrap it with the wrapper classes as needed. This is much more flexible than the old approach which required having subclasses for all of the combinations. We can even combine one particular condiment multiple times, which would have been much harder with the inheritance approach:

@Test
void coffeeWithTripleWhip() {
    Beverage coffee = new Coffee();
    coffee = new WhipWrapper(coffee);
    coffee = new WhipWrapper(coffee);
    coffee = new WhipWrapper(coffee);

    assertThat(coffee.description()).isEqualTo("Coffee, Whip, Whip, Whip");
    assertThat(coffee.price()).isEqualByComparingTo("2.44");
}

This design is also better in terms of maintainability. Introducing a new condiment doesn’t impact any other class in the diagram (i.e. there will be no class explosion). Those wrapper classes are highly reusable because they can be reused for anything which implements the Beverage interface. Also, they don’t depend on anything but the Beverage interface. They solely rely on the public methods in the Beverage interface. There will be no breakage as long as the contract of the Beverage interface stays the same.

Conclusion

We’ve seen how improper use of inheritance can make your code fragile and inflexible. Using inheritance just for code reuse can lead to an unforseen problem in the future. And inheritance is not flexible as you might expect. When you are tempted to use inheritance, I recommend considering if you can do it using composition instead. In my experience, inheritance is not the best option for most of such cases.

I would also like to mention that composition opens up a whole new world of clever design ideas in an object oriented programming language. If you want to learn more about it, I recommend checking out the following books:

Java Generics wildcards

2020-07-17T09:49:22+00:00

This entry is a quick introduction to Java Generics wildcards, which make Java code more flexible and type-safe.

Wildcards with extends

Wildcards with extends provide flexibility to code which gets a value out of a container object. For example, let’s consider the following variable definition:

List<? extends Number> list_extendsNumber;

You can think of it as a List which contains elements whose type is a subtype of Number. For example, we can assign the following instances to the variable:

list_extendsNumber = new ArrayList<Number>();
list_extendsNumber = new ArrayList<Integer>(); // Integer extends Number
list_extendsNumber = new ArrayList<BigDecimal>(); // BigDecimal extends Number

But the following doesn’t compile since neither of them is a subtype of Number:

list_extendsNumber = new ArrayList<String>(); // compilation error; a String is not a Number
list_extendsNumber = new ArrayList<Object>(); // compilation error; Number extends Object but not the other way around

As you can see from the examples above, it’s guaranteed that a List which is assigned to the list_extendsNumber variable contains a Number or a value whose type is a subtype of Number. We can use the variable for getting a value out of it as a Number. For example:

// It's guaranteed that it contains a Number or its subtype
Number firstNumber = list_extendsNumber.get(0);

You can even write some utility method like this:

double avg(List<? extends Number> nums) {
    return nums.stream().mapToDouble(Number::doubleValue).average().orElse(0);
}

The avg() method accepts any List whose type parameter is a subtype of Number. For example, the following code which calls the method works fine:

List<Number> nums = Arrays.asList(1, 0.5d, 3);
assert avg(nums) == 1.5d;

List<Integer> ints = Arrays.asList(1, 2, 3);
assert avg(ints) == 2d;

List<BigDecimal> bds = Arrays.asList(new BigDecimal(1), new BigDecimal(2), new BigDecimal(3));
assert avg(bds) == 2d;

But the following is invalid:

List<String> strs = Arrays.asList("foo", "bar", "baz");
avg(strs) // compilation error; a String is not a Number

List<Object> objs = Arrays.asList("foo", 0.5d, 3);
avg(objs) // compilation error; Number extends Object but not the other way around

What’s good about it? Let’s see what happens if we don’t use the wildcard here. Now the method looks like this:

double avg(List<Number> nums) {
    return nums.stream().mapToDouble(Number::doubleValue).average().orElse(0);
}

Now the method can accept only List<Number> that is much less flexible. The following still works:

List<Number> nums = Arrays.asList(1, 0.5d, 3);
assert avg(nums) == 1.5d; // still fine

But neither of them compiles anymore:

List<Integer> ints = Arrays.asList(1, 2, 3);
assert avg(ints) == 2d; // compilation error

List<BigDecimal> bds = Arrays.asList(new BigDecimal(1), new BigDecimal(2), new BigDecimal(3));
assert avg(bds) == 2d; // compilation error

Which means that when you write code which gets values out of a container object (e.g. List), using wildcards with extends provides more flexibility. In other words, your code (or method) will be able to accept a wider range of parameters if wildcards with extends are used appropriately.

The limitation imposed by the use of wildcards with extends is that you won’t be able to put any value except for null through a variable which uses extends. For example:

List<? extends Number> list_extendsNumber = new ArrayList<Integer>();
list_extendsNumber.add(null); // compiles; null is the only exception
list_extendsNumber.add(1); // compilation error

Why? Remember that we can assign any List whose type parameter is a subtype of Number. For example, you can also assign a List whose type parameter is BigDecimal to the list_extendsNumber variable. In that case adding an Integer to the List should be invalid since an Integer is not a BigDecimal. The compiler prevents it from happening thanks to generics and wildcards. Adding a null is fine since null is not tied to a particular type.

Wildcards with super

While wildcards with extends make code which gets a value more flexible, wildcards with super provide flexibility to code which puts a value into a container object. Let’s consider the following variable definition:

List<? super Integer> list_superInteger;

It means a List which contains elements whose type is a supertype of Integer. For example, we can assign the following instances into the variable:

list_superInteger = new ArrayList<Integer>();
list_superInteger = new ArrayList<Number>(); // Number is a supertype of Integer
list_superInteger = new ArrayList<Object>(); // Object is a supertype of Integer

But the following doesn’t compile:

list_superInteger = new ArrayList<String>(); // compilation error; String is not a supertype of Integer

What we can see from the example above is that it’s guaranteed that the List which is assigned to the list_superInteger variable can accept an Integer. We can use it for putting a value into it. For example:

// It's guaranteed that it can accept an Integer
list_superInteger.put(123);

Or you can write some method with it like this:

void addInts(List<? super Integer> ints) {
    Collections.addAll(ints, 1, 2, 3);
}

The method can accept any of the following:

List<Integer> ints = new ArrayList<>();
addInts(ints);
assert ints.toString().equals("[1, 2, 3]");

List<Number> nums = new ArrayList<>();
addInts(nums);
assert nums.toString().equals("[1, 2, 3]");

List<Object> objs = new ArrayList<>();
addInts(objs);
assert objs.toString().equals("[1, 2, 3]");

But the following doesn’t compile:

List<String> strs = new ArrayList<>();
addInts(strs); // compilation error; List<String> cannot accept an Integer

If we didn’t use the wildcard, the method would be less flexible. Let’s say now we have this without a wildcard:

void addInts(List<Integer> ints) {
    Collections.addAll(ints, 1, 2, 3);
}

The following still compiles:

List<Integer> ints = new ArrayList<>();
addInts(ints);
assert ints.toString().equals("[1, 2, 3]");

But the following does not compile anymore:

List<Number> nums = new ArrayList<>();
addInts(nums); // compilation error

List<Object> objs = new ArrayList<>();
addInts(objs); // compilation error

The limitation imposed by the use of wildcards with super is that you will be able to get a value out of a container object only with the Object type. In other words, if you want to get a value out of the list_superInteger variable, this is the only thing you can do:

List<Integer> ints = new ArrayList<>();
ints.add(123);
List<? super Integer> list_superInteger = ints;

Object head = list_superInteger.get(0); // only Object can be used as the type of head
assert head.equals(123);

The following doesn’t compile:

Integer head = list_superInteger.get(0); // compilation error

Why? Remember that we can also assign a List<Number> or a List<Object> to the list_superInteger variable, which means that the only type that we can safely use to get a value out of it is the Object type.

The Get and Put Principle

As we have seen, appropriate use of wildcards provides more flexibility to your code. To summarize when we should use which, there is a good principle to follow:

“The Get and Put Principle: Use an extends wildcard when you only get values out of a structure, use a super wildcard when you only put values into a structure, and don’t use a wildcard when you both get and put.” - Naftalin, M., Wadler, P. (2007). Java Generics And Collections, O’Reilly. p.19

Functional interfaces exemplify this principle. For example, consider a method which accepts objects that implement functional interfaces as follows:

void myMethod(Supplier<? extends Number> numberSupplier, Consumer<? super String> stringConsumer) {
    Number number = numberSupplier.get();
    String result = "I got a number whose value in double is: " + number.doubleValue();
    stringConsumer.accept(result);
}

Supplier is something which you can apply the get principle to; you only get values out of it. And Consumer is the same for the put principle; you only put values into it.

myMethod() can accept various types of parameters. The user of the method doesn’t necessarily have to pass a Supplier<Number> and a Consumer<String>. The user can also pass a Supplier<BigDecimal> and a Consumer<Object> as follows, but that’s possible only with the use of the wildcards:

Supplier<BigDecimal> bigDecimalSupplier = () -> new BigDecimal("0.5");
AtomicReference<Object> reference = new AtomicReference<>();
Consumer<Object> objectConsumer = reference::set;

myMethod(bigDecimalSupplier, objectConsumer);

assert reference.get().equals("I got a number whose value in double is: 0.5");

Conclusion

We have seen how we can use wildcards, what the benefits of them are and when to use them. To make your code more flexible and reusable, especially when you write a method or a constructor, it’s good practice to think if you can apply wildcards to code which handles an object that has a type parameter. Remembering the get and put principle will be helpful when you do so.

References

Naftalin, M., Wadler, P. (2007). Java Generics And Collections, O’Reilly

JUnit and Mockito tips

2020-07-03T09:40:03+00:00

In this entry I’ll share some tips about JUnit and Mockito for making unit tests better.

Mockito annotations

Mockito has some annotations that can be used for reducing redundancy of tests:

@Mock
@InjectMocks
@Captor

Before looking into the usage of those annotations, let’s assume we have the following production code which consists of 2 classes.

The first one is called MailServer, which has a method called send() that sends an SMTP message which this object receives as the parameter of the method. Note that the MailServer class most probably needs to be mocked out when you want to write a unit test for a class which uses the MailServer class because it really opens a TCP connection to an SMTP server, which is not a preferable thing for unit tests.

class MailServer {
    void send(String smtpMessage) throws IOException {
        // Opens a connection to an SMTP server, sends the SMTP message
    }
}

The other class is called Messenger, which depends on the MailServer class. This class requires an instance of the MailServer class in its constructor. This class has a method called sendMail(), which has 3 parameters. The responsibility of this method is first constructing an SMTP message based on those 3 parameters and then asking the MailServer object to send the SMTP message. It also does quick error handling which translates IOException into an unchecked one with embedding the content.

class Messenger {

    private final MailServer mailServer;

    Messenger(MailServer mailServer) {
        this.mailServer = mailServer;
    }

    void sendMail(String from, String to, String body) {
        String smtpMessage = String.join("\n", "From: " + from, "To: " + to, "", body);
        try {
            mailServer.send(smtpMessage);
        } catch (IOException e) {
            throw new UncheckedIOException("Error! smtpMessage=" + smtpMessage, e);
        }
    }
}

Let’s try writing a unit test for the Messenger class. But we don’t want to use the real MailServer class because it really tries to open a connection to an SMTP server. It will make testing harder because in order to test with the real MailServer class, we really need an SMTP server which is up and running. Let’s avoid doing that and try using a mocked version of a MailServer instance for the testing here.

A happy path test case would look like the following:

class MessengerPlainTest {

    MailServer mailServer;
    Messenger sut;

    @BeforeEach
    void setUp() {
        mailServer = Mockito.mock(MailServer.class);
        sut = new Messenger(mailServer);
    }

    @Test
    @DisplayName("Messenger constructs the SMTP message and feeds MailServer")
    void test() throws IOException {
        String expected = "From: joe@example.com\n"
                + "To: jane@example.com\n\n"
                + "Hello!";

        sut.sendMail("joe@example.com", "jane@example.com", "Hello!");

        Mockito.verify(mailServer).send(expected);
    }
}

In the setUp() method, a mock MailServer object is created and injected into the constructor of the Messenger class. And in the test() method, first we create the expected SMTP message which the Messenger class has to create, then we call the sendMail() method and finally we verify that the send() method of the mock MailServer object has been called with the expected SMTP message.

With annotations, the test above can be written as follows:

@ExtendWith(MockitoExtension.class)
class MessengerTest {

    @Mock
    MailServer mailServer;
    @InjectMocks
    Messenger sut;

    @Test
    @DisplayName("Messenger constructs the SMTP message and feeds MailServer")
    void test() throws IOException {
        String expected = "From: joe@example.com\n"
                + "To: jane@example.com\n\n"
                + "Hello!";

        sut.sendMail("joe@example.com", "jane@example.com", "Hello!");

        Mockito.verify(mailServer).send(expected);
    }
}

First we annotate the test class with @ExtendWith(MockitoExtension.class) (Note that it’s a JUnit5 specific annotation, so for JUnit4 tests we need something different). Having the test class annotated with that one, when there is a field annotated with @Mock in the test class, Mockito will automatically create a mock for the field and inject it. And when there is a field annotated with @InjectMocks, Mockito will automatically create a real instance of the declared type and inject the mocks that are created by the @Mock annotation.

This is especially beneficial when many mock objects are needed because it reduces the amount of repetitive mock() method calls and also removes the need for creating the object which gets tested and injecting the mocks into the object.

And also it provides a clean way to create a mock instance of a class which has a parameterized type. When we create a mock instance of the Consumer class, a straightforward way would be the following:

Consumer<String> consumer = Mockito.mock(Consumer.class);

The problem here is that it produces an unchecked assignment warning. Your IDE will complain about it and you will get this warning when the code compiles with -Xlint:unchecked :

Warning:(35, 49) java: unchecked conversion
  required: java.util.function.Consumer<java.lang.String>
  found:    java.util.function.Consumer

With the @Mock annotation, we can get rid of the warning:

@ExtendWith(MockitoExtension.class)
class MyTest {

    @Mock
    Consumer<String> consumer;
...

There is another useful annotation called @Captor. Let’s see the following test case:

@ExtendWith(MockitoExtension.class)
class MessengerCaptorTest {

    @Mock
    MailServer mailServer;
    @InjectMocks
    Messenger sut;

    @Captor
    ArgumentCaptor<String> captor;

    @Test
    @DisplayName("Messenger constructs the SMTP message and feeds MailServer")
    void test() throws IOException {
        sut.sendMail("joe@example.com", "jane@example.com", "Hello!");

        Mockito.verify(mailServer).send(captor.capture());
        String capturedValue = captor.getValue();
        assertTrue(capturedValue.endsWith("Hello!"));
    }
}

The @Captor annotation creates an object called ArgumentCaptor which captures a method parameter of a method call of a mock object. In order to capture a parameter with an ArgumentCaptor, first we need to call the capture() method in a method call chain of the Mockito.verify() method. Then we can get the captured value with the getValue() method and we can do any assertion against it. It’s especially useful in a situation where checking the equality is not sufficient and a complex verification is needed.

AssertJ

AssertJ is an assertion library for unit tests written in Java. It provides better readability and richer assertions than its older equivalents like the one shipped with JUnit. Let’s see some example code from the official website:

// entry point for all assertThat methods and utility methods (e.g. entry)
import static org.assertj.core.api.Assertions.*;

// basic assertions
assertThat(frodo.getName()).isEqualTo("Frodo");
assertThat(frodo).isNotEqualTo(sauron);

// chaining string specific assertions
assertThat(frodo.getName()).startsWith("Fro")
                           .endsWith("do")
                           .isEqualToIgnoringCase("frodo");

// collection specific assertions (there are plenty more)
// in the examples below fellowshipOfTheRing is a List<TolkienCharacter>
assertThat(fellowshipOfTheRing).hasSize(9)
                               .contains(frodo, sam)
                               .doesNotContain(sauron);

A unique feature of AssertJ is that all of the assertions here begin with the method assertThat() which receives the parameter that gets asserted. After that we specify the conditions the parameter needs to fulfill. An advantage of this approach is that we can specify multiple conditions with method call chain. It’s more readable and less verbose than the old way where repetitive assertTrue() or assertEquals() calls are involved. It also provides rich assertions for widely used classes like List, Set or Map.

Another useful feature of AssertJ is for verifying an unhappy path where an Exception is involved. Let’s remember the production code we used in the Mockito annotation section and consider a situation where the MailServer cannot connect to the SMTP server. Due to that, the send() method throws IOException. In this situation, the Messenger class is expected to catch the IOException and translate it into UncheckedIOException with the SMTP message embedded. A unit test for this can be written as follows with AssertJ:

@ExtendWith(MockitoExtension.class)
class MessengerUnhappyTest {

    @Mock
    MailServer mailServer;
    @InjectMocks
    Messenger sut;

    @Test
    @DisplayName("Messenger throws UncheckedIOException with the SMTP message when MailServer has thrown IOException")
    void test() throws IOException {
        doThrow(new IOException("The server is down")).when(mailServer).send(any());
        String expectedMessage = "From: joe@example.com\n"
                + "To: jane@example.com\n\n"
                + "Hello!";

        assertThatThrownBy(() -> sut.sendMail("joe@example.com", "jane@example.com", "Hello!"))
                .isInstanceOf(UncheckedIOException.class)
                .hasMessage("Error! smtpMessage=%s", expectedMessage)
                .hasCauseInstanceOf(IOException.class);
   }
}

First we make the mock MailServer instance throw IOException when its send() method is called. After that we pass a lambda expression which calls the sendMail() method to the assertThatThrownBy() method of AssertJ. After that we can do various assertions. What we are checking here is that the sendMail() method throws UncheckedIOException with the SMTP message embedded and it also contains a parent Exception whose class is IOException.

Conclusion

We’ve discussed some tips about Mockito and the basic uses of AssertJ for test cases that are written in JUnit. Both Mockito and AssertJ have extensive documents and rich functionality which greatly helps writing unit tests. I highly recommend checking the references below:

The pieces of code which are used in this entry can be found on GitHub: https://github.com/nuzayats/junittips

JPA Builder パターン

2016-12-03T15:00:00+00:00

このエントリは Java EE Advent Calendar 2016 の4日目の記事です．

昨日は opengl-8080 さんの「JPA マッピングカタログ」でした．明日は tyru さんです．

JPA のおかげで，多数のカラムや複雑なリレーションを持つテーブルへのレコード生成は，プレーンな JDBC を使っていた時代に比べると，たいへん楽になりました．しかし，いぜん頭を悩ませる局面もあります．例えば，以下の図のようなスキーマを考えてみてください:

このスキーマ中の Employee テーブルへレコードを生成するコードを考えてみてください．以下のようになると思います:

public class EmployeeService {

    private final EntityManager em;

    EmployeeService(final EntityManager em) {
        this.em = em;
    }

    public long create(long deptId,
                       String name,
                       boolean temporary,
                       Set<Long> projectIds,
                       Set<String> phoneNumbers) {

        // instantiating and setting attributes of employee
        final Employee employee = new Employee();
        employee.setName(name);
        employee.setTemporary(temporary);
        employee.setProjects(new HashSet<>());
        employee.setPhones(new HashSet<>());

        // making a relation between employee and dept
        final Dept dept = em.find(Dept.class, deptId);
        employee.setDept(dept);
        em.persist(employee);
        dept.getEmployees().add(employee);

        // making relations between employee and projects
        for (final Long projectId : projectIds) {
            final Project project = em.find(Project.class, projectId);
            project.getEmployees().add(employee);
            employee.getProjects().add(project);
        }

        // creating phones
        for (final String phoneNumber : phoneNumbers) {
            final Phone phone = new Phone();
            phone.setNumber(phoneNumber);
            phone.setEmployee(employee);
            em.persist(phone);
            employee.getPhones().add(phone);
        }

        em.flush(); // making sure a generated id is present

        return employee.getId();
    }
}

いま書いたメソッド create() を呼び出すコードは，以下のようになります:

final Set<Long> projectIds = new HashSet<>();
Collections.addAll(projectIds, project1Id, project2Id);
final Set<String> phoneNumbers = new HashSet<>();
Collections.addAll(phoneNumbers, "000-0000-0001", "000-0000-0002", "000-0000-0003");

final long savedEmployeeId = service.create(
        engineeringDeptId,
        "Jane Doe",
        true,
        projectIds,
        phoneNumbers);

悪くはありません．しかし，もっと複雑なリレーションや省略可能なカラムが多数存在するケースを考えてみてください．それらに対応するメソッドの引数も多くなるのにしたがって，省略可能な引数に対応するためのオーバーロードや null の引数が並ぶメソッド呼び出しも増殖していき，だんだんメンテナンスが大変になっていきます．

このようなケースでは，私が個人的に「JPA Builder パターン」と呼んでいる書き方がおすすめです．以下に示すような非 static のネストされた Builder クラスと，そのインスタンスを生成するためのメソッドを，前述の EmployeeService クラスに対して追加します:

...

public Builder builder(long deptId, String name) {
    return new Builder(deptId, name);
}

public final class Builder { // non-static
    private final long deptId;
    private final String name;
    private boolean temporary;
    private Set<Long> projectIds = new HashSet<>();
    private Set<String> phoneNumbers = new HashSet<>();

    private Builder(final long deptId, final String name) {
        this.deptId = deptId;
        this.name = name;
    }

    public Builder temporary(boolean temporary) {
        this.temporary = temporary;
        return this;
    }

    public Builder projectIds(Long... ids) {
        Collections.addAll(projectIds, ids);
        return this;
    }

    public Builder phoneNumbers(String... numbers) {
        Collections.addAll(phoneNumbers, numbers);
        return this;
    }

    public long build() {
        // In reality, passing "this" instead of actual values (deptId, name, ...) is recommended
        return EmployeeService.this.create(deptId, name, temporary, projectIds, phoneNumbers);
    }
}

これを呼び出すコードは，以下のようになります:

final long savedEmployeeId = service.builder(engineeringDeptId, "Jane Doe")
        .temporary(true)
        .projectIds(project1Id, project2Id)
        .phoneNumbers("000-0000-0001", "000-0000-0002", "000-0000-0003")
        .build();

今回の例のようにリレーションや省略可能なカラムの数がさほど多くない場合，あまりメリットがないように見えるかもしれませんが，現実には，もっと多い数になることは珍しくありません．そのようなケースでは，このパターンを使うことで保守性・可読性を維持・向上させることができます．

このエントリで使用したクラスとテストを含むサンプルの一式は以下にありますので，実際に動かしてみたい方はチェックしてみてください．EclipseLink と Hibernate での動作を確認しています．DB のセットアップなどは必要なく，単に "mvn test" を叩くとインメモリの Apache Derby 上でテストが走ります:

https://github.com/lbtc-xxx/jpa-builder-pattern

宣伝: 求人のご案内

現在，私の勤めている (株) L is B では，エンジニアを募集しています．主に "direct" という企業向けメッセンジャーの開発と運用を行っている会社です．

残念ながら，今のところ GlassFish や WildFly のようなフルの Java EE コンテナは使われていないのですが，Java EE の構成要素，例えば JPA や JAX-RS などはヘビーに使われています．「Java EE を使って自社サービス開発したい！」という方，ぜひ以下をチェックしてみてください:

Jersey 1.x ExceptionMapper examples

2016-10-29T03:42:56+00:00

CAUTION: this posting is about Jersey 1.x which is obsoleted. If you use more modern JAX-RS implementation such as Jersey 2.x or above, please check if there are any better approaches.

JAX-RS (Jersey) provides a mechanism called ExceptionMapper which is an universal way to map an exception that thrown by a JAX-RS implementation itself or application code, to any HTTP response. In this posting, I’ll introduce some basic but useful usages of it that I have found.

ExceptionMapper for RuntimeException

When your resource method throw an Exception, say, an unintentional NullPointerException which caused by a bug or something, typically this produces a 500 Error page which created by your application container. You can catch, log those exceptions and produce a customer-friendly response with an ExceptionMapper which is something like following:

@Provider
public class RuntimeExceptionMapper implements ExceptionMapper<RuntimeException> {

    private static final Logger LOGGER = Logger.getLogger(RuntimeExceptionMapper.class.getName());

    @Override
    public Response toResponse(final RuntimeException e) {
        // taken from http://stackoverflow.com/questions/13716793/jersey-how-to-register-a-exceptionmapper-that-omits-some-subclasses
        if (e instanceof WebApplicationException) {
            return ((WebApplicationException) e).getResponse();
        }

        LOGGER.log(Level.WARNING, "RuntimeException occurred", e);

        return Response.status(Response.Status.INTERNAL_SERVER_ERROR)
                .entity("Sorry, something went wrong")
                .build();
    }
}

ExceptionMapper for NotFoundException

When your app receive a request which has no corresponding resource method, typically this produces a 404 Error page created by your container as well as an uncaught Exception. You can handle this situation with an ExceptionMapper as follows:

@Provider
public class NotFoundExceptionMapper implements ExceptionMapper<NotFoundException> {

    private static final Logger LOGGER = Logger.getLogger(NotFoundExceptionMapper.class.getName());

    @Override
    public Response toResponse(final NotFoundException e) {
        LOGGER.log(Level.FINE, "NotFoundException occurred", e);

        return Response.status(Response.Status.NOT_FOUND)
                .entity("Check the destination path of your request - we have no API here")
                .build();
    }
}

ExceptionMapper for ParamExceptionMapper

Let’s say you have a value class which is something like following:

public class EmployeeId {

    private final long value;

    public EmployeeId(final long value) {
        if (value < 1) {
            throw new IllegalArgumentException("EmployeeId must be larger than zero");
        }

        this.value = value;
    }

    public EmployeeId(final String value) {
        this(Long.parseLong(value));
    }

    // getter and toString omitted
}

And you have a resource method which receives an EmployeeId as follows:

@Path("myresource")
public class MyResource {

    @GET
    @Path("emp")
    @Produces(MediaType.TEXT_PLAIN)
    public String emp(@QueryParam("id") EmployeeId id) {
        ...
    }
}

When the resource method receives a valid id, say 123, Jersey automatically constructs an EmployeeId instance and passes it to the application code. That’s fine, but consider if a malicious user has sent an invalid value, say -1. Typically this produces an error page which created by your container as well. You may want to return a more informational response with HTTP status code 400 with an ExceptionMapper which is something like:

@Provider
public class ParamExceptionMapper implements ExceptionMapper<ParamException> {

    private static final Logger LOGGER = Logger.getLogger(ParamExceptionMapper.class.getName());

    @Override
    public Response toResponse(final ParamException e) {
        LOGGER.log(Level.FINE, "ParamException occurred", e);

        final StringBuilder sb = new StringBuilder("Your parameter '" + e.getParameterName() + "' is invalid");

        final Throwable cause = e.getCause();
        if (cause instanceof IllegalArgumentException) {
            final String message = cause.getMessage();
            if (message != null && !message.isEmpty()) {
                sb.append(": ").append(message);
            }
        }

        return Response.status(Response.Status.BAD_REQUEST)
                .entity(sb.toString())
                .build();
    }
}

Conclusion

ExceptionMapper helps making error responses of your REST APIs more helpful. And it reduces repetitive exception handling code in your resource classes that tend to be tons of boilarplate.

You can obtain complete code based on Jersey 1.x and testcases that powered by Arquillian, Embedded Tomcat and Apache HttpClient, from my GitHub repository.

References

JPA Builder Pattern

2016-10-16T09:07:19+00:00

Thanks to JPA, creating an entity which has tons of fields and complex relations has become much easier than the plain old JDBC era. but there are still some difficulties with it. for example, consider that you have a database schema which is something like following diagram:

With those entities, let’s say you have to write some code for creating an employee. this would be something like:

public class EmployeeService {

    private final EntityManager em;

    EmployeeService(final EntityManager em) {
        this.em = em;
    }

    public long create(long deptId,
                       String name,
                       boolean temporary,
                       Set<Long> projectIds,
                       Set<String> phoneNumbers) {

        // instantiating and setting attributes of employee
        final Employee employee = new Employee();
        employee.setName(name);
        employee.setTemporary(temporary);
        employee.setProjects(new HashSet<>());
        employee.setPhones(new HashSet<>());

        // making a relation between employee and dept
        final Dept dept = em.find(Dept.class, deptId);
        employee.setDept(dept);
        em.persist(employee);
        dept.getEmployees().add(employee);

        // making relations between employee and projects
        for (final Long projectId : projectIds) {
            final Project project = em.find(Project.class, projectId);
            project.getEmployees().add(employee);
            employee.getProjects().add(project);
        }

        // creating phones
        for (final String phoneNumber : phoneNumbers) {
            final Phone phone = new Phone();
            phone.setNumber(phoneNumber);
            phone.setEmployee(employee);
            em.persist(phone);
            employee.getPhones().add(phone);
        }

        em.flush(); // making sure a generated id is present

        return employee.getId();
    }
}

And you will use the method create() as follows:

final Set<Long> projectIds = new HashSet<>();
Collections.addAll(projectIds, project1Id, project2Id);
final Set<String> phoneNumbers = new HashSet<>();
Collections.addAll(phoneNumbers, "000-0000-0001", "000-0000-0002", "000-0000-0003");

final long savedEmployeeId = service.create(
        engineeringDeptId,
        "Jane Doe",
        true,
        projectIds,
        phoneNumbers);

Not so bad, but think about if there are more complex relations or attributes that may be optional. the arguments of the method will be much longer, and hard to maintain.

In such a case, a pattern which I call "JPA builder pattern" would be nice. you create a non-static nested builder class and a method which creates a builder, into the class EmployeeService, as follows:

...

public Builder builder(long deptId, String name) {
    return new Builder(deptId, name);
}

public final class Builder { // non-static
    private final long deptId;
    private final String name;
    private boolean temporary;
    private Set<Long> projectIds = new HashSet<>();
    private Set<String> phoneNumbers = new HashSet<>();

    private Builder(final long deptId, final String name) {
        this.deptId = deptId;
        this.name = name;
    }

    public Builder temporary(boolean temporary) {
        this.temporary = temporary;
        return this;
    }

    public Builder projectIds(Long... ids) {
        Collections.addAll(projectIds, ids);
        return this;
    }

    public Builder phoneNumbers(String... numbers) {
        Collections.addAll(phoneNumbers, numbers);
        return this;
    }

    public long build() {
        // In reality, passing "this" instead of actual values (deptId, name, ...) is recommended
        return EmployeeService.this.create(deptId, name, temporary, projectIds, phoneNumbers);
    }
}

And you will use the builder as follows:

final long savedEmployeeId = service.builder(engineeringDeptId, "Jane Doe")
        .temporary(true)
        .projectIds(project1Id, project2Id)
        .phoneNumbers("000-0000-0001", "000-0000-0002", "000-0000-0003")
        .build();

It doesn’t make much sense if relations or attributes that may be optional are not that many as this example, but in reality, entities likely to have those much more. in such a case, I believe this pattern makes your code much clean, readable and maintainable.

You can obtain the entire project which contains entities, the service class and executable tests that run with an embedded database, from my GitHub repo.

Jukito integration with JPA and guice-persist

2016-07-31T02:28:48+00:00

A DI container such as Guice helps you to assemble loosely-coupled classes that easy to write unit tests.

But integration tests, that are necessary thing as well as unit tests, it’s a different story from writing unit tests. writing integration tests involve many cumbersome initialization code of frameworks such as DI container, JPA provider or an application server. they tend to spread across testcases and make testcases messy.

If you use Guice as DI container, Jukito helps you to write integration tests. its official documentation covers some simple usecases, but not mentioned about integration with guice-persist. so in this entry I’ll introduce you how to integrate and begin writing clean testcases without messy boilarplate code with them.

Create an application to be tested

Consider we have a simple JPA entity and a service class which uses it.

An entity class named Employee:

@Entity
public class Employee implements Serializable {
    @Id
    @GeneratedValue
    private Long id;
    private String name;

    public Employee(final String name) {
        this.name = name;
    }
...

The service class to be tested named EmployeeService:

public class EmployeeService {

    @Inject
    private EntityManager em;

    @Transactional
    public Long save(final String name) {
        final Employee employee = new Employee(name);
        em.persist(employee);
        em.flush();
        return employee.getId();
    }

    public List<String> findAllNames() {
        return em.createQuery("SELECT e.name FROM Employee e ORDER BY e.name", String.class).getResultList();
    }
}

Create a module for testing

Next, we will create a module for testing (BTW I recommend you to use Module overriding for creating one for testing, based on production one). this would be something like:

// Taken from https://gist.github.com/JWGmeligMeyling/785e459c4cbaab606ed8 , thanks!
public class DatabaseModule extends AbstractModule {

    @Override
    protected void configure() {
        install(new JpaPersistModule("myPU"));
        bind(JPAInitializer.class).asEagerSingleton();
    }

    @Singleton
    private static class JPAInitializer {
        @Inject
        public JPAInitializer(final PersistService service) {
            service.start();
        }
    }
}

Create JpaJukitoRunner

Then, create a special TestRunner which extends JukitoRunner as follows:

public class JpaJukitoRunner extends JukitoRunner {

    public JpaJukitoRunner(final Class<?> klass) throws InitializationError, InvocationTargetException, InstantiationException, IllegalAccessException {
        super(klass);
    }

    public JpaJukitoRunner(final Class<?> klass, final Injector injector) throws InitializationError, InvocationTargetException, InstantiationException, IllegalAccessException {
        super(klass, injector);
    }

    private UnitOfWork unitOfWork;

    @Override
    protected Object createTest() throws Exception {
        this.unitOfWork = getInjector().getInstance(UnitOfWork.class);
        this.unitOfWork.begin();
        return super.createTest();
    }

    @Override
    public void run(final RunNotifier notifier) {
        notifier.addListener(new RunListener() {

            @Override
            public void testFinished(final Description description) throws Exception {
                // this ensures every tests use distinct entity manager instances
                unitOfWork.end();
            }
        });
        super.run(notifier);
    }
}

This ensures every tests use distinct EntityManager instances for each execution. without this, only one EntityManager instance will be used for all of executions of test cases because it is stored in a ThreadLocal and JUnit uses only one Thread by default for all test executions.

That means that many entities will be kept managed during an execution. consider if you have thousands of test classes in your project - entities will be shared across all of test execution. it will make your EntityManager fat, also your tests may get affected by 1st level cache which got dirty by other test executions.

Create a testcase

Finally, you can write a testcase as follows:

@RunWith(JpaJukitoRunner.class)
@UseModules(DatabaseModule.class)
public class EmployeeServiceTest {

    @Inject
    private EmployeeService sut;
    @Inject
    private EntityManager em;

    @Before
    @Transactional
    public void setUp() throws Exception {
        em.createQuery("DELETE FROM Employee").executeUpdate();
    }

    @Test
    public void saveShouldPersistJohnDoe() throws Exception {
        final String name = "John Doe";

        final long id = sut.save(name);

        final Employee employee = em.find(Employee.class, id);
        assertThat(employee.getName(), is(name));
    }

    @Test
    public void findAllNamesShouldReturnExpectedResult() throws Exception {
        sut.save("Jane Doe");
        sut.save("John Doe");

        final List<String> result = sut.findAllNames();

        assertThat(result.size(), is(2));
    }
}

You can see that there are no any cumbersome initialization code of JPA or database stuff. and note that you can use declarative transaction management by @Transactional, for both the sut class and populating test data into your database.

Cleaning 1st level cache

I have mentioned about bad effect of 1st level cache of EntityManager earlier in this entry, so you may consider that you want to clean 1st level cache. you can do it with em.clear(), or also something like following:

@RunWith(JpaJukitoRunner.class)
@UseModules(DatabaseModule.class)
public class EmployeeServiceTestManagesUOW {

    @Inject
    private Provider<EmployeeService> sut;
    @Inject
    private Provider<EntityManager> em;
    @Inject
    private UnitOfWork unitOfWork;

    @Before
    @Transactional
    public void setUp() throws Exception {
        em.get().createQuery("DELETE FROM Employee").executeUpdate();
    }

    @Test
    public void saveShouldPersistJohnDoe() throws Exception {
        final String name = "John Doe";
        newEntityManager();

        final long id = sut.get().save(name);

        newEntityManager();
        final Employee employee = em.get().find(Employee.class, id);
        assertThat(employee.getName(), is(name));
    }

    private void newEntityManager() {
        unitOfWork.end();
        unitOfWork.begin();
    }
}

In this test, there are three distinct EntityManagers involved in a test execution. the one in setUp(), another one in invocation of sut#save() and finally one for em#find() which is used for assertion.

But in my opinion, this is an overkill and using one shared EntityManager for one test execution would be sufficient. if it didn’t work well, something may wrong with your usage of JPA.

Conclusion

We have seen how testcases which involve JPA and guice-persist could be written cleanly with Jukito. it enables us to eliminate cumbersome boilarplate code for managing and invoking Injector, JPA initialization and manual transaction management. now our testcases look pretty clean.

Executable testcases and the example project can be obtained from my GitHub repository.

References

https://gist.github.com/JWGmeligMeyling/785e459c4cbaab606ed8 - many part of this entry taken from this gist. thank you so much!

Managing multiple JPA persistence units with guice-persist

2016-06-19T09:15:35+00:00

Guice has an extension named guice-persist which aim for providing integration between Guice and a data persistence mechanism. it gives declarative transaction management functionality with the annotation @com.google.inject.persist.Transactional which works with a standalone environment or plain servlet containers such as Tomcat or Jetty.

guice-persist supports JPA and it’s simple to use with only one persistence unit, but to use it with multiple persistence units, it requires some tricks.

The official Guice wiki has only some brief description and I can’t find any complete example to implement it in an actual application. so, in this entry I’ll give you a complete example about how to write a module which manages multiple persistence units.

Module hierarchy

To manage multiple PUs in a module, you should create PrivateModule subclasses of the same number of your persistence units.

Let’s say we have two persistence units that one is named masterPU and another one named slavePU. For example, we have the following persistence.xml in an application:

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.1" xmlns="http://xmlns.jcp.org/xml/ns/persistence"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd">
    <persistence-unit name="masterPU" transaction-type="RESOURCE_LOCAL">
...
    </persistence-unit>
    <persistence-unit name="slavePU" transaction-type="RESOURCE_LOCAL">
...
    </persistence-unit>
</persistence>

In this case, we are going to create and assemble classes as the following diagram:

MasterPu and SlavePu are qualifier annotations that used for distinguish multiple bindings of JPA classes.

Writing modules

So, how do you write those modules? I’ll show you some important parts of them.

Qualifier annotations

First you need to create the two qualifier annotations something like this:

import static java.lang.annotation.ElementType.*;

@javax.inject.Qualifier
@java.lang.annotation.Target({FIELD, PARAMETER, METHOD})
@java.lang.annotation.Retention(java.lang.annotation.RetentionPolicy.RUNTIME)
public @interface MasterPu {
}

Don’t forget about SlavePu as well.

JpaPersistPrivateModule

Now here’s the most important part - this class installs JpaPersistModule and rebinds and exposes JPA class bindings:

public class JpaPersistPrivateModule extends PrivateModule {
    protected final String persistenceUnitName;
    protected final Properties props;
    protected final Class<? extends Annotation> qualifier;

    public JpaPersistPrivateModule(final String persistenceUnitName, final Properties props, final Class<? extends Annotation> qualifier) {
        this.persistenceUnitName = persistenceUnitName;
        this.props = props;
        this.qualifier = qualifier;
    }

    public JpaPersistPrivateModule(final String persistenceUnitName, final Class<? extends Annotation> qualifier) {
        this(persistenceUnitName, new Properties(), qualifier);
    }

    @Override
    protected void configure() {
        install(new JpaPersistModule(persistenceUnitName).properties(props));
        rebind(qualifier, EntityManagerFactory.class, EntityManager.class, PersistService.class, UnitOfWork.class);
        doConfigure();
    }

    private void rebind(Class<? extends Annotation> qualifier, Class<?>... classes) {
        for (Class<?> clazz : classes) {
            rebind(qualifier, clazz);
        }
    }

    private <T> void rebind(Class<? extends Annotation> qualifier, Class<T> clazz) {
        bind(clazz).annotatedWith(qualifier).toProvider(binder().getProvider(clazz));
        expose(clazz).annotatedWith(qualifier);
    }

    /**
     * bind your interfaces and classes as well as concrete ones that use JPA classes explicitly
     */
    protected void doConfigure() {
        // write your bindings in your subclasses
        // bindConcreteClassWithQualifier(MyTableService.class);
        // ...
    }

    /**
     * binds and exposes a concrete class with an annotation
     */
    protected <T> void bindConcreteClassWithQualifier(Class<T> clazz) {
        bind(clazz).annotatedWith(qualifier).to(clazz);
        expose(clazz).annotatedWith(qualifier);
    }

    /**
     * binds and exposes a concrete class without any annotation
     */
    protected void bindConcreteClass(Class<?> clazz) {
        bind(clazz);
        expose(clazz);
    }
}

First, this class installs JpaPersistModule and it creates bindings of JPA classes without any annotation but those bindings will not be exposed globally because we are in a PrivateModule. then, this class rebinds them with a qualifier annotation and exposes them with qualifier annotation. eventually, bindings of the four JPA classes will be created with a qualifier annotation.

MyModule, MasterPuModule and SlavePuModule

public class MyModule extends AbstractModule {
    @Override
    protected void configure() {
        install(new MasterPuModule());
        install(new SlavePuModule());
    }

    private static class MasterPuModule extends JpaPersistPrivateModule {
        public MasterPuModule() {
            super("masterPU", MasterPu.class);
        }

        @Override
        protected void doConfigure() {
            bindConcreteClassWithQualifier(MyTableService.class);
        }
    }

    private static class SlavePuModule extends JpaPersistPrivateModule {
        public SlavePuModule() {
            super("slavePU", SlavePu.class);
        }

        @Override
        protected void doConfigure() {
            bindConcreteClassWithQualifier(MyTableService.class);
        }
    }
}

This class installs two JpaPersistPrivateModule subclasses for persistence units and binds a service class named MyTableService which requires injection of EntityManager. this module creates two distinct annotated bindings for the class.

Note that if you need declarative transaction management by @Transactional for your service classes, you should create bindings of them inside doConfigure(). for example, if you creates such bindings in MyModule#configure(), declarative transactions won’t work.

How about PersistFilter?

If you need PersistFilter for those two modules, you need to create a binding for each modules inside doConfigure() as follows:

Key<PersistFilter> key = Key.get(PersistFilter.class, qualifier);
bind(key).to(PersistFilter.class);
expose(key);

Then, install all of modules inside your subclass of ServletModule. after that, create filter mappings inside configureServlets() as follows:

filter("/*").through(Key.get(PersistFilter.class, MasterPu.class));
filter("/*").through(Key.get(PersistFilter.class, SlavePu.class));

Conclusion

We have seen an example of a Guice module that manages two persistence units with guice-persist. check my GitHub repository for the complete example project and testcases.

In-container JMS consumer/producer example

2016-05-14T14:06:12+00:00

In this entry, I’ll show you a complete example of using JMS in a Java EE 7 compliant application container, through creating a webapp which consists of both a consumer and a producer. we’re going to deploy it to a container (WildFly 10.0.0.Final) and see a webapp produces and consumes a message.

Launch the container

Launch WildFly server with the following parameter so the bundled message queue broker (ActiveMQ Artemis) will be launched:

./standalone.sh -c standalone-full.xml

For IntelliJ IDEA, check how to launch WildFly with -c standalone-full.xml from this SO: http://stackoverflow.com/questions/25849810/how-to-run-wildfly-with-standalone-full-xml-from-intellij-idea

Define a queue

Launch jboss-cli and define a queue with this command:

jms-queue add --queue-address=testQueue --entries=queue/test,java:jboss/exported/jms/queue/test

Check if it’s successfully created:

[standalone@localhost:9990 /] /subsystem=messaging-activemq/server=default/jms-queue=testQueue:read-resource
{
    "outcome" => "success",
    "result" => {
        "durable" => true,
        "entries" => [
            "queue/test",
            "java:jboss/exported/jms/queue/test"
        ],
        "legacy-entries" => undefined,
        "selector" => undefined
    }
}

Create the webapp which contains consumer/producer

Here we’re going to create following three classes in the webapp:

MyProducer: a Stateless Session Bean which produces a message to the queue
MyConsumer: a Message-Driven Bean which consumes any messages being sent to the queue
MyServlet: Receives HTTP GET request and kicks MyProducer

The whole project can be obtained from My GitHub repository.

MyProducer

@Stateless
@LocalBean
public class MyProducer {

    @Resource(mappedName = "java:/queue/test")
    Queue testQueue;
    @Inject
    JMSContext jmsContext;

    public void enqueue(final String text) {
        jmsContext.createProducer().send(testQueue, text);
    }
}

MyConsumer

@MessageDriven(name = "MyMDB", activationConfig = {
        @ActivationConfigProperty(propertyName = "destination", propertyValue = "queue/test"),
        @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"),
        @ActivationConfigProperty(propertyName = "acknowledgeMode", propertyValue = "Auto-acknowledge")})
public class MyConsumer implements MessageListener {

    private final static Logger LOGGER = Logger.getLogger(MyConsumer.class.toString());

    @Override
    public void onMessage(final Message msg) {
        if (msg instanceof TextMessage) {
            try {
                final String text = ((TextMessage) msg).getText();
                LOGGER.info(() -> "Received: " + text);
            } catch (final JMSException e) {
                throw new RuntimeException(e);
            }
        }
    }
}

MyServlet

@WebServlet(urlPatterns = "/")
public class MyServlet extends HttpServlet {

    @EJB
    MyProducer myProducer;

    @Override
    protected void doGet(final HttpServletRequest req, final HttpServletResponse resp) throws ServletException, IOException {
        final String text = "Hello, JMS!";
        myProducer.enqueue(text);
        resp.getWriter().write("Published! check output of the consumer: " + text + "\n");
    }
}

Of course you don’t need to put MyConsumer to the webapp which contains MyProducer. this is just an example, and in fact, just for asynchronous/background processing in a webapp, you better use more simple EJB Asynchronous methods or ManagedExecutorService instead of JMS. for real use-case, you may create a dedicated app for queue consumers and place your MyConsumer equivalent into it.

Trigger producing and consuming

Deploy the app and submit HTTP GET request to it as follows.

$ curl http://localhost:8080/jms-example/

If it worked successfully, you’ll see following response from the Servlet:

Published! check output of the consumer: Hello, JMS!

Then check console/output/log of your application container. if the consumer worked successfully, you can see the output something like following:

13:55:18,168 INFO  [class jms.MyConsumer] (Thread-438 (ActiveMQ-client-global-threads-271921988)) Received: Hello, JMS!

Conclusion

As described above, thanks to JMS, we can develop a messaging app without tons of annoying, vendor-specific boilar plate code, in lean and simple semantics. there are no worries of maintaining logic of connection handling, polling loop and so forth in the application code.

And note that there are no use of vendor-specific classes - it uses only standardized API which comes from javaee-api. you may need to alter some portion of code when you deploy this example to other Java EE 7 compliant container such as GlassFish, WebLogic or WebSphere, but that should be few.

References

PrivateModule, TypeLiteral and AssistedInject - Google Guice techniques for complex scenarios

2016-02-07T02:44:32+00:00

In this entry I’ll introduce you some Google Guice techniques that can be used for some complex class designing scenarios, I’ve learned recently and found very helpful.

I’ve tested techniques that used in this entry works with Google Guice 3.0 and 4.0. example implementations and test cases can be obtained from my GitHub repository.

PrivateModule

Consider you have classes that something like following diagram:

You have a service class named MyService which depends on the interface named MyStrategy (DOC, Depended On Component). the DOC has two implementations, the one EnglishStrategy says "Hello", when the method of it named sayHello() is being called, and another one JapaneseStrategy says "Konnichiwa".

There are annotation classes English and Japanese to distinguish two strategy MyService depends on. these two annotations are written as follows:

import static java.lang.annotation.ElementType.*;

@javax.inject.Qualifier
@java.lang.annotation.Target({FIELD, PARAMETER, METHOD})
@java.lang.annotation.Retention(java.lang.annotation.RetentionPolicy.RUNTIME)
public @interface English {
}

On that situation, you want Guice to produce and inject two MyService instances for your client class. you would declare injection points as follows:

// (1) call for injection of MyService annotated as @English
@Inject
@English
MyService englishService;

// (2) call for injection of MyService annotated as @Japanese
@Inject
@Japanese
MyService japaneseService;

But it doesn’t work due to Guice failed to look MyService binding annotated as @English and @Japanese up.

In such case, you can create two PrivateModule and install them in your Module implementation as follows:

@Override
protected void configure() {
    install(new PrivateModule() {
        @Override
        protected void configure() {
            // bind MyService annotated as English to MyService PRIVATELY
            bind(MyService.class).annotatedWith(English.class).to(MyService.class);

            // expose MyService annotated as English GLOBALLY. this fulfills injection point (1)
            expose(MyService.class).annotatedWith(English.class);

            // bind MyStrategy to EnglishStrategy PRIVATELY
            bind(MyStrategy.class).to(EnglishStrategy.class);
        }
    });
    install(new PrivateModule() {
        @Override
        protected void configure() {
            // bind MyService annotated as Japanese to MyService PRIVATELY
            bind(MyService.class).annotatedWith(Japanese.class).to(MyService.class);

            // expose MyService annotated as Japanese GLOBALLY. this fulfills injection point (2)
            expose(MyService.class).annotatedWith(Japanese.class);

            // bind MyStrategy to JapaneseStrategy PRIVATELY
            bind(MyStrategy.class).to(JapaneseStrategy.class);
        }
    });
}

These PrivateModule create two bindings:

MyService annotated with English or Japanese.
MyStrategy without any annotation for EnglishStrategy or JapaneseStrategy privately

And, it exposes only the binding 1. If the binding 2 exposes as well or created in global scope, Guice will complain that there are two bindings for MyStrategy without any annotation. that’s point what PrivateModule helps.

TypeLiteral

Consider you have following injection points that with generics type parameter:

@Inject
List<String> stringList;
@Inject
List<Integer> integerList;

To bind instances to these injection points, you need to use TypeLiteral in your Module implementation as follows:

@Override
protected void configure() {
    bind(new TypeLiteral<List<String>>() {
    }).toInstance(new ArrayList<String>() {{
        add("hello");
    }});

    bind(new TypeLiteral<List<Integer>>() {
    }).toInstance(new ArrayList<Integer>() {{
        add(123);
    }});
}

With this module, Guice resolves correct the two injection points with use of generics type parameter.

Also you can leave constructing an instance to Guice if you have an implementation class. for example, consider you have a generic interface as follows:

public interface MyGenericService<T extends Number> {
    T get();
}

Injection points:

@Inject
MyGenericService<Integer> integerService;
@Inject
MyGenericService<Double> doubleService;

Two implementations:

public class MyIntegerService implements MyGenericService<Integer> {
    @Override
    public Integer get() {
        return 123;
    }
}

public class MyDoubleService implements MyGenericService<Double> {
    @Override
    public Double get() {
        return 0.5;
    }
}

To create bindings to these injection points without creating instances in the module, you can write as follows in your module implementation:

bind(new TypeLiteral<MyGenericService<Integer>>(){}).to(MyIntegerService.class);
bind(new TypeLiteral<MyGenericService<Double>>(){}).to(MyDoubleService.class);

AssistedInject

Consider you have classes that something like following diagram:

You have a service class named MyProduct, which requires a parameter name for its constructor. the parameter name varies in each injection, so we create the factory interface MyFactory for it. also, MyProduct depends on MyCollaborator.

You would create MyProduct instance in your client as follows:

@Inject
MyFactory factory;
...
public void someMethodInClient() {
    final MyProduct product = factory.create("foo");
    ...
}

So, anyway, an implementation of MyFactory is needed. it must be simple in this case but it brings another boilarplate code that should be avoided as possible. and imagine that if MyProduct has a hundreds of DOCs? it will be longer as numbers of DOCs grows. manual construction procedure will be something like new MyProduct(new MyCollaborator1(), new MyCollaborator2(), …) . obviously, it should be avoided.

With AssistedInject , Guice will do it instead of you.

First, create MyFactory as follows:

public interface MyFactory {
    MyProduct create(String name);
}

Note that you don’t need to create an implementation class yourself, just forget about it. next, put an additional dependency to your pom.xml:

<dependency>
    <groupId>com.google.inject.extensions</groupId>
    <artifactId>guice-assistedinject</artifactId>
    <version>4.0</version>
</dependency>

Put following procedure to your Module implementation:

@Override
protected void configure() {
    install(new FactoryModuleBuilder().build(MyFactory.class));
}

Finally, declare an injection point in the MyProduct class as follows. note that a constructor parameter name is annotated as @Assisted:

@Inject
MyProduct(@Assisted final String name, final MyCollaborator collaborator) {
    this.name = name;
    this.collaborator = collaborator;
}

With that, Guice automatically creates MyFactory implementation for you, and handles parameter name nicely.

References

jEdit conditional line deleting with regex

2016-01-31T09:03:26+00:00

How to delete lines that contains a keyword?

Open Search And Replace dialog and turn Regular expressions on then put ^.*KEYWORD.*$\n to Search for box, leave Replace with blank then hit Replace All.

How to delete lines that NOT contains a keyword?

With ^((?!KEYWORD).)*$\n do the same to the above. for detail check http://stackoverflow.com/questions/406230/regular-expression-to-match-line-that-doesnt-contain-a-word

My jEdit setup on OS X

2016-01-31T07:00:16+00:00

Make Option key work

Put a file contains following line to $HOME/Library/jEdit/startup so shortcuts that used Option work.

Debug.ALT_KEY_PRESSED_DISABLED = false;

Check http://www.nailedtothex.org/roller/kyle/entry/installing-and-configuring-jedit-5 for further information about setup jEdit on OS X.

Basic configurations

Just for my preferences:

Go to View section
- check Show full path of buffer in title bar
Go to Editing section
- set Word wrap to soft
- set Wrap margin to 100
- check Soft (emulated with spaces) tabs
Go to Toolbar
- uncheck Show tool bar
Go to Text Area
- check Caret: thick

Customize shortcut

First, choose keymap Mac OS X. Then customize maps that unusable by default because OS X takes precedence.

Incremental Search: CMD+, launches application’s preferences window. so I bind it to Option+, instead.

Note: Draw multi-key shortcuts on screen menu bar option makes some shortcut such as move to dockables unusable, I don’t know why.

Basic operations

According to http://comments.gmane.org/gmane.editors.jedit.user/6251 , C+c C+j : (Scroll to Current Line) can be used as "Back To Text Area" equivalent.

Macros

From Macros ⇒ New Macro, you can create a BeanShell macros. For example, the macro below puts current date to the position of caret:

textArea.setSelectedText(java.time.LocalDate.now().toString());

Recommended plugins

Project Viewer

This plugin enables managing set of files in a unit named project. imagine the same name thing that implemented in some IDEs for Java development such as Eclipse or NetBeans.

There are some plugin that requires it. For example, FastOpen is a companion plugin for Project Viewer that enables open files fast with keyboard. I’ve set Delay before searching option to the smallest value (0.5sec).

The documentation of this plugin can be obtained from http://plugins.jedit.org/plugindoc/ProjectViewer/

Console

This plugin integrates console as a jEdit dockable.

Its advantages are commands that for invoking jEdit’s functionalities. for example, %edit foo.txt opens a file as a jEdit buffer. another useful one is redirecting output to a new jEdit buffer. for example, typing echo hello, jEdit in the console and hitting Ctrl+ENTER yields a new jEdit buffer with contents of hello, jEdit. also there are many useful variable syntax. e.g. ${f} for the current buffer’s path name. BeanShell console is available as well.

Also SshConsole Plugin extends its functionality to support remote hosts can be connected with SSH.

I recommend that check all three options in Console ⇒ System Shell ⇒ Events to chdir at Plugin Options.

The documentation of this plugin can be obtained from http://plugins.jedit.org/plugindoc/Console/

WhiteSpace

As its name states it make whitespaces, tabs or control characters visible. I recommend you to set to show leading and trailing tabs / spaces by this plugin.

TextTools

It provides some useful text operations, such as Toggle Range Comment. I recommend you to replace shortcuts for built-in ones by this plugin.

jDiff Plugin

It shows difference between two files pretty nicely as IntelliJ.

Others

Some worth considering plugins are:

FTP
SQL
XML
MarkerSets

BeanShell recipies

2016-01-24T07:00:35+00:00

BeanShell is a handy lightweight scripting language for Java. In this entry, I’ll introduce you some useful snippets powered by BeanShell, and some recipies about it.

Setup and hello world

Grab a copy of bsh-2.0b4.jar from http://www.beanshell.org and put following shell script named bsh into your PATH:

#!/bin/sh
BEANSHELL_JAR=$HOME/Downloads/bsh-2.0b4.jar # replace path to suit your environment
java -cp $BEANSHELL_JAR bsh.Interpreter $@

Then fire up bsh from your console then just put print("hello, world!"); to confirm it works.

$ bsh
BeanShell 2.0b4 - by Pat Niemeyer (pat@pat.net)
bsh % print("hello, world!");
hello, world!

Hit Ctrl+D to exit interpreter.

You can launch your BeanShell script in a file as follows:

$ echo 'print("hello, world!");' > hello.bsh
$ bsh hello.bsh
hello, world!

Stdin

Text filtering script can be written as follows:

Scanner scanner = new Scanner(System.in);
while (scanner.hasNextLine()) {
  String line = scanner.nextLine();
  System.out.println(line.toUpperCase());
}

Save this script as toUpperCase.bsh . The script can be executed as follows:

$ echo foo | bsh toUpperCase.bsh
FOO

Command line arguments

Command line arguments can be used as follows:

sb = new StringBuilder();
for (arg : bsh.args) {
    sb.append(arg);
}
print(sb);

Save this script as args.bsh. The script can be executed as follows:

$ bsh args.bsh foo bar
foobar

Use of external jar

Any external jar can be added via addClassPath clause dynamically. For example, a SQL beautifier script powered by a Hibernate class can be written as follows:

addClassPath("/path/to/hibernate-core-4.3.7.Final.jar"); // replace path to suit your environment
import org.hibernate.engine.jdbc.internal.BasicFormatterImpl;

scanner = new Scanner(System.in);
sb = new StringBuilder();
while (scanner.hasNextLine()) {
  sb.append(scanner.nextLine()).append('\n');
}

beautifized = new BasicFormatterImpl().format(sb.toString());
print(beautifized);

Save this script as sql-beautifier.bsh then execute following command:

$ SQL="SELECT t0.content AS a2, t0.contenttype AS a3, t0.email AS a4 FROM roller_comment t0, weblogentry t1 WHERE ((t1.websiteid = 'f0588427-f2ca-4843-ac87-bbb31aa6013c') AND (t1.id = t0.entryid)) ORDER BY t0.posttime DESC LIMIT 31 OFFSET 0;"
$ echo $SQL | bsh sql-beautifier.bsh

This yields nicely formatted SQL:

SELECT
    t0.content AS a2,
    t0.contenttype AS a3,
    t0.email AS a4
FROM
    roller_comment t0,
    weblogentry t1
WHERE
    (
        (
            t1.websiteid = 'f0588427-f2ca-4843-ac87-bbb31aa6013c'
        )
        AND (
            t1.id = t0.entryid
        )
    )
ORDER BY
    t0.posttime DESC LIMIT 31 OFFSET 0;

Maven plugin

If you have Maven installed, you can execute any BeanShell script without obtaining bsh-2.0b4.jar by hand. Maven and the beanshell-maven-plugin takes care of it instead of you:

$ mvn com.github.genthaler:beanshell-maven-plugin:1.0:run -Dbsh.file="hello.bsh"
...
[INFO] --- beanshell-maven-plugin:1.0:run (default-cli) @ standalone-pom ---
[INFO] Executing Script
[INFO] file class java.lang.String
[INFO] script class java.lang.Object
[INFO] interpreting file hello.bsh
hello, world!

Note that you don’t need to create pom.xml to execute a simple BeanShell script.

For managing complex dependencies, you can leave that duty to Maven with pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>sql-beautifier</groupId>
    <artifactId>sql-beautifier</artifactId>
    <version>1.0-SNAPSHOT</version>
    <build>
        <plugins>
            <plugin>
                <groupId>com.github.genthaler</groupId>
                <artifactId>beanshell-maven-plugin</artifactId>
                <version>1.0</version>
                <configuration>
                    <script><![CDATA[
                    import java.nio.charset.Charset;
                    import org.apache.commons.io.FileUtils;
                    import org.hibernate.engine.jdbc.internal.BasicFormatterImpl;

                    file = new File(System.getProperty("sql"));
                    sql = FileUtils.readFileToString(file, Charset.defaultCharset());
                    result = new BasicFormatterImpl().format(sql);

                    print(result);
         ]]></script>
                </configuration>
                <dependencies>
                    <dependency>
                        <groupId>org.hibernate</groupId>
                        <artifactId>hibernate-core</artifactId>
                        <version>4.3.7.Final</version>
                    </dependency>
                    <dependency>
                        <groupId>commons-io</groupId>
                        <artifactId>commons-io</artifactId>
                        <version>2.4</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>
</project>

Save the SQL you want to beautify as original.sql and executing following command yields similar result:

$ mvn bsh:run -Dsql=original.sql

jEdit integration

jEdit has pretty nice integration with BeanShell. You can integrate that SQL beautifier as a jEdit macro. Put following snippet as ~/Library/jEdit/macros/FormatSQL.bsh (for OS X) or create it with Macros → New Macro from jEdit menu bar:

addClassPath("/path/to/hibernate-core-4.3.7.Final.jar"); // replace path to suit your environment
import org.hibernate.engine.jdbc.internal.BasicFormatterImpl;

sql = textArea.getSelectedText();
beautifized = new BasicFormatterImpl().format(sql);
textArea.setSelectedText(beautifized);

Paste SQL to any jEdit buffer, and select SQL statement and execute the macro with Macros → FormatSQL to trigger formatting.

References

Testing guice-persist nested @Transactional behavior

2015-12-23T04:41:47+00:00

There is no mention of nested use of @Transactional in the Transactions page of official Guice Wiki so I’ve done some testing about how transactions will be treated in nested use.

As it turns out, Guice seems ignore any of nested @Transactional and any of its rollbackOn attributes. It’s all up to the parent invoker.

For testing, I created two classes as following UML diagram:

EnclosingService holds a reference to EnclosedService and invokes a method of it from every method of EnclosingService. Each name of methods state that what exception will be thrown from the method. For example, EnclosedService#runtimeException() throws RuntimeException while EnclosedService#noException() won’t throw any of Exception. EnclosedService#ioExceptionWithRollbackOn() throws IOException and annotated as @Transactional(rollbackOn = IOException.class). And names of methods of EnclosingService states same but the part of after _ means that what method of EnclosedService will be invoked.

All of methods in both services save a Entity with em.persist() then invoke em.flush() so that every changes will be flushed immediately. i.e. EnclosingService#noException_noException() saves two entities that one in the method itself and another one in the enclosing invocation of EnclosedService#noException().

Then I created a test case which has 16 test methods that covers all of methods defined in EnclosingService and found that any of @Transactional annotation or rollbackOn attribute in EnclosedService are being ignored. I summarized the result in a table:

I can see all of operations are atomic and any Exception thrown in methods of EnclosedService doesn’t affect the result if the transaction will be committed or rolled back. When Guice detects that it’s a RuntimeException or a checked exception that defined as rollbackOn in the first method which annotated as @Transactional then rollback happen.

All resources that used in this test can be obtained from my GitHub repository. It used in-memory database of Apache Derby so that everyone can execute the testcase without any annoying preparation. Also EclipseLink 2.5.1, Guice 3.0 and log4jdbc are used so that I can see how transaction is going such as commit / rollback.

Java EE ブログエンジン Apache Roller のご紹介

2015-12-02T23:00:00+00:00

このエントリは Java EE Advent Calendar 2015 の3日目（12月3日）の記事です．

昨日は @empressia さんの「GlassFish上で動くWebアプリをGradleとNetBeansで開発するための準備」でした．明日は @kencharos さんの「jBatch RI を Java SE でまともに(JPA,CDI,Transactionalつきで)動かす」です．

自己紹介

業務ではJPA, JAX-RSなどを使っているソフトウェアエンジニアです．趣味ではWildFly, JBatch, JSFなどを使っていろいろ作ったり，Java/Java EE関連OSSの不具合を見つけてパッチやPull Requestを送ったりしています．

今回紹介する Apache Roller のコミッタをやっているのですが，日本ではあまり知名度がないので，少しでも日本の方に興味を持ってもらえればと思い，紹介記事を書くことにしました．

Apache Rollerとは？

Apache Rollerは，Javaサーブレットコンテナ向けのオープンソースブログエンジンで，WordPressやMovable Typeとほぼ同じジャンルのソフトウェアです．元Sun Microsystemsのエンジニア David M. Johnsonさんによって2002年に作られ，blogs.sun.com（現 blogs.oracle.com ）などで利用されてきました．

現在の最新バーションは，2015年3月にリリースされた5.1.2です．

誰が使っているの？

Rollerはさまざまな組織や個人によって使われています．組織での使用例としては，Javaエンジニアにはおなじみの Oracle blogs や， IBM developerworks blog， Apache Foundationのブログなどがあります．他にも世界中のさまざまな企業，官公庁などで使われています．

個人では，Java EEを追っかけている人にはおなじみの Adam Bienさん，Javaの父 James Goslingさんなどが，Rollerを使ってブログを書いています．コミッタもそれぞれ自分でRollerのブログを持っています．

その他多数のユーザについてはオフィシャルのWikiページにまとめられています．

使われている技術

DIコンテナとしてはGuiceが使われていて，パーシステンス層ではEclipseLink，ビュー・コントローラ層はStruts2 + JSP + Servlet，ブログのデザインなどはVelocityテンプレートで書けるようになっています．認証まわりではSpring Security，全文検索エンジンとしてはLuceneが使われています．

ServletとJSPのコードが多く，今となってはちょっと時代遅れな感は否めませんが，ソースは全て公開されているので，Javaで書かれていてリアルに運用されているWebシステムのサンプルとして一つの参考になるかもしれません．

特徴

マルチユーザー・マルチブログに対応

1つのインストレーションで複数のブログ，ユーザを作ることができます．

各ユーザーには3種類の権限のうちいずれかを割り当てることができ，具体的には，エントリの下書きだけができるユーザ(Limited)，エントリの投稿などはできるがブログやシステムの設定変更はできないユーザ(Author)，全ての操作ができるユーザ(Admin)のうちいずれかをユーザごとに割り当てることができます．組織用ブログなどで，日常的にレビュープロセスを取り入れているようなケースで便利に使うことができます． LDAP認証に対応しているのも組織で使いやすいポイントです．

個人で使う場合でも，複数のブログをかんたんに作れるので，たとえば技術系エントリはこのブログ，趣味の旅行の話題はまた別のブログ，といった感じで使い分けることができます．もちろん，それぞれのブログのデザインやテンプレートはべつべつに管理できます．

さまざまなDB・アプリケーションサーバに対応

DBスキーマ生成用スクリプトとして，MySQL，PostgreSQL，Oracle，DB2，SQL Server，Derby，HSQLDB用のものが用意されています．ほとんどのユーザおよび開発メンバが使っているのはMySQLなので，特に理由がなければMySQL 5.6.x以降をおすすめします（次のリリースからは5.6.xより前のサポートがなくなる予定なので）．個人的にはPostgreSQLを使っていて，このブログもPostgreSQLで動いています．

いわゆるサーブレットコンテナ向けに作られているので，TomcatやJettyをはじめ，JBoss(WildFly)，GlassFishなどのJava EEコンテナにもデプロイできます．ほとんどのユーザはTomcatを使っているようですが，個人的にはWildFly（このブログ）やJBoss EWSにデプロイして使っています．

Javaでプラグインが書ける

最近のブログサービスでは，冗長なHTMLだけではなくMarkdownや「はてな記法」のような簡潔なシンタックスを使ってエントリを書けるものが多くでてきています．Rollerがデフォルトでサポートしているのは標準的なHTMLだけですが，プラグインを使うことで，そういったシンタックスを使ってブログエントリを書けるようになります．

個人的には， Asciidoctorjプラグインというものを作り AsciiDoc シンタックスを使って日常的にエントリを書いています（このエントリもそうです）．他にも Markdown や JSPWiki プラグインなど，いくつかのプラグインが公開されており，別途インストールすることで利用できるようになります．

プラグインを書くのはとても簡単で，適用するプラグインはエントリごとにべつべつに設定できます．今後またなんらかの新たな記法が登場してそれを使いたくなったら，HTMLへの変換エンジンとRollerをつなぐ部分だけをプラグインとして書いてやれば，既存のエントリはそのままに新しい記法を使うことができるようになります．ブログエントリの記法の他にも，コメントの記法，コメント認証などもプラグインとして独自に追加することができるようになっています．

インストール

いわゆるオンプレミス環境へのインストール方法については，英語になりますが Wiki およびオフィシャルのインストレーションガイドを参照してください．また OpenShift で運用することもできます（無料枠でも可能）．詳細は，こちらも英語になりますが，このブログエントリを参照してください．

メンテナンスなど

まずバックアップですが，個人的にはPostgreSQLデータベースとRollerのデータファイル用ディレクトリをssh経由でバックアップするためのAntスクリプトを書いて，定期的にcronで実行しています．スクリプトは GitHubで公開しています．

他のブログエンジンからのデータ移行については，残念ながら確立されたやり方はまだ存在していませんが，WordPressのブログエクスポート形式であるWXR形式のファイルを取り込むためのツールを作っています．詳細はこちらをご確認ください．

また，SVNからのソース取得，ビルド，プラグインのインストール，自家製パッチ適用などを自動化するためのAntスクリプトを GitHub に置いてあります．

その他，運用ノウハウなどはこのブログエントリによくまとまっています．

最後に

日本語情報は少なめですが，最新リリースでは，実用上は日本語依存の問題というのは特にありません．

ただ，日本語ブログで使うと表示上の違和感があるところがいくつかあったり，メッセージの誤訳や不足，モバイル用テーマ・一部の利用者が少ないDBでのみ生じる細かいバグが残っています（詳細は JIRA を参照）．SVN上の最新版では，そのうちのいくつかは修正ずみですので，可能な方はぜひ開発中の最新版のソースをSVNからチェックアウトしてビルドしてみてください．mvn clean package でWARファイルが生成されます．前述のAntスクリプトも併用されると便利かもしれません．

バグレポート・パッチはもちろん大歓迎です．なにか日本語まわりで質問などあれば，このエントリにコメントをつけていただいても結構です．Javaエンジニアで，自分のブログを持ちたいとお考えの方，ぜひRollerを試してみてください！

リファレンスなど

オフィシャルサイト
オフィシャルブログ
メーリングリスト
Twitterアカウント
Apache Roller 4.0 – Beginner’s Guide: Rollerの入門書
Whats New In Roller5: 原作者Davidさんのスライド
Introduction to Apache Roller: コミッタMattさんのスライド
Tightblog: 元コミッタGlenさんによるRollerのFork．GuiceからSpringへの移行，テーブル構造の見直し，大規模なリファクタリングなどが行われています

宣伝：求人のご案内

現在，私の勤務先株式会社 L is B ではエンジニア・デザイナーを募集しています．主に direct というインスタントメッセージングサービスの開発と運用を行っている会社です．

会社所在地は東京（+徳島県に開発拠点があります）ですが，一部リモートワークも認められています．少しでも興味を持たれた方，ぜひ応募してみてください！

Adding indexes to tables of Apache James 3

2015-11-15T08:06:38+00:00

I’m using Apache James (3.0-beta5-SNAPSHOT) as the mail server for my domain. It’s running with JPA backend on PostgreSQL.

Today I’ve found that some slow queries are logged as the number of emails has been grown through the days. The slowest two are following:

SELECT t0.property_id, t0.property_line_number, t0.property_local_name, t0.property_name_space, t0.property_value FROM public.JAMES_MAIL_PROPERTY t0 WHERE t0.mailbox_id = $1 AND t0.mail_uid = $2 ORDER BY t0.property_line_number ASC
SELECT t0.userflag_id, t0.userflag_name FROM public.JAMES_MAIL_USERFLAG t0 WHERE t0.mailbox_id = $1 AND t0.mail_uid = $2 ORDER BY t0.userflag_id ASC

These queries are used sequential scan. It’s terrible for thousands of rows so I’ve created a couple of indexes to avoid sequential scan as follows:

create index on james_mail_property (mailbox_id, mail_uid);
create index on james_mail_userflag (mailbox_id, mail_uid);

They seems work expectedly as I’ve confirmed that now these queries are using index scan instead.

Notes about using UPSERT on RDBMS

2015-11-07T03:15:07+00:00

Recently I’ve investigated some ways to implement UPSERT which gets the following job done without problems of race conditions:

INSERT a row if there’s no duplicate one with same ID
UPDATE a row otherwise

Also another slightly differ requirement:

INSERT a row if there’s no duplicate one with same ID
Do nothing otherwise
Application needs to know whether the query has inserted a row because one is not exist

The above two requirements are needed to implement an application that works with an Amazon SQS which is configured as the destination of Amazon SES notification.

Table to use for experimentation

CREATE TABLE mytable (id INTEGER PRIMARY KEY, cnt INTEGER);

Solution for MySQL (5.6.x)

There’s an easy solution that uses a MySQL specific clause INSERT INTO … ON DUPLICATE KEY UPDATE …. For detail check http://dev.mysql.com/doc/refman/5.6/en/insert-on-duplicate.html

With the mytable, I’ve done some experimentation (on MySQL 5.6.27) as follows:

Launch two instances of mysql
Execute begin; on both so let a transaction begin for each instances
Execute INSERT INTO mytable (id, cnt) VALUES (1, 1) ON DUPLICATE KEY UPDATE cnt=cnt+1; on both. Note that the following execution will be blocked due to the preceding transaction is about to insert a row which has same ID
Execute commit; on the instance which executes the statement first
Execute commit; on the another instance which has been blocked

Then execute select * from mytable; you will see the desired result:

+----+------+
| id | cnt  |
+----+------+
|  1 |    2 |
+----+------+
1 row in set (0.00 sec)

If you don’t want to update any values if duplicated one already exists, Use following SQL instead:

INSERT INTO mytable (id, cnt) VALUES (1, 1) ON DUPLICATE KEY UPDATE id=id;

Also note that if you’re using JDBC to communicate with MySQL, You need to add useAffectedRows=true parameter to the JDBC URL so that executeUpdate() method will return the number of affected rows instead of found rows. For detail check https://dev.mysql.com/doc/connector-j/en/connector-j-reference-configuration-properties.html

Another solution

I found an interesting attempt that seems to work with generic SQL:

http://www.xaprb.com/blog/2005/09/25/insert-if-not-exists-queries-in-mysql/ http://stackoverflow.com/a/17067131/3591946

And I’ve confirmed it works as I expected. I’ve done the following experiment on MySQL 5.6.x:

Launch two instances of mysql
Execute begin; on both so let a transaction begin for each instances
Execute INSERT INTO mytable (id, cnt) SELECT 1, 0 FROM (select 0 as i) mutex LEFT JOIN mytable ON id = 1 WHERE i = 0 AND id IS NULL;. Note that the following execution will be blocked as well
Execute UPDATE mytable SET cnt = cnt + 1 WHERE id = 1; on the instance which executes the statement first, if incrementation is needed
Execute commit; on the instance which executes the statement first
Execute UPDATE mytable SET cnt = cnt + 1 WHERE id = 1; on the instance which executes the statement second as well
Execute commit; on the another instance

Note that I’ve tried the experimentation for PostgreSQL 9.3.4 as well but doesn’t work. It blocks the following query but produces ERROR: duplicate key value violates unique constraint "mytable_pkey" after issuing commit of the preceding transaction.

I have no idea why it doesn’t work for PostgreSQL (To be honest, I don’t exactly know why it does work for MySQL). If you know why, Let me know via posting a comment to this entry that would be greatly appreciated.

UPSERT functionality will be in the PostgreSQL 9.5 release (citation from https://wiki.postgresql.org/wiki/SQL_MERGE).

Excluding particular JUnit test cases that marked as slow in the time of execution

2015-11-01T06:43:16+00:00

Sometimes we need to create some slow test cases that involve some external resources such as databases, or external API servers. They are necessary to ensure that your application can integrate with such external resources while vast majority of test cases should stick with fast-running plain unit testing.

In such case, We usually wants to exclude such slow test cases from the ones that are frequently executed in local development environment so that we can get timely response from the tests. In this posting, I introduce you a solution that avoids maintenance of any hand-made listing of test cases.

Creating a suite that scans and runs all of test cases exist in classpath

First, assume we have a simple production class named Hello.

public class Hello {
    public String greetings(String name) {
        return name != null ? "hello, " + name : "hi, what's your name?";
    }
}

We also have a couple of test cases against the preceding class:

public class HelloTest1 {
    @Test
    public void test() {
        System.out.println("Running " + getClass().getSimpleName());
        Assert.assertEquals("hello, kyle", new Hello().greetings("kyle"));
    }
}

public class HelloTest2 {
    @Test
    public void test() {
        System.out.println("Running " + getClass().getSimpleName());
        Assert.assertEquals("hi, what's your name?", new Hello().greetings(null));
    }
}

Next, We’d like to introduce a test suite that automatically includes the preceding two test cases. Put a following dependency to your pom.xml:

<dependency>
    <groupId>io.takari.junit</groupId>
    <artifactId>takari-cpsuite</artifactId>
    <version>1.2.7</version>
    <scope>test</scope>
</dependency>

And create a test suite that named AllTests as follows. You can run this suite from your IDE or executing mvn -Dtest=AllTests test.

@RunWith(ClasspathSuite.class)
public class AllTests {
}

Involving a slow test case and exclude it

First, Create a marker interface which indicates that this test is slow:

public interface SlowTest {
}

Next create a slow test case which annotated with @Category(SlowTest.class) that we would like to avoid execute it frequently:

@Category(SlowTest.class)
public class HelloSlowTest {
    @Test
    public void test() throws Exception {
        System.out.println("Running " + getClass().getSimpleName());
        Thread.sleep(3000);
    }
}

Finally create a test suite that automatically excludes the test cases annotated as slow but executes rest of the test cases:

@RunWith(Categories.class)
@ExcludeCategory(SlowTest.class)
@SuiteClasses(AllTests.class)
public class AllExceptSlowTests {
}

You can run it on a daily basis instead of selecting root of your entire project and execute tests from your IDE or Maven without any hand maintenance of the listings of tests. For example, mvn -Dtest=AllExceptSlowTests test produces following output in very short-term execution time:

...
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Picked up _JAVA_OPTIONS: -Dfile.encoding=UTF-8
Running category.suite.AllExceptSlowTests
Running HelloTest1
Running HelloTest2
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.084 sec
...

The complete resources can be obtained from https://github.com/lbtc-xxx/junit-category

SwingIrc: A simple IRC client

2015-11-01T04:42:12+00:00

About 6 years ago I’ve made a simple IRC client that uses Swing. I’ve digged source code of the app from my old PC by chance so I’ve Mavenized it and put it to GitHub. To be hornest, it’s a toy app but may someone like a student can refer it as an example of Java based GUI application.

https://github.com/lbtc-xxx/swingirc

How To Launch

Refer README.md in the repository.

How To Use

Put name, hostname and port to the dialog and hit connect
Click File then select Join
Enter name of a channel to the dialog and hit OK
Chat with other users

Future plans

Rewrite entire the app with JavaFX
Implement more features
Involve automated GUI testing

Using JPQL IN clause with composite key

2015-10-25T11:16:44+00:00

Assume we have an entity named Employee:

@Entity
public class Employee implements Serializable {
    @Id
    @GeneratedValue
    private Long id;
    @Embedded
    private EmployeeName employeeName;

    // accessor omitted

And EmployeeName:

@Embeddable
public class EmployeeName implements Serializable {
    private String firstName;
    private String lastName;

    // accessor omitted

Using preceding entity, We want to execute following JPQL:

SELECT e FROM Employee e WHERE e.employeeName IN :employeeNames

Will it work? It works for Hibernate 4.3.11.Final but unfortunately not for EclipseLink 2.6.1.

Hibernate generates following SQL for the JPQL and the parameter of a List contains two elements:

Hibernate: select employee0_.id as id1_0_, employee0_.firstName as firstNam2_0_, employee0_.lastName as lastName3_0_ from Employee employee0_ where employee0_.firstName=? and employee0_.lastName=? or employee0_.firstName=? and employee0_.lastName=?
[Employee{id=1, employeeName=EmployeeName{firstName='Scott', lastName='Vogel'}}, Employee{id=2, employeeName=EmployeeName{firstName='Nick', lastName='Jett'}}]

EclipseLink failed to generate correct SQL for the JPQL. In such case, you need to create a JPQL by hand or Criteria API that uses each column separately (lastName and firstName). EclipseLink produces following Exception:

Exception in thread "main" javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.6.1.v20150916-55dc7c3): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLDataException: An attempt was made to get a data value of type 'BIGINT' from a data value of type 'entity.EmployeeName'.
Error Code: 20000
Call: SELECT ID, FIRSTNAME, LASTNAME FROM EMPLOYEE WHERE (ID IN (?,?))
	bind => [EmployeeName{firstName='Scott', lastName='Vogel'}, EmployeeName{firstName='Nick', lastName='Jett'}]
Query: ReadAllQuery(referenceClass=Employee sql="SELECT ID, FIRSTNAME, LASTNAME FROM EMPLOYEE WHERE (ID IN ?)")
	at org.eclipse.persistence.internal.jpa.QueryImpl.getDetailedException(QueryImpl.java:382)
	at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:260)
	at org.eclipse.persistence.internal.jpa.QueryImpl.getResultList(QueryImpl.java:473)
	at main.Main.main(Main.java:45)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Caused by: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.6.1.v20150916-55dc7c3): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLDataException: An attempt was made to get a data value of type 'BIGINT' from a data value of type 'entity.EmployeeName'.
Error Code: 20000
Call: SELECT ID, FIRSTNAME, LASTNAME FROM EMPLOYEE WHERE (ID IN (?,?))
	bind => [EmployeeName{firstName='Scott', lastName='Vogel'}, EmployeeName{firstName='Nick', lastName='Jett'}]
Query: ReadAllQuery(referenceClass=Employee sql="SELECT ID, FIRSTNAME, LASTNAME FROM EMPLOYEE WHERE (ID IN ?)")
	at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:340)
	at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:684)
	at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:560)
	at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:2055)
	at org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:570)
	at org.eclipse.persistence.sessions.server.ClientSession.executeCall(ClientSession.java:258)
	at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:242)
	at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:228)
	at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelectCall(DatasourceCallQueryMechanism.java:299)
	at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.selectAllRows(DatasourceCallQueryMechanism.java:694)
	at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectAllRowsFromTable(ExpressionQueryMechanism.java:2740)
	at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectAllRows(ExpressionQueryMechanism.java:2693)
	at org.eclipse.persistence.queries.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:559)
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1175)
	at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:904)
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1134)
	at org.eclipse.persistence.queries.ReadAllQuery.execute(ReadAllQuery.java:460)
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:1222)
	at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2896)
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1857)
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1839)
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1804)
	at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:258)
	... 7 more

Complete source code that has been used in the test can be obtained from https://github.com/lbtc-xxx/jpa-composite-in

How to bind / lookup DataSource via JNDI without container

2015-10-25T01:58:28+00:00

While I prefer deploying JPA based apps to a Java EE container and test it via Arquillian as integration testing, some occasions won’t allow it and need arises that using a Servlet container or Java SE environment.

To supply information that required to connect the database (e.g. JDBC URL or credentials), it’s preferable to use JNDI rather than using DriverManager or javax.persistence.jdbc.* properties in persistence.xml because using JNDI eliminates the need of managing such information in the application codebase, also it enables to use the container managed connection pool which is more flexible and scalable over another.

In such case, hard-coded JNDI name of a DataSource may be a problem in the time of testing because JNDI lookup doesn’t work without container as is. So we may need some considering of involve pluggable mechanism of acquiring java.sql.Connection instance or creating persistence.xml for unit testing.

These solutions are not much difficult to implement, but it’s preferable if JNDI lookup does work without container as well because it will decrease amount of testing specific code. In this posting, I’ll give you a complete example of looking up a DataSource without container using bare InitialContext and the non-jta-datasource persistence descriptor definition.

Environment

tomcat-catalina artifact of Apache Tomcat 8.0.28: Enables binding a resource to JNDI context in Java SE environment
Apache Commons DBCP 1.4: Supplies BasicDataSource class so make the example in database independent manner
Apache Derby 10.12.1.1
EclipseLink 2.6.1
Oracle JDK8u60

Dependencies

<dependencies>
    <dependency>
        <groupId>org.eclipse.persistence</groupId>
        <artifactId>eclipselink</artifactId>
        <version>2.6.1</version>
    </dependency>
    <dependency>
        <groupId>commons-dbcp</groupId>
        <artifactId>commons-dbcp</artifactId>
        <version>1.4</version>
    </dependency>
    <dependency>
        <groupId>org.apache.derby</groupId>
        <artifactId>derby</artifactId>
        <version>10.12.1.1</version>
        <scope>runtime</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.tomcat</groupId>
        <artifactId>tomcat-catalina</artifactId>
        <version>8.0.28</version>
        <scope>runtime</scope>
    </dependency>
</dependencies>

persistence.xml

Note that the non-jta-data-source is used with JNDI name of DataSource.

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.1" xmlns="http://xmlns.jcp.org/xml/ns/persistence"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd">
    <persistence-unit name="myPU" transaction-type="RESOURCE_LOCAL">
        <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
        <non-jta-data-source>java:comp/env/jdbc/database</non-jta-data-source>
        <class>entity.Employee</class>
        <shared-cache-mode>NONE</shared-cache-mode>
        <properties>
            <property name="javax.persistence.schema-generation.database.action" value="create"/>
            <property name="eclipselink.logging.level" value="FINE"/>
            <property name="eclipselink.logging.parameters" value="true"/>
        </properties>
    </persistence-unit>
</persistence>

Employee.java

This is a simple JPA entity class that will be used in testing.

@Entity
public class Employee implements Serializable {
    @Id
    private Long id;
    private String firstName;
    private String lastName;

    // accessors omitted

Main.java

This binds a DataSource of Embedded in-memory Apache Derby database to java:comp/env/jdbc/database, then lookup it via InitialContext and EntityManagerFactory.

public class Main {
    private static final String JNDI = "java:comp/env/jdbc/database";

    public static void main(String[] args) throws Exception {
        bind();
        lookup();
        final EntityManagerFactory emf = Persistence.createEntityManagerFactory("myPU");
        populate(emf);
        query(emf);
    }

    private static void bind() throws NamingException {
        System.setProperty(Context.INITIAL_CONTEXT_FACTORY, "org.apache.naming.java.javaURLContextFactory");
        System.setProperty(Context.URL_PKG_PREFIXES, "org.apache.naming");

        final BasicDataSource ds = new BasicDataSource();
        ds.setUrl("jdbc:derby:memory:myDB;create=true");

        final Context context = new InitialContext();
        try {
            context.createSubcontext("java:");
            context.createSubcontext("java:comp");
            context.createSubcontext("java:comp/env");
            context.createSubcontext("java:comp/env/jdbc");
            context.bind(JNDI, ds);
        } finally {
            context.close();
        }
    }

    private static void lookup() throws NamingException, SQLException {
        final Context context = new InitialContext();
        try {
            final DataSource ds = (DataSource) context.lookup(JNDI);
            try (final Connection cn = ds.getConnection();
                 final Statement st = cn.createStatement();
                 final ResultSet rs = st.executeQuery("SELECT CURRENT_TIMESTAMP FROM SYSIBM.SYSDUMMY1")) {
                while (rs.next()) {
                    System.out.println(rs.getTimestamp(1));
                }
            }
        } finally {
            context.close();
        }
    }

    private static void populate(final EntityManagerFactory emf) {
        final EntityManager em = emf.createEntityManager();
        try {
            final EntityTransaction tx = em.getTransaction();
            tx.begin();
            final Employee emp = new Employee();
            emp.setId(1l);
            emp.setFirstName("Jane");
            emp.setLastName("Doe");
            em.persist(emp);
            tx.commit();
        } finally {
            em.close();
        }
    }

    private static void query(final EntityManagerFactory emf) {
        final EntityManager em = emf.createEntityManager();
        try {
            System.out.println(em.find(Employee.class, 1l));
        } finally {
            em.close();
        }
    }
}

Log

You can see the lookup() method dumped CURRENT_TIMESTAMP and EclipseLink successfully acquired a DataSource as follows.

2015-10-25 10:33:24.235
[EL Fine]: server: 2015-10-25 10:33:24.478--Thread(Thread[main,5,main])--Configured server platform: org.eclipse.persistence.platform.server.NoServerPlatform
[EL Config]: metadata: 2015-10-25 10:33:24.633--ServerSession(1323434987)--Thread(Thread[main,5,main])--The access type for the persistent class [class entity.Employee] is set to [FIELD].
[EL Config]: metadata: 2015-10-25 10:33:24.654--ServerSession(1323434987)--Thread(Thread[main,5,main])--The alias name for the entity class [class entity.Employee] is being defaulted to: Employee.
[EL Config]: metadata: 2015-10-25 10:33:24.656--ServerSession(1323434987)--Thread(Thread[main,5,main])--The table name for entity [class entity.Employee] is being defaulted to: EMPLOYEE.
[EL Config]: metadata: 2015-10-25 10:33:24.666--ServerSession(1323434987)--Thread(Thread[main,5,main])--The column name for element [firstName] is being defaulted to: FIRSTNAME.
[EL Config]: metadata: 2015-10-25 10:33:24.668--ServerSession(1323434987)--Thread(Thread[main,5,main])--The column name for element [lastName] is being defaulted to: LASTNAME.
[EL Config]: metadata: 2015-10-25 10:33:24.668--ServerSession(1323434987)--Thread(Thread[main,5,main])--The column name for element [id] is being defaulted to: ID.
[EL Info]: 2015-10-25 10:33:24.7--ServerSession(1323434987)--Thread(Thread[main,5,main])--EclipseLink, version: Eclipse Persistence Services - 2.6.1.v20150916-55dc7c3
[EL Fine]: connection: 2015-10-25 10:33:24.706--Thread(Thread[main,5,main])--Detected database platform: org.eclipse.persistence.platform.database.JavaDBPlatform
[EL Config]: connection: 2015-10-25 10:33:24.714--ServerSession(1323434987)--Connection(1872973138)--Thread(Thread[main,5,main])--connecting(DatabaseLogin(
	platform=>JavaDBPlatform
	user name=> ""
	connector=>JNDIConnector datasource name=>java:comp/env/jdbc/database
))
[EL Config]: connection: 2015-10-25 10:33:24.715--ServerSession(1323434987)--Connection(1465346452)--Thread(Thread[main,5,main])--Connected: jdbc:derby:memory:myDB
	User: APP
	Database: Apache Derby  Version: 10.12.1.1 - (1704137)
	Driver: Apache Derby Embedded JDBC Driver  Version: 10.12.1.1 - (1704137)
[EL Config]: connection: 2015-10-25 10:33:24.715--ServerSession(1323434987)--Connection(1634387050)--Thread(Thread[main,5,main])--connecting(DatabaseLogin(
	platform=>JavaDBPlatform
	user name=> ""
	connector=>JNDIConnector datasource name=>java:comp/env/jdbc/database
))
[EL Config]: connection: 2015-10-25 10:33:24.716--ServerSession(1323434987)--Connection(1740223770)--Thread(Thread[main,5,main])--Connected: jdbc:derby:memory:myDB
	User: APP
	Database: Apache Derby  Version: 10.12.1.1 - (1704137)
	Driver: Apache Derby Embedded JDBC Driver  Version: 10.12.1.1 - (1704137)
[EL Info]: connection: 2015-10-25 10:33:24.747--ServerSession(1323434987)--Thread(Thread[main,5,main])--/file:/Users/kyle/src/jndi-se/target/classes/_myPU login successful
[EL Fine]: sql: 2015-10-25 10:33:24.784--ServerSession(1323434987)--Connection(762809053)--Thread(Thread[main,5,main])--CREATE TABLE EMPLOYEE (ID BIGINT NOT NULL, FIRSTNAME VARCHAR(255), LASTNAME VARCHAR(255), PRIMARY KEY (ID))
[EL Fine]: sql: 2015-10-25 10:33:24.85--ClientSession(1027495011)--Connection(1688470144)--Thread(Thread[main,5,main])--INSERT INTO EMPLOYEE (ID, FIRSTNAME, LASTNAME) VALUES (?, ?, ?)
	bind => [1, Jane, Doe]
[EL Fine]: sql: 2015-10-25 10:33:24.877--ServerSession(1323434987)--Connection(640808588)--Thread(Thread[main,5,main])--SELECT ID, FIRSTNAME, LASTNAME FROM EMPLOYEE WHERE (ID = ?)
	bind => [1]
Employee{id=1, firstName='Jane', lastName='Doe'}

The complete source code can be obtained from my GitHub repository.

References

Working example of EclipseLink static weaving

2015-10-23T18:37:42+00:00

These days I’m using EclipseLink at work. It runs on Servlet containers, unfortunately not Java EE containers at there so I have experienced some difference between them. A significant one is class weaving. The dynamic weaving is enabled by default in Java EE containers but not for Java SE environment. Weaving is a prerequisite of using some important functions such as Lazy Loading but it doesn’t work for default Java SE environment. In Java SE environment, EclipseLink requires a special prerequisite that set an agent in the time of launching JVM, or use static weaving to enable Lazy Loading.

Static weaving offers some performance benefit over Dynamic weaving because it doesn’t require runtime weaving step. I think it’s preferable so I tried it over another.

Environment

EclipseLink 2.6.1
Apache Derby 10.12.1.1
Oracle JDK8u60

Projects

eclipselink-entity

This project contains three simple entity classes. Dept has many Employee, and Employee has one Phone.

Dept

@Entity
public class Dept implements Serializable {
    @Id
    private Long id;
    private String deptName;
    @OneToMany(mappedBy = "dept")
    private List<Employee> employees;

    // accessors omitted

Employee

@Entity
public class Employee implements Serializable {
    @Id
    private Long id;
    @ManyToOne(fetch = FetchType.EAGER) // default
    @JoinColumn(nullable = false)
    private Dept dept;
    private String firstName;
    private String lastName;
    @OneToOne(mappedBy = "employee", fetch = FetchType.LAZY) // overridden by LAZY
    private Phone phone;

    // accessors omitted

Note that the relation Employee.dept is set to EAGER, and Employee.phone is set to LAZY as FetchType.

Phone

@Entity
public class Phone implements Serializable {
    @Id
    @OneToOne
    @JoinColumn(nullable = false)
    private Employee employee;
    private String phoneNumber;

    // accessors omitted

persistence.xml

The persistence descriptor requires a property called eclipselink.weaving with the value static to enable static weaving.

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.1" xmlns="http://xmlns.jcp.org/xml/ns/persistence"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd">
    <persistence-unit name="myPU" transaction-type="RESOURCE_LOCAL">
        <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
        <exclude-unlisted-classes>false</exclude-unlisted-classes>
        <shared-cache-mode>NONE</shared-cache-mode>
        <properties>
            <property name="javax.persistence.jdbc.driver" value="org.apache.derby.jdbc.EmbeddedDriver"/>
            <property name="javax.persistence.jdbc.url" value="jdbc:derby:memory:myDB;create=true"/>
            <property name="javax.persistence.jdbc.user" value="app"/>
            <property name="javax.persistence.jdbc.password" value="app"/>
            <property name="javax.persistence.schema-generation.database.action" value="drop-and-create"/>
            <property name="eclipselink.weaving" value="static"/>
            <property name="eclipselink.logging.level" value="FINE"/>
            <property name="eclipselink.logging.parameters" value="true"/>
        </properties>
    </persistence-unit>
</persistence>

pom.xml

The static weaving will be done by a convenient Maven plugin. Just put following plugin definition in your pom.xml and execute mvn clean install.

<build>
    <plugins>
        <plugin>
            <groupId>de.empulse.eclipselink</groupId>
            <artifactId>staticweave-maven-plugin</artifactId>
            <version>1.0.0</version>
            <executions>
                <execution>
                    <phase>process-classes</phase>
                    <goals>
                        <goal>weave</goal>
                    </goals>
                    <configuration>
                        <persistenceXMLLocation>META-INF/persistence.xml</persistenceXMLLocation>
                        <logLevel>FINE</logLevel>
                    </configuration>
                </execution>
            </executions>
            <dependencies>
                <dependency>
                    <groupId>org.eclipse.persistence</groupId>
                    <artifactId>org.eclipse.persistence.jpa</artifactId>
                    <version>${eclipselink.version}</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

eclipselink-example

This project is a client of the preceding eclipselink-entity project. It has a Main class which simply populates some records then fetches them.

public class Main {
    public static void main(String[] args) {
        EntityManagerFactory emf = null;
        try {
            emf = Persistence.createEntityManagerFactory("myPU");
            EntityManager em = null;

            // Populating data
            try {
                em = emf.createEntityManager();
                final EntityTransaction tx = em.getTransaction();
                tx.begin();

                Dept dept = new Dept();
                dept.setId(1l);
                dept.setDeptName("Engineering");
                dept.setEmployees(new ArrayList<>());
                em.persist(dept);

                Employee emp = new Employee();
                emp.setId(1l);
                emp.setFirstName("Jane");
                emp.setLastName("Doe");
                dept.getEmployees().add(emp);
                emp.setDept(dept);
                em.persist(emp);

                Phone phone = new Phone();
                phone.setPhoneNumber("000-1111-2222");
                phone.setEmployee(emp);
                emp.setPhone(phone);
                em.persist(phone);

                tx.commit();
            } finally { if (em != null) { em.close(); } }

            System.out.println("<<< Populating done >>>");

            try {
                em = emf.createEntityManager();
                final Employee emp = em.find(Employee.class, 1l);

                System.out.println(emp.getFirstName() + " " + emp.getLastName());

                // EAGER
                System.out.println(emp.getDept().getDeptName());
                // LAZY
                System.out.println(emp.getPhone().getPhoneNumber());
            } finally { if (em != null) { em.close(); } }
        } finally { if (emf != null) { emf.close(); } }
    }
}

Here you can see the Phone entity has lazily fetched while Dept entity was eagerly fetched:

<<< Populating done >>>
[EL Fine]: sql: 2015-10-24 03:18:23.389--ServerSession(1216590855)--Connection(1488298739)--Thread(Thread[main,5,main])--SELECT ID, FIRSTNAME, LASTNAME, DEPT_ID FROM EMPLOYEE WHERE (ID = ?)
	bind => [1]
[EL Fine]: sql: 2015-10-24 03:18:23.408--ServerSession(1216590855)--Connection(1488298739)--Thread(Thread[main,5,main])--SELECT ID, DEPTNAME FROM DEPT WHERE (ID = ?)
	bind => [1]
Jane Doe
Engineering
[EL Fine]: sql: 2015-10-24 03:18:23.413--ServerSession(1216590855)--Connection(1488298739)--Thread(Thread[main,5,main])--SELECT PHONENUMBER, EMPLOYEE_ID FROM PHONE WHERE (EMPLOYEE_ID = ?)
	bind => [1]
000-1111-2222

This example uses in-memory Apache Derby so you don’t need to set up any databases to execute this example. complete projects can be obtained from following GitHub repositories:

Also here’s build.gradle example: https://github.com/lbtc-xxx/eclipselink-entity/blob/master/build.gradle

References

Using DBUnit from Ant

2015-09-23T03:02:44+00:00

It’s a common usecase that using DBUnit from JUnit testcases but sometimes I need to use DBUnit in a standalone application manner to achive tasks such as exporting the data from a database to a XML file for creating test data from an existing table. In such case, I feel using DBUnit Ant task is better rather than launching it from Maven plugin or write a standalone Java application uses DBUnit API. in this entry, I introduce you a complete working example of an Ant script that uses DBUnit Ant task.

Environment

dbunit-2.5.1.jar (Not available on the download page on SourceForge. Obtain it from Maven Repository instead)
- With dependencies: slf4j-api-1.7.12.jar, slf4j-jdk14-1.7.12.jar
postgresql-9.4-1203.jdbc42.jar
Apache Ant™ version 1.9.6
PostgreSQL 9.3.5
Oracle JDK8u40

Complete JAR file list in my ~/.ant/lib:

$ ls -l ~/.ant/lib
total 1236
-rw-rw-r--. 1 kyle kyle 560755 Sep 23 10:56 dbunit-2.5.1.jar
-rw-rw-r--. 1 kyle kyle 660126 Sep 23 10:54 postgresql-9.4-1203.jdbc42.jar
-rw-rw-r--. 1 kyle kyle  32127 Sep 23 11:15 slf4j-api-1.7.12.jar
-rw-rw-r--. 1 kyle kyle   7892 Sep 23 11:16 slf4j-jdk14-1.7.12.jar
$

Exporting example

build.xml

This script exports the result of a SQL to a flat XML file. The format can be specified in dbunit.format property that has set to flat as default. You can add additional query or table elements inside the dbunit element to include multiple dataset in single XML file. JDBC connection information such as credentials or URL is expected to be stored as a separate file named dbunit.properties that is placed under the same directory to this file.

<project name="dbunit-export" basedir=".">
    <taskdef name="dbunit" classname="org.dbunit.ant.DbUnitTask"/>

    <loadproperties srcFile="dbunit.properties"/>

    <fail unless="dbunit.driver"/>
    <fail unless="dbunit.url"/>
    <fail unless="dbunit.userid"/>
    <fail unless="dbunit.password"/>
    <fail unless="dbunit.datatypeFactory"/>

    <property name="dbunit.format" value="flat"/> <!-- Possible values are "flat", "xml", "csv", "dtd", "xls". Defaults to "flat" -->

    <target name="export">
        <fail unless="dbunit.sql"/>
        <fail unless="dbunit.dest"/>
        <fail unless="dbunit.format"/>
        <fail unless="dbunit.query"/>

        <dbunit driver="${dbunit.driver}" url="${dbunit.url}" userid="${dbunit.userid}" password="${dbunit.password}">
            <dbconfig>
                <property name="datatypeFactory" value="${dbunit.datatypeFactory}" />
            </dbconfig>
            <export dest="${dbunit.dest}">
                <query name="${dbunit.query}" sql="${dbunit.sql}"/>
            </export>
        </dbunit>
    </target>
</project>

dbunit.properties

Set each properties to suit your environment. See this page to check what datatypeFactory are available.

dbunit.driver=org.postgresql.Driver
dbunit.url=jdbc:postgresql://localhost:5432/mydb
dbunit.userid=someone
dbunit.password=somepass
dbunit.datatypeFactory=org.dbunit.ext.postgresql.PostgresqlDataTypeFactory

How to run

The following command exports a XML file from a SQL.

$ ant export "-Ddbunit.sql=select * from job_instance where jobinstanceid >= 399341 order by jobinstanceid" -Ddbunit.query=job_instance -Ddbunit.dest=job_instance.xml
Buildfile: /home/kyle/dbunit-ant-example/build.xml

export:
   [dbunit] Executing export:
   [dbunit]       in format: flat to datafile: /home/kyle/dbunit-ant-example/job_instance.xml
Successfully wrote file '/home/kyle/dbunit-ant-example/job_instance.xml'

BUILD SUCCESSFUL
Total time: 0 seconds

This produces a flat XML dataset named job_instance.xml something like:

<?xml version='1.0' encoding='UTF-8'?>
<dataset>
  <job_instance jobinstanceid="399341" jobname="somejob1" applicationname="someapp"/>
  <job_instance jobinstanceid="399342" jobname="somejob2" applicationname="someapp"/>
  <job_instance jobinstanceid="399343" jobname="somejob3" applicationname="someapp"/>
...
  <job_instance jobinstanceid="400004" jobname="somejob4" applicationname="someapp"/>
</dataset>

Exporting / importing an entire database

<project name="mydbunit" basedir=".">
    <taskdef name="dbunit" classname="org.dbunit.ant.DbUnitTask"/>

    <loadproperties srcFile="dbunit.properties"/>

    <fail unless="dbunit.driver"/>
    <fail unless="dbunit.url"/>
    <fail unless="dbunit.userid"/>
    <fail unless="dbunit.password"/>
    <fail unless="dbunit.datatypeFactory"/>

    <property name="dbunit.format" value="flat"/> <!-- Possible values are "flat", "xml", "csv", "dtd", "xls". Defaults to "flat" -->

    <target name="export">
        <fail unless="dbunit.dest"/>
        <fail unless="dbunit.format"/>

        <dbunit driver="${dbunit.driver}" url="${dbunit.url}" userid="${dbunit.userid}" password="${dbunit.password}">
            <dbconfig>
                <property name="datatypeFactory" value="${dbunit.datatypeFactory}" />
                <!-- <property name="escapePattern" value="`?`" /> -->
            </dbconfig>
            <export dest="${dbunit.dest}" ordered="true"/>
        </dbunit>
    </target>

    <target name="import">
        <fail unless="dbunit.src"/>
        <fail unless="dbunit.format"/>

        <dbunit driver="${dbunit.driver}" url="${dbunit.url}" userid="${dbunit.userid}" password="${dbunit.password}">
            <dbconfig>
                <property name="datatypeFactory" value="${dbunit.datatypeFactory}" />
                <!-- <property name="escapePattern" value="`?`" /> -->
            </dbconfig>
            <operation type="CLEAN_INSERT" src="${dbunit.src}" ordered="true"/>
        </dbunit>
    </target>

</project>

To export:

$ ant export -Ddbunit.dest=mydb.xml

To import:

$ ant import -Ddbunit.src=mydb.xml

Note that you need commons-collections3 jar file in your classpath.

Other tasks

There are more operations that can be achieved through Ant tasks available such as:

Data comparison
INSERT or UPDATE based on the file

References

http://dbunit.sourceforge.net/anttask.html

Using RabbitMQ with Java

2015-08-05T08:04:54+00:00

RabbitMQ is an open source message broker software made with Erlang. RabbitMQ is sponsored by Pivotal Software, Inc. in this entry, we’ll see how to launch it, publish or receive simple messages with Java client on OS X.

Installation

I’m using OS X 10.9.5. download rabbitmq-server-mac-standalone-3.5.4.tar.gz for Macs then extract it on somewhere.

Start the server

The server can be launched via execute sbin/rabbitmq-server as follows.

$ ./rabbitmq-server

              RabbitMQ 3.5.4. Copyright (C) 2007-2015 Pivotal Software, Inc.
  ##  ##      Licensed under the MPL.  See http://www.rabbitmq.com/
  ##  ##
  ##########  Logs: ./../var/log/rabbitmq/rabbit@kyle-no-MacBook.log
  ######  ##        ./../var/log/rabbitmq/rabbit@kyle-no-MacBook-sasl.log
  ##########
              Starting broker... completed with 0 plugins.

To shutdown, hit Ctrl+C then press a.

Next, we’ll look how to interact with the server from Java client.

Dependencies

RabbitMQ supplies the client library for Java in Maven Central. just put following dependency to your pom.xml.

<dependency>
	<groupId>com.rabbitmq</groupId>
	<artifactId>amqp-client</artifactId>
	<version>3.5.4</version>
</dependency>

Java examples in official tutorial

There is a complete Hello World example of a publisher in https://www.rabbitmq.com/tutorials/tutorial-one-java.html . just copy and paste it to your workspace and execute it after launch the server.

Complete a consumer example is here: https://github.com/rabbitmq/rabbitmq-tutorials/blob/master/java/Recv.java . just execute it after execution of publisher is done.

The whole project can be obtained from my GitHub repository, nothing interesting here though.

Remarks

It’s very easy to use, I haven’t experienced any pitfall at all.

I wanted to interact with the server using JMS, but the JMS client needs a commercial license to download and unfortunately I don’t have one.

http://rabbitmq.1065348.n5.nabble.com/RabbitMQ-broker-with-JMS-Client-td31418.html

References

https://www.rabbitmq.com/getstarted.html

Lean example of Tomcat 8 + Guice 4 + Jersey 2.19

2015-08-05T05:36:09+00:00

Jersey is the RI of JAX-RS. in this entry, I introduce you how to use Jersey 2.19 with Guice 4 on Tomcat 8. it looks like there are some issues exist as follows but thanks to https://github.com/Squarespace/jersey2-guice , I’ve succeeded to use them anyway.

The entire project which based on Maven can be obtained from My GitHub repository.

Dependencies

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.glassfish.jersey</groupId>
            <artifactId>jersey-bom</artifactId>
            <version>2.19</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <dependency>
        <groupId>javax.servlet</groupId>
        <artifactId>javax.servlet-api</artifactId>
        <version>3.1.0</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.glassfish.jersey.containers</groupId>
        <artifactId>jersey-container-servlet</artifactId>
    </dependency>
    <dependency>
        <groupId>com.google.inject</groupId>
        <artifactId>guice</artifactId>
        <version>4.0</version>
    </dependency>
    <dependency>
        <groupId>com.google.inject.extensions</groupId>
        <artifactId>guice-servlet</artifactId>
        <version>4.0</version>
    </dependency>
    <dependency>
        <groupId>com.squarespace.jersey2-guice</groupId>
        <artifactId>jersey2-guice</artifactId>
        <version>0.10</version>
    </dependency>
</dependencies>

web.xml

<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd"
         version="3.1">

    <servlet>
        <servlet-name>Jersey Web Application</servlet-name>
        <servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
        <init-param>
            <param-name>jersey.config.server.provider.packages</param-name>
            <param-value>guice.tomcat.jersey</param-value>
        </init-param>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>Jersey Web Application</servlet-name>
        <url-pattern>/webapi/*</url-pattern>
    </servlet-mapping>

    <listener>
        <listener-class>guice.tomcat.MyJerseyGuiceServletContextListener</listener-class>
    </listener>

    <filter>
        <filter-name>guiceFilter</filter-name>
        <filter-class>com.google.inject.servlet.GuiceFilter</filter-class>
    </filter>

    <filter-mapping>
        <filter-name>guiceFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

</web-app>

MyJerseyGuiceServletContextListener.java

In this example, we use the servlet context listener named JerseyGuiceServletContextListener which comes from jersey2-guice artifact as parent class.

public class MyJerseyGuiceServletContextListener extends JerseyGuiceServletContextListener {
    @Override
    protected List<? extends Module> modules() {
        return Collections.singletonList(new ServletModule() {
            @Override
            protected void configureServlets() {
                bind(MyService.class).to(MyServiceImpl.class);
            }
        });
    }
}

Service class

We use very simple pair of an interface and implementation that creates a simple greeting message, which used in a past entry so omitted for simplicity.

MyResource.java

This is an simple implementation of JAX-RS resource class. placed under guice.tomcat.jersey package. the preceding service class named MyService will be injected by @javax.inject.Inject annotation.

@Path("myresource")
public class MyResource {

    @Inject
    private MyService myService;

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String getIt() {
        return myService.hello("Jersey");
    }
}

Test run

$ curl http://localhost:8080/webapi/myresource
Hello, Jersey

Lean example of Tomcat 8 + Guice 4 + EclipseLink 2.6.0

2015-08-04T10:26:46+00:00

I wrote a Lean example of Tomcat 8 + Guice 4 in previous entry. this time, I’ll try JPA(EclipseLink) integration with automatic transaction management.

The entire project which based on Maven can be obtained from My GitHub repository.

Prerequisites

For database connection, we’ll use a DataSource which is defined on Tomcat server. to define a Embedded Derby DataSource, This entry would help.

Dependencies

<dependencies>
    <dependency>
        <groupId>javax.servlet</groupId>
        <artifactId>javax.servlet-api</artifactId>
        <version>3.1.0</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.eclipse.persistence</groupId>
        <artifactId>eclipselink</artifactId>
        <version>2.6.0</version>
    </dependency>
    <dependency>
        <groupId>com.google.inject</groupId>
        <artifactId>guice</artifactId>
        <version>4.0</version>
    </dependency>
    <dependency>
        <groupId>com.google.inject.extensions</groupId>
        <artifactId>guice-servlet</artifactId>
        <version>4.0</version>
    </dependency>
    <dependency>
        <groupId>com.google.inject.extensions</groupId>
        <artifactId>guice-persist</artifactId>
        <version>4.0</version>
    </dependency>
</dependencies>

web.xml

No changes against previous entry except addition of resource-ref element.

<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd"
         version="3.1">

    <!-- taken from https://github.com/google/guice/wiki/Servlets -->

    <listener>
        <listener-class>guice.tomcat.MyGuiceServletConfig</listener-class>
    </listener>

    <filter>
        <filter-name>guiceFilter</filter-name>
        <filter-class>com.google.inject.servlet.GuiceFilter</filter-class>
    </filter>

    <filter-mapping>
        <filter-name>guiceFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

    <resource-ref>
        <res-ref-name>jdbc/derby</res-ref-name>
        <res-type>javax.sql.DataSource</res-type>
        <res-auth>Container</res-auth>
    </resource-ref>

</web-app>

META-INF/persistence.xml

If you don’t want to use a DataSource on Tomcat, specify the connection information in javax.persistence.jdbc.* properties instead of non-jta-data-source.

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.1" xmlns="http://xmlns.jcp.org/xml/ns/persistence"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd">
    <persistence-unit name="myJpaUnit" transaction-type="RESOURCE_LOCAL">
        <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
        <non-jta-data-source>java:comp/env/jdbc/derby</non-jta-data-source>
        <class>guice.tomcat.MyEntity</class>
        <properties>
            <property name="eclipselink.ddl-generation" value="drop-and-create-tables"/>
            <property name="eclipselink.logging.level" value="FINE"/>
            <property name="eclipselink.logging.level.sql" value="FINE"/>
        </properties>
    </persistence-unit>
</persistence>

MyGuiceServletConfig.java

The last two statements will take care of JPA integration.

public class MyGuiceServletConfig extends GuiceServletContextListener {
    @Override
    protected Injector getInjector() {
        return Guice.createInjector(new ServletModule() {
            @Override
            protected void configureServlets() {
                serve("/*").with(MyServlet.class);
                bind(MyService.class).to(MyServiceImpl.class);

                install(new JpaPersistModule("myJpaUnit"));
                filter("/*").through(PersistFilter.class);
            }
        });
    }
}

MyEntity.java

This is a very simple JPA entity which keeps only generated id and ts (a timestamp).

@Entity
@NamedQuery(name = "MyEntity.findAll", query = "SELECT e FROM MyEntity e ORDER BY e.id DESC")
public class MyEntity implements Serializable {

    @Id
    @GeneratedValue
    private Long id;
    @Column(nullable = false)
    @Temporal(TemporalType.TIMESTAMP)
    private Date ts;

... accessors are omitted

Service class

MyService.java

public interface MyService {
    void save(MyEntity e);
    List<MyEntity> findAll();
}

MyServiceImpl.java

Note that you need to annotate a method with @com.google.inject.persist.Transactional if you need a transaction on it.

public class MyServiceImpl implements MyService {

    // Transactions doesn't start if EntityManager is directly injected via @Inject.
    // I have no idea why...

    // According to https://github.com/google/guice/wiki/JPA,
    // "Note that if you make MyService a @Singleton, then you should inject Provider<EntityManager> instead."
    @Inject
    private Provider<EntityManager> emp;

    @Override
    // @javax.transaction.Transactional is not supported yet. https://github.com/google/guice/issues/797
    @com.google.inject.persist.Transactional
    public void save(MyEntity e) {
        EntityManager em = emp.get();
        em.persist(e);
    }

    @Override
    public List<MyEntity> findAll() {
        return emp.get().createNamedQuery("MyEntity.findAll", MyEntity.class).getResultList();
    }
}

MyServlet.java

This servlet saves a MyEntity with current timestamp, then fetches all of rows and returns them to the client on every request.

@javax.inject.Singleton
public class MyServlet extends HttpServlet {

    @javax.inject.Inject
    private MyService myService;

    @Override
    public void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        MyEntity myEntity = new MyEntity();
        myEntity.setTs(new Date());
        myService.save(myEntity);

        PrintWriter writer = resp.getWriter();
        writer.write("<html><body><ul>");
        for(MyEntity e : myService.findAll()){
            writer.write("<li>");
            writer.write(e.toString());
            writer.write("</li>");
        }
        writer.write("</ul></body></html>");

    }
}

Test run

A row will be saved on every request as follows:

References

Lean example of Tomcat 8 + Guice 4

2015-08-03T09:56:46+00:00

I prefer Java EE stack usually, but I need to learn a Tomcat based stack for some reasons these days. in this entry, I introduce you a very simple example of Tomcat 8 + Guice 4 app which uses GuiceFilter and GuiceServletContextListener so that bring Guice to a Servlet based web application.

The entire project which based on Maven can be obtained from My GitHub repository.

Dependencies

<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>javax.servlet-api</artifactId>
    <version>3.1.0</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>com.google.inject</groupId>
    <artifactId>guice</artifactId>
    <version>4.0</version>
</dependency>
<dependency>
    <groupId>com.google.inject.extensions</groupId>
    <artifactId>guice-servlet</artifactId>
    <version>4.0</version>
</dependency>

web.xml

<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd"
         version="3.1">

    <listener>
        <listener-class>guice.tomcat.MyGuiceServletConfig</listener-class>
    </listener>

    <filter>
        <filter-name>guiceFilter</filter-name>
        <filter-class>com.google.inject.servlet.GuiceFilter</filter-class>
    </filter>

    <filter-mapping>
        <filter-name>guiceFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

</web-app>

MyGuiceServletConfig.java

We need to declare servlet mappings and the mappings between interfaces and implementations here.

public class MyGuiceServletConfig extends GuiceServletContextListener {
    @Override
    protected Injector getInjector() {
        return Guice.createInjector(new ServletModule() {
            @Override
            protected void configureServlets() {
                serve("/*").with(MyServlet.class);
                bind(MyService.class).to(MyServiceImpl.class);
            }
        });
    }
}

Service class

We use a pair of a very simple implementation of a service class. it simply creates a greetings which uses an argument.

MyService.java

public interface MyService {
    String hello(String name);
}

MyServiceImpl.java

public class MyServiceImpl implements MyService {
    @Override
    public String hello(String name) {
        return "Hello, " + name;
    }
}

MyServlet.java

Note that we use @javax.inject.Singleton and @javax.inject.Inject annotations, that are standardized JSR API.

@javax.inject.Singleton
public class MyServlet extends HttpServlet {

    @javax.inject.Inject
    private MyService myService;

    @Override
    public void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        resp.getWriter().write(myService.hello("Guice"));
    }
}

Test run

After deployment, send a request to check the response:

$ curl http://localhost:8080
Hello, Guice

Other things/functions need to be studied

Transaction management
CDI Producer equivalent
AOP
Automatic implementation scanning/binding