Kohei Nozaki's blog 

The State Pattern


Posted on Thursday Aug 11, 2022 at 01:21PM in Technology


What is the State Pattern?

It’s one of the GoF Design Patterns which helps to organize code that handles multiple states. If you have a code base where there are many boolean fields or an enum field used to determine the state, similar if/switch branches and unclear state transitions, a giant class where there is all of the stuff in messy code causing a maintainability issue, applying the State Pattern might be worth considering.

The State Pattern consists of the following participants:

fa37dd6b 3a52 4a9a afa4 ccc5079d9f10

The Context class provides the clients of your code with the public API. All of the other participants are implementation details which the clients of your class don’t have to know about.

The Context class holds a reference to an object which implements the State interface. The Context class forwards some of the API calls to the state object. The State interface defines part of the API whose behavior has to be changed depending on the state.

The ConcreteState classes implement the State interface. Each implementation exists for each possible state of your application. It handles the API calls forwarded from the Context class appropriately according to the requirement of the state it corresponds to. A ConcreteState class can trigger a state transition.

Case study: the SMTP protocol

As an example application for the State Pattern, let’s think about an SMTP server. SMTP is a protocol used for email transport. Usually email client software like Mozilla Thunderbird supports this protocol. When such software sends an email, it connects to an SMTP server and talks to the server using this protocol to send an email.

This is a typical flow of an SMTP communication session (taken from Wikipedia):

S: 220 smtp.example.com ESMTP Postfix
C: HELO relay.example.com
S: 250 smtp.example.com, I am glad to meet you
C: MAIL FROM:<bob@example.com>
S: 250 Ok
C: RCPT TO:<alice@example.com>
S: 250 Ok
C: RCPT TO:<theboss@example.com>
S: 250 Ok
C: DATA
S: 354 End data with <CR><LF>.<CR><LF>
C: From: "Bob Example" <bob@example.com>
C: To: Alice Example <alice@example.com>
C: Cc: theboss@example.com
C: Date: Tue, 15 Jan 2008 16:02:43 -0500
C: Subject: Test message
C:
C: Hello Alice.
C: This is a test message with 5 header fields and 4 lines in the message body.
C: Your friend,
C: Bob
C: .
S: 250 Ok: queued as 12345
C: QUIT
S: 221 Bye
{The server closes the connection}

The lines beginning with "S:" are sent by the server and the ones beginning with "C: " are sent by the client.

Typically, after establishing a TCP connection, the server sends a short message which indicates it is ready to accept a command from the client. The client first sends an HELO message with its hostname, then begins an email transaction with specifying the email address of the sender of the email with a MAIL FROM command. After that, the client sends the email addresses of the recipients of the email with RCPT TO commands. Then finally, the client sends a DATA command, sends the content of the email and finishes it with a line which contains only a period. If everything goes fine, the server responds with an OK message which means the email is accepted by the server and the email will be delivered to the recipients specified.

What we can see from here is that in this protocol a client has to send necessary information to the server in a specific order. The server reacts differently for each state of the communication process. For example, the client must provide a sender identification first. Otherwise, any other command from the client doesn’t get processed and the server returns an error. It means that the server remembers the current communication state at any given moment.

So, what are those states? A good way to find that out is drawing a state diagram. It will look like the following:

0334ad5d 100e 4b11 98a3 8705c9cb832b

There are 4 states in the communication process. The first one is the idle state where the client has to start a session with an HELO command. Then it proceeds to the initial state, where the client has to provide the email address of the sender of the email with a MAIL FROM command. Then we proceed to the transaction started state where the client provides the recipients of the email with an RCPT TO command. The client might stay in this state until it finishes sending all of the recipients and that is why there is this transition which goes back to the same state. After that, finally it proceeds to the data transfer state with a DATA command where the client sends the body of the email and finishes the transaction with a line which contains only a period. Then it gets back to the initial state and if the client wants to send another email, it can start over from there.

Implementing an SMTP server with the State Pattern

In order to implement such a communication process which consists of multiple stages or states, applying the State Pattern can be a good way to keep the implementation clean and organized. Otherwise, we might end up having a giant class where there are a lot of mutable states and if/switch conditionals which are highly likely to be a maintenance problem.

Let’s look at one possible design based on the State Pattern which can handle the SMTP protocol:

53b553bd 9d02 4c68 965b 2adad582e346

The SMTPSessionHandler class corresponds to the Context class in the previous class diagram. An instance of this class exists for each SMTP conversation session. There is a public method called handleCommand() which receives a command from the client and returns the response for the command. This method has to handle the commands from the client appropriately depending on the current state. In order to achieve that, we have those 4 concrete state classes that correspond to the states in the state diagram given earlier, and what this method does is basically just forwarding the method calls to the currentState object.

In this design, state transitions are done by each concrete state class. For that purpose, the SMTPSessionHandler class has the setCurrentState() method. And when the SMTPSessionHandler class forwards the method calls to the currentState object, it also passes its own reference as an additional parameter. It allows each concrete state class to call the setCurrentState() method.

Let’s look at the source code of each participant. The SMTPSessionHandler class just holds the currentState object and forwards any handleCommand() calls to the currentState object with a reference to this object. It also has the setCurrentState() method for the concrete state classes.

public class SMTPSessionHandler {
    private State currentState = new IdleState();

    public String handleCommand(String command) {
        return currentState.handleCommand(command, this);
    }

    void setCurrentState(State newState) {
        this.currentState = newState;
    }
}

The SMTPSessionHandler class calls the underlying current state object through this State interface, which has 4 concrete implementations.

interface State {
    String handleCommand(String command, SMTPSessionHandler context);
}

The IdleState class corresponds to the very first state when a connection is established with the client. The only command it accepts is the HELO command. When it receives the HELO command, it makes a state transition by calling the setCurrentState() method of the context object with a new instance of the InitialState class and returns a greeting to the client. Otherwise, it returns an error.

class IdleState implements State {
    @Override
    public String handleCommand(String command, SMTPSessionHandler context) {
        if (command.startsWith("HELO")) {
            context.setCurrentState(new InitialState());
            return "250 smtp.example.com, I am glad to meet you";
        } else {
            return "500 5.5.1 Invalid command";
        }
    }
}

When the state transition in the IdleState class happens, the InitialState class takes over. The only command it accepts is the MAIL FROM command. When it happens, it extracts the email address which the client has sent and makes another state transition with a new instance of the TransactionStartedState class. When it creates the instance, it passes the email address it has extracted in order to let the subsequent process use it. Otherwise it returns an error.

class InitialState implements State {

    private static final Pattern PATTERN_FOR_EXTRACTING_EMAIL =
            Pattern.compile("MAIL FROM:<([^>]+)>");

    @Override
    public String handleCommand(String command, SMTPSessionHandler context) {
        Matcher matcher = PATTERN_FOR_EXTRACTING_EMAIL.matcher(command);
        if (matcher.find()) {
            String from = matcher.group(1);
            context.setCurrentState(new TransactionStartedState(from));
            return "250 Ok";
        } else {
            return "500 5.5.1 Invalid command";
        }
    }
}

The TransactionStartedState class is where we receive the destinations of the email. After specifying at least one destination, we can proceed to the next state but if there is none, it returns an error. The client has to send the destinations with the RCPT TO command. When it receives the RCPT TO command, it extracts the email address and keeps it in the List object. After sending at least one destination, the client can proceed to the next state with the DATA command. At this point, we have the address of the sender and the destinations and those are passed as the parameters of the constructor of the DataTransferState class.

class TransactionStartedState implements State {
    private static final Pattern PATTERN_FOR_EXTRACTING_EMAIL =
            Pattern.compile("RCPT TO:<([^>]+)>");
    private final String from;
    private final List<String> destinations = new ArrayList<>();

    TransactionStartedState(String from) {
        this.from = from;
    }

    @Override
    public String handleCommand(String command, SMTPSessionHandler context) {
        if (command.equals("DATA")) {
            if (destinations.isEmpty()) {
                return "500 5.5.1 Invalid command";
            } else {
                context.setCurrentState(new DataTransferState(from, destinations));
                return "354 End data with <CR><LF>.<CR><LF>";
            }
        }

        Matcher matcher = PATTERN_FOR_EXTRACTING_EMAIL.matcher(command);
        if (matcher.find()) {
            String to = matcher.group(1);
            destinations.add(to);
            return "250 Ok";
        } else {
            return "500 5.5.1 Invalid command";
        }
    }
}

The DataTransferState class corresponds to the final state of this communication process. What it does is just accumulating the body of the email in a StringBuilder object until it receives one single period which means the end of the body. After that, it will trigger the email delivery process which involves a DNS lookup, relaying the message to another SMTP server using the pieces of the data we received from the client during the process and making a state transition to the initial state. If the client wants to send another email, the client can start all over from there.

class DataTransferState implements State {
    private final String from;
    private final List<String> destinations;
    private final StringBuilder body = new StringBuilder();

    static DeliverySystem deliverySystem = (from, destinations, body) -> {
        // looks up MX records, connects to external SMTP servers and relays the message
    };

    DataTransferState(String from, List<String> destinations) {
        this.from = from;
        this.destinations = destinations;
    }

    @Override
    public String handleCommand(String command, SMTPSessionHandler context) {
        if (command.equals(".")) {
            deliverySystem.deliver(from, destinations, body.toString());
            context.setCurrentState(new InitialState());
            return "250 Ok: the email has been delivered";
        } else {
            body.append(command);
            body.append('\n');
            return null;
        }
    }
}

Conclusion: benefits of the State Pattern

That’s all of the implementation. The responsibility of each concrete state class is clear and it helps to make each class concise and short. The state transitions are also clear because when they happen, they are done in a clear way, which is the setCurrentState() method in our example. There are no cryptic boolean flags that maintain states in an unclear way. It makes the code readable, easy to maintain and extend.

For example, when you need to add a new state, what you need to do is write a new class which corresponds to the new state and add state transitions to the relevant existing state classes that are likely to be much simpler than a giant single class maintaining all of the states. It will reduce the risk of degradation.

In my experience, there are not so many use cases where this pattern is a great match, but it greatly improves the maintainability if it matches the use case and is applied appropriately.


Synchronize access to shared mutable data


Posted on Thursday Dec 17, 2020 at 06:00PM in Technology


In Java, an update on mutable data made by one thread may not be visible by other threads unless an appropriate synchronization is applied. Such unsynchronized mutable data can be the cause of a nasty bug which is difficult to spot, reproduce and fix.

Why do we need a synchronization?

Let’s say we have a class called Counter which holds mutable data:

class Counter {
    int count = 0;
    int getCount() { return count; }
    int increment() { return ++count; }
}

There is an int field named count. There are also a simple getter and a method which increments the counter. The increment method can change the value in the count field which means that this class holds mutable data.

Now let’s see what happens when an instance of the Counter class is shared across 2 threads without any synchronization. We have this client code of the Counter class:

class ThisNeverFinishesOnMyLaptop {
    public static void main(String[] args) throws InterruptedException {
        Counter counter = new Counter();
        Thread backgroundThread = new Thread(() -> {
            while (true) { // wait until the counter gets incremented
                if (counter.getCount() != 0) break;
            }
            System.out.println("Finished");
        });
        backgroundThread.start();
        TimeUnit.SECONDS.sleep(1);
        counter.increment();
    }
}

First we create an instance of the Counter class then we launch a background thread which starts a while loop and waits until the counter gets incremented. Then the main thread sleeps for 1 second and calls the increment method. This increment() call is supposed to make the background thread get out of the while loop. So what this code is supposed to do is to finish after 1 second with having the "Finished" message printed into the console.

But in some environment, this code never even finishes which means the while loop becomes an infinite loop. The problem here is that due to lack of synchronization, the background thread fails to see the new value which the main thread set. In Java, without synchronization, it’s not guaranteed that updates that have been made by one thread will be visible by other threads. Another thread might see the update at some point or might not see at all like in this example on my laptop. The behavior might vary depending on the environment where this code runs in and it is totally unpredictable.

How do we do a synchronization?

Then how can we make it work properly? One way to do that is the following:

class NowItFinishesOnMyLaptop {
    public static void main(String[] args) throws InterruptedException {
        Counter counter = new Counter();
        Object lock = new Object();
        Thread backgroundThread = new Thread(() -> {
            while (true) { // wait until the counter gets incremented
                synchronized (lock) {
                    if (counter.getCount() != 0) break;
                }
            }
            System.out.println("Finished");
        });
        backgroundThread.start();
        TimeUnit.SECONDS.sleep(1);
        synchronized (lock) {
            counter.increment();
        }
    }
}

Now we create an object named lock and when we read or write data in the counter object, we do it in a synchronized block with the lock object. This guarantees that any update which is done in a synchronized block will be seen by any other code inside a synchronized block that uses the same lock object.

There is also another way to fix this. We can add the volatile keyword to the count field:

class VolatileCounter {
    volatile int count = 0;
    int getCount() { return count; }
    int increment() { return ++count; }
}

And we can replace the use of the Counter class in the ThisNeverFinishesOnMyLaptop class by the VolatileCounter class. The volatile keyword guarantees that updates made by one thread will be visible to other threads.

class ThisAlsoFinishesOnMyLaptop {
    public static void main(String[] args) throws InterruptedException {
        VolatileCounter counter = new VolatileCounter();
        Thread backgroundThread = new Thread(() -> {
            while (true) { // wait until the counter gets incremented
                if (counter.getCount() != 0) break;
            }
            System.out.println("Finished");
        });
        backgroundThread.start();
        TimeUnit.SECONDS.sleep(1);
        counter.increment();
    }
}

But there are some cases where the volatile keyword is not enough. Let’s think about the following code:

class IncrementByMultipleThreads {
    public static void main(String[] args) {
        VolatileCounter counter = new VolatileCounter();
        Set<Integer> ints = Collections.synchronizedSet(new HashSet<>());
        Runnable incrementer = () -> {
            while (true) {
                int increment = counter.increment();
                boolean added = ints.add(increment);
                if (!added) System.out.println("duplicate number detected");
            }
        };
        Thread t1 = new Thread(incrementer), t2 = new Thread(incrementer), t3 = new Thread(incrementer);
        t1.start(); t2.start(); t3.start();
    }
}

First we create a volatile counter object and a Set of integers. We use a synchronized wrapper for the set in order to make it work properly in a concurrent usecase like this. And create a Runnable object which launches an infinite loop. In the loop, it increments the counter object and puts the return value of the increment method into the set. The set will be keeping all of the numbers the counter returned. And when it fails to add the number to the set, which means the same number was already added to the set, it prints that "duplicate number detected" message. And we launch 3 threads that execute the same Runnable object.

It prints out a lot of the "duplicate number detected" messages immediately when I run it on my laptop. Which means the counter object has returned duplicate numbers. The reason behind this behavior is that the incrementation done by ++count is not atomic. What it does is that first it reads the number stored in the count field, adds 1 to it and stores the result into the count field and those are not atomically executed. And since we have 3 threads that are executing the increment method concurrently, there are chances that some of those threads read, calculate and return the same value.

If we want the counter to return unique values for all of the threads, volatile is not sufficient. We can use a synchronized block to make it happen instead:

class IncrementByMultipleThreadsWithLock {
    public static void main(String[] args) {
        Counter counter = new Counter();
        Object lock = new Object();
        Set<Integer> ints = Collections.synchronizedSet(new HashSet<>());
        Runnable incrementer = () -> {
            while (true) {
                int increment;
                synchronized (lock) {
                    increment = counter.increment();
                }
                boolean added = ints.add(increment);
                if (!added) System.out.println("duplicate number detected");
            }
        };
        Thread t1 = new Thread(incrementer), t2 = new Thread(incrementer), t3 = new Thread(incrementer);
        t1.start(); t2.start(); t3.start();
    }
}

We have introduced the lock object again, and put the increment() call inside the synchronized block. Since we have the synchronized block, we don’t need the counter to be volatile anymore, therefore we can simply use the plain Counter class instead.

Now we get unique numbers from the counter and it doesn’t show the "duplicate number detected" message anymore at least until the overflow of the counter field. In addition to the guarantee about the memory synchronization of mutable data, the synchronized block also guarantees that only one thread can execute the code placed inside the block at any given time. It effectively prevents the race condition where the counter returns duplicate numbers.

Which classes in the JDK require synchronization?

So we have seen that we need to use an appropriate way of synchronization for mutable data when it’s shared across multiple threads. There are many classes in the Java standard library that contain mutable data or require synchronization if they are shared across multiple threads. It includes some of collection classes (HashMap, ArrayList etc.) and the Format class and its subclasses (e.g. SimpleDateFormat). To find out if a class requires synchronization, the first thing you can do is to consult the documentation of the class.

If we fail to apply necessary synchronization, the consequences can be horrible. I once saw code which uses a HashMap shared across multiple threads without a synchronization. The HashMap was used as some kind of cache and there are multiple threads that update the HashMap and it could happen simultaneously. It eventually broke the underlying data structure of the HashMap and triggered an infinite loop under heavy load. Due to the fact that this kind of problem rarely shows up, it passed all of the QA processes we had and unfortunately ended up appearing in the production environment. I will never forget this incident because I was the one to fix the issue in midnight. There is a great article on the web which explains how an unsynchronized HashMap triggers an infinite loop: https://mailinator.blogspot.com/2009/06/beautiful-race-condition.html

I also sometimes see a SimpleDateFormat object stored in a static field and shared across multiple threads. This is also dangerous. It might seem to be working fine for most of the time but occasionally it produces a wrong result or throws a mysterious exception under heavy load.

How do we do synchronization for an unsynchronized class in the JDK?

Whenever we encounter a situation where we need to use some class which requires a synchronization in concurrent usecases, the first thing we can do is to look for a thread-safe equivalent.

As for collection classes, there are quite a few thread-safe classes which cover almost all of the interfaces in the collection framework (e.g. ConcurrentHashMap, CopyOnWriteArrayList). If one of those suits your usecase, simply using one of them is the safest and easiest way to apply the synchronization. But one thing we need to keep in mind is that just using thread-safe classes doesn’t always make your code work properly in concurrent usecases. For example, sometimes we need to make certain operations atomic like we did in one of the examples earlier. In such cases, we still need to use some kind of mutual execution exclusion mechanism like we did with the synchronization blocks.

As for SimpleDateFormat, if the version of your Java is 8 or greater, please check the DateTimeFormatter class, which is immutable and thread-safe.

Also one important thing to remember is that just putting volatile doesn’t make unsynchronized objects thread-safe. For example, when you have a HashMap field shared across multiple threads and you put volatile into the declaration of the HashMap field, it doesn’t guarantee that the HashMap will work property in concurrent use cases.

Conclusion

We have discussed why we need to synchronize mutable data, how we can do that and what happens when we fail to do that in concurrent usecases. The consequences can be horrible and a nasty thing about the failure of applying an appropriate synchronization is that everything looks fine for most of the time and the problem shows up only occasionally and sometimes it’s even hard to reproduce. So I recommend paying extra attention when you have to deal with mutable data in concurrent usecases.

Another good way to reduce this kind of risk is minimizing mutability in the code. Immutable data is always safe to share across threads which means we don’t have to even think about it if no mutable data is shared. One good starting point is that whenever you write a class, I recommend trying making the fields in your class final as much as possible. It will make your class one step closer to immutable.

There are a lot of more things to learn for writing safe concurrent applications in Java. If you are interested, I recommend checking this book out: https://jcip.net


The Singleton Pattern in Java


Posted on Tuesday Sep 22, 2020 at 12:47PM in Technology


In this entry, I’ll introduce some common ways to apply the Singleton Pattern in Java and a problem that can harm maintainability of your code base and a solution for the problem.

What is it?

The Singleton Pattern is a commonly used idiom to create and maintain objects called singletons. A singleton is the only instance of a class.

Common use cases are an object which keeps the user’s preferences in a desktop application, cache or a resource pool. One example would be a TCP connection pool where there are multiple active connections to a backend service like a database server or a web service. Typically there should be only one pool object which maintains all of the connections. Having multiple separate pool objects in one application doesn’t make much sense in most cases because it will make the use of the pool less effective.

As an example, let’s imagine that there is the following interface which allows us to fetch the weather data from some external web service:

interface WeatherFetcher {
    String fetchWeather(String cityCode) throws IOException;
    int fetchTemperature(String cityCode) throws IOException;
}

There are a couple of methods to fetch the weather (atmospheric conditions) and the temperature of a city. Since it involves network communcation, there can be an IOException.

Now imagine that you are working on implementing a feature which uses this interface. You are told that the performance requirement for this feature is strict, therefore you have decided to maintain a pool of active TCP connections to the external web service in order to keep the latency low. Applying the Singleton Pattern here sounds like a good idea because it will make sure that you will have only one pool object in your application.

Class with a private constructor

Let’s see how we can implement this with the Singleton Pattern. One simple implementation would be something like this:

public class SimpleSingleton implements WeatherFetcher {
    private static final SimpleSingleton INSTANCE = new SimpleSingleton();
    public static WeatherFetcher getInstance() { return INSTANCE; }
    private SimpleSingleton() { System.out.println("Populating the connection pool..."); }

    @Override
    public int fetchTemperature(String cityCode) {
        // gets an idle connection from the pool, sends a request
        return 71; // please assume it's from a response
    }

    @Override
    public String fetchWeather(String cityCode) {
        // gets an idle connection from the pool, sends a request
        return "Sunny"; // please assume it's from a response
    }

    // client code
    public static void main(String[] args) throws IOException {
        WeatherFetcher singleton = SimpleSingleton.getInstance();
        System.out.printf("Weather in New York: %s, %dF\n",
                singleton.fetchWeather("NewYork"), singleton.fetchTemperature("NewYork"));
    }
}

The class has a static final field called INSTANCE, where the only instance of this class is kept. There is a simple getter for that.

This class has a constructor populating the pool. Let’s imagine that it opens a lot of connections to an external web service and makes those ready to use. The constructor is private, which prevents creation of another instance of the class. Without this, any other class can create an instance of this class, which means that there is no guarantee that there is only one instance of this class. It also prevents inheritance because there is no constructor a subclass can call.

In order to get the singleton, the client needs to call the static method getInstance() first. After that, the client can fetch the weather data via the methods defined in the WeatherFetcher interface.

Class with a static holder class for lazy initialization

The SimpleSingleton class works fine, but there is one common requirement you might encounter. Populating the connection pool can be expensive and there might be a situation where the feature we just implemented will be used by only a very small number of your users, and you want to make sure that the connection pool gets populated only when it’s really needed.

We cannot guarantee that with the SimpleSingleton class. Let’s add a System.out.println() call to the main method and see what is happening there:

public static void main(String[] args) throws IOException {
    System.out.println("Doing some other stuff - I don't need the singleton yet"); // added
    WeatherFetcher singleton = SimpleSingleton.getInstance();
    System.out.printf("Weather in New York: %s, %dF\n",
            singleton.fetchWeather("NewYork"), singleton.fetchTemperature("NewYork"));
}

The main method yields the following output:

Populating the connection pool...
Doing some other stuff - I don't need the singleton yet
Weather in New York: Sunny, 71F

As you can see, the pool has got initialized even though it’s not needed yet; the constructor gets executed before the getInstance() method is called because the constructor invocation is written in the static initializer of the class. It can lead to a situation where unnecessary resource consumption is imposed on the user even though the user might not need the feature. We can work around this with having a simple private static holder class like the following:

public class LazySingleton implements WeatherFetcher {
    private static class SingletonHolder {
        static final LazySingleton INSTANCE = new LazySingleton();
    }
    public static WeatherFetcher getInstance() { return SingletonHolder.INSTANCE; }
    private LazySingleton() { System.out.println("Populating the connection pool..."); }
    ...
    // client code
    public static void main(String[] args) throws IOException {
        System.out.println("Doing some other stuff - I don't need the singleton yet");
        WeatherFetcher singleton = LazySingleton.getInstance();
        System.out.printf("Weather in New York: %s, %dF\n",
                singleton.fetchWeather("NewYork"), singleton.fetchTemperature("NewYork"));
    }
}

The LazySingleton singleton instance is kept in the static field in the static inner class called SingletonHolder and the getInstance() method returns that. Executing the main method will show the following output, as expected:

Doing some other stuff - I don't need the singleton yet
Populating the connection pool...
Weather in New York: Sunny, 71F

A problem that can harm maintainability: getInstance() everywhere

Let’s imagine that now you need to implement some code for generating a report about the weather of some cities with the singleton we just wrote. It might be implemented like this:

public class WeatherReporter {
    String generateWeatherReport(List<String> cityCodeList) {
        StringBuilder sb = new StringBuilder("=== Weather Report ===\n");
        for (String cityCode : cityCodeList) {
            try {
                String weather = LazySingleton.getInstance().fetchWeather(cityCode);
                int temperature = LazySingleton.getInstance().fetchTemperature(cityCode);
                sb.append(String.format("%s: %s, %dF\n", cityCode, weather, temperature));
            } catch (IOException e) {
                sb.append(String.format("%s: Failed to fetch data\n", cityCode));
            }
        }
        return sb.toString();
    }

    public static void main(String[] args) {
        WeatherReporter reporter = new WeatherReporter();
        System.out.println(reporter.generateWeatherReport(
                Arrays.asList("NewYork", "Montreal", "Tokyo")));
    }
}

It receives a List of city code which we need to create the report for. Then it constructs the header, iterates the list of city code and fetches the weather and the temperature for each city and builds the String which represents the report. If there is an I/O problem, we just mention that it failed to fetch the data for the city.

Executing the main method should produce the following output:

Populating the connection pool...
=== Weather Report ===
NewYork: Sunny, 71F
Montreal: Cloudy, 68F
Tokyo: Rainy, 69F

That’s a fairly complicated piece of code. Therefore, having some unit tests for the WeatherReporter class would be nice, but unfortunately it’s almost impossible because the current implementation is hard-coded to directly call the static getInstance() method to get the WeatherFetcher object, which means it always sends requests to the real external web service.

We cannot make a reliable assertion if it depends on the output of the real external service that way. One option would be running a fake of the external service, but it would require a lot of effort. And it would be nice if we could write a test case for IOException handling, but reproducing IOException in a real environment is also too much work. And the connection pool population might take a lot of time. The LazySingleton class might be hard-coded to create thousands of TCP connections and in that case it might take a few tens of seconds to populate. Waiting for such a long time with every unit test execution doesn’t sound reasonable.

So, the problem here is that we have business logic tightly coupled to an external data source. Having that kind of a method call in the middle of business logic makes writing unit tests very difficult or almost impossible because in order to make that kind of thing work, in many cases it requires some sort of specific setup or configuration and oftentimes that is too much work.

One thing we can try to work around this problem is replacing the static getInstance() method call by a functional interface. In our case, we can do something like this:

public class ImprovedWeatherReporter {

    private final Supplier<? extends WeatherFetcher> weatherFetcherSupplier;

    public ImprovedWeatherReporter(Supplier<? extends WeatherFetcher> weatherFetcherSupplier) {
        this.weatherFetcherSupplier = weatherFetcherSupplier;
    }

    String generateWeatherReport(List<String> cityCodeList) {
        StringBuilder sb = new StringBuilder("=== Weather Report ===\n");
        for (String cityCode : cityCodeList) {
            try {
                String weather = weatherFetcherSupplier.get().fetchWeather(cityCode);
                int temperature = weatherFetcherSupplier.get().fetchTemperature(cityCode);
                sb.append(String.format("%s: %s, %dF\n", cityCode, weather, temperature));
            } catch (IOException e) {
                sb.append(String.format("%s: Failed to fetch data\n", cityCode));
            }
        }
        return sb.toString();
    }

    public static void main(String[] args) {
        ImprovedWeatherReporter reporter = new ImprovedWeatherReporter(LazySingleton::getInstance);
        System.out.println(reporter.generateWeatherReport(
                Arrays.asList("NewYork", "Montreal", "Tokyo")));
    }
}

The ImprovedWeatherReporter class is not tightly coupled to the getInstance() method anymore. Instead, now it has a field for a Supplier object which returns a WeatherFetcher object. The client of this class can inject the Supplier via its constructor. In our case, we can inject LazySingleton.getInstance() for the production use.

That seems to be not much change, but it is a great improvement from the perspective of unit testing. Now we can inject any Supplier returning a WeatherFeather object, and it enables us to write reliable, fast and stable unit tests like the following:

@ExtendWith(MockitoExtension.class)
class ImprovedWeatherReporterTest {

    @Mock
    WeatherFetcher weatherFetcher;
    ImprovedWeatherReporter sut;

    @BeforeEach
    void setUp() {
        sut = new ImprovedWeatherReporter(() -> weatherFetcher);
    }

    @Test
    void producesReports() throws IOException {
        when(weatherFetcher.fetchTemperature("NewYork")).thenReturn(71);
        when(weatherFetcher.fetchWeather("NewYork")).thenReturn("Sunny");
        when(weatherFetcher.fetchTemperature("Montreal")).thenReturn(68);
        when(weatherFetcher.fetchWeather("Montreal")).thenReturn("Cloudy");

        String actual = sut.generateWeatherReport(Arrays.asList("NewYork", "Montreal"));

        assertThat(actual).isEqualTo("=== Weather Report ===\n" +
                "NewYork: Sunny, 71F\n" +
                "Montreal: Cloudy, 68F\n");
    }

    @Test
    void exception() throws IOException {
        when(weatherFetcher.fetchTemperature("Tokyo")).thenThrow(new IOException("catch this"));

        String actual = sut.generateWeatherReport(Collections.singletonList("Tokyo"));

        assertThat(actual).isEqualTo("=== Weather Report ===\n" +
                "Tokyo: Failed to fetch data\n");
    }
}

First, we create a mock of WeatherFetcher and inject a Supplier which returns the mock. With that, we can have complete control of the WeatherFetcher object the ImprovedWeatherReporter class relies on. Now we can specify what the WeatherFetcher object returns for what parameter with the mock framework you use. It will even enable us to test a case where the IOException is thrown, which was impossible with the tightly-coupled version of the WeatherReporter class. Just having a simple abstraction layer between your business logic and an external service makes writing unit tests much easier.

Another solution you might want to check is using a dependency injection framework to maintain singletons. You can easily make an object singleton and let the framework inject the singleton to the objects the framework maintains. It will reduce a lot of boilerplates, and some of the code I introduced here will be unnecessary. Especially if your code base has a dependency injection framework already, I recommend checking the documentation of your framework. For Google Guice, check this out: https://github.com/google/guice/wiki/Scopes

Conclusion

We’ve seen a couple of common idioms that can be used to apply the Singleton Pattern in Java, and discussed a common maintainability issue that is sometimes caused by applying the pattern. When you see code where there are a lot of getInstance() calls in the middle of business logic, it might be good to take some time to see if you would be able to run your business logic in isolation for unit tests. If your code was unit test friendly, your code would be likely to have wider coverage of tests and it would make future maintenance easier and the code base more stable.


Favor composition over inheritance


Posted on Friday Aug 14, 2020 at 04:52PM in Technology


In this entry, I’ll discuss problems of inheritance, which is often overused in software written in an object oriented programming language, and how we can do better with composition, which is usually a much better alternative to inheritance.

Inheritance can make your code fragile

Let’s think about some piece of software used by some cafe. It contains the following classes:

1e95a7ea 3a54 4fcc ad6b 798b5b3d1ebf

There is an interface called Beverage, which can return the price and description of a beverage. Most probably there are Coffee or Tea classes that implement the interface. And there is a class called Order, where you can add Beverage objects to it in order to calculate the grand total on the bill for a customer of the cafe. The code of these looks like this:

interface Beverage {
    BigDecimal price();
    String description();
}

class Order {

    private static final BigDecimal TAX_RATE = new BigDecimal("0.1");
    private BigDecimal subTotal = BigDecimal.ZERO;

    void add(Beverage beverage) {
        subTotal = subTotal.add(beverage.price());
    }

    void addAll(Collection<? extends Beverage> beverages) {
        for (Beverage beverage : beverages)
            subTotal = subTotal.add(beverage.price());
    }

    BigDecimal grandTotal() {
        BigDecimal tax = subTotal.multiply(TAX_RATE);
        return subTotal.add(tax);
    }
}

Now, let’s consider the following scenario: the owner of the cafe wants to start a campaign to boost their sales. The idea of the campaign is that if a customer orders more than 2 beverages at the same time, he/she will get 20% discount from the grand total. In order to implement this requirement, a programmer comes up with the CampaignOrder class which extends the Order class. It looks like the following:

class CampaignOrder extends Order {

    private static final BigDecimal DISCOUNT_RATE = new BigDecimal("0.2");
    private int numberOfBeverages;

    @Override
    void add(Beverage beverage) {
        super.add(beverage);
        numberOfBeverages++;
    }

    @Override
    void addAll(Collection<? extends Beverage> beverages) {
        super.addAll(beverages);
        numberOfBeverages += beverages.size();
    }

    @Override
    BigDecimal grandTotal() {
        BigDecimal grandTotal = super.grandTotal();
        if (numberOfBeverages > 2) {
            BigDecimal discount = grandTotal.multiply(DISCOUNT_RATE);
            grandTotal = grandTotal.subtract(discount);
        }
        return grandTotal;
    }
}

It captures add() and addAll() method calls, forwards them to its superclass and keeps track of the number of the beverages which are added in the numberOfBeverages variable. And it also captures grandTotal() method calls, forwards them to its superclass and applies the 20% discount to the grand total the superclass calculated depending on the numberOfBeverages variable.

It might look reasonable as it reuses the Order class effectively so that it won’t introduce any duplicate code. But this approach can lead to an unforseen issue due to the fact that the CampaignOrder class relies on a hidden behavior of the Order class, which is that the add() method and the addAll() method work independently. Consider that at some point some other programmer has done some quick refactoring in the addAll() method:

class Order {
    ...
    void add(Beverage beverage) {
        subTotal = subTotal.add(beverage.price());
    }

    void addAll(Collection<? extends Beverage> beverages) {
        for (Beverage beverage : beverages)
            // Someone has done refactoring. Original code was:
            // subTotal = subTotal.add(beverage.price());
            // Now:
            add(beverage);
    }
    ...
}

It hasn’t broken anything in terms of the functionality of the Order class but unfortunately it has just broken the CampaignOrder class. Remember the implementation of the CampaignOrder class which captures both of the add() and the addAll() method calls and counts the number of the beverage objects it receives. Due to the fact that now the Order class calls the add() method from the addAll() method, whenever the addAll() method of the CampaignOrder class gets called, the number of the beverage objects gets counted twice in the CampaignOrder class. In other words, now the following test case fails:

@ExtendWith(MockitoExtension.class)
class CampaignOrderTest {

    @Mock
    Beverage coffee, tea;
    CampaignOrder sut = new CampaignOrder();

    @Test
    void addAll() {
        when(coffee.price()).thenReturn(new BigDecimal("2.0"));
        when(tea.price()).thenReturn(new BigDecimal("3.0"));

        sut.addAll(Arrays.asList(coffee, tea));

        // It fails after the refactoring. Now grandTotal() returns 4.4
        // The discount is applied unexpectedly since the logic which counts beverages is broken
        // Now it's counted as 4, which is over the threshold of the discount
        assertThat(sut.grandTotal()).isEqualByComparingTo("5.5");
    }
}

The person who has done this refactoring should not be blamed. In fact this person removed a duplicate piece of code, which is a good thing, and it’s not easy to catch such an error. The real problem here is the wrong use of inheritance, which is writing a subclass that relies on an implementation detail of its super class. That introduces fragility to the codebase. Hence using inheritance this way should be avoided.

And also there can be another problematic case where a new method has been added to the Order class. If the new method can be used for adding a beverage, we need to make sure that the CampaignOrder class captures method calls to the new method, but chances are we would not even notice that there was a subclass which we might have to change.

What could have been done instead of inheritance then? The most obvious approach is using composition instead. Let’s rework the class hierarchy and make it like this:

1d07a6e0 893d 4104 bfdb 177221cd449b

The implementation:

interface Order {
    void add(Beverage beverage);
    void addAll(Collection<? extends Beverage> beverages);
    BigDecimal grandTotal();
}

class RegularOrder implements Order {

    private static final BigDecimal TAX_RATE = new BigDecimal("0.1");
    private BigDecimal subTotal = BigDecimal.ZERO;

    @Override
    public void add(Beverage beverage) {
        subTotal = subTotal.add(beverage.price());
    }

    @Override
    public void addAll(Collection<? extends Beverage> beverages) {
        for (Beverage beverage : beverages)
            subTotal = subTotal.add(beverage.price());
    }

    @Override
    public BigDecimal grandTotal() {
        BigDecimal tax = subTotal.multiply(TAX_RATE);
        return subTotal.add(tax);
    }
}

class CampaignOrder implements Order {

    private static final BigDecimal DISCOUNT_RATE = new BigDecimal("0.2");
    private int numberOfBeverages;

    private final Order delegate;

    CampaignOrder() {
        this(new RegularOrder());
    }

    private CampaignOrder(Order delegate) {
        this.delegate = delegate;
    }

    @Override
    public void add(Beverage beverage) {
        delegate.add(beverage);
        numberOfBeverages++;
    }

    @Override
    public void addAll(Collection<? extends Beverage> beverages) {
        delegate.addAll(beverages);
        numberOfBeverages += beverages.size();
    }

    @Override
    public BigDecimal grandTotal() {
        BigDecimal grandTotal = delegate.grandTotal();
        if (numberOfBeverages > 2) {
            BigDecimal discount = grandTotal.multiply(DISCOUNT_RATE);
            grandTotal = grandTotal.subtract(discount);
        }
        return grandTotal;
    }
}

Now the Order class has become an interface which has 2 implementations. One is the RegularOrder class, which was formerly the Order class, and the other one is the CampaignOrder class. An instance of the CampaignOrder class has a reference to a RegularOrder instance in order to reuse its functionality, but the CampaignOrder class doesn’t have any superclass anymore. With the new design, no refactoring of the RegularCampaign class can break the CampaignOrder class as long as the RegularOrder class keeps the contract of the public methods.

One important difference from the inheritance approach is that now the CampaignOrder class no longer relies on any of the implementation details of the RegularOrder class. What it relies on now is the behavior of the public methods of the RegularOrder class, which are all defined in the Order interface.

The good thing about sticking with this approach is that as long as a class keeps its functionality on the public interface level the same, it can change its internal structure without you worrying about breaking something accidentally. It greatly reduces chances of unforseen breakage. Therefore, it will make your codebase more stable. Also, with having the interface, when a new method has been added to the interface, it will make the compilation of its implementors fail since the implementation is missing. With that, unlike with the inheritance solution, we can notice that we have to add the missing implementation to its implementors immediately.

Inheritance is inflexible

Now let’s think about some details of the Beverage interface in the codebase. It has an implemention class called Coffee. The customer can add a condiment such as milk, whip or sugar into it if they want to. For some reason, the original programmer implemented this requirement using inheritance:

faea3261 015d 43d8 be9a 69c44d88c388
interface Beverage {
    BigDecimal price();
    String description();
}

class Coffee implements Beverage {
    @Override public BigDecimal price() { return new BigDecimal("1.99"); }
    @Override public String description() { return "Coffee"; }
}

class CoffeeWithMilk extends Coffee {
    @Override public BigDecimal price() { return super.price().add(new BigDecimal("0.10")); }
    @Override public String description() { return super.description() + ", Milk"; }
}

class CoffeeWithWhip extends Coffee {
    @Override public BigDecimal price() { return super.price().add(new BigDecimal("0.15")); }
    @Override public String description() { return super.description() + ", Whip"; }
}

class CoffeeWithSugar extends Coffee {
    @Override public BigDecimal price() { super.price().add(new BigDecimal("0.05")); }
    @Override public String description() { return super.description() + ", Sugar"; }
}

Now we’ve got a new requirement to implement, which is that a customer should be able to add multiple condiments into coffee. Let’s see if we can do that with inheritance. It would look like the following:

2d51746f 83fd 43ba b0c5 3628272b7d88

We ended up creating a lot of subclasses for all of the combinations. That will do for the time being but think about the future expansion. We will need to create many subclasses everytime we introduce a new condiment. And what if we want to reuse code which is responsible for a condiment for another beverage class, say, a Tea class? It’s not clear if we can do that in a reasonable way with this approach.

Let’s see if we could do better with composition. It would look like the following:

9b0b3122 c78b 4f45 b358 b30e682519dc

The implementation (the Beverage interface and the Coffee class are the same as the ones we have seen before):

class MilkWrapper implements Beverage {
    private final Beverage delegate;

    MilkWrapper(Beverage delegate) { this.delegate = delegate; }

    @Override public BigDecimal price() { return delegate.price().add(new BigDecimal("0.10")); }
    @Override public String description() { return delegate.description() + ", Milk"; }
}

class WhipWrapper implements Beverage {
    private final Beverage delegate;

    WhipWrapper(Beverage delegate) { this.delegate = delegate; }

    @Override public BigDecimal price() { return delegate.price().add(new BigDecimal("0.15")); }
    @Override public String description() { return delegate.description() + ", Whip"; }
}

class SugarWrapper implements Beverage {
    private final Beverage delegate;

    SugarWrapper(Beverage delegate) { this.delegate = delegate; }

    @Override public BigDecimal price() { return delegate.price().add(new BigDecimal("0.05")); }
    @Override public String description() { return delegate.description() + ", Sugar"; }
}

In the new implementation, the classes responsible for condiments hold a reference to a Beverage object instead of extending the Coffee class. They can still reuse the code of the Coffee class but through the Beverage interface. Instead of creating an instance of a subclass of the Coffee class, you need to provide a Coffee instance to the constructor of one of the wrapper classes. Let’s do it like this:

@Test
void coffeeWithMilk() {
    Beverage coffeeWithMilk = new MilkWrapper(new Coffee());
    assertThat(coffeeWithMilk.description()).isEqualTo("Coffee, Milk");
    assertThat(coffeeWithMilk.price()).isEqualByComparingTo("2.09");
}

@Test
void coffeeWithWhip() {
    Beverage coffeeWithWhip = new WhipWrapper(new Coffee());
    assertThat(coffeeWithWhip.description()).isEqualTo("Coffee, Whip");
    assertThat(coffeeWithWhip.price()).isEqualByComparingTo("2.14");
}

@Test
void coffeeWithSugar() {
    Beverage coffeeWithSugar = new SugarWrapper(new Coffee());
    assertThat(coffeeWithSugar.description()).isEqualTo("Coffee, Sugar");
    assertThat(coffeeWithSugar.price()).isEqualByComparingTo("2.04");
}

Now, let’s see how we can implement the new requirement about multiple condiments with the new design. In fact, no change is needed in the Coffee class and the other classes in the diagram. We can do that like this:

@Test
void coffeeWithMilkAndWhip() {
    Beverage coffee = new Coffee();
    coffee = new MilkWrapper(coffee);
    coffee = new WhipWrapper(coffee);

    assertThat(coffee.description()).isEqualTo("Coffee, Milk, Whip");
    assertThat(coffee.price()).isEqualByComparingTo("2.24");
}

@Test
void coffeeWithMilkAndSugar() {
    Beverage coffee = new Coffee();
    coffee = new MilkWrapper(coffee);
    coffee = new SugarWrapper(coffee);

    assertThat(coffee.description()).isEqualTo("Coffee, Milk, Sugar");
    assertThat(coffee.price()).isEqualByComparingTo("2.14");
}

@Test
void coffeeWithMilkAndWhipAndSugar() {
    Beverage coffee = new Coffee();
    coffee = new MilkWrapper(coffee);
    coffee = new WhipWrapper(coffee);
    coffee = new SugarWrapper(coffee);

    assertThat(coffee.description()).isEqualTo("Coffee, Milk, Whip, Sugar");
    assertThat(coffee.price()).isEqualByComparingTo("2.29");
}

First, we create an instance of the Coffee class and then wrap it with the wrapper classes as needed. This is much more flexible than the old approach which required having subclasses for all of the combinations. We can even combine one particular condiment multiple times, which would have been much harder with the inheritance approach:

@Test
void coffeeWithTripleWhip() {
    Beverage coffee = new Coffee();
    coffee = new WhipWrapper(coffee);
    coffee = new WhipWrapper(coffee);
    coffee = new WhipWrapper(coffee);

    assertThat(coffee.description()).isEqualTo("Coffee, Whip, Whip, Whip");
    assertThat(coffee.price()).isEqualByComparingTo("2.44");
}

This design is also better in terms of maintainability. Introducing a new condiment doesn’t impact any other class in the diagram (i.e. there will be no class explosion). Those wrapper classes are highly reusable because they can be reused for anything which implements the Beverage interface. Also, they don’t depend on anything but the Beverage interface. They solely rely on the public methods in the Beverage interface. There will be no breakage as long as the contract of the Beverage interface stays the same.

Conclusion

We’ve seen how improper use of inheritance can make your code fragile and inflexible. Using inheritance just for code reuse can lead to an unforseen problem in the future. And inheritance is not flexible as you might expect. When you are tempted to use inheritance, I recommend considering if you can do it using composition instead. In my experience, inheritance is not the best option for most of such cases.

I would also like to mention that composition opens up a whole new world of clever design ideas in an object oriented programming language. If you want to learn more about it, I recommend checking out the following books:


Java Generics wildcards


Posted on Friday Jul 17, 2020 at 06:49PM in Technology


This entry is a quick introduction to Java Generics wildcards, which make Java code more flexible and type-safe.

Wildcards with extends

Wildcards with extends provide flexibility to code which gets a value out of a container object. For example, let’s consider the following variable definition:

List<? extends Number> list_extendsNumber;

You can think of it as a List which contains elements whose type is a subtype of Number. For example, we can assign the following instances to the variable:

list_extendsNumber = new ArrayList<Number>();
list_extendsNumber = new ArrayList<Integer>(); // Integer extends Number
list_extendsNumber = new ArrayList<BigDecimal>(); // BigDecimal extends Number

But the following doesn’t compile since neither of them is a subtype of Number:

list_extendsNumber = new ArrayList<String>(); // compilation error; a String is not a Number
list_extendsNumber = new ArrayList<Object>(); // compilation error; Number extends Object but not the other way around

As you can see from the examples above, it’s guaranteed that a List which is assigned to the list_extendsNumber variable contains a Number or a value whose type is a subtype of Number. We can use the variable for getting a value out of it as a Number. For example:

// It's guaranteed that it contains a Number or its subtype
Number firstNumber = list_extendsNumber.get(0);

You can even write some utility method like this:

double avg(List<? extends Number> nums) {
    return nums.stream().mapToDouble(Number::doubleValue).average().orElse(0);
}

The avg() method accepts any List whose type parameter is a subtype of Number. For example, the following code which calls the method works fine:

List<Number> nums = Arrays.asList(1, 0.5d, 3);
assert avg(nums) == 1.5d;

List<Integer> ints = Arrays.asList(1, 2, 3);
assert avg(ints) == 2d;

List<BigDecimal> bds = Arrays.asList(new BigDecimal(1), new BigDecimal(2), new BigDecimal(3));
assert avg(bds) == 2d;

But the following is invalid:

List<String> strs = Arrays.asList("foo", "bar", "baz");
avg(strs) // compilation error; a String is not a Number

List<Object> objs = Arrays.asList("foo", 0.5d, 3);
avg(objs) // compilation error; Number extends Object but not the other way around

What’s good about it? Let’s see what happens if we don’t use the wildcard here. Now the method looks like this:

double avg(List<Number> nums) {
    return nums.stream().mapToDouble(Number::doubleValue).average().orElse(0);
}

Now the method can accept only List<Number> that is much less flexible. The following still works:

List<Number> nums = Arrays.asList(1, 0.5d, 3);
assert avg(nums) == 1.5d; // still fine

But neither of them compiles anymore:

List<Integer> ints = Arrays.asList(1, 2, 3);
assert avg(ints) == 2d; // compilation error

List<BigDecimal> bds = Arrays.asList(new BigDecimal(1), new BigDecimal(2), new BigDecimal(3));
assert avg(bds) == 2d; // compilation error

Which means that when you write code which gets values out of a container object (e.g. List), using wildcards with extends provides more flexibility. In other words, your code (or method) will be able to accept a wider range of parameters if wildcards with extends are used appropriately.

The limitation imposed by the use of wildcards with extends is that you won’t be able to put any value except for null through a variable which uses extends. For example:

List<? extends Number> list_extendsNumber = new ArrayList<Integer>();
list_extendsNumber.add(null); // compiles; null is the only exception
list_extendsNumber.add(1); // compilation error

Why? Remember that we can assign any List whose type parameter is a subtype of Number. For example, you can also assign a List whose type parameter is BigDecimal to the list_extendsNumber variable. In that case adding an Integer to the List should be invalid since an Integer is not a BigDecimal. The compiler prevents it from happening thanks to generics and wildcards. Adding a null is fine since null is not tied to a particular type.

Wildcards with super

While wildcards with extends make code which gets a value more flexible, wildcards with super provide flexibility to code which puts a value into a container object. Let’s consider the following variable definition:

List<? super Integer> list_superInteger;

It means a List which contains elements whose type is a supertype of Integer. For example, we can assign the following instances into the variable:

list_superInteger = new ArrayList<Integer>();
list_superInteger = new ArrayList<Number>(); // Number is a supertype of Integer
list_superInteger = new ArrayList<Object>(); // Object is a supertype of Integer

But the following doesn’t compile:

list_superInteger = new ArrayList<String>(); // compilation error; String is not a supertype of Integer

What we can see from the example above is that it’s guaranteed that the List which is assigned to the list_superInteger variable can accept an Integer. We can use it for putting a value into it. For example:

// It's guaranteed that it can accept an Integer
list_superInteger.put(123);

Or you can write some method with it like this:

void addInts(List<? super Integer> ints) {
    Collections.addAll(ints, 1, 2, 3);
}

The method can accept any of the following:

List<Integer> ints = new ArrayList<>();
addInts(ints);
assert ints.toString().equals("[1, 2, 3]");

List<Number> nums = new ArrayList<>();
addInts(nums);
assert nums.toString().equals("[1, 2, 3]");

List<Object> objs = new ArrayList<>();
addInts(objs);
assert objs.toString().equals("[1, 2, 3]");

But the following doesn’t compile:

List<String> strs = new ArrayList<>();
addInts(strs); // compilation error; List<String> cannot accept an Integer

If we didn’t use the wildcard, the method would be less flexible. Let’s say now we have this without a wildcard:

void addInts(List<Integer> ints) {
    Collections.addAll(ints, 1, 2, 3);
}

The following still compiles:

List<Integer> ints = new ArrayList<>();
addInts(ints);
assert ints.toString().equals("[1, 2, 3]");

But the following does not compile anymore:

List<Number> nums = new ArrayList<>();
addInts(nums); // compilation error

List<Object> objs = new ArrayList<>();
addInts(objs); // compilation error

The limitation imposed by the use of wildcards with super is that you will be able to get a value out of a container object only with the Object type. In other words, if you want to get a value out of the list_superInteger variable, this is the only thing you can do:

List<Integer> ints = new ArrayList<>();
ints.add(123);
List<? super Integer> list_superInteger = ints;

Object head = list_superInteger.get(0); // only Object can be used as the type of head
assert head.equals(123);

The following doesn’t compile:

Integer head = list_superInteger.get(0); // compilation error

Why? Remember that we can also assign a List<Number> or a List<Object> to the list_superInteger variable, which means that the only type that we can safely use to get a value out of it is the Object type.

The Get and Put Principle

As we have seen, appropriate use of wildcards provides more flexibility to your code. To summarize when we should use which, there is a good principle to follow:

“The Get and Put Principle: Use an extends wildcard when you only get values out of a structure, use a super wildcard when you only put values into a structure, and don’t use a wildcard when you both get and put.” - Naftalin, M., Wadler, P. (2007). Java Generics And Collections, O’Reilly. p.19

Functional interfaces exemplify this principle. For example, consider a method which accepts objects that implement functional interfaces as follows:

void myMethod(Supplier<? extends Number> numberSupplier, Consumer<? super String> stringConsumer) {
    Number number = numberSupplier.get();
    String result = "I got a number whose value in double is: " + number.doubleValue();
    stringConsumer.accept(result);
}

Supplier is something which you can apply the get priciple to; you only get values out of it. And Consumer is the same for the put principle; you only put values into it.

myMethod() can accept various types of parameters. The user of the method doesn’t necessarily have to pass a Supplier<Number> and a Consumer<String>. The user can also pass a Supplier<BigDecimal> and a Consumer<Object> as follows, but that’s possible only with the use of the wildcards:

Supplier<BigDecimal> bigDecimalSupplier = () -> new BigDecimal("0.5");
AtomicReference<Object> reference = new AtomicReference<>();
Consumer<Object> objectConsumer = reference::set;

myMethod(bigDecimalSupplier, objectConsumer);

assert reference.get().equals("I got a number whose value in double is: 0.5");

Conclusion

We have seen how we can use wildcards, what the benefits of them are and when to use them. To make your code more flexible and reusable, especially when you write a method or a constructor, it’s good practice to think if you can apply wildcards to code which handles an object that has a type parameter. Remembering the get and put principle will be helpful when you do so.