Kotlin: A tale of cyclical static initialization

Take this seemingly simple piece of Kotlin code, which we will run on the JVM platform:

Seems like it might be a good guessing strategy for a multiple choice test (assuming a computer didn’t generate it, of course). Since all good code comes with tests, let’s make sure our guesser meets those criteria:

Wonderful, it printed true! If only testing all code was this easy. Just to be safe, let’s test the other way too!

Just as before, it prints… wait what? false!? What’s going on here?

Let’s first pull apart these statements, to make the expression evaluation order explicit. For the way that seemed to work as expected (Choices.defaultChoice == Choices.C), Choices.defaultChoice is evaluated first, so let’s try that one that first:

Alright, that one makes some sense… what about the other way?

null. Looks like our good friend null is showing up to ruin our day again. But why? And why does changing the order matter?

To answer both, we’ll have to dive into… the bytecode. The Kotlin code that you write isn’t what’s being executed directly. If you are working on a JVM platform (like Android), the Kotlin first gets compiled into JVM bytecode. Usually you don’t have to worry about what gets generated, but Intellij IDEA and Android Studio both have an extremely helpful Kotlin bytecode viewer, which you can also decompile back into Java. It is here where we will find our answer.

First, let’s strip away the unused objects:

Running this through the bytecode viewer, decompiling to Java, and removing the Kotlin metadata for brevity, the above Kotlin code is equivalent to this:

The two static blocks of code are responsible for the behavior we are seeing, depending on how they are executed. Referring to the detailed JVM specification for the initialization of classes¹, let’s take a look at what happens in each case.

(Choices.Companion.getDefaultChoice() == Choices.C.INSTANCE) == true

Choices.Companion.getDefaultChoice() is evaluated first. Because Companion is a nested static class of Choices, Choices has to be initialized before we can run the method. Thus, that static block starts to run first:

defaultChoice = Choices.C.INSTANCE;

Now, before we can access C.INSTANCE, Choices.C needs to be initialized. Because C is a subclass of Choices, that means we need to initialize Choices first— but we’re already in the process of initializing Choices! Because static blocks must only happen exactly once, we notice that this is a recursive initialization of Choices, and the current thread is initializing Choices. Therefore, we proceed to initializing Choices.C and its static block:

Choices.C var0 = new Choices.C();
INSTANCE = var0;

This proceeds as expected, and INSTANCE is initialized to a non-null instance of Choices.C.

Since the initialization of Choices.C is done, we return to finishing the static block of Choices, and we assign the newly created Choices.C instance to defaultChoice.

Finally, we’ve finished initializing Choices, so we can run the getDefaultChoice() method:

public final Choices.C getDefaultChoice() {
    return Choices.defaultChoice;
}

After a ton of steps, we finally return our new, non-null instance of Choices.C. Whew!

Moving on to the second call, on the right side of the equation: Choices.C.INSTANCE. Because Choices.C is already initialized, no static initialization is necessary. INSTANCE returns the same instance of Choices.C as before, and thus the equals returns true.

(Choices.C.INSTANCE == Choices.Companion.getDefaultChoice()) == false

So what happens differently when switching the evaluation order?

Choices.C.INSTANCE is evaluated first, which means we need to initialize Choices.C first.

We mark that we are initializing Choices.C, and like above, since Choices.C is a subclass of Choices, we have to initialize Choices before we can initialize Choices.C. The key difference here is that we weren’t already initializing Choices. So now we actually get to evaluate Choices’s static block while initializing Choices.C:

defaultChoice = Choices.C.INSTANCE;

Now, this static block refers to Choices.C, so we need to initialize Choices.C first. But we already are! So we continue running Choice’s static block, and set defaultChoice to whatever is in Choice.C.INSTANCE. At this point, we haven’t been able to set it yet, so we get null! And that’s where the null came from.

We’ve finished initializing Choices, so now we can run the static block for Choices.C:

Choices.C var0 = new Choices.C();
INSTANCE = var0;

Uneventfully, this sets INSTANCE as we’d expect, and Choices.C.INSTANCE expression evaluates to our freshly created non-null instance of Choices.C.

Now, when we evaluate the right side, Choices.Companion.getDefaultChoice(), we find that Choices is already initialized. Great! We return the value of Choices.defaultChoice… and even though that method getDefaultChoice() is marked as @NotNull by the Kotlin metadata, it returns the null that was assigned to it previously, leading to our expression evaluating to false.

The Solution

Taking a step back, it seems like this should be a bug. Surely, it should be possible to figure out what the “correct” value should be?

Unfortunately, there is no general solution to handle cyclical initialization², and its pretty easy to come up with examples as to why:

This will compile, but is completely non-sensical.

Since static code could run anything it wants or do complicated logic, the best we might be able to hope for is some sort of lint or compiler warning, which might be able to suggest a fix for extremely simple situations for the one above.

Programming languages are extremely powerful abstractions for instructing our computers, and their inherent ability to hide away the gory details is the reason why we don’t write everything in machine code anymore. However, our code still eventually runs on something, so we can’t pretend to forget those translation steps, and its helpful to be able to dive deeper to figure out what’s really going on.

It’s also worth noting that this behavior is specific to the JVM platform, and in my testing this issue doesn’t happen with the defaultChoice case above with Kotlin Native or the JavaScript platform:

For the actual in-practice solution for the JVM, we can prevent the cyclical weirdness for this simple case by just making defaultChoice use a getter:

Alex works for Livefront, where we aren’t afraid to look at the bytecode.

¹: https://docs.oracle.com/javase/specs/jls/se14/html/jls-12.html#jls-12.4

²: https://stackoverflow.com/questions/47648689/sealed-classs-objects-mysteriously-becoming-null-when-referenced-by-other-compa
Also linked in that question:
https://youtrack.jetbrains.com/issue/KT-21614#comment=27-2583294
https://www.reddit.com/r/Kotlin/comments/7hoytl/kotlin_team_ama_ask_us_anything/dqssfg9/