April 12, 2022

    Keep Rules in the Age of Kotlin

    Kotlin is relatively young – only 11 years old – and Android has only had official support for Kotlin for five years. Yet Kotlin is now the go-to language for Android development, and is becoming more popular for desktop and server application development. Kotlin is used by over 60% of Android professional Android developers and it's estimated that 1 million professional software developers use Kotlin as 1 of their 3 primary languages.

    Clearly, we are truly in a new age: the Age of Kotlin.

    ProGuard in the Age of Kotlin

    ProGuard, which turns 20 this year, was originally created in the era of Java 1.4, Java Applets, J2ME and Sun Microsystems. Though Kotlin was designed to be interoperable with Java from the start and designed to run on the JVM, much has changed in 20 years. Kotlin is a much more modern language which has many features that Java does not.

    In my presentation at Droidcon London, Keep Rules in the Age of Kotlin, I laid out some potential security concerns, such as leaking data via Kotlin assertions, and the potential impact of Kotlin metadata on the size of apps. I recommend you watch the full presentation for an overview of the potential hidden pitfalls in the Age of Kotlin as well as some ideas for mitigations.

    In this post, we’ll dive deeper into one section of the Droidcon presentation: the complications of writing keep rules for Kotlin code. You’ll learn why you need to think in terms of Java when writing keep rules for Kotlin code, using examples from the Droidcon London 2021: Keep Rules in the Age of Kotlin ProGuard Playground; you’ll also find the answers to the playground challenges there.

    What is a keep rule anyway and why do we need them?

    ProGuard, and similar tools like R8 and Redex, aim to shrink and optimize code as much as possible. One shrinking and obfuscation technique that can cause issues is the renaming of identifiers, such as class names, methods or fields, to shorter names (which obfuscates names while also making apps smaller). This can lead to problems when using reflection to dynamically access classes or members at runtime.

    Consider the following Java class that contains a field namedmyFieldand uses reflection to access the field by name:

    import java.lang.reflect.*;
    
    public class Reflection {
        private String myField = "Hello World";
    
        public static void main(String[] args) throws Exception {
            Reflection obj = new Reflection();
            Field field = obj.getClass().getDeclaredField("myField");
            System.out.println(field.get(obj)); // prints Hello World
        }
    }

    If a shrinker, like ProGuard, renames the field to some shorter name (a, for example) then getDeclaredField(“myField”)will no longer work.

    Though this case is simple, the field name could be generated dynamically at runtime. For example, libraries such as jackson rely on reflection to map JSON to model classes, so we need a way to tell ProGuard “don’t rename or remove this field”. In other words, we need to tell ProGuard to keep the field!

    So, if we have a fieldmyFieldin a class namedReflectionthat we want to tell ProGuard to keep, we can express this with the following keep rule:

    -keep class Reflection {
        private java.lang.String myField;
    }

    ProGuard introduced keep rules 20 years ago, and they are now the de facto standard in the Android world; in fact, they are also used by R8, Redex and DexGuard. ProGuard keep rules were originally designed for, and inspired by, the Java class file format which explains why the rules look suspiciously like Java signatures (and why they don’t look like Kotlin signatures!).

    ProGuard keep rules also support various wildcards, and alongside other configuration rules make them a very powerful tool for configuring ProGuard. A good place to get started is the ProGuard manual, along with the ProGuard Playground, which helps visualize the classes, methods and fields in your app that are kept or not by keep rules.

    But Kotlin Compiles to Java bytecode, right?

    Kotlin compiles to Java bytecode and is interoperable with Java, so there shouldn’t be any problem writing keep rules … in theory. But Kotlin is not Java – what you write in Kotlin code and what comes out of the Kotlin compiler can seem disconnected. And you need to write keep rules for the latter, not the former, since shrinkers operate on the Java bytecode!

    Top-level declarations: Keeping the Kotlinmainfunction

    In Java, all declarations must be within a class, even a simple “Hello World” application needs a class declaration. Kotlin, in contrast, supports top-level declarations. For example, in the following Kotlin file,Main.kt, a top-level constant, a property and main function are defined:

    const val MESSAGE = "Hello World"
    
    val message: String
        get() = MESSAGE
    
    fun main(): Unit {
        println(message)
    }

    If these declarations are not in a class, how can we write a keep rule for such top-level declarations given that all keep rules need a class name, or at least a wildcard matching a class?

    Of course, since Kotlin compiles to Java, there is an actual class introduced behind the scenes! Such classes (known as file facades) take on the name of the Kotlin file with the suffixKt. In this case, a class file is generated namedMainKt.class.

    So, we now know how to write a keep rule to keep the file facade class:

    -keep class MainKt

    But what about the main function? Surely it’ll be simple?

    If you compare the keep rule below, which seems reasonable when thinking in terms of Kotlin code, with the actual methods in theMainKtclass, you’ll notice that the keep rule doesn’t match eithermainmethod generated by the Kotlin compiler.

    # Incorrect keep rule
    -keep class MainKt {
        kotlin.Unit main();
    }
    $ javap MainKt.class 
    Compiled from "Main.kt"
    public final class MainKt {
      public static final void main();
      public static void main(java.lang.String[]);
    }
    Remember, we need to write keep rules in terms of the compiled Java class files, not Kotlin code. An additional reminder: the main entry point for the JVM has the signaturepublic static void main(java.lang.String[]).

    The Kotlin compiler generates a syntheticmainmethod that has the correct signature, which delegates to the secondmainmethod without parameters (also notice that the Kotlinkotlin.Unittype becomes the Javavoidtype). Therefore, a keep rule that keeps the main Kotlin function would be:
    -keep class MainKt {
        public static void main(java.lang.String[]);
        # or using a wildcard for parameters
        # public static void main(...);
    }
    Challenge

    The constant propertyMESSAGEand the property getter for themessageproperty are also compiled to members in the file facadeMainKtclass. How would you write keep rules for them? Give it a go in the playground below, using the entity tree to get an idea of how the members are compiled from Kotlin code to Java classes.

     

    HINT: Constant properties are compiled to static final fields and the property getter is compiled to a method.

    Kotlin Types

    We’ve already seen how a return type ofkotlin.Unitcompiles to a Javavoidreturn type, but what about when a function has a parameter of typekotlin.Unit? And how about other Kotlin specific types? In the following snippet, we haveUnit,Any,Nothingand some reified type parameters.

    package types
    
    // Unit type as parameter type and return type
    fun process(unit: Unit): Unit { }
    
    // Any type
    fun convertToString(any: Any): String = any.toString()
    
    // Nothing type
    fun fail(message: String): Nothing {
        throw IllegalArgumentException(message)
    }
    
    // Reified types
    open class MyFoo
    inline fun <reified T> foo(foo: T) = foo
    inline fun <reified T : MyFoo> foo(foo: T) = foo

    Unit

    In the example above, theprocessfunction has a return type ofkotlin.Unitand the parameter is also of typekotlin.Unit. The return type will compile to void, but the parameter will remain akotlin.Unitparameter! Taking this in account, a keep rule for this function would look like:

    -keep class types.TypesKt {
        void process(kotlin.Unit);
    }

    Any

    Thekotlin.Anyclass is the root of the Kotlin class hierarchy, just likejava.lang.Objectis the root of the Java class hierarchy. Thekotlin.Anytype is compiled to thejava.lang.Objecttype, so the keep rule for theconvertToString(Any): Stringfunction is simply:

    -keep class types.TypesKt {
        java.lang.String convertToString(java.lang.Object);
    }

    Nothing

    Kotlin also has a typeNothingthat represents a value that never exists. This is useful, for example, to tell the compiler that a function never returns because it always throws an exception, like thefailfunction in the example above.

    So if it never returns, the compiled Java return type would bevoid, right? Well … almost! Under the hood, the Java method actually returns the wrapper typejava.lang.Void! So a keep rule would look something like:

    -keep class types.TypesKt {
        java.lang.Void fail(java.lang.String);
    }
    Challenge

    How would you write keep rules for thefoofunctions that have reified type parameters in the example? Try writing a keep rule for thefoofunctions in the playground below, using the entity tree to discover what the compiledtypes.TypesKtclass looks like.

    HINT: The Kotlin compiler may or may not be able to determine the concrete type, depending on how much information it has!

    Companion objects

    Kotlin companion objects are singleton objects tied to a class but not a specific instance of a class. In this way, they provide a similar feature (though not the same) as Java static methods. Companion objects can be either unnamed or named, and they can contain property and function declarations.

    package companion
    
    class Foo {
        // Unnamed companion
        companion object {
            const val CONSTANT = "bar"
            fun foo() = println("foo")
        }
    }
    
    class Bar {
        // Named companion
        companion object MyCompanion {
            fun bar() = println("bar")
        }
    }

    It begs the question: if a companion object is not a class and it has no name, how can we write a keep rule for it?

    As before, we must think in terms of Java class files and not Kotlin code! The important thing to know about companion objects is that they are compiled to Java inner classes using the nameCompanionor their specified name (MyCompanionin theBarexample above).

    As a reminder, the naming convention for compiled inner classes in Java combines both the outer class and inner class names using the dollar symbol as a separator (e.g.OuterClass$InnerClass).

    So the companion class ofcompanion.Foowill be compiled to a class namedcompanion.Foo$Companion. This means that a keep rule to keepcompanion.Foo’s companion would look like this:

    -keep class companion.Foo$Companion
    # or for named companions:
    -keep class companion.Bar$MyCompanion
    Challenge

    How would you write a keep rule for the propertyCONSTANTor the functionfoo? Try to write a keep rule in the playground below, using the entity tree to get an idea of howCONSTANTandfooare compiled to Java entities.

    HINT: Although companion functions are compiled to Java methods in the companion object class, constant properties are compiled to fields in the outer class!

    Object declarations

    We’ve already seen one type of object – the companion – but object declarations allow the general declaration of singleton classes. For example, theLoggerclass in the example below declares a singleton logger and looks similar to how you would create such a class in Java containing static methods for different log levels.

    package logger
    
    // Object declaration
    object Logger {
        fun verbose(message: String) = println(message)
        fun info(message: String) = println(message)
    }
    
    fun useLogger() {
        Logger.verbose("foo")
    }

    As usual, if you want to keep theLoggermethods, you need to know what it looks like when compiled to a Java class file.

    First, the easy part: an object declaration is compiled to a class of the same name.

    Secondly, the functions: they look like they are called like Java static methods (since you call them likeLogger.verbose(“foo”)). This would imply they are compiled to Java static methods. A keep rule would then look something like this:

    -keep class logger.Logger {
        public static void verbose(java.lang.String);
        public static void info(java.lang.String);
    }

    But, this keep rule won’t work! An important thing to know for object declarations is that the methods are instance methods.

    Under the hood, the Java class also contains a fieldINSTANCEwhich holds a singleton instance; all calls to these methods, even though they look like Java static method calls at first glance, are calls to these instance methods on the object in theINSTANCEfield. This means that the keep rules should not include the static modifier:

    -keep class logger.Logger {
        public void verbose(java.lang.String);
        public void info(java.lang.String);
    }
    Challenge

    Did you know that ProGuard can be used to remove logging calls from your release app? Given what you’ve learned so far about writing keep rules, how would you remove the logging calls to theLoggersingleton using-assumenosideeffects?

    HINT: Other ProGuard configuration rules, like-assumenosideeffectshave the same format as the keep rules!

    Extension functions & properties

    Kotlin, unlike Java, allows adding new functionality to existing classes without using inheritance. This is achieved via the use of extension functions and properties – they allow extending any class, including classes from the Kotlin runtime library and third-party libraries. The following example shows extensions on theStringclass:

    // File: extension/Ext.kt
    package extension
    
    
    fun String.foo(): String = "foo"
    // println(“Hello World”.foo()) prints foo
    
    var String.foo: String
        get() = "foo"
        set(value) { /* do something with value */ }
    // println(“Hello World”.foo)  prints foo

    Given that functions or properties can be added to classes that you don’t even have the source code for, how do they get compiled? And how do we write keep rules for them? Like this?

    -keep class java.lang.String {
        java.lang.String foo();
    }

    Of course,java.lang.Stringis a Java library class, and a shrinker won’t (and can’t!) touch it, so writing such a keep rule doesn’t make sense (ProGuard will actually warn you about keep rules for library classes). The important thing to know about extensions is that though they are compiled to Java methods like any other Kotlin function, they have an extra first parameter for the receiver object (which is an instance of the class or property that is being extended).

    For example, the String extension functionfooin the example is compiled to a Java method with the signaturejava.lang.String foo(java.lang.String). Using this knowledge, combined with what we’ve already learned about top-level declarations (and given that the function is in the fileextension/Ext.kt), we can craft a keep rule to keep this extension function:

    -keep class extension.ExtKt {
        public java.lang.String foo(java.lang.String);
    }
    Challenge

    How would you write keep rules for the getter & setters of thefooextension property on theStringclass? Try to write a keep rule in the playground below, using the entity tree to get an idea of how extension properties are compiled to Java entities.

    HINT: The getter has the receiver as its only parameter but the setter will have two parameters: the receiver and the value.

    JVM annotations

    Kotlin was designed with Java interoperability in mind, so it’s natural to call Kotlin code from Java. But there are some differences between Kotlin and Java that require some attention so Kotlin provides a way to tweak the generation of Java code using annotations.

    In the following example, three annotations are used:

    • @JvmName to change the name of a property
    • @JvmOverloads to generate multiple Java methods instead of a single method with all parameters, since Java doesn’t have the concept of default parameters
    • @JvmStatic to tell the compiler to create a static method in the outer class that delegates to the companion function
    package jvm
    
    class MyClass {
        @JvmName("foobar") fun foo() = "bar"
    
        @JvmOverloads
        fun overloaded(a: Int = 1, b: String = "foo") { }
    
        companion object {
            @JvmStatic fun jvmstatic() = "bar"
        }
    }

    Given that@JvmNameis used to change the name a keep rule is straightforward, simply use the name specified in the annotation:

    -keep class jvm.MyClass {
        public java.lang.String foobar();
    }

    For@JvmOverloads, presumably, when keeping this function all overloaded versions should be kept. This is easy to do with the...wildcard, which means “any number and any type of parameters”:

    -keep class jvm.MyClass {
        public void overloaded(...);
    }

    For@JvmStatic, a static method is created in the outerjvm.MyClassclass that delegates to the companion method. A keep rule for this is simply:

    -keep class jvm.MyClass {
        public static java.lang.String jvmstatic();
    }
    Challenge

    The three annotations shown here aren’t the only JVM platform annotations. How do other annotations, such as @JvmField, @JvmMultifileClass, @JvmWildcard etc, affect the generated code and therefore how you write keep rules?

    HINT: Create some samples using these annotations and upload them to the ProGuard Playground: you can then see what the generated entities look like in the entity tree and interactively write keep rules for them.

    Conclusion

    We’ve seen how, in the Age of Kotlin, writing keep rules is not always easy or intuitive. It requires some understanding of how Kotlin code is compiled to Java class files but the task is made a lot easier with the help of the ProGuard Playground.

    Every example in this post, including solutions to the challenges with embedded playgrounds, can be found in the Droidcon London 2021: Keep Rules in the Age of Kotlin Playground – have a look and play around with the keep rules yourself

    Remember: you need to write keep rules in terms of the compiled Java class files, not Kotlin code!

    We also mentioned that writing keep rules is not the only complication brought about by Kotlin – to find out more, watch the full Droidcon London Keep Rules in the Age of Kotlin presentation, and keep an eye out for our upcoming deep dive into the security implications of Kotlin assertions.

    webinar-keep_rules_in_the_age_of_kotlin_

    James Hamilton - Software Engineer

    Do you have any Kotlin keep rule tips of your own?

    Let us know on the Guardsquare community!

    Other posts you might be interested in