Bytecode Manipulation with a Java Agent and Byte Buddy

Editor's Notes

• #2: Hello, everyone! Today, I'd like to talk about manipulating Java bytecode. I'll introduce Java Agent and a library called Byte Buddy.
• #3: Let me briefly introduce myself. I'm Koichi Sakata. I'm a KanJava JUG leader. Kan means Kansai region in Japan. It includes Kyoto. Do you know Kyoto? Kyoto is visited by many tourists. I became a Java champion this year. I work at PONOS Corporation, that is a mobile game company.
• #4: This session is for beginners, not for experts.
• #5: Moving on to our main topic. Java bytecode is generated by the javac compiler. It's in a class file. It's an intermediate representation of Java programs. We can visualize bytecode with javap command.
• #6: What is bytecode manipulation? It's to edit bytecode, such as add, delete and replace bytecode.
• #7: Why do we manipulate bytecode? Because it's fun!
• #8: But not only fun. It's also useful to change code at runtime. For example, Hibernate uses bytecode manipulation to generate entity objects, and Mockito uses it to create mock objects. Of course at runtime.
• #9: It's hard to change bytecode by hand. Because a class file is a binary file and JVM has a verification process for a class file.
• #10: So there are many libraries for bytecode manipulation. Byte Buddy, Byteman, Javassist, cglib, ASM and BCEL. Today I'll introduce Byte Buddy.
• #11: Why Byte Buddy? Byte Buddy has a Easy-to-use API, supports Java 11 already and won a Duke's Choice Award.
• #12: Many major libraries are using Byte Buddy, such as Hibernate, Mockito, Jackson, etc.
• #13: This is a hello world program with Byte Buddy. It creates a new class that extends Object class, and overrides toString() method to return "Hello World". I'll explain the details of this code.
• #14: First, we create a subclass of the Object class and choose the toString() method.
• #15: Next, we intercept the metod to return a fixed string value "Hello World".
• #16: Make method builds a new type. The new type isn't loaded by a classloader yet, so the type is an unloaded class. Then the classloader loads the type. Finally we can get the new loaded class object. It's easy to manipulate bytecode with Byte Buddy. That's the reason why Byte Buddy is so popular.
• #17: OK, I explained all parts of this code.
• #18: Let's run the hello world. This code is a bit different from the previous code. I'll save a generated class to a file and outputs the method return. (run HelloWorld) Let's see the generated file. It is in a temp directory. Byte Buddy names the dynamic class randomly.
• #19: By the way, the demo code used today is available on GitHub. Just access "bitly/oc1bcm"
• #20: Now I'd like to show code in Mockito. Mockito is a famous mocking framework. This method creates a mock class. As you can see, it's similar to our hello world example.
• #21: How to use? You can run bytecode manipulation code directly in your application. But you need to change your code. When you choose the second option, you need to rebuild. So I'd like to introduce Java Agent. It doesn't require the change. It's also detachable.
• #22: What is Java Agent? The word "Agent" invokes the "Mission: Impossible" spy series. Java Agent do espionage activities in your application, like profiling, logging and even changing the behavior of the application itself.
• #23: Java Agent can instrument programs running on the JVM. It's defined as JSR 163. The JVM calls Java Agent methods with proper timing.
• #24: To create a Java Agent, just define one of the these methods or both, premain() and agentmain().
• #25: Premain() method is called by the JVM before the application's main method(). Agentmain() is called when the agent is attached after the JVM has started.
• #26: So with Java Agent, we can run bytecode manipulation code at runtime.
• #27: To create a Java Agent, first, define the Agent Class method. Second, write a MANIFEST.MF file. Third, package the Agent class to a JAR file. Finally, run java command with javaagent option.
• #28: A signature of premain() is "public static void premain()". We can have an opportunity to modify classes before the class is loaded. Instrumentation class in method parameters provides useful methods to modify classes.
• #29: In the manifest file, we specify the Agent class and set the path to the Byte Buddy's JAR file.
• #30: Let's create a simple Java Agent. The Agent class is a Java agent class. It has the premain() method. This is the Main class. Run the main() method with javaagent option. As you can see, premain() executed before main method.
• #31: Let's combine Java Agent with Byte Buddy. I'll show another example. It adds outputting logs before executing a particular method on a particular class.
• #32: I'll do two things. First, define the premain() method in a class. Second, write code in premain() method to replace the class file to output a log. Note that I don't create a subclass.
• #33: This is the Target class. We want to output a log when foo() method is called.
• #34: Next, replace the class with Byte Buddy. Let me explain the details of this code.
• #35: This code uses rebase() method, instead of previous subclass() method. Rebase() method doesn't create a subclass. It replaces the original implementation with new one and saves it under a different method name.
• #36: Byte Buddy provides three method to enhance existing methods. Subclass() creates a subclass and overrides the method. Redefine() replaces the implementation with new one. Rebase() also replaces it and saves the original implementation.
• #37: Note that we can't use a class literal. If we use a class literal for the Target class, the JVM will load the class before changing the class. We can use the TypePool class and its describe() method, to represent the class without the class literal.
• #38: Also, note that when using unloaded classes, we need to specify a ClassFileLocator. It allows to locate a class file. In this example, we simply create a class file locator with ofClassPath() method. It scans the running application's class path for class files.
• #39: This time I'd like to delegate outputting to another class, LoggingInterceptor class.
• #40: To do so, We can use MethodDelegation.to() method. We simply specify a class to delegate. A best match method is chosen automatically by Byte Buddy.
• #41: After the method is invoked, the original call will continue, thanks to andThen() method with SuperMethodCall.INSTANCE. SuperMethodCall.INSTANCE represents the original method.
• #42: This is the interceptor class. This code outputs the intecepted method name.
• #43: In delegated method, various annotations can be used. The Origin annotation is a reference to the original method or constructor. So we can get the method object of the original method.
• #44: Other annotations are shown here. More details can be found in the Byte Buddy's website.
• #45: Go back to the premain(). How do we run that code in Java Agent? Instrument API provides ClassFileTransformer class to change bytecode. To manipulate bytecode in Java Agent, We need to override transform(). Then pass the ClassFileTransformer instance to addTransformer() of the Instrumentation class. The transform() will be called by the JVM before loading.
• #46: Let's go on to the transform() method. We can call the manipulation code in transform() method.
• #47: In parameters, class name is the name to be loaded. Class file buffer is the content of the class file.
• #48: We return the class file content from the transform().
• #49: Finally we can run the demo. Let's see the complete code. (Show LoggingAgent class) (run Target class) OK, foo() is intercepted.
• #50: So we covered the basics of Byte Buddy and Java Agent.
• #51: Of course, there are many features in Byte Buddy. Add fields and methods, manipulate even stacks. Also, we can generate CONDY code with Byte Buddy. Have you ever heard about condy? CONDY means constant dynamic. It defines as JEP 309. It's included from Java 11, but the Java compiler doesn't use CONDY yet. We can try to use CONDY directly with Byte Buddy.
• #52: Acutually Byte Buddy provides AgentBuilder class to simplify Java Agent. We can rewrite the previous Logging Agent with AgentBuilder.
• #53: Just use AgentBuilder class instead of ByteBuddy class, and use AgentBuilder.Transformer instead of ClassFileTransformer. Call install() method at the end.
• #54: Use AgentBuilder, Use AgentBuilder.Transformer and call installOn().
• #55: Then We can use builder object to intercept the method. Let's see the complete code. (show LoggingAgent class) This code is more readable, isn't it? (mvn package & run Target class)
• #56: Additionally, when we use AgentBuilder, we can remove javaagent option. Just call ByteBuddyAgent's install() and AgentBuilder's installOnByteBuddyAgent() instead of installOn() method. To use ByteBuddyAgent, we need to add byte-buddy-agent JAR file to the class path.
• #57: Call ByteBuddyAgent.install() and AgentBuidler's installOnByteBuddyAgent().
• #58: (show LoggingMainWithoutAgent class) Run this code without javaagent option. (run LoggingMainWithoutAgent)
• #59: Let's wrap up. Bytecode manipulation is fun, and it's close to us. I'd like to stress this point. Many famous libraries are using bytecode manipulation. As application examples, there are How Swap, which swap classes at runtime, and advanced dynamic proxy.