Devnexus presentation slides

1. A Glance at
The Java Performance
Toolbox

2. Java Champion Alumni, Certified Architect
Senior Developer Advocate at Oracle
“DevOps Tools for Java Developers” guest author
ammbra1508 ammbra1508.mastondon.social

3. What does application
performance mean?

4. Performance experienced
by end-users
Computational resources used by the
application

5. Performance Metrics
CPU Usage
Memory Usage
Error rates
User satisfaction (Apdex scores)
Average response time
Request rates
Latency and uptime
Throughput

7. More Questions Using Java Applications in
Containers
• Which JDK tools help you create container images with only what is needed at runtime?
• How to run the JDK tools in containers and proxy their output?
• Which tools help you fine tune the JVM flags and keep application overhead at minimum?
• How to capture performance relevant JVM and application events?
• How to correlate data from JVM monitoring tools with the one from tools like Prometheus,
Grafana?

8. Which tools help you create
container images with only what
is needed at runtime?

10. Tools to build an OCI Compliant Image
• Docker build
• Buildpacks
• Jib
• kaniko
• buildah
• s2i
• And many more J

11. The Good Old Dockerfile
FROM openjdk:10-jre-slim
# Continue with your application deployment
COPY ./target/spring-todo-app.jar /app.jar
COPY entrypoint.sh /entrypoint.sh
RUN groupadd -r appuser && useradd -r -g appuser appuser
RUN chmod +x /entrypoint.sh
USER appuser
CMD ["/entrypoint.sh"] #CMD ["java", "-jar", "app.jar"]

12. The Good Old Dockerfile
FROM openjdk:10-jre-slim
# Continue with your application deployment
COPY ./target/spring-todo-app.jar /app.jar
COPY entrypoint.sh /entrypoint.sh
RUN groupadd -r appuser && useradd -r -g appuser appuser
RUN chmod +x /entrypoint.sh
USER appuser
CMD ["/entrypoint.sh"] #CMD ["java", "-jar", "app.jar"]
Use a small base image
Create images with common layers

13. The Good Old Dockerfile
FROM openjdk:10-jre-slim
# Continue with your application deployment
COPY ./target/spring-todo-app.jar /app.jar
COPY entrypoint.sh /entrypoint.sh
RUN groupadd -r appuser && useradd -r -g appuser appuser
RUN chmod +x /entrypoint.sh
USER appuser
CMD ["/entrypoint.sh"] #CMD ["java", "-jar", "app.jar"]
Install only what is strictly needed

14. JDK tools that shape
efficient Java Runtimes for
Container Images

15. Jlink
JEP 282 “Link time is an opportunity to do whole-world optimizations that are
otherwise difficult at compile time or costly at run-time”
jlink: Command Line Tool to Link Modules
• Modules in a jimage file
• Generate runtime images

16. Jlink Packaging
Legacy JDK image
JDK > 9 generated image
bin jre lib
bin conf ……..
Modular runtime image
jlink

17. Maintain The Good Old Dockerfile with jlink
The runtime stage build
• From a JDK base image
• Create your own custom JRE with jlink

18. Maintain The Good Old Dockerfile with jlink
The runtime stage build
• From a JDK base image
• Create your own custom JRE with jlink
The application stage build
• From an OS base image
• Copy the custom JRE from the runtime stage build
• Copy the artifacts needed by your application
• Run the application

19. The Runtime Stage Build
FROM container-registry.oracle.com/java/openjdk:21-oraclelinux8 as jre-build
# Create a custom Java runtime
RUN $JAVA_HOME/bin/jlink
--add-modules <modules>
--no-man-pages
--no-header-files
--compress=zip-9
--output /javaruntime
jdeps --ignore-missing-deps -q -recursive
--multi-release 21 --print-module-deps
--class-path 'target/libs/*’
target/spring-todo-app.jar

20. The Runtime Stage Build
FROM container-registry.oracle.com/java/openjdk:21-oraclelinux8 as jre-build
# Create a custom Java runtime
RUN $JAVA_HOME/bin/jlink
--add-modules <modules>
--no-man-pages
--no-header-files
--compress=zip-9
--output /javaruntime
Exclude man pages and header files

21. The Runtime Stage Build
FROM container-registry.oracle.com/java/openjdk:21-oraclelinux8 as jre-build
# Create a custom Java runtime
RUN $JAVA_HOME/bin/jlink
--add-modules <modules>
--no-man-pages
--no-header-files
--compress=zip-9
--output /javaruntime
Enable compression of resources zip[0-9]:
zip-0: No compression
zip-6: Default value
zip-9: Best compression

22. The Application Stage Build
FROM oraclelinux:8-slim
ENV JAVA_HOME /usr/java/openjdk-21
ENV PATH $JAVA_HOME/bin:$PATH
COPY --from=jre-build /javaruntime $JAVA_HOME
# Continue with your application deployment
ENV JDK_JAVA_OPTIONS "--enable-preview"
CMD ["/entrypoint.sh"]
Copy the custom JRE

23. The Application Stage Build
FROM oraclelinux:8-slim
ENV JAVA_HOME /usr/java/openjdk-21
ENV PATH $JAVA_HOME/bin:$PATH
COPY --from=jre-build /javaruntime $JAVA_HOME
# Continue with your application deployment
ENV JDK_JAVA_OPTIONS "--enable-preview"
CMD ["/entrypoint.sh"]
Copy the artifacts needed by
your application

24. The Application Stage Build
FROM oraclelinux:8-slim
ENV JAVA_HOME /usr/java/openjdk-21
ENV PATH $JAVA_HOME/bin:$PATH
COPY --from=jre-build /javaruntime $JAVA_HOME
# Continue with your application deployment
ENV JDK_JAVA_OPTIONS "--enable-preview"
CMD ["/entrypoint.sh"] Run the application

25. How about fine tuning
JVM Flags?

26. Conserving Dynamic Footprint by
Minimizing Java Heap Size
Start by looking at the JVM ergonomics

27. Conserving Dynamic Footprint by
Minimizing Java Heap Size
Start by looking at the JVM ergonomics
At initialization of the virtual machine, the entire space for the heap is reserved.
Eg: setting for minimum and maximum heap size: -Xms768m -Xmx768m

28. Conserving Dynamic Footprint by
Minimizing Java Heap Size
Start by looking at the JVM ergonomics
At initialization of the virtual machine, the entire space for the heap is reserved.
Eg: setting for minimum and maximum heap size: -Xms768m -Xmx768m
Minimize Java heap size by lowering the values of the options:
• -XX:MaxHeapFreeRatio (default value is 70%)
-XX:MinHeapFreeRatio (default value is 40%)
• Eg:-XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=70

29. What if the problem is not
the amount of resources
allocated to/by the JVM?

30. Tracking Native Memory With jcmd
FROM container-registry.oracle.com/java/openjdk:21-oraclelinux8 as jre-build
# Create a custom Java runtime
RUN $JAVA_HOME/bin/jlink
--add-modules <modules, jdk.jcmd>
--no-man-pages
--no-header-files
--compress=zip-9
--output /javaruntime
Add jdk.jcmd module

31. Tracking Native Memory With jcmd
WARNING: Overusing jcmd to send diagnostic commands can affect the performance of the VM.

32. Tracking Native Memory With jcmd
WARNING: Overusing jcmd to send diagnostic commands can affect the performance of the VM.
Add -XX:NativeMemoryTracking={off|summary|detail} to JDK_JAVA_OPTIONS
• kubectl set env deployment/app JDK_JAVA_OPTIONS=“-XX:NativeMemoryTracking=summary”
• Please consider that having NMT enabled can add application performance overhead.

33. Tracking Native Memory With jcmd
WARNING: Overusing jcmd to send diagnostic commands can affect the performance of the VM.
Add -XX:NativeMemoryTracking={off|summary|detail} to JDK_JAVA_OPTIONS
Create a native memory baseline
kubectl exec pod_name –it --/bin/bash -c "jcmd llvmid VM.native_memory baseline"

34. Tracking Native Memory With jcmd
WARNING: Overusing jcmd to send diagnostic commands can affect the performance of the VM.
Add -XX:NativeMemoryTracking={off|summary|detail} to JDK_JAVA_OPTIONS
Create a native memory baseline
kubectl exec pod_name –it --/bin/bash -c "jcmd llvmid VM.native_memory baseline"
Create a native memory summary diff
kubectl exec pod_name –it --/bin/bash -c "jcmd llvmid VM.native_memory summary.diff"

35. Performance and
Resource Consumption
Overview information about the Java VM and
monitored values.
Information about memory use.
Information about thread use.
Information about class loading.
Information about the Java VM.
Information about MBeans

36. Get Statistics About
Running Java Processes

37. Obtain Statistics with jstat and jmap
Obtain statistics about garbage collectors by running jstat
kubectl exec pod_name –it --/bin/bash -c "jstat -gcutil llvmid 500 50"

38. Obtain Statistics with jstat and jmap
Obtain statistics about garbage collectors by running jstat
kubectl exec pod_name –it --/bin/bash -c "jstat -gcutil llvmid 500 50"
Print heap summaries
kubectl exec pod_name –it --/bin/bash -c "jmap -histo llvmid"

39. Capture Performance
Relevant JVM And
Application Events

40. When to Use Profiling
“To consume the data today, a user must start a recording, stop it, dump the
contents to disk and then parse the recording file. This works well for
application profiling, where typically at least a minute of data is being
recorded at a time, but not for monitoring purposes.“
source: JEP 349

41. FROM container-registry.oracle.com/java/openjdk:21-oraclelinux8 as jre-build
# Create a custom Java runtime
RUN $JAVA_HOME/bin/jlink
--add-modules <modules, jdk.jfr>
--no-man-pages
--no-header-files
--compress=zip-9
--output /javaruntime
Add JFR To Your Container Image
Add jdk.jfr module

42. How to Use JFR
Enable JFR in JDK_JAVA_OPTIONS
kubectl set env deployment/app JDK_JAVA_OPTIONS=“--XX:StartFlightRecording:..”
For example, start a time fixed recording with default profile
-XX:StartFlightRecording=delay=10s,duration=10m,name=Default,settings=default
Use the diagnostic command:
kubectl exec pod_name –it --/bin/bash -c "jcmd 1 JFR.start…"

43. Monitor and React
…..
AppTarget 1
AppTarget 2
AppTarget n
Prometheus server
Pull metrics
Pull metrics
Pull metrics
Gathering metrics
Visualizing metrics
Alerting

44. Streaming selected metrics to your
monitoring service
Where to Use JFR
Streaming

45. Stream selected metrics to your
monitoring service
Send an average, min, max, etc of a metric
Where to Use JFR
Streaming

46. Stream selected metrics to your
monitoring service
Send an average, min, max, etc of a metric
Expose JFR data through other
management APIs
Where to Use JFR
Streaming

47. Make your settings by extending
jdk.jfr.SettingControl
Recording Custom
JFR Events

48. Make your settings by extending
jdk.jfr.SettingControl
Create your custom event by extending
jdk.jfr.Event
Recording Custom
JFR Events

49. Recording Custom
JFR Events
Make your settings by extending
jdk.jfr.SettingControl
Create your custom event by extending
jdk.jfr.Event
Register the custom settings in your custom
event via
@jdk.jfr.SettingsDefinition

50. Recording Custom Events in Spring Boot
(example)
Capture the custom JFR events in a dedicated filter and register it

51. Recording Custom Events in Spring Boot
(example)
Listen the custom JFR events and record their duration

52. Thank You!
https://github.com/ammbra/performance-glance

53. Useful Links
• Article https://www.javaadvent.com/2022/12/a-sneak-peek-at-the-java-performance-toolbox.html
• Tools https://docs.oracle.com/en/java/javase/21/docs/specs/man/index.html
• jlink https://dev.java/learn/jlink/
• Troubleshoot Memory Leaks: https://docs.oracle.com/en/java/javase/21/troubleshoot/troubleshooting-memory-
leaks.html
• Innovative JFR use by @gunnarmorling: https://www.morling.dev/blog/finding-java-thread-leaks-with-jdk-flight-
recorder-and-bit-of-sql/
• @gunnarmorling article on custom JDK Flight Recorder Events
• Great intro by @BillyKorando https://www.youtube.com/watch?v=K1ApBZGiT-Y
• Stack walker episode by @BillyKorando: https://inside.java/2023/05/14/stackwalker-02/