Posts
2022
OpenJDK JVM Deep Dive: Java Memory Model - A Comprehensive Guide to Concurrency and Memory Barriers
·12058 words·57 mins
A deep dive into Java Memory Model (JMM) from specification to implementation, covering memory barriers, CPU reordering, and Java 9+ VarHandle APIs. Learn about coherence, causality, consensus, and how volatile, final, and other synchronization mechanisms work under the hood with practical jcstress examples.
The Hidden Performance Killer: Why Code Location in Logs Can Destroy Your Microservice Performance
·898 words·5 mins
Discover how enabling code location in logs can cause severe CPU performance issues in microservices, especially reactive applications. This deep-dive analysis reveals the hidden costs of stack walking in Log4j2 and provides actionable solutions for high-throughput systems.
Troubleshooting Memory Issues After Spring Boot Upgrade: A Deep Dive into ResolvableType Object Creation
·1180 words·6 mins
An investigation into excessive memory allocation and YoungGC frequency after upgrading to Spring Boot 2.4.6 + Spring Cloud 2020.0.x, revealing how BeanUtils.copyProperties creates massive ResolvableType objects without caching in Spring 5.3.x versions.
MySQL Optimizer Statistics: Why Your Queries Choose the Wrong Index
·1600 words·8 mins
A deep dive into MySQL’s InnoDB optimizer statistics and how sampling inaccuracies can lead to poor index selection, causing dramatic performance differences between similar queries. Learn practical solutions to prevent slow SQL queries caused by optimizer misjudgments.
Configuring Spring Data Redis with Lettuce for Effective Pipeline Operations
·1158 words·6 mins
A comprehensive guide on how to properly configure Spring Data Redis with Lettuce to enable pipeline functionality. Learn about connection sharing, AutoFlushCommands, and PipeliningFlushPolicy configurations to optimize your Redis batch operations and reduce network round-trip time.
2021
Spring Data Redis Connection Leak Mystery: When Your Microservice Goes Rogue
·1820 words·9 mins
A production incident investigation revealing how Spring Data Redis + Lettuce can leak connections when mixing SessionCallback and executeWithStickyConnection operations. Deep dive into connection management mechanisms, JFR analysis techniques, and practical solutions to prevent your Redis connection pool from becoming a black hole.
Gateway Avalanche Crisis: How Synchronous Redis Calls Nearly Brought Down Our System
·1662 words·8 mins
A deep dive into a production incident where our Spring Cloud Gateway experienced cascading failures due to blocking Redis operations. Learn how synchronous API calls in reactive environments can cause thread starvation, leading to health check failures and system-wide avalanches, plus the complete solution using async patterns.
A Peculiar Bug Hunt: When Exceptions Lose Their Voice
·1195 words·6 mins
A deep dive into a mysterious production issue where exception logs mysteriously disappeared, leading us through Arthas debugging, Log4j2 internals, and the discovery that an exception’s getMessage() method was itself throwing exceptions due to Guava-Guice version incompatibility.
A Hidden Production Issue Discovered Through SQL Optimization
·1101 words·6 mins
When our operations team brought us a complex SQL query that was taking forever to execute, we thought it was just a performance issue. Little did we know, this investigation would uncover a deeply hidden character encoding mismatch that had been silently causing full table scans in our production database.
Troubleshooting a SSL Performance Bottleneck Using JFR
·395 words·2 mins
In-depth analysis of a microservice performance issue with CPU spikes and database connection anomalies. Through JFR profiling, we discovered the root cause was Java SecureRandom blocking on /dev/random and provide solutions using /dev/urandom.