What Is the Impact of Big Keys or Hot Keys?

Category

Impact

Big key

Instance specifications fail to be modified.

Specification modification of a Redis Cluster instance involves rebalancing (data migration between nodes). Redis has a limit on key migration. If the instance has any single key bigger than 512 MB, the modification will fail when big key migration between nodes times out. The bigger the key, the more likely the migration will fail.

Data migration fails.

During data migration, if a key has many elements, other keys will be blocked and will be stored in the memory buffer of the migration ECS. If they are blocked for a long time, the migration will fail.

Cluster shards are unbalanced.

  • The memory usage of shards is unbalanced. For example, if a shard uses a large memory or even uses up the memory, keys on this shard are evicted, and resources of other shards are wasted.

  • The bandwidth usage of shards is unbalanced. For example, flow control is repeatedly triggered on a shard.

Latency of client command execution increases.

Slow operations on a big key block other commands, resulting in a large number of slow queries.

Flow control is triggered on the instance.

Frequently reading data from big keys exhausts the outbound bandwidth of the instance, triggering flow control. As a result, a large number of commands time out or slow queries occur, affecting services.

Master/standby switchover is triggered.

If the high-risk DEL operation is performed on a big key, the master node may be blocked for a long time, causing a master/standby switchover.

Hot key

Cluster shards are unbalanced.

If only the shard where the hot key is located is busy processing service queries, there may be performance bottlenecks on a single shard, and the compute resources of other shards may be wasted.

CPU usage surges.

A large number of operations on hot keys may cause high CPU usage. If the operations are on a single cluster shard, the CPU usage of the shard where the hot key is located will surge. This will slow down other requests and the overall performance. If the service volume increases sharply, a master/standby switchover may be triggered.

Cache breakdown may occur.

If Redis cannot handle the pressure on hot keys, requests will hit the database. The database may break down as its load increases dramatically, affecting other services.

last updated: 2024-05-19 03:18