Lock Module
This module provides distributed locking mechanisms using Redis as the backend.
Overview
The lock module implements distributed locking patterns for coordinating access to shared resources in a distributed environment. It offers two primary implementations:
Redis-based locks (RedisLock, RedisLockPool): Distributed locks using Redis
Thread-local locks (ThreadLock, ThreadLockPool): In-memory locks for single-process applications
Key Features
Automatic Expiry: Locks automatically expire after a configured timeout
Active Update: Lock owners must periodically update locks to maintain ownership
Reentrant Locks: Same owner can re-acquire its own locks with rlock
Lock Pools: Manage collections of locks for allocation and tracking
Implementation Notes
Redis-based locks require a single Redis instance (not compatible with Redis Cluster)
Locks have a timeout to prevent deadlocks in case of client failures
Thread identification is used to determine lock ownership
RedisLock
- class redis_allocator.lock.RedisLock[source]
Redis-based lock implementation.
Uses standard Redis commands (SET with NX, EX options) for basic locking and Lua scripts for conditional operations (set/del based on value comparison).
- redis
StrictRedis client instance (must decode responses).
- Type:
- key_status(key, timeout=120)[source]
Get the status of a key.
- Parameters:
- Returns:
The current status of the key.
- Return type:
- update(key, value='1', timeout=120)[source]
Lock a key for a specified duration without checking if the key is already locked.
- rlock(key, value='1', timeout=120)[source]
Try to lock a key for a specified duration.
When the value is the same as the current value, the function will return True.
- deleq(key, value)
Deletes a key when the comparison value is equal to the current value.
- delge(key, value)
Deletes a key when the comparison value is greater than or equal to the current value.
- delgt(key, value)
Deletes a key when the comparison value is greater than the current value.
- delle(key, value)
Deletes a key when the comparison value is less than or equal to the current value.
- dellt(key, value)
Deletes a key when the comparison value is less than the current value.
- delne(key, value)
Deletes a key when the comparison value is not equal to the current value.
- seteq(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is equal to the current value.
- setge(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is greater than or equal to the current value.
- setgt(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is greater than the current value.
- setle(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is less than or equal to the current value.
- setlt(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is less than the current value.
- setne(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is not equal to the current value.
RedisLockPool
- class redis_allocator.lock.RedisLockPool[source]
Manages a collection of RedisLock keys as a logical pool.
Uses a Redis Set (<prefix>|<suffix>|pool) to store the identifiers (keys) belonging to the pool. Inherits locking logic from RedisLock.
Provides methods to add (extend), remove (shrink), replace (assign), and query (keys, __contains__) the members of the pool. Also offers methods to check the lock status of pool members (_get_key_lock_status).
- __init__(redis, prefix, suffix='lock-pool', eps=1e-06)[source]
Initialize a RedisLockPool instance.
- __iter__()
Iterate over the keys in the pool.
- __len__()
Get the number of keys in the pool.
- deleq(key, value)
Deletes a key when the comparison value is equal to the current value.
- delge(key, value)
Deletes a key when the comparison value is greater than or equal to the current value.
- delgt(key, value)
Deletes a key when the comparison value is greater than the current value.
- delle(key, value)
Deletes a key when the comparison value is less than or equal to the current value.
- dellt(key, value)
Deletes a key when the comparison value is less than the current value.
- delne(key, value)
Deletes a key when the comparison value is not equal to the current value.
- health_check()
Check the health status of the keys in the pool.
- is_locked(key)
Check if a key is locked.
- items_locked_status()
Get (key, lock_status) pairs for all keys in the pool.
- key_status(key, timeout=120)
Get the status of a key.
- Parameters:
- Returns:
The current status of the key.
- Return type:
- lock(key, value='1', timeout=120)
Try to lock a key for a specified duration.
- lock_value(key)
Get the value of a locked key.
- rlock(key, value='1', timeout=120)
Try to lock a key for a specified duration.
When the value is the same as the current value, the function will return True.
- seteq(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is equal to the current value.
- setge(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is greater than or equal to the current value.
- setgt(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is greater than the current value.
- setle(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is less than or equal to the current value.
- setlt(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is less than the current value.
- setne(key, value, set_value=None, ex=None)
Sets a new value when the comparison value is not equal to the current value.
- unlock(key)
Forcefully release a key without checking if the key is locked.
- update(key, value='1', timeout=120)
Lock a key for a specified duration without checking if the key is already locked.
LockStatus
- class redis_allocator.lock.LockStatus[source]
Enumeration representing the status of a Redis lock.
The LockStatus enum defines the possible states of a Redis lock:
FREE: The lock is not being used.
UNAVAILABLE: The lock is being used by another program, or it has been marked as unavailable for a certain period of time.
LOCKED: The lock is being used by the current program.
ERROR: The lock is being used permanently, indicating a potential issue with the program.
- FREE = 0
- UNAVAILABLE = 1
- LOCKED = 2
- ERROR = 4
- __new__(value)
Base Classes
- class redis_allocator.lock.BaseLock[source]
Abstract base class defining the interface for lock implementations.
- __init__(eps=1e-06)[source]
Initialize a BaseLock instance.
- Parameters:
eps (float) – Epsilon value for floating point comparison.
- abstract key_status(key, timeout=120)[source]
Get the status of a key.
- Parameters:
- Returns:
The current status of the key.
- Return type:
- abstract update(key, value='1', timeout=120)[source]
Lock a key for a specified duration without checking if the key is already locked.
- abstract rlock(key, value='1', timeout=120)[source]
Try to lock a key for a specified duration.
When the value is the same as the current value, the function will return True.
- setgt(key, value, set_value=None, ex=None)[source]
Sets a new value when the comparison value is greater than the current value.
- setlt(key, value, set_value=None, ex=None)[source]
Sets a new value when the comparison value is less than the current value.
- setge(key, value, set_value=None, ex=None)[source]
Sets a new value when the comparison value is greater than or equal to the current value.
- setle(key, value, set_value=None, ex=None)[source]
Sets a new value when the comparison value is less than or equal to the current value.
- seteq(key, value, set_value=None, ex=None)[source]
Sets a new value when the comparison value is equal to the current value.
- setne(key, value, set_value=None, ex=None)[source]
Sets a new value when the comparison value is not equal to the current value.
- delgt(key, value)[source]
Deletes a key when the comparison value is greater than the current value.
- delge(key, value)[source]
Deletes a key when the comparison value is greater than or equal to the current value.
- delle(key, value)[source]
Deletes a key when the comparison value is less than or equal to the current value.
- class redis_allocator.lock.BaseLockPool[source]
Abstract base class defining the interface for lock pool implementations.
A lock pool manages a collection of lock keys as a group, providing methods to track, add, remove, and check lock status of multiple keys.
Thread-Local Implementations
- class redis_allocator.lock.ThreadLock[source]
In-memory, thread-safe lock implementation conforming to BaseLock.
Simulates Redis lock behavior using Python’s threading.RLock for concurrency control and a defaultdict to store lock data (value and expiry timestamp). Suitable for single-process scenarios or testing.
- _locks
defaultdict storing LockData(value, expiry) for each key.
- _lock
threading.RLock protecting access to _locks.
- __init__(eps=1e-06)[source]
Initialize a ThreadLock instance.
- Parameters:
eps (float) – Epsilon value for floating point comparison.
- update(key, value='1', timeout=120)[source]
Lock a key for a specified duration without checking if already locked.
- class redis_allocator.lock.ThreadLockPool[source]
In-memory, thread-safe lock pool implementation.
Manages a collection of lock keys using a Python set for the pool members and inherits the locking logic from ThreadLock.
- _pool
Set containing the keys belonging to this pool.
- _lock
threading.RLock protecting access to _locks and _pool.
Usage Patterns
Basic Lock Usage
lock = RedisLock(redis_client, "app", "lock")
# Acquire lock with thread ID as value and 60-second timeout
if lock.lock("resource-1", value=thread_id, timeout=60):
try:
# Resource is locked for 60 seconds
# Process the resource...
# Extend lock timeout by updating it
lock.update("resource-1", value=thread_id, timeout=60)
# Continue processing...
finally:
# Always release the lock when done
lock.unlock("resource-1")
Reentrant Lock Usage
# First acquire the lock normally
if lock.lock("resource-1", value=thread_id, timeout=60):
try:
# Later, the same thread can re-acquire the lock
if lock.rlock("resource-1", value=thread_id):
# Process the resource again...
pass
finally:
# Release the lock once
lock.unlock("resource-1")
Using Lock Pools
pool = RedisLockPool(redis_client, "app", "pool")
# Add resources to the pool
pool.extend(["resource-1", "resource-2"])
# Lock a specific resource
if pool.lock("resource-1"):
try:
# Use the resource
pass
finally:
# Release the lock
pool.unlock("resource-1")
Simplified Lock Flow
This diagram illustrates the typical sequence for acquiring, updating, and releasing a lock using RedisLock.
sequenceDiagram
participant Client
participant RedisLock
participant Redis
Client->>RedisLock: lock("key", "id", timeout=60)
note right of Redis: Attempts SET key id NX EX 60
RedisLock->>Redis: SET key id NX EX 60
alt Lock Acquired
Redis-->>RedisLock: OK
RedisLock-->>Client: True
Client->>RedisLock: update("key", "id", timeout=60)
note right of Redis: Refreshes expiry: SET key id EX 60
RedisLock->>Redis: SET key id EX 60
Redis-->>RedisLock: OK
Client->>RedisLock: unlock("key")
note right of Redis: Removes lock: DEL key
RedisLock->>Redis: DEL key
Redis-->>RedisLock: 1 (deleted)
RedisLock-->>Client: True
else Lock Not Acquired (Already Locked)
Redis-->>RedisLock: nil
RedisLock-->>Client: False
end