Git Commit History

This page provides a detailed history of all changes to the RedisAllocator project, automatically generated from Git commit messages.

Quick Summary

• docs: clarify docker deployment scope by Qingrui Hu at 2026-06-30 05:36:52

• bump: version 0.6.1 → 0.7.0 by Qingrui Hu at 2026-06-29 18:55:07

• docs: add repository guidance and diagnostics plan by Qingrui Hu at 2026-06-29 18:54:56

• docs: clarify orphan lock recovery by Qingrui Hu at 2026-06-29 18:49:12

• test: stabilize assign performance benchmark by Qingrui Hu at 2026-06-29 18:12:47

Full History

• docs: clarify docker deployment scope by Qingrui Hu at 2026-06-30 05:36:52

• bump: version 0.6.1 → 0.7.0 by Qingrui Hu at 2026-06-29 18:55:07

• docs: add repository guidance and diagnostics plan by Qingrui Hu at 2026-06-29 18:54:56

• docs: clarify orphan lock recovery by Qingrui Hu at 2026-06-29 18:49:12

• test: stabilize assign performance benchmark by Qingrui Hu at 2026-06-29 18:12:47

• test: expand allocator performance contracts by Qingrui Hu at 2026-06-29 18:06:39

• style: satisfy diagnostics lint by Qingrui Hu at 2026-06-29 17:42:30

• docs: document allocator diagnostics by Qingrui Hu at 2026-06-29 17:40:50

• feat: add live diagnostics dashboard UI by Qingrui Hu at 2026-06-29 17:38:10

• feat: serve allocator diagnostics dashboard by Qingrui Hu at 2026-06-29 17:35:58

• feat: add allocator diagnostics CLI by Qingrui Hu at 2026-06-29 17:34:19

• feat: collect allocator diagnostics snapshots by Qingrui Hu at 2026-06-29 17:33:00

• feat: add allocator diagnostics invariants by Qingrui Hu at 2026-06-29 17:29:19

• feat: add diagnostics snapshot models by Qingrui Hu at 2026-06-29 17:27:48

• docs: design allocator diagnostics dashboard by Qingrui Hu at 2026-06-29 16:59:15

• feat: improve allocator cleanup and task timeouts by Qingrui Hu at 2026-06-29 16:06:06

• bump: version 0.6.0 → 0.6.1 by Qingrui Hu at 2026-04-28 11:30:21

• refactor: extract sentinels, derive prefixes from helpers, trim narrative comments by Qingrui Hu at 2026-04-28 10:35:42

  Code-review pass over the recent allocator hardening + test infrastructure
  work. Changes are zero-behavior — just simplification:
  
    - Extract ``ALLOCATED_SENTINEL`` and ``POOL_VALUE_SEP`` as module-level
      constants in redis_allocator/allocator.py and inject them into the
      Lua scripts via the existing f-string template. Both the Python
      ``_validate_keys`` path and ``tests/_bughunt_helpers.py`` now import
      the constants instead of re-hardcoding ``"#ALLOCATED"`` and ``"||"``.
  
    - In ``tests/_bughunt_helpers.py`` derive the lock-key and soft-bind
      prefixes from ``allocator._key_str("")`` and
      ``allocator._soft_bind_name("")`` rather than reformatting the
      ``prefix|suffix:`` shape inline (would silently drift if the producer
      changed its key layout). Switch ``_SNAPSHOT_SCRIPT_CACHE`` to a
      ``WeakKeyDictionary`` so cached Script entries are released with
      their Redis client instead of pinning by ``id()``.
  
    - ``tests/test_allocator_forensics.py``: bound the trace via
      ``collections.deque(maxlen=TRACE_TAIL + 200)`` so long runs cannot
      grow it without limit. Trim the file/test docstrings to drop the
      historical narrative.
  
    - Trim retrospective / "this fixes the user-reported X" comments out
      of allocator.py and the test files. The bug-history belongs in
      BUGHUNT_REPORT.md and the commit log, not in production source.
      The cleanup checklist in tests/conftest.py is reduced to one
      sentence pointing at the upstream PR.
  
  Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
  

• docs: surface behavior contracts, testing layout, and recent hardening in README by Qingrui Hu at 2026-04-28 10:09:29

  Three additions to README.md:
  
    - "Behavior Contracts (Important for API Users)" section right after
      Core Features. Spells out the four intentional surprises a caller
      needs to know (timeout<=0 permanent locks, cache_timeout=0 permanent
      soft bindings, assign() not deleting evicted lock keys, extend/assign
      rejecting malformed pool keys with ValueError).
  
    - "Concurrency Safety" subsection after the architecture description,
      explaining the Lua-EVAL atomicity + lock-TTL + GC self-heal model
      so users understand what survives process crashes.
  
    - "Testing" top-level section before Roadmap, with the default unit
      suite, opt-in markers (concurrency / fuzz / benchmark), Docker
      stress driver, and a note about the temporary fakeredis monkey-patch
      bridging upstream PR #473.
  
  Roadmap "Recently Completed" gains three bullets covering the pool
  hardening work, the test infrastructure additions, and the validated
  O(1) performance contract — pointing to BUGHUNT_REPORT.md for detail.
  
  Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
  

• build: relax fakeredis dep + restore HSET-O(1) monkey-patch as upstream-release bridge by Qingrui Hu at 2026-04-28 09:32:16

  Previous commit ab5b8ac dropped the conftest monkey-patch and bumped the
  fakeredis dep to >= 2.36.0 on the assumption that the upstream fix would
  be released. The fix has been merged
  (https://github.com/cunla/fakeredis-py/pull/473, commit 1e1173b on master)
  but no release tag yet contains it — the latest fakeredis on PyPI is
  still 2.35.1, so `pip install` against >= 2.36.0 cannot resolve.
  
  This commit restores the bridge:
    - setup.py: fakeredis[lua] dep relaxed back to >= 2.35.1 (PyPI-resolvable)
    - tests/conftest.py: re-add the idempotent _install_fakeredis_hset_o1_patch
      with a CLEANUP CHECKLIST comment that pinpoints the exact follow-up
      work (delete the block + bump dep + update CLAUDE.md) so it can be done
      in one clean commit when fakeredis releases the fix
    - CLAUDE.md: mention the temporary monkey-patch and link to PR #473
  
  The patch is no-op-equivalent on a future fakeredis where the upstream
  fix is already in place — re-installing the same len(h) logic is
  harmless.
  
  Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
  

• build: bump fakeredis to >=2.36 and drop in-tree HSET monkey-patch by Qingrui Hu at 2026-04-26 16:53:53

  Switch the test dependency from fakeredis 2.20.1 to 2.36.0+. The local
  HSET-O(1) monkey-patch in tests/conftest.py was a workaround for a
  fakeredis upstream bug; the equivalent fix has been prepared as a PR
  against cunla/fakeredis-py and once merged + released will land
  naturally via this version bump.
  
  CLAUDE.md is updated to note that the benchmark suite needs the
  upstream fix to pass; for fresh installs from PyPI today the benchmark
  will report ~12x scaling instead of flat O(1) until the fakeredis PR
  ships. Local development against `pip install -e` of the patched fork
  (or any fakeredis ≥ next-release) keeps benchmark green.
  
  Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
  

• docs: add bug-hunt report summarizing the three investigation rounds by Qingrui Hu at 2026-04-26 07:04:56

  Summarises every bug found, every fix applied, and every behavior
  classified as by-design (with an asserting test pinning the contract).
  
  Round 1 hunted single-process / static bugs: input validation,
  shrink-of-unknown-key, and the three by-design timeout/binding/lock
  contracts.
  
  Round 2 added concurrency tests and revealed that the original
  non-atomic watcher was producing false-positive INV violations under
  load. Replacing it with an atomic Lua-EVAL snapshot eliminated the
  noise and let the real bugs surface as Lua asserts.
  
  Round 3 chased the remaining real concurrency bugs to root cause:
  pop_from_head not maintaining headNext.prev across head removal,
  soft-binding lock-set ordered before chain mutation, and
  extend/assign re-adding keys without clearing residual lock state.
  After these fixes, real-Redis stress (60s/90s/16-worker/0.4s-kill)
  sustains zero invariant violations.
  
  Includes the marker policy, test categories, and a "what was not a
  bug" section so future maintainers don't re-investigate the same
  false positives.
  
  Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
  

• test: add comprehensive bug-hunt, concurrency, fuzz, benchmark, and stress suites by Qingrui Hu at 2026-04-26 07:04:42

  Test infrastructure built up alongside the allocator hardening work:
  
  - tests/_bughunt_helpers.py: PoolSnapshot helpers and check_invariants
    for INV-1..5 (linked-list integrity, pointer well-formedness,
    half-allocated detection, free/locked overlap, soft-binding refs).
    snapshot() uses a single Lua EVAL so head/tail/entries/locks are
    read atomically — eliminates the false-positive violations the
    original non-atomic watcher produced under concurrent load.
  
  - tests/test_allocator_bughunt.py: per-bug regression + by-design
    contract tests (~30 cases). Covers input validation rejections,
    shrink/free no-op, zombie resurrection, and the by-design contracts
    for timeout<=0 permanent locks, cache_timeout=0 permanent bindings,
    and assign() preserving evicted lock keys.
  
  - tests/test_allocator_fuzz.py: random operation sequences across
    parametrized seeds. Exercises serialized EVAL ordering edge cases.
  
  - tests/test_allocator_concurrency.py: multi-thread fakeredis stress
    asserting invariants are preserved under interleaved EVALs.
  
  - tests/test_allocator_forensics.py: same as concurrency but with
    per-method-call tracing so any future violation dumps the offending
    EVAL sequence.
  
  - tests/test_allocator_benchmark.py: O(1) contract for malloc/free
    and per-key extend cost; threshold MAX_SCALE_FACTOR=3.
  
  - tests/stress_pool_corruption.py: standalone Docker-driven stress
    driver (workers + abandoners + atomic watcher + murderer). Not a
    pytest test; run directly when validating against real Redis.
  
  - tests/_perf_real_redis.py: cross-checks the benchmark against a
    real Redis container, demonstrating the allocator's true O(1) and
    the fakeredis non-linearity that the conftest patch addresses.
  
  - tests/_profile_fakeredis.py: localised the upstream fakeredis
    HSET-is-O(n) issue (worth filing upstream).
  
  - tests/conftest.py: monkey-patches fakeredis HashCommandsMixin._hset
    to use len(h) instead of len(h.keys()) — Hash.keys() builds a
    fresh list each call (O(n)), so each HSET was O(n) on the hash
    size, masking the allocator's true O(1) cost on the benchmark.
    The patched _hset is semantically identical (same _expire_keys()
    and same returned count).
  
  - pyproject.toml: registers concurrency / fuzz / benchmark / stress
    pytest markers and sets default addopts to exclude all four.
    Default `pytest` runs only the fast unit suite (~7s); explicit
    `pytest -m <marker>` opts into each category.
  
  Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
  

• fix(allocator): pool key validation, race-driven structural corruption, zombie key resurrection by Qingrui Hu at 2026-04-26 07:04:18

  Eight related fixes that together eliminate the user-reported "pool
  completely corrupted under multi-process load with mid-sequence kills":
  
  1. _validate_keys: reject keys equal to "", "#ALLOCATED", containing
     "||" (the on-disk separator), NUL bytes, or non-str types from
     extend()/assign(). Each pattern previously caused silent linked-list
     corruption (greedy regex mis-parse, sentinel collision, ambiguous
     empty-pointer).
  
  2. pop_from_head: locked-skip and expired branches now run a new
     _advance_head_after_removal() helper that updates headNext.prev = ""
     before recursing. The old code left headNext referencing a key that
     was either HDEL'd (expired branch) or now in #ALLOCATED state
     (locked-skip branch). A later set_item_allocated on the neighbor
     then wrote half-allocated "#ALLOCATED || real_key || ..." entries,
     tripping cascading Lua asserts ("prev value should not be nil",
     "next item should also be allocated", "head value should not be
     nil") that crashed every subsequent malloc — the headline corruption
     symptom in production.
  
  3. _free_script: HEXISTS guard before push_to_tail prevents free() from
     resurrecting a key that assign() already evicted from the pool.
  
  4. _shrink_script: HEXISTS guard before set_item_allocated makes
     shrink([unknown_key]) a silent no-op instead of a Lua assert.
  
  5. _malloc_script soft-binding: reorder set_item_allocated before SET
     lock. If set_item_allocated raised on a stale chain reference, the
     prior order left the lock in place while the entry stayed in free
     state — INV-4 (free + locked). New order aborts the EVAL cleanly
     with no orphan lock.
  
  6/7. _extend_script and _assign_script: DEL the lock key before
     push_to_tail when adding a fresh free node. Without this, a key
     re-extended after a previous incarnation was allocated-then-evicted
     would surface as free + locked (INV-4), since by-design #4 keeps
     locks alive across pool eviction.
  
  8. check_item_health: enable the previously commented-out
     "if locked then set_item_allocated(...)" branch so any future race
     producing INV-4 self-heals on the next gc() pass.
  
  Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
  

• docs: add CLAUDE.md project guide for future Claude Code sessions by Qingrui Hu at 2026-04-26 07:03:54

  Captures the load-bearing architectural facts that take time to
  re-derive each session: the doubly-linked-list-in-a-hash pool layout,
  the _lua_required_string shared helper contract, shared-vs-exclusive
  allocation modes, the parametrized fixture convention in conftest, the
  minimal-change coding rule from .cursor/rules, the _version.py vs
  __init__.py version drift, and several "looks-like-a-bug-is-design"
  contracts (timeout<=0 permanent locks, cache_timeout=0 permanent
  bindings, assign() preserving lock keys).
  
  Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
  

• test: add comprehensive Redis allocator test suite with thread health and allocation scenarios by Qingrui Hu at 2025-05-19 07:39:42

• bump: version 0.5.2 → 0.6.0 by Qingrui Hu at 2025-05-19 07:32:24

• feat: implement Redis-based distributed memory allocator with health checks and soft binding by Qingrui Hu at 2025-05-18 16:43:44

• bump: version 0.5.1 → 0.5.2 by Qingrui Hu at 2025-04-22 16:34:27

• fix: update refresh_pool_all to use length of updater parameters by Qingrui Hu at 2025-04-22 16:34:23

  - Changed the loop in `refresh_pool_all` method to iterate based on the length of `updater.params` instead of the parameter itself, ensuring correct iteration count.
  

• bump: version 0.5.0 → 0.5.1 by Qingrui Hu at 2025-04-22 16:30:08

• fix: implement clear method in RedisAllocator and update refresh_pool_all by Qingrui Hu at 2025-04-22 16:30:01

  - Added a `clear` method in `RedisAllocator` to clear resources from the allocation pool and cache using a Lua script.
  - Updated the `refresh_pool_all` method in `DefaultRedisAllocatorPolicy` to call the new `clear` method on the allocator.
  

• bump: version 0.4.6 → 0.5.0 by Qingrui Hu at 2025-04-22 16:22:47

• feat: add refresh_pool_all method to RedisAllocatorPolicy and DefaultRedisAllocatorPolicy by Qingrui Hu at 2025-04-22 16:22:42

  - Introduced an abstract method `refresh_pool_all` in `RedisAllocatorPolicy` to define a contract for refreshing the allocation pool.
  - Implemented the `refresh_pool_all` method in `DefaultRedisAllocatorPolicy` to iterate and refresh the pool based on the updater parameters.
  

• bump: version 0.4.5 → 0.4.6 by Qingrui Hu at 2025-04-22 16:04:31

• fix: adjust timeout calculation in RedisAllocatorPolicy health check by Qingrui Hu at 2025-04-22 16:03:53

  - Modified the timeout parameter in the executor.map call to use only lock_duration, simplifying the logic and ensuring consistent health check behavior.
  

• bump: version 0.4.4 → 0.4.5 by Qingrui Hu at 2025-04-21 21:17:25

• fix: comment out assertion in RedisAllocator to prevent runtime errors by Qingrui Hu at 2025-04-21 21:17:20

  - Commented out the assertion that checks if both head and tail are empty, allowing for smoother operation when the free list is empty.
  - This change aims to prevent potential runtime errors during allocation processes.
  

• bump: version 0.4.3 → 0.4.4 by Qingrui Hu at 2025-04-20 11:59:53

• fix: refactor pointer management in RedisAllocator by Qingrui Hu at 2025-04-20 11:59:45

  - Introduced helper functions to streamline head and tail pointer retrieval and updates, improving code readability and maintainability.
  - Enhanced the push_to_tail and pop_from_head functions to ensure correct handling of the free list and item states.
  - Added a new get_free_list function to return the order of items in the free list without modifying the list structure.
  

• fix: remove redundant head pointer update in RedisAllocator by Qingrui Hu at 2025-04-19 17:39:49

  - Eliminated unnecessary logic that updated the head pointer when the current head was removed, streamlining the allocation process.
  

• bump: version 0.4.2 → 0.4.3 by Qingrui Hu at 2025-04-19 17:16:02

• fix: enhance head and tail management in RedisAllocator by Qingrui Hu at 2025-04-19 17:15:56

  - Added logic to ensure the head and tail pointers are correctly updated when items are allocated or removed from the free list.
  - Implemented checks to handle corrupted pointers and maintain the integrity of the linked list structure.
  - Improved the handling of the current head's previous pointer to ensure it is always set to an empty string.
  

• fix: update tail assertion in RedisAllocator to allow for allocated items by Qingrui Hu at 2025-04-19 10:00:00

  - Modified the assertion in the RedisAllocator class to check if the next item is either an empty string or marked as '#ALLOCATED', ensuring correct handling of the free list.
  

• test: remove test for __del__ method in TestRedisAllocatorObject by Qingrui Hu at 2025-04-19 09:49:41

  - Removed the test_del method that tested the __del__ method of RedisAllocatorObject.
  - This change is part of the ongoing refactoring to improve test clarity and focus.
  

• bump: version 0.4.1 → 0.4.2 by Qingrui Hu at 2025-04-18 14:27:48

• fix: improve error logging and comment out __del__ method in RedisAllocatorObject by Qingrui Hu at 2025-04-18 14:26:45

  - Added error logging for failed object refresh attempts.
  - Commented out the __del__ method to prevent automatic closing of objects.
  

• bump: version 0.4.0 → 0.4.1 by Qingrui Hu at 2025-04-17 13:44:02

• refactor: enhance RedisAllocatorPolicy and DefaultRedisAllocatorPolicy with generic type support by Qingrui Hu at 2025-04-17 13:43:58

  - Updated RedisAllocatorPolicy to accept a generic type parameter.
  - Modified methods in RedisAllocatorPolicy and DefaultRedisAllocatorPolicy to utilize the generic type for improved type safety.
  - Adjusted the allocator initialization and method signatures accordingly.
  

• bump: version 0.3.3 → 0.4.0 by Qingrui Hu at 2025-04-17 05:31:31

• docs: update README files to clarify soft binding unbinding and correct PoolHash content representation by Qingrui Hu at 2025-04-17 05:31:15

• feat: enhance RedisAllocator with health management and automatic closing features by Qingrui Hu at 2025-04-17 05:22:13

  - Added set_healthy method to mark objects as healthy.
  - Introduced refresh_until_healthy method to refresh objects until they are healthy.
  - Updated refresh method to include cache_timeout parameter.
  - Modified DefaultRedisAllocatorPolicy to support auto_close functionality for non-unique objects.
  

• bump: version 0.3.2 → 0.3.3 by Qingrui Hu at 2025-04-16 13:36:45

• fix: update open method to return self in RedisAllocatableClass and RedisAllocatorObject for improved chaining by Qingrui Hu at 2025-04-16 13:33:44

• bump: version 0.3.1 → 0.3.2 by Qingrui Hu at 2025-04-16 13:11:13

• fix: replace os.time with custom time function in RedisAllocator for improved accuracy in Lua scripts by Qingrui Hu at 2025-04-16 13:11:08

• bump: version 0.3.0 → 0.3.1 by Qingrui Hu at 2025-04-16 08:56:06

• refactor: move versioning to a dedicated _version.py file and update setup.py to read version dynamically by Qingrui Hu at 2025-04-16 08:56:01

• feat: add health check methods to RedisAllocator for improved object management and reliability by Qingrui Hu at 2025-04-16 07:57:25

• feat: add cachetools dependency and enhance RedisAllocator with new methods for object management and soft binding retrieval by Qingrui Hu at 2025-04-15 16:37:51

• docs: update README and Chinese documentation to clarify RedisAllocator’s features, including resource pooling, atomic operations, and enhanced roadmap details; add example for managing a proxy pool with automatic updates by Qingrui Hu at 2025-04-14 20:01:06

• docs: enhance README and Chinese documentation with simplified flow diagrams for RedisLock and RedisAllocator, improving clarity and understanding by Qingrui Hu at 2025-04-14 19:20:39

• feat: enhance RedisAllocator with custom cache timeout for soft binding and improve documentation clarity by Qingrui Hu at 2025-04-14 18:49:46

• feat: add unique ID handling and object lifecycle management in RedisAllocator by Qingrui Hu at 2025-04-14 15:42:18

• refactor: simplify allocation logic in RedisAllocator and enhance Lua script handling for improved performance and clarity by Qingrui Hu at 2025-04-14 15:20:05

• refactor: update tox configuration to use python instead of python3 and modify cursor rules to always apply; enhance RedisLock and RedisLockPool docstrings for clarity by Qingrui Hu at 2025-04-14 03:12:13

• docs: introduce shared and soft binding modes in RedisAllocator, enhancing resource allocation flexibility and documentation by Qingrui Hu at 2025-04-13 05:07:51

• docs: enhance README and documentation with RedisLock usage examples and key concepts for better clarity by Qingrui Hu at 2025-04-12 18:35:37

• refactor: update tox configuration and improve test commands for flake8 and pytest by Qingrui Hu at 2025-04-12 17:26:00

• feat: enhance RedisAllocator with a new allocation policy and refactor RedisAllocatableClass for improved configurability by Qingrui Hu at 2025-04-12 14:03:28

• feat: implement timeout management in RedisAllocator tests with extend, assign, and free methods by Qingrui Hu at 2025-04-11 15:07:48

• refactor: replace LUA script properties with cached properties in RedisAllocator for improved performance by Qingrui Hu at 2025-04-11 15:00:02

• feat: add cursor rules for coding standards and enhance RedisAllocator with timeout management by Qingrui Hu at 2025-04-11 14:54:28

• docs: add contributing guidelines and code of conduct to CONTRIBUTING.md by Qingrui Hu at 2025-04-11 10:37:18

• style: fix whitespace inconsistencies in docstrings across allocator and task queue modules by Qingrui Hu at 2025-04-11 10:35:30

• docs: update setup for redis-allocator and add documentation structure with Sphinx by Qingrui Hu at 2025-04-11 10:31:47

• chore: add initial project structure with configuration files, README, and core modules for Redis-based resource allocation system by Qingrui Hu at 2025-04-11 10:03:31