Skip to content

fix(community): async label_propagation with oscillation detection#1388

Open
Ataxia123 wants to merge 1 commit intogetzep:mainfrom
NERDDAO:fix/label-propagation-async
Open

fix(community): async label_propagation with oscillation detection#1388
Ataxia123 wants to merge 1 commit intogetzep:mainfrom
NERDDAO:fix/label-propagation-async

Conversation

@Ataxia123
Copy link
Copy Markdown

Summary

  • Replace label_propagation with the asynchronous form from Raghavan et al. (2007) — visit nodes in a fresh random order each pass and update the community map in place, so neighbors immediately see the new label.
  • Deterministic tie-breaking by community id + strict-improvement rule prevents churn on symmetric graphs.
  • Oscillation safeguard via a short state-hash window catches any stable cycle that somehow survives the async form.
  • Adds 10 unit tests covering the regression case and common graph shapes.

The bug

The current implementation uses synchronous batch updates: it snapshots the community map at the start of each pass, computes new labels for all nodes from that snapshot, then replaces the map. This form is vulnerable to flip-flop oscillation on graphs with high-degree hub nodes — tied candidate scores cause groups of nodes to swap labels symmetrically every iteration, which repeats forever.

Observed failure: a real-world knowledge graph with 48 entities and a central hub connected to 14+ peers. 19 nodes kept flipping between two states indefinitely. The while True: loop at the top of label_propagation never terminated, which froze the caller (in our case, a FastAPI worker serving build_communities). No try/except in the call chain could recover because the function is purely synchronous and never yields control.

The fix

Implement the asynchronous form described in the canonical LPA paper (Raghavan, Albert, Kumara — Near linear time algorithm to detect community structures in large-scale networks, 2007):

  1. Visit nodes in a fresh random order each pass. A deterministic seed keeps results reproducible.
  2. For each node, read the current community map and update it in place before moving to the next node. Neighbors immediately see the new label, which breaks the ping-pong pattern that causes batch LPA to oscillate.
  3. Break ties deterministically by preferring the higher community id, and only move on strict improvement over the current support (so well-connected nodes stay put when candidates tie).
  4. Terminate on natural convergence (no changes in a full pass). As a belt-and-suspenders safeguard, also break if the exact community_map repeats within a short recent window — async LPA is known to converge in practice, but a cycle detector covers any pathological input.

Behavior change

  • Graphs that already converged: unchanged qualitative behavior, still fast.
  • Graphs that previously oscillated forever: now converge in milliseconds and produce sensible partitions.
  • Results are deterministic across runs (fixed RNG seed).

Verification

New test file at tests/utils/maintenance/test_community_operations.py (10 tests, all passing):

  • test_empty_projection_returns_empty
  • test_single_isolated_node
  • test_two_disconnected_triangles
  • test_complete_graph_collapses_to_one_community
  • test_hub_with_leaves_converges — regression case with a central hub + 20 leaves
  • test_two_stars_joined_by_bridge
  • test_real_world_pathological_graph_converges — minimized reproduction of the 48-node production failure (hub + 4 heavy satellites + 10 light satellites + 2 dyads)
  • test_deterministic_under_seed
  • test_ring_graph_of_varying_sizes[50]
  • test_ring_graph_of_varying_sizes[200]

Before: test_real_world_pathological_graph_converges hangs indefinitely on the old implementation.
After: all 10 tests pass in 0.69s.

Test plan

  • All new unit tests pass locally
  • Verified regression case (48-node hub graph) now completes in <1s instead of hanging
  • Verified existing call sites in get_community_clusters and build_communities still produce valid community partitions

References

  • Raghavan, U.N., Albert, R., Kumara, S. (2007). Near linear time algorithm to detect community structures in large-scale networks. Physical Review E, 76(3), 036106. https://arxiv.org/abs/0709.2938

🤖 Generated with Claude Code

The current label_propagation implementation uses synchronous batch
updates: it snapshots the community map at the start of each pass,
computes new labels for all nodes from that snapshot, then replaces
the map. This form is vulnerable to flip-flop oscillation on graphs
with high-degree hub nodes. Tied candidate scores cause groups of
nodes to swap labels symmetrically every iteration, which repeats
forever and blocks the caller indefinitely.

Observed on a real knowledge graph with 48 entities and a central
hub connected to 14+ peers: 19 nodes kept flipping between two
states forever. The main `while True:` loop never terminated.

Replace with the Raghavan et al. (2007) asynchronous form described
in "Near linear time algorithm to detect community structures in
large-scale networks":

1. Visit nodes in a fresh RANDOM order each pass (deterministic
   seed for reproducibility).
2. For each node, read the CURRENT community map and update it IN
   PLACE before moving to the next node. Neighbors immediately see
   the new label, which breaks the ping-pong pattern.
3. Break ties deterministically by preferring the higher community
   id, and only move when a candidate strictly improves on the
   current support — so well-connected nodes stay put under ties.
4. Terminate on natural convergence (no changes in a full pass).
   As a safeguard, also break if the exact community_map repeats
   within a short recent window — async LPA converges in O(log n)
   on real-world graphs but a cycle detector covers any edge case.

Verified on synthetic graphs (disconnected, stars, complete graphs,
rings, bridged stars, barbells) and a real-world pathological case
(hub + heavy/light satellites) — all converge in milliseconds and
produce sensible partitions.

Adds tests/utils/maintenance/test_community_operations.py with 10
unit tests covering the regression case and common graph shapes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant