Skip to content

algo.wcc: min-label propagation rescans every edge on each pass; a parallel union-find would need far fewer edge visits #9133

Description

@lvca

GraphAlgorithms.connectedComponents (the kernel behind algo.wcc) is a synchronous min-label propagation. Every
pass re-reads every edge of every edge type, in both directions, and the number of passes grows with the diameter of
the largest component. A parallel union-find would touch each edge about once, so it should be noticeably cheaper
on large graphs. This is a proposal to measure and, if it holds up, replace the kernel.

Where it stands

engine/src/main/java/com/arcadedb/graph/olap/GraphAlgorithms.java, connectedComponents(view, edgeTypes...)
(unchanged between 02ac27327d, the build measured below, and current main):

  • two int[n] arrays, label and newLabel, initialised to the node id;
  • each iteration does System.arraycopy(label -> newLabel), then a parallelForRange in which every node takes
    the minimum label over its forward and backward CSR neighbours, for every edge type, then copies newLabel back
    into label;
  • it loops until a pass changes nothing.

So the cost is passes x (E + 2n copies), and passes is bounded below by how far the smallest label has to
travel. Because the update is double-buffered (Jacobi style), a label moves one hop per pass even when the node
that holds the smaller label has already been visited in the same pass.

What it looks like in practice

Embedded, GAV enabled, -Xms12g -Xmx12g, Eclipse Temurin 25 with -XX:+UseCompactObjectHeaders, MacBook Pro M5 Pro,
LDBC Graphalytics graphs treated as undirected (ArcadeDB 26.10.1-SNAPSHOT, main at 02ac27327d):

graph vertices edges PageRank (10 iterations) WCC
datagen-7_5-fb 633,432 34,185,747 0.17 s 0.07 s
graph500-22 2,396,657 64,155,735 0.31 s (single run) 0.54 s (single run)

On the first graph WCC is cheaper than ten PageRank iterations. On the second, which has a much heavier-tailed
degree distribution, WCC costs more than ten full PageRank iterations, which suggests it needs more than ten edge
passes there. I have not instrumented the pass count, so that part is an inference; a counter
would settle it.

The same procedure over HTTP (CALL algo.wcc() YIELD componentId RETURN count(*), server in Docker) takes 0.20 s
and 0.81 s on the two graphs.

Proposal

Replace the label-propagation loop by a lock-free parallel union-find over the CSR:

  • parent[] as an int[] (or AtomicIntegerArray), find with path halving, union with a CAS loop;
  • always hook the larger root under the smaller one, so the final root is the minimum node id of the component. That
    keeps the current contract (component id = min node id), so callers and countComponents do not change;
  • process each edge once, in parallel over node ranges using the existing parallelForRange; only one direction of
    each stored edge is needed because the relation is symmetric;
  • finish with a parallel compression pass (label[u] = find(u)).

Cheap variants worth trying first, in increasing effort:

  1. keep the current structure but update in place (Gauss-Seidel style, a single label array) so a smaller label
    can travel several hops in one pass;
  2. union-find as above;
  3. union-find with neighbour sampling (the Afforest idea): union each node with a couple of neighbours, find the
    largest component, then only process the remaining edges of nodes outside it. This skips most edges on graphs
    with one giant component.

Memory stays at O(n) (one extra int[n], the same as today).

Acceptance

  • same output as today for every existing GraphAlgorithmsTest case and for the LDBC Graphalytics WCC reference
    output on the two graphs above (component ids are compared as equivalence classes there);
  • measured before/after on both graphs, embedded and through algo.wcc, with the pass count of the current
    implementation reported alongside;
  • no regression on small graphs (the kernel only goes parallel above PARALLEL_THRESHOLD today; keep that).

The harness used for the numbers above is ArcadeData/ldbc_graphalytics_platforms_arcadedb (ldbc-native/ArcadeDBEmbeddedBenchmark.java
for the embedded runs, with datagen-7_5-fb and graph500-22 from the LDBC dataset repository).

Activity

  1. added
    engineCore engine: document/schema API
    severity:minorCosmetic, documentation, edge case, or a workaround exists
    on Oct 3, 2026
  2. self-assigned this
    on Oct 3, 2026
  3. added theissue type on Oct 3, 2026
  4. added this to the 26.10.1 milestone on Oct 3, 2026
  5. added
    in progressThe team is actively working on this issue
    on Oct 3, 2026
  6. removed
    in progressThe team is actively working on this issue
    on Oct 4, 2026
  7. lvca commented on Oct 4, 2026

    @lvca
    MemberAuthor

    Done in #9135, merged.

    algo.wcc no longer rescans every edge on each pass. The old label propagation was replaced with a parallel union-find (with neighbour sampling) that touches each edge about once, so the cost no longer grows with the graph's diameter. Results are unchanged: the component id is still the lowest vertex id in the component, memory use is the same single array, and small graphs behave as before.

    On a synthetic 300k-vertex / 3M-edge graph, WCC went from about 7.5 ms to about 2.5 ms. I did not measure the LDBC graph500-22 and datagen-7_5-fb graphs from the report, so please re-run your Graphalytics harness and share the numbers if you can.


    ❤️ Thanks for helping make ArcadeDB better! ArcadeDB is free and open source, sustained by its community. If it's useful to you or your company, please consider becoming a sponsor to keep development going.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

engineCore engine: document/schema APIenhancementNew feature or requestperformanceseverity:minorCosmetic, documentation, edge case, or a workaround exists

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions