Repository navigation
Description
GraphAlgorithms.connectedComponents (the kernel behind algo.wcc) is a synchronous min-label propagation. Every
pass re-reads every edge of every edge type, in both directions, and the number of passes grows with the diameter of
the largest component. A parallel union-find would touch each edge about once, so it should be noticeably cheaper
on large graphs. This is a proposal to measure and, if it holds up, replace the kernel.
Where it stands
engine/src/main/java/com/arcadedb/graph/olap/GraphAlgorithms.java, connectedComponents(view, edgeTypes...)
(unchanged between 02ac27327d, the build measured below, and current main):
- two
int[n]arrays,labelandnewLabel, initialised to the node id; - each iteration does
System.arraycopy(label -> newLabel), then aparallelForRangein which every node takes
the minimum label over its forward and backward CSR neighbours, for every edge type, then copiesnewLabelback
intolabel; - it loops until a pass changes nothing.
So the cost is passes x (E + 2n copies), and passes is bounded below by how far the smallest label has to
travel. Because the update is double-buffered (Jacobi style), a label moves one hop per pass even when the node
that holds the smaller label has already been visited in the same pass.
What it looks like in practice
Embedded, GAV enabled, -Xms12g -Xmx12g, Eclipse Temurin 25 with -XX:+UseCompactObjectHeaders, MacBook Pro M5 Pro,
LDBC Graphalytics graphs treated as undirected (ArcadeDB 26.10.1-SNAPSHOT, main at 02ac27327d):
| graph | vertices | edges | PageRank (10 iterations) | WCC |
|---|---|---|---|---|
| datagen-7_5-fb | 633,432 | 34,185,747 | 0.17 s | 0.07 s |
| graph500-22 | 2,396,657 | 64,155,735 | 0.31 s (single run) | 0.54 s (single run) |
On the first graph WCC is cheaper than ten PageRank iterations. On the second, which has a much heavier-tailed
degree distribution, WCC costs more than ten full PageRank iterations, which suggests it needs more than ten edge
passes there. I have not instrumented the pass count, so that part is an inference; a counter
would settle it.
The same procedure over HTTP (CALL algo.wcc() YIELD componentId RETURN count(*), server in Docker) takes 0.20 s
and 0.81 s on the two graphs.
Proposal
Replace the label-propagation loop by a lock-free parallel union-find over the CSR:
parent[]as anint[](orAtomicIntegerArray),findwith path halving,unionwith a CAS loop;- always hook the larger root under the smaller one, so the final root is the minimum node id of the component. That
keeps the current contract (component id = min node id), so callers andcountComponentsdo not change; - process each edge once, in parallel over node ranges using the existing
parallelForRange; only one direction of
each stored edge is needed because the relation is symmetric; - finish with a parallel compression pass (
label[u] = find(u)).
Cheap variants worth trying first, in increasing effort:
- keep the current structure but update in place (Gauss-Seidel style, a single
labelarray) so a smaller label
can travel several hops in one pass; - union-find as above;
- union-find with neighbour sampling (the Afforest idea): union each node with a couple of neighbours, find the
largest component, then only process the remaining edges of nodes outside it. This skips most edges on graphs
with one giant component.
Memory stays at O(n) (one extra int[n], the same as today).
Acceptance
- same output as today for every existing
GraphAlgorithmsTestcase and for the LDBC Graphalytics WCC reference
output on the two graphs above (component ids are compared as equivalence classes there); - measured before/after on both graphs, embedded and through
algo.wcc, with the pass count of the current
implementation reported alongside; - no regression on small graphs (the kernel only goes parallel above
PARALLEL_THRESHOLDtoday; keep that).
The harness used for the numbers above is ArcadeData/ldbc_graphalytics_platforms_arcadedb (ldbc-native/ArcadeDBEmbeddedBenchmark.java
for the embedded runs, with datagen-7_5-fb and graph500-22 from the LDBC dataset repository).
Activity
fix(engine): compare against the giant's current root in WCC finish p…
test(engine): WCC edge cases (self-loops, parallel edges, single node…
test(engine): WCC restricted to an edge type subset (#9133)
test(engine): deterministic CSR-level WCC tests, many-small-component…
docs(engine): reflow WCC Javadoc, document kernel array contract and …
test(engine): WCC hubs outside the giant exercise the remaining-forwa…
test(engine): WCC kernel with a null CSR type, repeated runs for dete…
Merge pull request #9135 from ArcadeData/perf/9133-wcc-union-find
lvca commentedon Oct 4, 2026
Done in #9135, merged.
algo.wccno longer rescans every edge on each pass. The old label propagation was replaced with a parallel union-find (with neighbour sampling) that touches each edge about once, so the cost no longer grows with the graph's diameter. Results are unchanged: the component id is still the lowest vertex id in the component, memory use is the same single array, and small graphs behave as before.On a synthetic 300k-vertex / 3M-edge graph, WCC went from about 7.5 ms to about 2.5 ms. I did not measure the LDBC graph500-22 and datagen-7_5-fb graphs from the report, so please re-run your Graphalytics harness and share the numbers if you can.
❤️ Thanks for helping make ArcadeDB better! ArcadeDB is free and open source, sustained by its community. If it's useful to you or your company, please consider becoming a sponsor to keep development going.
perf(engine): parallel union-find with neighbour sampling for algo.wcc (
perf(engine): parallel union-find with neighbour sampling for algo.wcc (
fix(engine): compare against the giant's current root in WCC finish p…
test(engine): WCC edge cases (self-loops, parallel edges, single node…
test(engine): WCC restricted to an edge type subset (#9133)
test(engine): deterministic CSR-level WCC tests, many-small-component…
docs(engine): reflow WCC Javadoc, document kernel array contract and …
test(engine): WCC hubs outside the giant exercise the remaining-forwa…
test(engine): WCC kernel with a null CSR type, repeated runs for dete…