Starfold held-out benchmark

Starfold chooses UMAP and HDBSCAN settings without labels and reports how far its groups can be trusted. The benchmark checks whether the chosen setting recovers known classes better than the library defaults. Six development tables informed the selection rules, and four held-out tables were not used until the rules were fixed.

Recovery is the adjusted Rand index (floored at zero) against the true classes on the assigned rows, times the share of rows assigned. The defaults score is the median over eight seeds, and a difference below 0.02 counts as a match. Starfold matched or beat the defaults on 6 of 6 development tables and on 2 of 4 held-out tables, losing on COIL-20 and Isolet.

Results per table

Table	Set	Rows	Features	Classes	Starfold	Defaults	Result
Rice	development	3,810	7	2	0.292	0.008	starfold
Dry Bean	development	13,611	16	7	0.500	0.445	starfold
Landsat	development	6,430	36	6	0.354	0.048	starfold
Digits	development	1,797	64	10	0.740	0.637	starfold
PenDigits	development	10,992	16	10	0.819	0.753	starfold
Waveform	development	5,000	40	3	0.111	0.000	starfold
COIL-20	held-out	1,440	1,024	20	0.518	0.581	defaults
Isolet	held-out	7,797	617	26	0.198	0.453	defaults
Gas drift	held-out	13,910	128	6	0.038	0.023	match within 0.02
Letter	held-out	20,000	16	26	0.203	0.108	starfold

Recorded at tag v0.2.0 (commit f93dea0) in bench/results/summary.json. The CSV lists each table's source, and the provenance record is in JSON.

Figures

Starfold gains most where the classes are large compared with the finest structure its graph can resolve. Left, recovery minus default recovery; right, correlation of the label-free score with recovery. Data: resolution.json.

Groups smaller than the resolution scale are not found; those found stay pure as the support threshold rises. A method check on synthetic groups injected into a real table and never kept. Data: injection.json, provenance.