Metrics¶

This page lists every number Starfold prints or returns, says what question each answers, and sends you to its page.

Numbers about the groups¶

Metric	Question it answers	Range	Where printed
Group recurrence	How often does a group come back in the refits?	0 to 1	`result.worlds.group_recurrence`, `group_recurrence_band`
Support and own support	How much of a row’s refit outcome lands in each group?	0 to 1	`result.support`, `result.worlds.own_support`
Unresolved mass	How much of the table do the refits fail to place in a group?	0 to 1	`result.unresolved_mass`, `result.worlds.unresolved`
Count interval	How much does the number of groups move across refits?	whole numbers	`result.worlds.count_interval`, `count_values`
Co-association contrast	Do pairs in one group stay together across refits, and pairs in different groups stay apart?	-1 to 1	`result.consensus.contrast`, `within_association`, `between_association`, `agreement`
Calibration p-value	Does the table beat structureless copies on the largest search gap among trials 0 to 16 (the defaults and the 16-trial Sobol design; 0.710 for the scikit-learn digits dataset, the search gap of trial 7)?	0 to 1	`result.null.p_value`, `p_intervals`
Strengths	How firmly does HDBSCAN hold each row inside its group in one fit?	0 to 1	`result.strengths`
Properties	What are the typical column values of each group, with a band over refits?	column units	`result.worlds.property_median`, `property_band`

Numbers about the choice of setting¶

Metric	Question it answers	Range	Where printed
Prediction agreement	Do groups fitted on one subset predict the groups of unseen rows?	0 to 1	`result.search.trials[i].stability`
Coverage	What share of the tested rows do both fits assign to a group?	0 to 1	`result.search.trials[i].coverage`
Modularity	Do the groups follow the 15-nearest-neighbour graph of your table?	below 1, clipped at 0	`result.search.trials[i].modularity`
Score	How good is one setting on the three terms together?	0 to 1	`raw_score`, `fresh_score`, `result.search.score_selected`
Gap	How much better is the setting on the table than on a structureless copy?	-1 to 1	`result.search.trials[i].score`, `candidates[i].gap`, `gap_selected`
Default comparison	Does the chosen setting beat the library defaults, and by more than noise?	True or False, differences	`beats_default_under_objective`, `paired_differences`, flag `default_within_noise`

Other numbers Starfold returns¶

These have no page of their own. The table below defines each in one line.

Number	What it is	Where printed
Group count and noise rows	groups and rows with label -1 in the fit on all rows	`result.n_groups`, `result.labels`
Refit agreement	for each refit, the adjusted Rand index against the full-data labels times the joint coverage of the two	`result.worlds.world_agreement`
Backend agreement	CPU against cuML partitions, when `check_backends` is on	`result.backends.mean_normalised`
Flags	short names for weaknesses in the run	`result.flags`
Run record	rows kept, seconds and fits per stage, peak memory	`result.provenance`

Four meanings of agreement¶

The word “agreement” names four different numbers. Starfold never uses it alone in a printed name.

Name	What it compares	Where
Prediction agreement	labels predicted for unseen rows against the fit on those rows, ARI only	Prediction agreement
Refit agreement	each refit against the full-data labels, ARI times joint coverage	`result.worlds.world_agreement`
Consensus agreement	the co-association cut against the full-data labels, ARI times coverage (flag `consensus_disagrees` below 0.5)	Co-association, contrast and the consensus check
Agreement with known classes	full-data labels against classes you hold, computed after the fact, ARI times coverage	notebooks and bench, never the search

Which page answers which decision¶

Decision	Page
Is this setting worth keeping in the search?	Score and its three terms
Does the table beat a structureless copy?	Gap
Do I use the chosen setting or the defaults?	Default comparison
Do I use this group?	Group recurrence
Do I quote a group count?	Count interval

How to read a result in order¶

Read result.flags first. Each flag names one weakness or event.
Read the group count and the number of noise rows in result.labels.
Read the default comparison: did the search beat the defaults, and by more than noise?
Read unresolved mass: how much of the table the groups describe.
Read group recurrence for each group you plan to use.
Read the count interval before you quote a number of groups.
Read support for single rows you care about.
Read the co-association contrast, which ignores group numbers.
Read the calibration p-value if you ran the test. Its absence raises the flag null_not_run.
Read properties last, to describe the groups in column units.

Read the three terms of the score, the Score and the Gap when you want to know why the search chose its setting. The fresh re-score of the top trials reduces the winner’s curse (the best of many noisy scores is partly luck); it does not remove it. A stable partition is not a true partition, and no single number here proves the groups are real.