Set the budgets and run on a laptop¶

This guide shows how to set the number of trials, resamples and refits, how many fits each choice costs, and how to shrink a run to laptop size.

Prerequisites¶

A prepared table (Prepare your table for recover).
A fit here means one run of UMAP followed by HDBSCAN on one set of rows.

The budgets¶

Field	Default	Allowed	What it controls
`n_trials`	48	above `n_explore`	most trials (trial) the search may run
`n_explore`	16	power of two, at least 2	trials of the Sobol design, never pruned
`n_resamples`	3	at least 2	resamples every trial is scored on
`resample_fraction`	0.8	strictly between 0 and 1	share of rows in each resample
`n_worlds`	32	2 to 32767	number of refits (refit) of the selected setting
`world_fraction`	0.8	strictly between 0 and 1	share of rows in each refit

A value outside its range raises ValueError when you build the Config.

What a run costs in fits¶

The cost of a run is counted in fits. A complete trial fits each of its n_resamples resamples once on your data. With the default structureless copy (gap_family="gaussian_copula") it fits the same number again on the copy. A pruned trial stops after 2 fits. The search then re-scores the best trials on new resamples. After the search come 2 full-data fits and n_worlds refits.

Recorded runs, from each run’s provenance, with the calibration test off:

Table	Search fits	Reference fits	Refit fits	Total
scikit-learn digits, 1,797 rows	280	2	32	314
APOGEE DR19 example table, 9,242 rows	300	2	32	334

With cuML on a laptop RTX 3060 under WSL 2, the Digits run took 93.9 s (search 77.4 s, refits 7.2 s) and the APOGEE run 163 s (search 140 s, refits 13 s). No CPU time is recorded, so the CPU time of the same runs is not measured. The calibration test costs about six times the rest of a run.

Steps to a laptop-sized run¶

Test your pipeline on a random sample of rows, so a mistake costs seconds.

import numpy as np

rng = np.random.default_rng(0)
X_small = X[np.sort(rng.choice(X.shape[0], size=2000, replace=False))]

Shrink the budgets. Keep n_explore a power of two and n_trials above it.

import starfold

config = starfold.Config(n_trials=24, n_explore=8, n_worlds=16)
result = starfold.recover(X_small, config=config, random_state=0)

Read result.provenance["stage_fits"] and result.provenance["stage_seconds"] to see where the time went.

print(result.provenance["stage_fits"], result.provenance["total_fits"])
print(result.provenance["stage_seconds"])

Read the flags. With fewer trials the search is more likely to stop at its budget, which raises search_budget_exhausted. It means the score was still improving. It does not invalidate the result.
Run the full budget on the whole table for the result you report.

Expected result¶

The fit counts follow your budgets, and the times depend on your machine. No recorded run uses the reduced budgets above, so no output is shown for them. A run with n_trials=24, n_explore=8 and n_worlds=16 searches fewer settings and measures recurrence over fewer refits. Its count interval and recurrence bands are based on 16 refits instead of 32.

Variations¶

n_resamples. Each trial costs n_resamples fits on the data, so the cost scales with it. The prediction agreement is scored on ordered pairs of resamples, so 3 resamples give 6 pairs.
resample_fraction. The held-out rows of a pair are the rows of one resample that the other did not see. A fraction near 1 leaves few such rows.
Calibration cost. Each copy replays n_explore + 1 trials on n_resamples fits, for the data and for the copy. At the defaults that is 17 x 3 x 2 = 102 fits per copy. With n_explore=8 it is 9 x 3 x 2 = 54, derived from the same structure. The calibration guide has the rest.
Cluster. For a sweep over seeds, see Run a seed sweep on a SLURM cluster.