Cross-validation and data splitting

class GroupKFoldShuffle(n_splits=5, *, shuffle=False, random_state=None)

split(X, y=None, groups=None)

Generate indices to split data into training and test set.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Training data, where n_samples is the number of samples and n_features is the number of features.

• y (array-like of shape (n_samples,), default=None) – The target variable for supervised learning problems.

• groups (array-like of shape (n_samples,), default=None) – Group labels for the samples used while splitting the dataset into train/test set.

Yields:

• train (ndarray) – The training set indices for that split.

• test (ndarray) – The testing set indices for that split.

get_scaffold(smi)

Generate the Bemis-Murcko scaffold for a given molecule.

Parameters:

smi (Union[str, Mol]) – A SMILES string or an RDKit molecule object representing the molecule for which to generate the scaffold.

Return type:

str

Returns:

A SMILES string representing the Bemis-Murcko scaffold of the input molecule. If the scaffold cannot be generated, the input SMILES string is returned.

get_random_clusters(smiles_list)

Assign every SMILES to its own unique group.

This makes a grouped splitter (e.g. GroupKFoldShuffle) behave like a random (non-grouped) split, since no two molecules are forced into the same fold.

Parameters:

smiles_list (List[str]) – A list of SMILES strings.

Return type:

ndarray

Returns:

Array of integers from 0 to the length of the input list.

get_butina_clusters(smiles_list, dist_cutoff=0.65)

Cluster a list of SMILES strings using the Butina clustering algorithm.

Parameters:

• smiles_list (List[str]) – List of SMILES strings

• dist_cutoff (float) – distance cutoff (1 - Tanimoto similarity), see useful_rdkit_utils.taylor_butina_clustering()

Return type:

ndarray

Returns:

Array of cluster labels corresponding to each SMILES string in the input list.

get_bemis_murcko_clusters(smiles_list)

Cluster a list of SMILES strings based on their Bemis-Murcko scaffolds.

Parameters:

smiles_list (List[str]) – List of SMILES strings

Return type:

ndarray

Returns:

List of cluster labels corresponding to each SMILES string in the input list.

get_kmeans_clusters(smiles_list, n_clusters=10)

Cluster a list of SMILES strings using the KMeans clustering algorithm.

Parameters:

• smiles_list (List[str]) – List of SMILES strings

• n_clusters (int) – The number of clusters to use for clustering

Return type:

ndarray

Returns:

Array of cluster labels corresponding to each SMILES string in the input list.

cross_validate(df, model_list, y_col, group_list, n_outer=5, n_inner=5, random_state=None)

Perform nested cross-validation on a dataset using multiple models and grouping strategies.

For each grouping strategy the data is split into n_outer outer folds. For every outer fold the inner models are cross-validated on the outer TRAINING set only (group assignments are computed there as well, to avoid leaking information from the held-out fold). The models are then refit on the full outer training set and evaluated on the outer test fold.

The resulting fold column uses the label outer * (n_inner + 1) + inner for the inner splits; the outer test evaluation occupies slot outer * (n_inner + 1) + n_inner so that inner and outer metrics never collide.

Parameters:

• df (DataFrame) – The input dataframe containing the data.

• model_list (List[Tuple[str, Callable[[str], object]]]) – A list of tuples where each tuple contains a model name and a class that, when called with y_col, returns a model with a validate(train, test) method.

• y_col (str) – The name of the target column.

• group_list (List[Tuple[str, Callable[[Series], Series]]]) – A list of tuples where each tuple contains a group name and a callable that assigns groups based on the SMILES column.

• n_outer (int) – The number of outer folds for cross-validation. Default is 5.

• n_inner (int) – The number of inner folds for cross-validation. Default is 5.

• random_state (Optional[int]) – Seed for the shuffled splits. Default is None.

Return type:

DataFrame

Returns:

A dataframe containing predictions (and model names as columns) for each fold, model, and group.