A new foundation model named Bonbon, trained on a vast dataset of protein-ligand sequence pairs, demonstrates emergent biological capabilities without explicit instruction. The model, detailed in a paper published on arXiv, learned to predict molecular interaction properties at four distinct levels of resolution: functional, residue, bond, and atom. These capabilities include identifying a molecule's mechanism of action, localizing binding sites, distinguishing between covalent and non-covalent engagement, and pinpointing active moieties at single-bond resolution.
Bonbon's development extends the concept of emergent capabilities, previously observed in large language models, to multimodal biology. The model was trained exclusively on protein-ligand sequences, totaling approximately 3.2 trillion tokens, without any task-specific supervision or structural coordinates. This approach contrasts with traditional methods that often rely on separate supervised models or structure-based computations requiring three-dimensional coordinates.
The researchers conducted controlled scaling experiments with Bonbon across four model sizes, ranging from 42 million to 1.6 billion parameters. They observed a sharp increase in performance for each emergent capability at the largest scale, even as the pretraining loss decreased smoothly. This finding suggests that loss-based scaling laws alone may not fully predict when zero-shot emergent capabilities will appear. The capabilities emerged after training at a 1:2,000 parameter-to-token ratio, which the authors note accommodates the sparsity of paired biological interaction data.
Beyond its emergent properties, Bonbon's frozen representations achieved state-of-the-art performance in binding and affinity predictions. The model performed these predictions at speeds five orders of magnitude faster than structure-based methods. In prospective wet-lab validation, Bonbon demonstrated a 70% hit rate on de novo compounds, successfully identifying novel chemical structures not previously documented as inhibitors for the tested targets. These compounds had less than 60% structural similarity to molecules present in the training corpus, indicating the model's ability to generalize to out-of-distribution proteins.
The paper emphasizes that Bonbon's zero-shot emergent capabilities mean the model was never explicitly trained to produce these specific labels. Instead, the learned representations appear to contain sufficient information to support these predictions. This generalization to out-of-distribution complexes is a critical aspect for practical applications in drug discovery, where models must identify novel chemistry rather than merely recalling familiar interactions. The evaluation targets used for out-of-distribution testing were excluded from training through sequence identity filtering to ensure the tests probed generalization rather than memorization.