Machine Learning for Protein Engineering Part 1

We introduce “Lab-in-the-loop,” a paradigm shift for antibody design that orchestrates generative machine learning models, multi-task property predictors, active learning ranking and selection, and in vitro experimentation in a semiautonomous, iterative optimization loop. We apply lab-in-the-loop to four clinically relevant antigen targets: EGFR, IL-6, HER2, and OSM.

Epitope competition assays are a routine part of therapeutic antibody discovery. Modern high-throughput technologies have enabled the generation of complete pairwise binning matrices on large antibody panels. However, running the experiment for large panels is time-consuming and costly. The ML model in the workflow is trained on experimentally obtained pairwise binning information for a subset of the interactions, and the trained model predicts the remainder of the interactions. Blind testing across twelve distinct panels of antibodies, including IgGs and HCABs, targeting a variety of antigens, places the accuracy of the approach in the range of 85-90%.

Accelerating antibody-based medicine development requires better understanding and prediction of antibody-antigen binding. Affinity datasets from mutational scans are valuable but poorly understood. We present a multi-modal dataset of antigen and VHH interface variants, capturing affinity, stability, and expression changes. Most affinity changes stem from stability shifts. Structure-conditioned inverse folding models predict stability well but struggle with interface changes, underscoring the need for high-quality datasets in protein engineering.

In machine learning, protein sequences are usually processed by Transformers/LLMs, while protein structures are typically represented as graphs and processed by GNNs. However, AlphaFold3 and recent studies on protein LLMs show that Transformers alone can make sense of 3D inputs. Here I will share our recent work on deciphering the inner workings of Transformers when given 3D coordinates. These insights can guide the rational design of hybrid sequence/structure protein models.

As traditional protein development is resource-intensive, Merck is leveraging digital approaches to design and assess new proteins, increasing the hit rate of laboratory screens. We combine state-of-the-art tools for protein structure prediction (AlphaFold3-type models), representation (ESM), and de novo generation (diffusion models) with classical computational biochemistry and bioinformatics methods. We will present the application of our pipeline and discuss current gaps and potential solutions based on ongoing developments in the field.

We present Boltz, a family of open models for biomolecular modelling and design. Combining advances in structure prediction, affinity learning, and generative modelling, Boltz enables accurate reconstruction of molecular interactions and the creation of new functional biomolecules across diverse targets. Experimental validations demonstrating high-affinity binders illustrate the potential of large, open biomolecular models to accelerate molecular discovery.

Dual-specific antibodies are a promising but challenging modality. We demonstrate a combination of experimental and computational techniques to discover and optimise a TIGIT/LILRB4 dual-specific binder. Starting from a synthetic humanoid phage library, we identified compatible scFvs, used multiplexed yeast display affinity data to confirm binding and generate a training dataset, and applied a fine-tuned deep learning affinity oracle for optimisation, yielding a molecule with improved dual-target affinity and developability.

Typical ML models are trained by masking and predicting experimentally determined labels. However, in novel drug discovery the goal is often to design antibodies that are better than previously observed ones or even have entirely new characteristics. In this work, we demonstrate that integrating pretrained LLMs with datasets featuring continuous labels allows prediction of binders with novel specificities and with much better affinities than those seen previously in experiments.

The OpenFold Consortium brings together academic and industrial teams to build state-of-the-art protein structure and co-folding prediction models optimised for use on commercial computational hardware. We develop fully open sourced models and support creation of new experimental datasets, aiming to build more powerful models that can accurately predict complex systems of significance to life sciences. With our latest release, we aim to reproduce the full scale and training regimen of AlphaFold3 and provide open source model weights, extensive datasets, and a permissively licensed code library for developing novel architectures and custom training pipelines.