Microsoft open sources EvoDiff, a novel protein-generating AI | TechCrunch

Proteins, the natural molecules that carry out key cellular functions within the body, are the building blocks of all diseases. Characterizing proteins can reveal the mechanisms of a disease, including ways to slow it or potentially reverse it, while creating proteins can lead to entirely new classes of drugs and therapeutics.

But the current process for designing proteins in the lab is costly — both from a computational and human resource standpoint. It entails coming up with a protein structure that could plausibly perform a specific task inside the body, then finding a protein sequence — the sequence of amino acids that make up a protein — likely to “fold” into that structure. (Proteins must correctly fold into three-dimensional shapes to carry out their intended function.)

It doesn’t necessarily have to be this complicated.

This week, Microsoft introduced a general-purpose framework, EvoDiff, that the company claims can generate “high-fidelity,” “diverse” proteins given a protein sequence. Different from other protein-generating frameworks, EvoDiff doesn’t require any structural information about the target protein, cutting out what’s typically the most laborious step.

Available in open source, EvoDiff could be used to create enzymes for new therapeutics and drug delivery methods as well as new enzymes for industrial chemical reactions, Microsoft senior researcher Kevin Yang says.

“We envision that EvoDiff will expand capabilities in protein engineering beyond the structure-function paradigm towards programmable, sequence-first design,” Yang, one of the co-creators of EvoDiff, told TechCrunch in an email interview. “With EvoDiff, we’re demonstrating that we may not actually need structure, but rather that ‘protein sequence is all you need’ to controllably design new proteins.”

Core to the EvoDiff framework is a 640-million parameter model trained on data from all different species and functional classes of proteins. (“Parameters” are the parts of an AI model learned from training data and essentially define the skill of the model on a problem — in this case generating proteins.) The data to train the model was sourced from the OpenFold data set for sequence alignments and UniRef50, a subset of data from UniProt, the database of protein sequence and functional information maintained by the UniProt consortium.

EvoDiff is a diffusion model, similar in architecture to many modern image-generating models such as Stable Diffusion and DALL-E 2. EvoDiff learns how to gradually subtract noise from a starting protein made almost entirely of noise, moving it closer — slowly, step by step — to a protein sequence.

Diffusion models have been increasingly applied to domains outside of image generation, from conjuring up designs for novel proteins, like EvoDiff, to creating music and even synthesizing speech.

“If there’s one thing to take away [from EvoDiff], I think it’d be this idea that we can — and should — do protein generation over sequence because of the generality, scale and modularity that we’re able to achieve,” Microsoft senior researcher Ava Amini, another co-contributor on EvoDiff, said via email. “Our diffusion framework gives us the ability to do that and also to control how we design these proteins to meet specific functional goals.”

To Amini’s point, EvoDiff can not only create new proteins but fill in the “gaps” in an existing protein design, so to speak. Provided a part of a protein that binds to another protein, the model can generate a protein amino acid sequence around that part that meets a set of criteria, for example.

Because EvoDiff designs proteins in the “sequence space” rather than the structure of proteins, it can also synthesize “disordered proteins” that don’t end up folding into a final three-dimensional structure. Like normal functioning proteins, disordered proteins play important roles in biology and disease, like enhancing or decreasing other protein activity.

Now, it should be noted that the research behind EvoDiff hasn’t been peer reviewed — at least not yet. Sarah Alamdari, a data scientist at Microsoft who contributed to the project, admits that there’s “a lot more scaling work” to be done before the framework can be used commercially.

“This is just a 640-million-parameter model, and we may see improved generation quality if we scale up to billions of parameters,” Alamdari said via email. “While we demonstrated some coarse-grained strategies, to achieve even more fine-grained control, we would want to condition EvoDiff on text, chemical information or other ways to specify the desired function.”

As a next step, the EvoDiff team plans to test the proteins that the model generated in the lab to determine whether they’re viable. If they turn out to be, they’ll begin work on the next generation of the framework.