AlphaFold Decoded: iGEM Education Initiative
Introduction
In recent years, AI tools have reshaped the landscape of biology, offering insights once thought beyond our grasp. Among these tools, AlphaFold has been a game-changer in protein structure prediction. However, while biologists are quickly adapting to use such tools, there's a gap in learning how to build them. This is where our education project, AlphaFold Decoded, steps in. We aim to make this AI marvel more accessible by guiding learners through the process of reconstructing AlphaFold from scratch.
Why AlphaFold Decoded Matters
Transforming
Biology
AlphaFold changed the game, but we’re not done yet. Imagine pushing molecular prediction further - where designing enzymes or predicting mutations could become nearly automatic. By mastering AlphaFold, you're stepping into a future where computational power drives breakthroughs in biology.
Building on
AlphaFold
AlphaFold is massive, complex, and expensive, making it challenging to replicate in typical research settings. However, mastering its principles opens up exciting possibilities. For example, you could connect AlphaFold's output to language models to generate detailed descriptions of protein structures or predict their functions.
Machine Learning
in Action
Even if you're not looking to advance AlphaFold itself, the skills you gain are invaluable. You’ll learn how to tackle complex, real-world problems using machine learning, understand and manipulate 3D structures, and gain practical coding experience that translates theoretical concepts into actionable solutions.
Challenges to Overcome
Mastering AlphaFold comes with genuine, often overlooked challenges – hurdles that likely explain why projects of this scale haven’t been attempted more frequently. Let’s explore the two most significant difficulties that stand in the way.
Bridging Missing
Prerequisites
AlphaFold isn't just another machine learning model – it demands knowledge beyond the usual toolkit. A solid grasp of machine learning fundamentals like attention mechanisms is required, but so is understanding more specialized topics like quaternions and homogeneous coordinates in structural biology. These gaps can be daunting, especially for those without a structured learning path.
The Realization
Challenge
AlphaFold’s extensive codebase can make it seem impossible to replicate as an individual, even more so when one realizes that a comprehensive understanding alone doesn’t suffice. The sheer volume of interconnected components can feel overwhelming, making it difficult to know where to start or what’s essential. This raises the question of whether all the details matter if you can’t fully commit to reproducing or applying them afterward.
The first challenge led us to carefully craft a structured content line, which we’ll discuss in later sections as our chosen roadmap. We assume only basic Python knowledge and aim to guide you through the rest. The second challenge is a deeper question: How complex does an AlphaFold implementation really need to be? This brings us to our next section.
Implementation Code
Quality education starts with quality code. Our goal was to provide an implementation of AlphaFold that prioritizes readability and clarity, allowing learners to focus on understanding the underlying concepts. Our version condenses the core functionality into 1,411 logical lines of code, closely following the original paper’s pseudocode, compared to 5,287 lines in AlphaFold’s key files. This direct approach from theory to practice is a hallmark of our implementation.
Following is a code comparison of selected components of AlphaFold, comparing the pseudocode from the paper against our implementation and the original AlphaFold repository. It’s important to note that the code taken from the AlphaFold repository has been cropped as best as possible to match the parts covered in the pseudocode and our implementation. However, AlphaFold’s code isn’t just bloat – it contains more efficient implementations and features like ensembling, template stacks, and detailed loss calculations. These additional features, while valuable for production, contribute to the complexity that can make learning from the code more challenging than necessary.
The AlphaFold code snippets displayed on this page are from DeepMind's AlphaFold repository. We have modified the displayed content by cropping it from the surrounding code to highlight relevant sections. The AlphaFold repository is licensed under the Apache 2.0 License, and we adhere to its terms. The code snippets labeled as AlphaFold Decoded were created by us. Additionally, the pseudocode images shown are reproductions of algorithms presented in the original AlphaFold paper by DeepMind.
Model
Evoformer
Geometry
Input Embedder
AlphaFold Decoded
AlphaFold Original
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
def forward(self, batch):
"""
Forward pass for the Alphafold model.
"""
N_cycle = batch['msa_feat'].shape[-1]
N_seq, N_res = batch['msa_feat'].shape[-4:-2]
batch_shape = batch['msa_feat'].shape[:-4]
device = batch['msa_feat'].device
dtype = batch['msa_feat'].dtype
c_m = self.c_m
c_z = self.c_z
outputs = {}
prev_m = torch.zeros(batch_shape+(N_seq, N_res, c_m),
device=device, dtype=dtype)
prev_z = torch.zeros(batch_shape+(N_res,N_res,c_z),
device=device, dtype=dtype)
prev_pseudo_beta_x = torch.zeros((N_res, 3),
device=device, dtype=dtype)
for i in range(N_cycle):
print(f'Starting iteration {i}...')
current_batch = {
key: value[...,i] for key, value in batch.items()
}
m, z = self.input_embedder(current_batch)
m_rec, z_rec = self.recycling_embedder(prev_m, prev_z,
prev_pseudo_beta_x)
m[..., 0, :, :] += m_rec
z += z_rec
e = self.extra_msa_embedder(current_batch)
z = self.extra_msa_stack(e, z)
del e
m, z, s = self.evoformer(m, z)
F = current_batch['target_feat'].argmax(dim=-1)
structure_output = self.structure_module(s, z, F)
prev_m = m
prev_z = z
prev_pseudo_beta_x = structure_output['pseudo_beta_positions']
for key,value in structure_output.items():
if key in outputs:
outputs[key].append(value)
else:
outputs[key] = [value]
outputs = {
key: torch.stack(value, dim=-1) for key, value in outputs.items()
}
return outputs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
def __call__(self,
ensembled_batch,
non_ensembled_batch,
is_training,
compute_loss=False,
ensemble_representations=False,
return_representations=False):
num_ensemble = jnp.asarray(ensembled_batch['seq_length'].shape[0])
if not ensemble_representations:
assert ensembled_batch['seq_length'].shape[0] == 1
def slice_batch(i):
b = {k: v[i] for k, v in ensembled_batch.items()}
b.update(non_ensembled_batch)
return b
# Compute representations for each batch element and average.
evoformer_module = EmbeddingsAndEvoformer(
self.config.embeddings_and_evoformer, self.global_config)
batch0 = slice_batch(0)
representations = evoformer_module(batch0, is_training)
# MSA representations are not ensembled so
# we don't pass tensor into the loop.
msa_representation = representations['msa']
del representations['msa']
# Average the representations (except MSA) over the batch dimension.
if ensemble_representations:
def body(x):
"""Add one element to the representations ensemble."""
i, current_representations = x
feats = slice_batch(i)
representations_update = evoformer_module(
feats, is_training)
new_representations = {}
for k in current_representations:
new_representations[k] = (
current_representations[k] + representations_update[k])
return i+1, new_representations
if hk.running_init():
# When initializing the Haiku module, run one iteration of the
# while_loop to initialize the Haiku modules used in 'body'.
_, representations = body((1, representations))
else:
_, representations = hk.while_loop(
lambda x: x[0] < num_ensemble,
body,
(1, representations))
for k in representations:
if k != 'msa':
representations[k] /= num_ensemble.astype(representations[k].dtype)
representations['msa'] = msa_representation
batch = batch0 # We are not ensembled from here on.
heads = {}
for head_name, head_config in sorted(self.config.heads.items()):
if not head_config.weight:
continue # Do not instantiate zero-weight heads.
head_factory = {
'masked_msa': MaskedMsaHead,
'distogram': DistogramHead,
'structure_module': functools.partial(
folding.StructureModule, compute_loss=compute_loss),
'predicted_lddt': PredictedLDDTHead,
'predicted_aligned_error': PredictedAlignedErrorHead,
'experimentally_resolved': ExperimentallyResolvedHead,
}[head_name]
heads[head_name] = (head_config,
head_factory(head_config, self.global_config))
total_loss = 0.
ret = {}
ret['representations'] = representations
def loss(module, head_config, ret, name, filter_ret=True):
if filter_ret:
value = ret[name]
else:
value = ret
loss_output = module.loss(value, batch)
ret[name].update(loss_output)
loss = head_config.weight * ret[name]['loss']
return loss
for name, (head_config, module) in heads.items():
# Skip PredictedLDDTHead and PredictedAlignedErrorHead until
# StructureModule is executed.
if name in ('predicted_lddt', 'predicted_aligned_error'):
continue
else:
ret[name] = module(representations, batch, is_training)
if 'representations' in ret[name]:
# Extra representations from the head. Used by the structure module
# to provide activations for the PredictedLDDTHead.
representations.update(ret[name].pop('representations'))
if compute_loss:
total_loss += loss(module, head_config, ret, name)
if self.config.heads.get('predicted_lddt.weight', 0.0):
# Add PredictedLDDTHead after StructureModule executes.
name = 'predicted_lddt'
# Feed all previous results to give access to structure_module result.
head_config, module = heads[name]
ret[name] = module(representations, batch, is_training)
if compute_loss:
total_loss += loss(module, head_config, ret, name, filter_ret=False)
if ('predicted_aligned_error' in self.config.heads
and self.config.heads.get('predicted_aligned_error.weight', 0.0)):
# Add PredictedAlignedErrorHead after StructureModule executes.
name = 'predicted_aligned_error'
# Feed all previous results to give access to structure_module result.
head_config, module = heads[name]
ret[name] = module(representations, batch, is_training)
if compute_loss:
total_loss += loss(module, head_config, ret, name, filter_ret=False)
if compute_loss:
return ret, total_loss
else:
return ret
Comparing Features and Predictions
While our implementation doesn’t include some of AlphaFold’s advanced features – such as ensembling, the use of protein structures as templates, or training capabilities (our version focuses solely on inference and does not implement loss calculation or backpropagation) – it still accomplishes its primary goal of predicting protein structures. This comparison shows the prediction of the hemoglobin beta subunit by AlphaFold (in blue, generated with ColabFold) and our implementation (in red). Despite the differences in features, our model achieves a result that captures the essential structure.

Comparison of Hemoglobin Beta Subunit Predictions by AlphaFold and Our Implementation
Guided Learning with Jupyter Notebooks
Our Jupyter Notebooks are at the heart of AlphaFold Decoded’s educational experience, offering a structured, hands-on approach to learning. We’ve divided our implementation into nine separate folders, each containing a Python file with skeleton code for the core methods and a corresponding Jupyter Notebook that guides students through the implementation process. This setup ensures that learners can follow along step-by-step, building their understanding in a structured way.
One of the biggest challenges in understanding or building complex projects is their non-linear structure. In the original AlphaFold repository, you often find yourself jumping from one file to another, tracking down various methods, which can be overwhelming. Our material addresses this issue by providing a clear, linear pathway, allowing students to focus on building their understanding without getting lost in the complexity.
Skeleton Implementations & Concrete Guidance
We provide detailed skeleton implementations of key methods, complete with concrete tips and explanations to guide students. From these foundations, they are tasked with implementing all the logic themselves, ensuring they engage deeply with the material and develop a strong grasp of how AlphaFold operates, rather than simply following pre-written code.
Automatic Error Checking
Our automatic error-checking system verifies each step of the user’s implementation, providing immediate feedback. This feature is crucial because constructing a project as complex as AlphaFold from scratch is incredibly challenging without continuous validation. By pinpointing issues early, students can progress confidently, learning from their mistakes and refining their code incrementally.
Google Colab Integration
Integration with Google Colab was proposed by one of our professors, who highlighted the difficulties students often face in setting up their development environments. Using Colab, students can bypass these challenges and work on AlphaFold Decoded without any installation hurdles. Furthermore, Colab’s cloud-based infrastructure allows students without access to high-performance hardware to still participate in running computationally demanding tasks, like protein structure prediction.
Try It Out Yourself
If you'd like to explore AlphaFold Decoded, you can try it out yourself! Simply download the tutorials
folder from our GitLab repository and upload it to your Google Drive. Make sure to add the Colab extension from the Google Workspace Marketplace. When you open a notebook, give Google Colab all necessary permissions when prompted. Once everything is set up, you can double-click the first notebook, tensor_introduction.ipynb
, in Google Drive to open it in Colab and work through the guided exercises.
Alternatively, if you prefer to see the full AlphaFold model in action, you can download the solutions
folder from our GitLab repository, upload it to Google Drive, and open model/model.ipynb
in Colab. Don’t forget to select a GPU under Runtime > Change runtime type
to experience running AlphaFold on a precomputed alignment. This provides a great way to engage with AlphaFold's protein structure prediction process firsthand!
Rolling Out AlphaFold Decoded
Our journey to share AlphaFold Decoded started with offline workshops at our university, where we engaged students from biology and computer science. These sessions included lectures and interactive coding workshops, where participants completed individual lessons from our AlphaFold Decoded curriculum. The workshops provided a hands-on learning experience, combining theoretical and practical elements.

Offline Workshop for AlphaFold Decoded Lesson 1 on May 14th, 2024

Offline Workshop for AlphaFold Decoded Lesson 1 on May 14th, 2024
As we received enthusiastic responses at international iGEM meetups, we recognized the need for a more flexible learning option, which led us to adopt hybrid workshops. However, even in these workshops, many participants expressed a desire for an asynchronous format, as fixed timelines often clashed with their academic schedules or individual pacing. This feedback pushed us to develop a more comprehensive solution, resulting in a nine-part video series on YouTube. This format allows learners worldwide to engage with the material at their own pace, ensuring that no one is left behind regardless of their other commitments.
Our YouTube Series
The transition to a video format was a significant undertaking, requiring us to produce nine detailed videos packed with rich visualizations and animations. This effort ensured that complex concepts were presented in an engaging and accessible manner, making it easier for viewers to grasp everything from the foundational elements to the more advanced aspects of AlphaFold. By leveraging this medium, we’ve been able to reach a much wider audience and offer a more immersive learning experience than traditional workshops alone could provide.
Bridging Fields: The Reach of AlphaFold Decoded

Biology students will find AlphaFold Decoded valuable as it deepens their understanding of AI-driven protein structure prediction, directly enhancing their knowledge in molecular biology. Computer science students can benefit from AlphaFold Decoded by applying machine learning concepts to real-world biological challenges, expanding their expertise in AI applications. For the iGEM community, AlphaFold Decoded provides a powerful tool for integrating protein modeling into synthetic biology projects, potentially boosting their research and competition outcomes. Young professionals can leverage AlphaFold Decoded to gain cutting-edge skills in AI and bioinformatics, positioning themselves for success in rapidly evolving interdisciplinary fields.
Course Content Overview
Our educational series follows a structured pathway, ensuring learners progress from foundational knowledge to complex AlphaFold-specific content. Here’s a visual representation of our nine-part roadmap:

1. Introduction to Tensors
In this introductory lesson on tensors, we learned that tensors are n-dimensional arrays fundamental to machine learning and deep learning models like AlphaFold. PyTorch provides efficient operations on tensors and automatic differentiation for gradient-based learning. We covered how to create and manipulate tensors in PyTorch, perform axis-wise operations, and use advanced methods like broadcasting and torch.einsum for efficient tensor operations. These concepts are essential as we move forward in implementing AlphaFold and understanding protein structure prediction.
Learn more about the lesson
+
This lesson introduced the concept of tensors, explaining that they are n-dimensional arrays crucial for representing data in deep learning models like AlphaFold. Tensors generalize familiar structures like vectors and matrices, allowing us to work with multi-dimensional data efficiently. In PyTorch, tensors are the core data structure, and the framework offers efficient tensor operations and automatic differentiation to facilitate model training. We began by learning how to create tensors in PyTorch using functions like torch.tensor, torch.linspace, and torch.ones, and explored how tensors support mathematical and logical operations. Understanding tensor shapes and manipulation, including reshaping and indexing, is key to working with multi-dimensional data.
The lesson also introduced axis-wise operations, where computations are performed along specific dimensions of a tensor. Operations like torch.sum, torch.argmax, and torch.mean were explained in the context of summing or reducing along axes. Non-reducing operations like the softmax function, which is vital for generating probability distributions, were also covered. We discussed broadcasting, a powerful technique that allows operations on tensors with mismatched shapes by implicitly expanding smaller tensors. The lesson demonstrated how broadcasting simplifies computations, such as adding a vector to each row of a matrix, and introduced the ellipsis operator for handling higher-dimensional tensors.
Finally, the torch.einsum function was introduced as a concise and flexible way to perform complex tensor operations like matrix multiplication and the outer product. This function uses Einstein notation to handle operations across multiple dimensions efficiently. Throughout the lesson, we emphasized that these tensor operations form the foundation for more complex tasks in AlphaFold, from handling input data to computing model predictions. By mastering tensor creation, manipulation, and advanced operations, learners will be well-prepared to tackle the upcoming challenges in protein structure prediction.
2. Introduction to Machine Learning
In this lesson, we introduced the fundamentals of machine learning, focusing on building a two-layer feed-forward neural network for handwritten digit classification using basic tensor operations in PyTorch. We discussed hierarchical feature extraction, non-linearities like ReLU and sigmoid, and how to optimize models using gradient descent. Additionally, we explored how the loss function helps evaluate model performance and how gradients are computed to adjust parameters during training.
Learn more about the lesson
+
This lesson introduced machine learning concepts, focusing on building a two-layer feed-forward neural network to classify handwritten digits. Starting with the idea of pattern recognition through dot products, we demonstrated how a machine learning model identifies hierarchical features, such as specific image parts like circles or lines. The network processes images by flattening them into vectors, which are multiplied by weight matrices that represent feature templates, followed by matrix multiplications to derive predictions. To prevent collapsing into a single matrix multiplication, non-linear activation functions like ReLU are applied between layers.
We discussed the importance of using biases in linear layers to improve flexibility and model accuracy. Next, we covered the loss function, focusing on L2-loss, which measures the difference between predicted outputs and actual labels. We explained how the model optimizes its weights using gradient descent, where gradients—calculated via backpropagation—inform the direction and magnitude of parameter updates to minimize the loss.
We then delved into the math of calculating derivatives for various layers, from the L2-loss and sigmoid to ReLU and affine linear layers. While the gradient computation can be complex, PyTorch handles these calculations automatically through its autograd feature. The overall training process involves multiple iterations of forward passes, loss computation, gradient calculation, and weight updates. Finally, we emphasized how this foundational machine learning process prepares us for the more advanced models, including AlphaFold. The lesson provided hands-on learning with a tutorial notebook to apply these concepts and create a machine learning model from scratch.
3. Introduction to Attention
In this lesson, we explored the concept of attention in machine learning, a crucial mechanism for working with sequential data like text or proteins. We discussed how attention determines which parts of an input sequence are most relevant, using key, query, and value vectors to compute attention scores. These scores help focus on important input elements, making it easier to generate accurate outputs. Additionally, we covered advanced topics such as self-attention, multi-head attention, and the practical applications of attention in models like AlphaFold.
Learn more about the lesson
+
This lesson provided an in-depth introduction to attention in machine learning, starting with its core idea of focusing on relevant parts of an input sequence. Attention uses key, query, and value vectors, which are created from the input data, to compute attention scores based on their dot-product similarities. These scores, normalized with softmax, help the model decide which inputs to prioritize when making predictions. This process allows for parallel processing of sequences and makes attention an ideal tool for tasks involving language, music, or protein sequences, like in AlphaFold.
We discussed several methods for applying attention to machine learning models, including self-attention, where both the queries and keys come from the same inputs. Self-attention is particularly useful in tasks like protein structure prediction or natural language processing, as it helps models understand the relationships between different parts of a sequence. We also introduced multi-head attention, which runs several attention mechanisms in parallel to capture more diverse aspects of the input.
One of the practical challenges of attention is its high memory usage due to the N² scaling of attention scores. AlphaFold addresses this with techniques like global attention, which reduces memory requirements by averaging the query vectors. Another challenge is handling sequential data, especially in tasks like next-word prediction, which requires causal attention masks to prevent the model from "cheating" by looking at future tokens. These masks ensure the model only attends to past tokens during inference.
We also explored the issue of permutation invariance in attention models, where the order of input tokens can be ignored by the model. To solve this, positional encodings are added to the input, allowing the model to learn the sequence structure. These encodings can be either static (like sinusoidal functions) or learned, both of which are used effectively in models like AlphaFold.
The lesson highlighted the transformative impact of attention in machine learning, as seen in the widely-used Transformer architecture. Transformers, which rely on attention, revolutionized natural language processing by enabling faster and more efficient sequence-to-sequence tasks like translation. We covered the key components of the Transformer, including multi-head attention, residual connections, and feed-forward layers, which together form the foundation of modern AI models.
Finally, we emphasized the importance of attention in AlphaFold, where it plays a crucial role in processing the sequences of amino acids to predict protein structures. With this foundational understanding of attention, the next steps in our series will focus on implementing these concepts in AlphaFold, starting with feature extraction from protein data files. The tutorial notebook for this lesson will help reinforce these ideas by guiding you through building a multi-head attention module and a Transformer encoder for sequence classification tasks.
4. Feature Extraction
This lesson introduces feature extraction in AlphaFold, where biological data is converted into machine learning-ready tensors. We focus on three key inputs: the amino acid sequence, multiple sequence alignment (MSA) data, and template structures. We explain how evolutionary information from MSA is used to predict protein structures and walk through the steps to process this data, including deletion handling, clustering, and sequence masking. The extracted features are critical inputs to AlphaFold’s Evoformer module, which will be covered in upcoming lessons.
Learn more about the lesson
+
In this lesson, we dive into AlphaFold’s feature extraction process, focusing on converting biological data into tensors for machine learning. AlphaFold uses three primary inputs: the amino acid sequence, MSA (multiple sequence alignment) data, and 3D structures of templates, though we simplify our approach by omitting the template stack. The amino acid sequence is straightforwardly converted to a one-hot encoding, but MSA data requires more complex processing due to variations like insertions, deletions, and evolutionary mutations across sequences. We introduce the Needlemann-Wunsch algorithm for aligning sequences and explain how deletions are handled by counting them while disregarding specific deleted amino acids.
The MSA data is processed into several key features, such as amino acid profiles and deletion counts. Unique sequences are clustered, and cluster centers are chosen, always including the target sequence. These clusters undergo a masking process where amino acids are randomly replaced to enhance model robustness. After masking, extra sequences are assigned to the clusters based on their similarity to the cluster centers, and features like deletion counts and amino acid types are averaged across clusters.
We then calculate the features for cluster centers and extra sequences, which are used as inputs to AlphaFold’s Evoformer module. These features include the one-hot encoding of amino acids, deletion counts normalized by an arctan function, and amino acid profiles averaged over clusters. The process involves randomness in selecting cluster centers and during masking, ensuring varied inputs across multiple prediction runs. Finally, we concatenate the features to create inputs like msa_feat and extra_msa_feat, ready for AlphaFold’s further processing.
This lesson emphasizes how feature extraction in AlphaFold leverages evolutionary data to inform protein structure prediction. It also highlights the importance of domain-specific data transformation for effective machine learning. In the next videos, we’ll explore how AlphaFold’s Evoformer module transforms these features into predictions, and later, how it outputs protein structures from these tensor inputs.
5. The Evoformer
In this lesson, we explore AlphaFold's Evoformer, which is responsible for most of the model's parameters and processes evolutionary and structural information about proteins. The Evoformer consists of multiple identical blocks, with both an MSA (multiple sequence alignment) stack and a pair stack that exchange information via row- and column-wise attention mechanisms. This lesson breaks down the Evoformer's components, including its specialized triangle attention mechanism that helps encode relationships between residues, and discusses how it integrates these features to help predict protein structure.
Learn more about the lesson
+
This lesson delves into AlphaFold’s Evoformer, the largest component of the model, accounting for 88M of the 93M parameters. The Evoformer plays a key role in transforming the MSA (multiple sequence alignment) and pair representations into features useful for protein structure prediction. It consists of multiple identical blocks that include an MSA stack (working on the MSA representation) and a pair stack (working on pairwise residue interactions). The main challenge of processing large proteins is tackled by employing row-wise and column-wise attention mechanisms instead of full attention, which would be computationally expensive.
The MSA stack performs row-wise attention using the pair representation as a bias, followed by column-wise attention and a feed-forward transition module. The pair stack features a unique triangle attention mechanism, which incorporates information from other residue pairs to ensure the structural consistency of the protein. Specifically, the triangle attention computes updates between residues using information from their shared neighbors, mimicking the geometric relationships in a protein's 3D structure.
The Evoformer block has an “Outer Product Mean” operation, which reshapes the MSA representation into the pair representation, facilitating information exchange between the two. The pair stack’s attention mechanisms, triangle multiplicative updates, and transition modules help refine the representation of pairwise interactions to generate the final prediction. Residual connections throughout the Evoformer ensure stable training, allowing the model to add learned corrections rather than replace inputs completely.
Finally, the MSA representation is reduced to a single sequence, the target sequence, which is passed on to the structure module for further refinement. The Evoformer, though more complex than a traditional transformer, is composed of straightforward operations that handle the enormous protein data by simplifying attention mechanisms. The next video will explore the missing input embedding step, linking the extracted features to the Evoformer’s processing pipeline, while this lesson's notebook provides a hands-on guide to implementing the Evoformer.
6. Feature Embedding
In this lesson, we focus on feature embedding in AlphaFold, which bridges the gap between feature extraction and the Evoformer. Feature embedding involves transforming extracted features, such as MSA and target features, into learned embeddings and adding positional encodings. Additionally, we explore the Recycling Embedder, which updates the pair and MSA representations using predicted outputs from previous iterations, and the Extra MSA Stack, which refines these features similarly to the Evoformer but with optimizations for large datasets.
Learn more about the lesson
+
This video focuses on feature embedding in AlphaFold, which converts extracted data into a format suitable for the Evoformer. Feature embedding is done in three main steps: the input embedder, the Recycling Embedder, and the Extra MSA Stack. The input embedder processes the target and MSA features, embedding them with linear layers and adding positional encodings. The positional encodings are derived from residue indices, and relative positions are one-hot encoded to ensure the model can understand residue relationships.
The Recycling Embedder, used during iterative predictions, updates the pair and MSA representations using the predicted pseudo-beta carbon positions from the last iteration. By calculating pairwise distances between these carbons, encoding them, and normalizing the representations, the Recycling Embedder integrates previously predicted structural information into the next prediction. During training, AlphaFold uses this recycling approach to improve efficiency by only optimizing parameters for the final prediction, skipping backpropagation for intermediate steps.
The Extra MSA Stack updates the extra MSA and pair representations and is almost identical to the Evoformer but uses fewer blocks (4 instead of 48) and includes global attention for better memory efficiency. Global attention reduces the computational load by averaging queries and broadcasting outputs, making it suitable for large datasets like the extra MSA sequences. The lesson wraps up by introducing key concepts like global attention and recycling while setting the stage for the upcoming construction of the structure module.
The next video will dive into 3D geometry concepts like rotation matrices and quaternions, which are essential for building the protein’s 3D structure, marking a deeper dive into the final parts of the AlphaFold model.
7. Geometry
This video delves into 3D geometry concepts essential for AlphaFold’s Structure Module. It covers how to convert the model's tensor outputs into atomic positions using quaternions for rotations and torsion angles for side chains. We also explore how transformations (rotations and translations) are applied to construct and globalize the atomic coordinates for amino acids. These concepts are crucial for building the protein’s 3D structure in AlphaFold.
Learn more about the lesson
+
In this video, we discuss the advanced 3D geometry concepts required to implement AlphaFold’s Structure Module, focusing on how to transform predicted tensor outputs into atomic positions. AlphaFold predicts three scalar coordinates for backbone translations and a quaternion for backbone orientation. Quaternions, rather than rotation matrices, are used for rotations because they are better suited to neural networks and maintain uniform distributions of rotations. Quaternions are scaled to a unit length, which allows a uniform distribution of rotation angles around an axis.
For torsion angles, AlphaFold predicts two values (cosine and sine), representing bond rotations, and these angles are applied to the side-chain atoms. The video explains how transformations are composed of a rotation and a translation, modeled using 4x4 matrices in homogeneous coordinates. This allows efficient application of rotations and translations as matrix multiplications, which is crucial for handling complex protein structures. The concept of transforming local frames to global frames is key to understanding how atomic positions are constructed in AlphaFold.
The backbone frame is defined using the backbone atoms, such as the C-alpha, nitrogen, and carbonyl groups, while side-chain frames are defined relative to these. We use transformations to update the frames as we apply the torsion angles for different residues. Additionally, the video explains how to use the Gram-Schmidt process to orthonormalize vectors for constructing rotation matrices, ensuring that rotations are well-behaved in 3D space.
After constructing the rotation and translation matrices, we apply these to calculate global frames for each amino acid. Finally, these global frames are used to compute the exact 3D positions of atoms within the protein. With the foundation of these geometric transformations, AlphaFold can build the entire protein structure from the predicted backbone and side-chain transformations. The accompanying Jupyter notebook provides hands-on implementation of these concepts, bridging the gap between theoretical geometry and practical AlphaFold development.
8. Structure Module
This video covers AlphaFold's Structure Module, which translates Evoformer outputs into 3D atom positions. It introduces Invariant Point Attention (IPA), where 3D distance between predicted points, rather than dot-products, determines attention weights. The module iteratively refines the backbone's transforms and torsion angles over several rounds, with each iteration adjusting the backbone's orientation and position, ultimately predicting atom coordinates for the final protein structure.
Learn more about the lesson
+
In this video, we explore the Structure Module of AlphaFold, which takes the Evoformer outputs and converts them into actual atom positions. A core part of this module is Invariant Point Attention (IPA), which replaces standard attention with a mechanism that computes attention weights based on the 3D distance between residues, treating key and query vectors as points in space. These distances are calculated using the current residue positions and transforms, updated iteratively across several rounds. Instead of only considering dot-product similarities for attention, AlphaFold integrates spatial proximity by globalizing predicted local coordinates and weighting closer residues more heavily.
Each iteration of the Structure Module refines the backbone’s rotation and translation using quaternion-based updates, gradually improving the protein’s predicted structure. AlphaFold refines the backbone transforms in eight iterations, initializing all residues at the origin in the first round (referred to as "black hole initialization"). Along with updating positions, the module computes torsion angles for the protein backbone and side chains. During training, predictions from each iteration contribute to the final loss, ensuring the model can produce accurate outputs at each step.
The Backbone Update module predicts a quaternion and translation for each residue. AlphaFold uses a simplified method of predicting quaternions, initially setting the scalar part to one, to ensure small, incremental rotations across iterations. After calculating the final backbone transforms and torsion angles, the method computeAllAtomCoordinates from the previous video is used to compute the exact atom positions.
The Structure Module outputs the torsion angles, backbone transforms, heavy atom positions, atom masks, and pseudo-beta positions. The pseudo-beta positions are used for recycling into subsequent AlphaFold iterations. AlphaFold also manages unit conversion between nanometers and angstroms, scaling the translation vectors accordingly. With this, the Structure Module completes the protein structure prediction, and in the next video, we'll combine all parts for the full AlphaFold implementation.
9. Full Model
This final video of the AlphaFold implementation series brings together all the modules previously built into a complete model. The modules include the Input Embedder, Extra MSA Embedder, Recycling Embedder, Evoformer Stack, Extra MSA Stack, and Structure Module, all working in tandem to predict protein structures. The AlphaFold inference pipeline processes inputs iteratively, using previous outputs to refine predictions. The video concludes with a note on potential troubleshooting and thanks to the viewers for completing the series.
Learn more about the lesson
+
In this final video, we put together the entire AlphaFold model from the modules we've built over the series. The model is composed of six main modules: Input Embedder, Extra MSA Embedder, Recycling Embedder, Evoformer Stack, Extra MSA Stack, and the Structure Module. These modules process inputs, create embeddings, and refine representations to predict accurate atom positions for protein structures. The process starts with Input Embedding, which creates the MSA and pair representations. The Recycling Embedder adjusts these representations using outputs from prior model iterations, particularly based on pseudo-beta carbon distances.
The Evoformer Stack, a transformer-like module, is the core of the network, performing updates through row-wise and column-wise attention layers. The Extra MSA Stack processes sequences not selected as cluster centers, contributing via a shallower model. After Evoformer, the Structure Module transforms the enriched tensors into precise 3D atom positions, using Invariant Point Attention based on residue proximity.
The full AlphaFold Inference Method combines these modules, iterating over them multiple times to improve prediction accuracy. The video mentions simplifying the inference process by excluding ensembling and template stacks. Key points include how recycling is handled by zero-initializing tensors in the first iteration and how new inputs are generated for each recycling step, adding variability to the process. Some troubleshooting tips are provided, including adjusting tensor datatypes and device settings. The series concludes with congratulations for completing the journey into AlphaFold and a promise of future developments, potentially with AlphaFold 3.
Reception and Impact
So, this is the content – it’s certainly comprehensive and covers a lot of ground. Despite the complexity and depth of the topic, we were genuinely excited to see that our videos reached a much wider audience than expected, gathering an impressive 2,920 views. This level of engagement shows there’s a real interest in diving deep into such challenging material. Even more rewarding were the comments we received from viewers, sharing their thoughts, questions, and appreciation. Here’s a look at some of the feedback that’s helped shape and motivate our journey.
"Wonderful project. Wondering if you're going to Cover AF3?"
@lialexwei509
"Nice, dude! And I don't care what they say about you recording this while being held hostage at gunpoint by DeepMind, with blood saturated with Adderall, I believe you're just really passionate about this stuff, and I'm gonna binge watch all your videos!"
@matveyshishov
"What a time to be alive!"
@ufuoma833
"Thank you so much for this amazing series! I’ve found the in-depth exploration of AlphaFold’s theory and implementation incredibly helpful. I’m curious to know if you have any plans to create a series on AlphaFold 3 in the future? I’m really interested in this field and would love to see more content on the latest developments. Thanks for all your hard work!"
@xiangtianlin8006
"That's a very comprehensive summary for the input feature of AlphaFold2, which helps clarify things a lot. Thanks very much for your work!"
@于睿-r6e
"This is wonderful. Thank you brother."
@rizbaruah752
"Wow this video made my day! Thank you!"
@fjgkjkojk
"Please finish this series - looking forward to the whole thing. Breaking down AlphaFold is going to have huge value.
I have one suggestion. While you can, start off the series differently with the high-level on AlphaFold first/videos on that and a more first principled breakdown of what we're building/high-level understanding before going into the code. Right now, the video about tensors without context on much else is less interesting. Especially because things that lack context don't tap into the right motivation circuits to make people interested, whereas if you explain first, then talk about the topics, it will be far more engaging.
Good luck with this channel!"
@adammajmudar889
"Can't wait for more lessons."
@cariyaputta
"Wow! I saw this video and instantly subscribed! That is great value content. Thanks for sharing. Greetings from Brazil!"
@douglasespindola5185
"Nice and simple explanation."
@zandacr0ss86
Prof. Edmund Beck, an accomplished scientist with extensive experience in both academia and industry, shared his thoughts on our videos. As a Distinguished Science Fellow at Bayer AG and an honorary Professor at Technical University Dortmund he is actively involved in the open source initiative OpenFold. With expertise ranging from quantum theory to life sciences, Prof. Beck offers a unique perspective on modern science communication. Here’s what he had to say about our work:
You can see the new generation of scientists at work here. This is how communication should be today - accessible and engaging, yet scientifically rigorous.
AlphaFold Decoded is more than just a project; it’s a community-driven effort to empower the next generation of scientists with the tools and knowledge to actively contribute to AI-driven breakthroughs in biology.
Outlook
As we wrap up this project, it’s clear that many participants are eager to keep going, with numerous questions already about the future and potential developments like AlphaFold 3. This enthusiasm reinforces our belief that there’s a real need for accessible, in-depth education in computational biology. Our goal isn’t just to share our work but to help establish a format for learning that engages and inspires others in the community. Toward this goal, we made our entire project, including the build logic of tutorial materials and the scripts for the chapters, available on the iGEM Aachen Gitlab. This kind of comprehensive education is still rare in our field, and we hope that by pioneering this approach, we can encourage others to take the lead, build on it, and create their own initiatives. Together, we can shape the future of computational biology education.
References
References
+
1. Mandon, K., AlphaFold Decoded Codebase. A comprehensive implementation and educational resource for understanding AlphaFold's architecture and functioning. Available from: https://gitlab.igem.org/2024/software-tools/aachen.
2. Jumper, J., et al., Highly accurate protein structure prediction with AlphaFold. Nature, 2021. 596(7873): p. 583-589. Available from: https://doi.org/10.1038/s41586-021-03819-2.
3. Ahdritz, G., et al., OpenFold: An Open-Source Implementation of AlphaFold. A valuable resource for code verification and utilizing pre-trained weights. Available from: https://github.com/aqlaboratory/openfold.
4. Senior, A.W., et al., AlphaFold Codebase. The official code repository that inspired and supported our implementation with code snippets and structural insights. Available from: https://github.com/google-deepmind/alphafold.
5. Mandon, K., AlphaFold Decoded YouTube Channel. Educational video series on AlphaFold, offering a step-by-step breakdown and explanation. Available from: http://www.youtube.com/@AlphaFoldDecoded.