To learn about the Engineering workflows for each part of out project, click the different tabs below!
Introduction
Phage-assisted continuous evolution (PACE) offers a new experimental machine for protein engineering with DE. By leveraging the rapid replication of bacterial cultures and bacteriophages, PACE enabled simultaneous generation and selection from mutant libraries in culture, eliminating the need for synthetic mutant libraries, the most resource intensive component of any DE experiment. While it held great promise for lowering the barrier to entry to DE, several challenges remained. Regulating PACE mechanisms relies on chemical inducers, which are costly, limiting, and difficult to maintain over multi-day experiments. More practically, running a PACE experiment requires chemo or turbidostat bioreactors, which can cost thousands of dollars even with only the minimum commercial components.
For our goal of making DE methods accessible, we designed a biological tool for regulating gene expression during a PACE experiment that uses optogenetics and a complimentary hardware platform to maintain continuous culture and light-based induction conditions. Here is a breakdown of the costs for running a PACE experiment using our optogenetic tools, and two chemical inducers, IPTG and Arabinose:
Through this cost analysis, we identified that one would have to run at least 3 experiments before OptoPACE becomes cost-efficient. However, based on Liu et al. 2020 and Shin et al. 2020, we recognized that it often takes more than just 3 experiments to test the component system and perform full runs with high-resolution data. Thus, there is a need to have an alternative solution, like OptoPACE, that can cut down experimental costs. Now, better informed on the costs associated with maintaining PACE systems over long periods of time, we moved on to characterizing and quantifying gene expression of our tools.
Optogenetic Tools For Control of Directed Evolution
Phytochromes and cyanobacteriochromes (CBCRs) are closely related prokaryotic photoreceptors. Phytochromes are found in bacteria and respond to red light wavelengths, while CBCRs, which originate from cyanobacteria, respond to a much broader range of wavelengths including ultraviolet, red, far-red, and green light. This wide range makes CBCRs very valuable tools for protein engineering. With such versatility, they enable the design of photoreversible and multiplexed systems that are capable of responding to different wavelengths. Such systems offer greater control and tunability over experimental procedures.
To build an optogenetic control system for DE, we utilized the CBCRs UirS/UirR from Synechocystis sp. PCC 6803. UirS/UirR is activated by UV-violet light and suppressed by green light. This ON/OFF optogenetic switch offers faster control than chemical inducers and should allow for specific tunability within a narrow range of wavelengths (Ramakrishnan, Tabor, 2016). When combined with other CBCRs, the subsequent multicomponent optogenetic system could independently control multiple sets of genes, allowing for more sophisticated protein engineering than previously possible.
Design:
Before we started testing our optogenetic constructs, we needed to conduct a preliminary experiment to quantify the baseline levels of our chosen reporter gene for all expression assays, sfGFP, in a positive control plasmid. As such, we measured the fluorescence of E. Coli containing the construct GlpT-sfGFP, which constitutively expresses sfGFP. We also used this experiment to confirm the emission and excitation values of sfGFP, and test the type of protocol we would later use to test our optogenetic constructs, which was adapted from Nakajima et al.
Build:
To test the relationship between the relative fluorescence (RFU) of sfGFP and optical density at 600 nm (OD600) of E coli expressing our positive control plasmid, we used NEB Stable Competent E. Coli cells containing GlpT-sfGFP, which constitutively expresses sfGFP. Black 96 well plates with clear bottoms were used to reduce background noise from scattering light and allow for OD600 measurements. We washed and resuspended the cells in Phosphate Buffered Saline (PBS) to remove the background noise of residual media for the plate reading. We then performed serial dilutions using the cultures and PBS buffer, to measure the OD600. Additionally, for a negative control, we included serial dilutions of an E. Coli culture that constitutively expresses mScarlet, a protein that emits fluorescence at different wavelengths to compare RFU values.
Test:
We tested the relationship between the relative fluorescence (RFU) and optical density at 600 nm (OD600) for sfGFP expression of our positive control plasmid. Using an overnight culture of the GlpT-sfGFP containing cells, the cells were initially washed twice with PBS and resuspended. This culture was distributed among four rows to make replicates for more accurate data collection, and across each row, the culture was serial diluted twofold. The same process was performed for the bottom four rows of the plate using the culture of the cells that expressed mScarlet fluorescence. To confirm commonly agreed upon literature values of excitation and emission for sfGFP, the spectra for fluorescence excitation and emission for an sfGFP expressing well were taken using the plate reader. Then, the plate was read to measure sfGFP fluorescence at an excitation value of 480 nm, emission of 515 nm, with OD600 measured in parallel.
Learn:
Looking at the fluorescence spectra of an sfGFP expressing culture, there is a positive slope up in excitation RFU around the end of the graph at 500 nm, and a peak around 515 nm for emission. Thus, it was determined that an excitation value of 480 nm and emission value of 515 nm was sufficient to measure the fluorescence of sfGFP, as any value higher than 480 nm for excitation would be too close to the emission value. These values would subsequently be used for future fluorescence readings of sfGFP.
The fluorescence, an average of each dilution across the four replicates, was taken and plotted against the average OD600. As expected, fluorescence in the sfGFP expressing cells increased as the OD600 increased, while the mScarlet expressing cells showed little expression. As noted by the R2 value of 0.9907, the OD600 and the fluorescence measurement had a strong linear correlation. This relationship was used to standardize the results of our future experiments, since by measuring the RFU per OD600 we could account for the number of bacteria in each sample.
Design:
In order to test the activity of the UirS/UirR output promoter itself (PcsiR1-109), we designed an experiment that would compare it to some other existing promoter systems. As part of UirS/UirR, this promoter is activated at 400 nm and deactivated at 530 nm (Ramakrishnan, Tabor, 2016). The sensor and regulator are both bound to a membrane and when exposed to the appropriate wavelength, the sensor phosphorylates the regulator. A chromophore is also a necessary component to enable light absorption of the system. The proper wavelength absorption induces isomerization of the chromophore, turning the CBCR into an activated state. Conversely, the absorption of the reversion wavelength will bring it back to the ground state. When UirS phosphorylates UirR, transcription is activated from the promoter (PcsiR1-109) towards the gene of interest. We designed an experiment to compare this promoter to our positive control plasmid and a well characterized repressor system (araC) as a negative control.
Build:
Three constructs were needed for this experiment. Those were the UirS/UirR system (FUExp), one under a constitutive promoter, and one under a repressor. The positive control construct contained the constitutive promoter GlpT driving sfGFP (GlpT-sfGFP). For a negative control, we used an arabinose inducible promoter, araBad, driving sfGFP (Ara-sfGFP). Under standard culture conditions, the araBad promoter is not induced, for minimal sfGFP signal under all conditions tested.
GlpT-sfgfp:
FUexp:
Ara sfGFP:
Test:
As a control experiment, we tested the activity of each of these promoters under non-induced conditions. UirS/UirR was never exposed to the activation or deactivation wavelengths (kept in dark), Ara-sfGFP was never induced by arabinose, and GlpT-sfGFP promoted as usual since the construct is constitutively on by design (always expressing). Three flasks containing cultures were kept in the dark and within a shaking incubator at 37 degrees C for 6 hours. Samples were taken every 2 hours.
Learn:
The cells containing GlpT-sfGFP expressed the highest levels of fluorescence due to the constitutive promoter, which is on at all times. Our negative control, Ara-sfGFP cells had the lowest level of expression, which was expected as it wasn’t induced by arabinose. Compared to GlpT-sfGFP, the cells containing FUexp showed significantly less expression. Likely because the promoter was never exposed to the inducing light condition, UV-Violet. This experiment was extremely important in characterizing the behavior of the promoter itself. FUexp does show some level of expression even in the dark condition; however, it is extremely damped compared to a constitutively on system (8 fold less). Using these controls, moving forward we can better understand how our expression under UV-Violet and green light compares to the baseline expression under dark conditions.
Design:
This experiment was the first iteration of characterizing the UirS/UirR’s expression under a combination of inducible and repressive conditions. The UirS/UirR photoreceptor has an excitation wavelength of 400 nm and a reversion wavelength of 530 nm. This construct also includes a PCB chromophore, a necessary cofactor for UirS/UirR and other CBCRs that absorb these wavelengths of light. To evaluate the functionality of this system, the corresponding promoter (PcsiR1-109) was linked to a backbone containing sfGFP. This way, we could measure the resulting fluorescence of the system under varying wavelengths. The entire system was designed with a constitutive T7 promoter for consistent expression.
Build:
This construct was built using synthetic DNA, PCR, and assembly methods. Each sensor and regulator block was ordered individually due to size. We did a mutual primer reaction to assemble the UirR and UirS synthetic fragments. We then used PCR to amplify our product. To attach the chromophore, we first PCR amplified the PCB g-block and then used Gibson Assembly to attach it to UirS/UirR, making FU1. FU1 was put into our backbone containing Kan resistance, T7 terminator, sfGFP, and an origin of replication via Gibson assembly. We then transformed FUexp into NEB Stable Competent E. Coli cells. The other component of our build cycle was creating a system of boxes that allowed us to easily manipulate the light conditions. To do this, we designed two small boxes big enough to fit two flasks inside. The first box was configured with a green light that shines inside the box. The second small box had no light attachments and was used to block light for the darkness condition. Lastly, we made a single “big box” that shines UV light inside and fits over the two small boxes.
Test:
We tested the success of FUexp by running multiple optogenetic experiments. The goal of this first experiment was to characterize the gene expression of UirS/UirR in response to different wavelengths over a 16 hour period. This experimental setup consisted of 7 different LB cultures. One of these flasks, TD, was kept in the dark for the whole 16 hours. The other 6 flasks began in darkness for 4 hours. They were then induced under UV-violet light for 2 hours. After these 2 hours, the flasks were moved into different light conditions with two replicates for each condition. The first condition, flasks UV1 and UV2, were kept in UV light for the remainder of the 14 hours. The second condition, flasks D1 and D2, were moved to darkness after the two hour induction in UV. The third condition, flasks G1 and G2, were moved to green light after the initial 2 hours in UV. Samples were taken every two hours, PBS washed, and frozen. The next day, samples were thawed on ice and put onto a plate reader to measure absorbance at 600 nm and fluorescence at an excitation value of 480 nm and emission of 515 nm.
Learn:
Plate reader fluorescence measurements turned out largely inconclusive after this first experiment. It was difficult to differentiate any trends and some of the data was conflicting as fluorescence was unexpectedly high for green light and unexpectedly low for UV light conditions. Additionally, since we started all flasks in the induction wavelength it was difficult to determine if we were seeing the effects of each condition or simply the effects of induction. It was determined that some experimental design factors could be revised to better suit the optogenetic switch we are working with. For one, the collection method we were using required us to open the light-blocking boxes at each time point, which could have induced all flasks during the experiment. We also had concerns over our PBS washing step since our microcentrifuge tubes were clear, and samples were exposed to ambient light during every collection time which may have skewed our results. With further reflection upon this experiment, we created a revised protocol to hopefully yield more conclusive data regarding the effectiveness of this optogenetic tool.
Design:
After reviewing the UirS/UirR paper published by Ramakrishnan and Tabor (2016), we noticed key experimental design choices that we decided to implement into our second experiment. First, we reduced the duration of the experiment to 2 hours, collecting the samples in 15-minute increments to see the immediate effects of the lights in greater detail. Secondly, due to concern that the PBS washes were taking too long and introducing noise to our samples, we changed the media from LB to M9 minimal media. M9 minimal media was also chosen because of its benefits for fluorescence measurements such as low autofluorescence.
Build:
We used the same FUexp constructs and boxes as before.
Test:
The goal of this experiment was to characterize the gene expression of UirS/UirR in response to different wavelengths and to determine the switch-off time of UirS/UirR between UV violet and green light. The cultures were grown in the dark condition for 4 hours to avoid initial induction effects. The conditions are listed below.
Learn:
One unforeseen factor was the amount of time it would take to collect the samples. Rather than a matter of seconds, the entire collection process took around 8 minutes to complete. Due to the frequent sampling periods, the 15-minute incubations were not as precise as we planned, which led us to look for an alternative solution for our next iteration. Additionally, our cultures did not grow in M9, thus we did not read the fluorescence data for this iteration. A decision to use LB media in the subsequent experiments was made.
Design:
After the previous experiment, we recognized a need to implement a staggered start method that would eliminate the need to collect aliquots during the experiment. Rather, it would allow us to collect all of the samples for each timepoint and condition at the end, limiting the light contamination of our samples. Additionally, we switched from using flasks to culture tubes to better accommodate the staggered start method.
Build:
We used the same FUexp constructs as our first experiment. We built new boxes for the new light condition, UV Violet to Green as well as improved boxes to fully ensure light leakage was minimized and that light would fully reflect inside the box using foil-lined walls.
Test:
Our goal was to again characterize the gene expression of UirS/UirR in response to different wavelengths and to determine the switch-off time of UirS/UirR between UV violet and green light. The cultures were grown in the dark condition for 4 hours to avoid initial induction effects. In this setup, the samples with the longest incubation time were added first and the samples with the shortest time were added last, to emulate a staggered start. The tubes were then collected altogether at the end of the 2 hour period and PBS washed. The conditions are listed below.
Learn:
While consistent suppression by green light gave the lowest fluorescence values for most of the experiment, the effect was lost after 1.5h. Early UV induction with green suppression, consistent UV and the dark condition were mostly indistinguishable over the time course. While light contamination was minimized using the staggered start method it was still possible. We further hypothesized that the switch-off time from UV violet to green in our system was too long for our intended purpose and prevented detectable changes.
Design:
Identifying possible sources of error in our first two FUexp experiments allowed us to better assess our experimental flaws. In this experiment, we wanted to determine if this optogenetic switch was a viable option for future experiments and if the three conditions stated in the original paper were valid. To determine this, we chose to simplify the experiment to reduce the effects of light contamination or other unknown confounding factors.
Build:
We used the same FUexp constructs as our previous experiments and built two new boxes (one shining green and one shining UV) to ensure that light leakage was minimized and that the lights were shining directly on our cultures using foil lined walls.
Test:
In order to reduce any light leakage or background noise, we took only one sample after two hours of growing in their light conditions. We started all cultures in the dark conditions and then moved them to their light conditions. Each condition had four replicates. The UV flasks were kept under UV violet light for two hours, and the G flasks under green light for two hours. The TD flasks were kept under total darkness for two hours.
Learn:
Plate reader fluorescence measurements showed significant suppression by green light, however, the mechanism was not as repressive as reported in the literature, with only a 13% change in expression between the two conditions. Fluorescence remained high across all conditions, suggesting that UirS/UirR was functional for activation but did not effectively switch off under dark or green light conditions. Reviewing the construct, published UirS/UirR results made a careful choice in constitutive low active promoters that we did not replicate. Future iterations of this construct will test the originally published promoter system.
Introduction
The central aim of this part of the project is to answer the question: “Is it possible to generate a small peptide with good binding affinity without using large-scale experimental approaches?” To explore this possibility, we focused on optimizing peptides rather than ambiguous small-molecule drugs due to the relatively low cost associated with their development. In the field of Machine Learning-based De Novo Design, the main modalities for generation focus either on monoclonal antibodies, or on small molecule drugs. However, the pitfall these modalities fall into is the lack of viable synthesis pathways for designed solutions.
Furthermore, the composition and size of peptides allow them to be easily conjugated with a new and emerging delivery system: cell-penetrating peptides (CPPs) (2). With this in mind, we aim to develop software and machine-learning-based platforms to facilitate the accessible design of these short-chain peptide binders.
Target for De Novo Design Workflow:
To demonstrate the efficacy of our model, we selected a structurally relevant target that has been extensively modeled and studied: HIV reverse transcriptase (HIV-RT). According to paper (3), it was experimentally determined that HIV complexes with endogenous human proteins, predominantly eEF1A, as part of its transcriptional activities. Therefore, for our application, we aim to design small peptides that can bind to and inhibit the protein-protein interactions between HIV-RT and endogenous eEF1A. Thus, we will use the structures of HIV-RT from the RCSB PDB (Research Collaboratory for Structural Bioinformatics Protein Data Bank) as our templates for design.
In this section, you will find common metrics in AlphaFold2 and some for ColabDesign
-
plddt: per-residue measure of local confidence
pLDDT measures confidence in the local structure by estimating how well the prediction aligns with an experimental structure. It is based on the local distance difference test Cα (lDDT-Cα), which evaluates the correctness of local distances without relying on superposition. pLDDT is scaled from 0 to 100, with higher scores reflecting greater confidence and typically more accurate predictions.
-
i_ptm
ipTM measures the accuracy of the predicted relative positions of the subunits forming the protein-protein complex. Values higher than 0.8 represent confident high-quality predictions, while values below 0.6 suggest likely a failed prediction. ipTM values between 0.6 and 0.8 are a grey zone where predictions could be correct or wrong. These values assume modelling with multiple recycling steps, so the process of prediction reaches a degree of convergence.
-
i_pae: median predicted aligned error at the interface
iPAE measures how confident AlphaFold2 is in the relative positions of two residues within the predicted structure. PAE is defined as the expected positional error at residue X, measured in angströms (Å), assuming that the predicted and actual structures are aligned at residue Y.
-
rmsd: root mean square deviation
RMSD measures the difference between the AlphaFold2 predicted structure of re-folded ProteinMPNN results and the predicted structure given by RFDiffusion.
-
mpnn: negative log likelihood
MPNN is a score of how well ProteinMPNN evaluates its performance based on validation scores.
Assuming that AF2 is infallible, we then want to maximize plddt, i_ptm and minimize i_pae and rmsd.
Design
With accessibility in mind, we focused on bridging the gap between computationally intensive approaches for determining binding interfaces and more accessible ones. In this pursuit, we streamlined the workflow to enhance the user experience. Since many of our team members have little to no experience with machine learning or modeling, we decided to utilize pre-trained models to establish a baseline “state of the art” for current approaches. To this end, we selected the following:
- RFDiffusion(4): RoseTTAFold diffusion is a diffusion model built on the well-established RoseTTAFold model developed by the Institute of Protein Design.
- ProteinMPNN: ProteinMPNN employs a graph neural network trained on beta-carbon distances to predict side chains and assist in “redesigning” protein scaffolds based on its weights.
Our approach was to first use RFDiffusion to generate arbitrary protein structures that are complementary to our target protein interface and then use ProteinMPNN as an inverse folding model to create the most stable side chains possible. This strategy is heavily inspired by the work of Professor Sergei Ovchinnikov in his repository, ColabDesign (https://github.com/sokrypton/ColabDesign). With this planning in place, we were now ready to move on to generation.
Make
Conceptual
As mentioned in the previous section, much of the work in this cycle was inspired by the ColabDesign repository.
First, we utilized RFDiffusion to generate short (10-15 AA) peptides that are complementary to the regions experimentally determined in paper (3). It is important to note that RFDiffusion generates structures based solely on beta-carbon structure, so before any inverse folding is performed, the generated structure is composed of purely glycines. This approach reduces computational costs but is also an artifact of the way in which RFDiffusion performs diffusion.
The RFDiffusion process injects Brownian noise into the coordinates of the atoms within a structural file, and then subsequently relearns the "negative" noise of those coordinates. This process effectively converts a fully bonded, experimentally generated protein structural file into a 3D point cloud of atoms. When generating structures, it is important to understand that RFDiffusion will only attempt to create complementary structures based on the similarity of the input structure to the data it has been trained on.
With this in mind, after each hypothetical peptide structure is generated, we must employ ProteinMPNN as an inverse folding model to generate valid amino acids. ProteinMPNN generates based on beta-carbon distances and then the predicted interactions. As such, there is a high likelihood that ProteinMPNN may produce clashing structures or even completely non-interacting side chain conformations at this step, which would result in poor binding affinities.
In Practice
Regarding binder design, we implemented a generation scheme involving iterations and batches. For the first batch, we generated 7 iterations of approximately 8 possible structures each. For each of these structures, 8 possible amino acid sequences were designed using ProteinMPNN. All of these runs were conducted using the Google Colab free research Python servers and took about 7 days to generate each scheme.
Test
Once we had finished generation sequences and structures for our potential peptides, it was time to determine their validity. To start, we began by comparing the newly folded ProteinMPNN structures with the original templates created by RFDiffusion. To avoid tunnel vision, we utilized AlphaFold2-multimer to predict the structures of both our protein and the binder. Then, we calculated the RMSD to evaluate how different the input structure was from the final predicted structure. Many of our predicted peptides did not perform well (see Fig.2 below).
However, structure prediction alone is insufficient to justify the interactions of our binder with the protein interface. To ameliorate this, we decided to use protein-ligand interaction models to assess these interactions. After extensive testing and research, we chose to use GAABind because it was quick and robust enough for our purposes, with well-rationalized model formulation and testing (5). We evaluated all of the generated peptides using the target residues we generated against as our binding pocket. This approach provides a secondary methodology of peptide viability control while helping to avoid overfitting models with similar goals, such as artifacts from structure-based prediction workflows.
Learn
From this first set of generated structures, we observed a common issue: all of them exhibited remarkably low predicted structural robustness. This indicated that the predicted structures all had low confidence scores, suggesting that both AlphaFold2 and RoseTTAFold were not confident in the generated structures. An explanation for the scores presented in the plot above can be found in the preceding section under the dropdown titled "What do the Different Metrics Mean?".
Conclusion
Given the resulting low structural stability and relatively low binding affinities, we determined that the best course of action moving forward would be to change our generation scheme and adjust the parameters we input into the diffusion process. Since our primary focus is on validating designs using a variety of models, our goal is to achieve the best possible outcome without altering the parameters of the machine learning models themselves. This approach helps us avoid increasing computational costs and ensures the accessibility of this technology. We aim to develop a single-input platform that individuals with limited computational resources or machine learning experience can use to solve protein design problems.
Design
Building on the lessons learned from the last cycle, this cycle, we focused primarily on dynamically evaluating binding affinities regarding flexible protein conformations. Additionally, we wanted to refurbish and standardize the generation scheme, so we returned to the Protein Data Bank and selected a higher-resolution structure that was directly referenced in this paper (3). In designing the code for this cycle, we incorporated more accessible side chains, and, rather than explicitly defining residues, we defined a surface on the outside of the protein for diffusing binders.
Make
Conceptual
We determined that the length of the binding peptide would be essential for both the structural stability score and the potential for novel side combinations. Furthermore, we discovered that ProteinMPNN has a num_recycles parameter, which enables designs to be cycled through AlphaFold2 and optimized based on the lowest RMSD. We also realized that using GAABind as a singular binding method would be too limiting and could potentially cause us to overlook possible aspects of the protein. Therefore, we decided to also implement DynamicBind (6), a protein-ligand interaction model that allows the protein to go through multiple possible conformational states depending on the energy landscape of protein folding. With this dual-model approach, we determined that the best practice would be to first test with GAABind to filter out stereochemically invalid peptides and then proceed to evaluate with DynamicBind.
In Practice
We modified the code to allow for the recycling of ProteinMPNN outputs through AlphaFold2 and increased the size of the binders. A significant amount of time was spent implementing and attempting to use DynamicBind on our local systems, only to realize how computationally expensive it is. Despite this, we persisted in our efforts to transition away from relying on a singular binding assessment metric to avoid overfitting the binders to one model.
Test
We decided to implement a more singular generation heuristic with consistent parameters to reduce inter-experiment variance. To this end, we decided on a schema in which 8 structural files would be generated, followed by the creation of 8 sequences from each structural file, with 12 recycling iterations for each sequence. This meant that each sequence would be predicted through AlphaFold2 12 times, with each iteration altering the design bias introduced by ProteinMPNN. The resulting 64 structural files and sequences would then be formatted and evaluated by GAABind and DynamicBind. This entire process took approximately 3 days and required extensive computational resources. At this point, we began exploring different ways to establish better measures of success with minimal computational intervention. If it were possible to deterministically identify good/valid binders during the sequence generation step, this methodology could be scaled down significantly.
Furthermore, building on the findings from the last cycle, we concluded that increasing the length of the binders would enhance structural confidence for AlphaFold2. Because AlphaFold2 is trained on the RCSB PDB, most of the structures that AlphaFold2 encounters are full-length protein functional units. This means that, unless the peptide we are predicting exists in a similar state within the PDB, we are unlikely to achieve high confidence scores. To address this, we increased the length of the peptide binders to 18-23 AA, which is expected to result in greater structural robustness.
Learn
Unfortunately, increasing the length of the binding peptides led to many conformationally unacceptable structures that AlphaFold2 predicted as feasible but were unusable for GAABind and DynamicBind. A recent post on X (formerly Twitter) by Professor Ovchinnikov revealed that AlphaFold often struggles to accurately predict the interactions of alanines and arginines in protein folding. In fact, Dr. Ovchinnikov demonstrated that he could add over 20 alanines to a single structure without AlphaFold2 changing the structural prediction. This indicates that AlphaFold2 may unconsciously introduce bias by substituting functional amino acids with alanines, resulting in no functional or structural changes in its prediction. Consequently, this bias affected the ProteinMPNN redesign cycles, leading to the generation of peptides composed of too many Alanines.
Conclusion
In this cycle of Design-Build-Test-Learn, we established a more standardized workflow for our project. With a better understanding of the uncertainties inherent in structural predictions made by AlphaFold2, we realized that binder length does not scale linearly with structural stability. Additionally, we discovered that because of the way that RFDiffusion was trained, it tends to bias its predictions toward forming alpha helices for proteins shorter than 100 AA. Thus, the expected cyclic peptides that we aimed to create could not be generated within our workflow.
Design
Now, equipped with a better understanding of the task at hand, we set out to replicate the results from the last cycle. We identified several key takeaways:
- AlphaFold2 is an unreliable source for structure prediction, and its influence on ProteinMPNN should be minimized.
- Due to implicit bias in RFDiffusion, we need to increase structural diversity by diffusing a larger set of structures
- Binder length does not scale linearly with stability or stereochemical feasibility
- The current workflow is becoming computationally demanding
Despite these challenges, the experimental design and focus of our project remain unchanged. Our goal is to implement an accessible platform for peptide binder generation that requires minimal computational resources.
Make
Conceptual
This cycle, we needed to revise some of our previous work to minimize design bias in the ProteinMPNN step caused by AlphaFold2. Based on Dr. Ovchinnikov’s findings, we decided to bias against alanines and arginines during ProteinMPNN’s redesign process. Furthermore, we decided to utilize GAABind, the relatively less computationally demanding model, as a filtering method to remove stereochemically implausible peptides. After GAABind filtered a set of peptides, we planned to take a subset of the originally generated sequences and structures before inputting them into DynamicBind. By narrowing the scope of our testing, we risk the possibility of discarding potentially valid sequences and structures that may be incorrectly filtered out.
In Practice
Fortunately, ColabDesign includes a --to-exclude flag in its implementation of ProteinMPNN, allowing us to avoid adding undesired residues during sequence generation. We also scaled back the number of recycles to 3 to align with the recommended amount. To increase structural variation, we generated 32 different structures using RFDiffusion and produced 16 different sequences from each, resulting in a total of 512 unique sequences. This shift moved us away from a batch-iteration methodology to a single iteration process.
Furthermore, we noted that many of the generated structures fell outside of the initially defined "hotspots." To avoid the inefficiency of discarding these sequences, we modified one of the formatting steps to include the k-nearest interface residues as the binding pocket. This adjustment allowed us to measure whether the output to RFDiffusion + ColabDesign would display reasonable binding affinity.
Regarding pre-filtering designs through GAABind, we established a triplicate threshold. From the various metrics produced by the ProteinMPNN process, we selected subsets of GAABind results with -log(Kd) > 8 (predicted binding affinity), ipTM > 0.1, and plddt > 0.19. This approach minimizes the inclusion of completely infeasible peptides and focuses on those deemed mediocre by AlphaFold2. In this manner, we can leverage some AlphaFold2 metrics to refine our peptide selection while also reducing computational overhead in our workflow.
Test
As described above, we adopted an inference method that enhanced structural variance while reducing the influence of AlphaFold2 in our redesign steps. In addition, we developed an intermediate script to format inputs for GAABind from the folders generated by ColabDesign and to merge metrics from ColabDesign/AlphaFold2 and ProteinMPNN with the outputs of GAABind. This allowed us to filter out completely unviable outputs from consideration in DynamicBind. Each of these designs was then written into a small subset CSV that is compatible with DynamicBind, facilitating parallelization and significantly lowering computational costs.
Learn
This time, the results showed significantly higher predicted binding scores, indicating an affinity comparable to canonical monoclonal antibodies. We recognized that our current generation and validation method had produced usable results, prompting us to consider potential in vitro experiments. Even if the predicted binding affinities of the peptides did not align with the experimental results, we could still identify a combination of different metrics that would help predict the experimentally validated outcomes.
Conclusion
In this cycle, we decided to modify certain aspects of the formatting scripts to preserve more of the potentially viable generated sequences and structures. In previous cycles, we demonstrated that AlphaFold2 is an unreliable metric for the structural prediction of short-chain proteins like binding peptides, leading us to conclude that the RMSD metric provided by AlphaFold2 is largely arbitrary. Furthermore, we reasoned that even if the binding peptide does not visually resemble our hotspots, the only true method of validating these results would be through in vitro experiments. To ensure we are satisfied with the peptides we create, we must produce only the highest quality candidates: ideally, every peptide we test should exhibit binding affinities in the nanomolar binding affinity. With this objective in mind, we moved on to the next cycle.
Design
To validate the workflow and results of our code, we decided to generate a large number of structures and sequences before potentially using them for experimental validation. Our goal is to establish a correlation between the generated metrics of success and the experimental results. This approach stems from the limited representation of short-chain peptides in the training sets of the models we utilize. We opted to use our university's HTC supercomputer services to create scripts that could run on multiple A100-80 GPUs. It is important to emphasize that we do not intend for our workflow to rely on the use of university resources. We are only doing so to develop a benchmarking test set for experimental validation.
Make
Using our original code as a template, we revamped the majority of the logic to streamline its execution. As such, the scripts designed for HTC require minimal user interaction, needing only a few initial prompts and a file path to the protein of interest. To facilitate this conversion, we created intermediate filtering steps beyond the initial sequence generation, which helped constrain our evaluation space for the binding model steps. We quickly realized that of all the steps, the most constraining were the generation of the initial structures and the evaluation of binding characteristics. In contrast, generating sequences and evaluating them with AlphaFold2 was computationally efficient and parallelized thanks to the implementation of ColabDesign.
Test
Over the course of approximately 4 weeks, we ran our scripts on over 8192 different designs. Almost all of the constraints from previous cycles have been maintained and represented in the code. For each set, we varied the length of each binder design. We found that, through the HTC workflow, the rate of unviable binders remained largely the same. We hypothesized that one of two issues persisted within our workflow: either the influence of structure-based systems is completely incompatible with RDKit-based binding prediction models, or the predicted structure generated by RFDiffusion does not account for peptide-based treatments. Currently, the filtering rate from initial designs and sequences to viable peptides through RDKit is around 1.5%. This means that if 100 binders are generated using RFDiffusion, ProteinMPNN, and backpropagation through AlphaFold2, only 1-2 binders will successfully load in RDKit. We also observed that the success rate decreased with increased torsion in the peptide structure. That is, if the generated structure is bent enough to be pseudo-cyclic, then RDKit is unable to load the structure at all.
Learn
Compared to the previous cycle, there was significantly less manual file handling on the user side, resulting in a much more streamlined workflow. Additionally, the results were considerably improved, with consistently high predicted binding values. Approximately 77 designs achieved a predicted Kd of 8 or higher, with one even reaching 9. This suggests a solid point of consistency, prompting us to pursue experimental validation of our designs To create a test set with greater variability, we also selected some designs with lower predicted binding affinity but higher predictions for structural stability.
Conclusion
We have successfully completed a full run, yielding the expected results based on insights from previous cycles. With these sequences in hand, we will proceed to create an experimental validation of our model’s results. The development of HTC scripts has also facilitated data collection regarding runtime and efficiency. We will compare these metrics against those from Google Colab on the software page.
References
- Therapeutic peptides: current applications and future directions
- Prospect of cell penetrating peptides in stem cell tracking
- HIV-1 Uncoating and Reverse Transcription Require eEF1A Binding to Surface-Exposed Acidic Residues of the Reverse Transcriptase Thumb Domain
- De novo design of protein structure and function with RFdiffusion
- GAABind: a geometry-aware attention-based network for accurate protein-ligand binding pose and binding affinity prediction
- DynamicBind: predicting ligand-specific protein-ligand complex structure with a deep equivariant generative model
Introduction
Continuous culture and selection in a directed evolution, or PACE, experiment requires automation to maintain conditions over multiple days. Commercial automated bioreactors cost $1-100k US dollars, putting these essential tools out of reach for most researchers looking to start a protein engineering project. To remove this barrier to accessible protein engineering, we designed a simple, open source bioreactor for turbidostat culture using an Arduino microcontroller, inexpensive electronics, and 3D printed components. Our design includes high-intensity UV, Green, and Red LEDs for optogenetic control of cultures and turbidity sensing, offering new users an automated bioreactor capable of PACE for under $200.
Component-Based Design Build Test Learn
Our turbidostat design consists of 3D printed components, Arduino-controlled electronics, and fluid-handling pumps, with control software written in the Arduino programming language. Given the modularity of the design, we followed the DBTL rubric and have compiled our learning here in design cycles. When we noted a fault with a single component, we created new iterations to test. Our DBTL approach for individual components of our system is below and is divided into four sections: 3D Parts, Electronics, Pumps, and Control Software.
3D Parts
Design:
For our 3D part design we used Onshape as our Computer Aided Design (CAD) program. Onshape can be accessed through a web browser, something we felt would make our designs even more accessible and easy to modify.Our first 3D part design combined three culture flasks onto one solid plate-like structure
Build:
This iteration was only modeled in CAD and never made it into the physical world. However it did allow us to set up our CAD model and get everything organized.
Test:
We could tell during our initial renderings and thinking about how the user would interact with the system that the single plate would restrict the modular intent of our design. With the single plate design it would be hard to change the size of the system without changing all the other aspects of the system, and it would make it difficult to expand the system in the future, should the user want to add an additional turbidostat or lagoon.
Learn:
The valuable lesson we learned from this round was the importance of making the system modular so it could fit into different spaces and be more easily adapted to a user's specific needs.
Design:
To increase modularity, we created a system where each culture flask is contained in a “silo,” a cylindrical 3D part that prevents external light from penetrating the culture. The “silo” is broken down into four segments: a base segment housing the stir motor and wiring connections, an optical density segment housing the photoresistor and laser, a normal segment to cover the remaining flask, and a lid segment to seal the “silo.” These detachable segments can be added and rearranged to fit the user's needs. Our second design also included a way to connect all the individual silos through the use of a base plate that had a series of holes; feet on the bottom of the base segments allowed them to slot into place on the base plate.
Build:
When building the system, we were not concerned with the size of the culture flasks we intended to use. Instead, we designed several systems of arbitrary size to test how the electronic components would fit at various system sizes.
Test:
We tested all the various electronic components for fit and identified areas for improvement. The stir motors risked overheating over time relevant for PACE experiments and wiring prevented the motor from sitting level, requiring better wire management. Additionally, the lid segment often got in the way, the small caps were easy to misplace,and the LED housings often broke. Finally, the feet we designed for the base sections became an inconvenience for testing and the plate system was bulky, hindering use of secondary containment, as all the pieces were interconnected .
Learn:
While modularity was good, we needed a way to interconnect all the parts so the system could easily move around and in or out of incubators and secondary containment. Additionally, wire management and heat dissipation could be improved.
Design:
Our third design included a simplified lid design, airflow vents for the stir motor, and a slot for better wire management so the motor can sit flush. Additionally, we redesigned LED placement, removing the thin walls. We designed a new snap-fit system to connect all of our segments, further improving wire management and designed entirely new parts to hold our microcontroller, protoboard/PCB, LCD and a rotary encoder using a snap fit design.
Build:
We 3D-printed all of the new parts and transferred over all of the electronic components, integrating the new LCD screen and rotary encoder.
Test:
The new system was easier to get into and out of incubators and secondary containment and the new snap-to-fit system helped streamline the design and improve wire management. The snap to fit pieces needed improvement and the rotary encoder had mechanical issues with its housing.
Learn:
The snap-to-fit parts are great for reducing the amount of hardware needed and for easy assembly and disassembly, however, 3D printed parts are not as robust as injection molded components that snap to fit connections were originally designed for. Taking into account the different mechanical properties of 3D printed plastic will be important to create more reliable and robust parts.
Electronics
Design:
We made our first designs using TinkerCAD, including our laser, photoresistor, pump, and stir motors. To deliver more power to our pumps and motors we used our microcontroller PWM channels to trigger MOSFETs. For all motors we placed flyback diodes to prevent motor feedback and included pull-down resistors to prevent power surges.
Build:
We transferred our TinkerCAD design to a breadboard to allow for changes during the build phase.
Test:
Our system could stir, run pumps and measure optical density. It was not robust however and lacked functionality like LED controls.
Learn:
This cycle allowed us to validate the optical density measurements with the laser and photoresistor and ensure both the pump and stir motors worked as intended. We also learned that in subsequent iterations it would be necessary for us to add the additional components such as the LEDs for optogenetic control, one of the core functionalities of our system.
Design:
Expanding on our TinkerCAD design, we added NeoPixel LEDs, and integrated an SD card module so that optical density values could be recorded without a laptop connected to the system at all times.
Build:
With most of the underlying circuitry for the essential components completed we were ready for a permanent circuit, we shifted to a soldered protoboard for more secure connections when moving the system into incubators and secondary containment.
Test:
We successfully integrated the LEDs and SD card module with all other components. The protoboard created a more robust system with no connections coming loose. However, we still found areas for improvement now that we had a fully functional prototype. Interacting with the electronics was difficult and unintuitive, and we knew users might have a hard time using the system if they did not have a solid understanding of code and electronics.
Learn:
The underlying circuitry we developed worked well; however, it was not yet user friendly and could be intimidating for users to interact with or set up.
Design:
Several key innovations came in the third cycle of our electronics design. To make the system more user-friendly, we integrated two new components: an LCD screen and a rotary encoder. These would allow the user to interact with the system without interacting with any code. We also made a series of smaller innovations which included running all of the stir motors using one transistor and PWM pin. Additionally, we added a capacitor to smooth voltage fluctuations in the power lines for our LEDs.
Build:
This build cycle built on our previous iteration with the addition of an LCD screen and rotary encoder to the protoboard, requiring us to move components on the board and switch them to different Arduino output pins. We also integrated all of these components in with our new 3D printed hardware and had to rewire and connect various components to fit properly.
Test:
Testing the components resulted in several fried LCD screens, but eventually we were able to get the code and all of the connections working properly. However, we found that the system wires were difficult to follow and rewire because they were cluttered in a compact area.
Learn:
The LCD screen and the rotary encoder improved the user friendliness of the system. However, the wires, which were now contained in the control unit, were cluttered and we sought a way to streamline them so the system was easy to assemble and use.
Design:
To make our design more accessible for future builders, we designed a printed circuit board for our system and simplified the wiring of components. Taking the circuit diagrams from TinkerCAD, we created a schematic and then completed the PCB layout for our custom circuit board in KiCad. While designing our circuit we added labels for where wires were supposed to connect and redesigned how our system used capacitors. We included a current regulating capacitor for each power bus (5V and 12V) and an additional capacitor next to our stir motors to absorb voltage spikes. Additionally, we added additional pins for the system to be expanded upon while still using the same circuit board. Additional pins remain available for extra stir motors, photoresistors, lasers, and pumps.
Build:
Our design was sent out for PCB printing and we received the board in time to solder all of the components in and connect all the wires.
Test:
We continue to test our PCB for power distribution, light and LCD controls. With a fully tested PCB we will be able to finalize our build kit and share the design with anyone by sharing the physical PCB, or offering the design as a download for printing.
Pumps
Design:
Our first design involved the use of gravity and a siphon to move liquid from one culture bottle to the next. This design was inspired by our conversation with Dr. Takahashi commented in our interview that one of his own favorite designs simply used gravity to move the media. Our first design made use of gravity to move media through the system until the lagoon, where we put a pump to move media from the lagoon to the waste.
Build:
Building the system required us to make new 3D printed hardware. We created “stilts,” minimal normal ring sections, that allowed us to raise up each of the culture bottles to a height that allowed for a siphon effect.
Test:
During the testing of the gravity powered system we found that it did work as intended and media moved successively through each of the culture bottles. However, we also found the system to be quite bulky and cumbersome to work with. We were also concerned that the system may not be able to fit certain user’s incubators.
Learn:
While the gravity fed system worked, its overall height and the space it took up did not make the system as accessible as it could be.
Design:
Our second design involved a positive air pressure pump system that was inspired by (McGeachy, et al., 2019). An air pump provided constant positive air pressure to the culture flasks to equilibrate the liquid levels while a single peristaltic pump at the final flask determined the flow rates.
Build:
The first time constructing the system, we found that the air lines found within the lab provided too much pressure, requiring additional hardware to accurately modulate the air flow. Our next design involved the use of a low pressure air pump which allowed us better, but imperfect modulation of the air flow.
Test:
During preliminary testing, the rubber stoppers sealing the flasks occasionally popped off as too much pressure built up in the system. Additionally, while altering the flow of air into the system loosely controlled the flow rate, it had to be adjusted manually using physical regulators as our air pump was connected to a wall outlet and was ultimately incompatible with a microcontroller.
Learn:
We determined that the rubber stoppers could pose a safety hazard of physical harm and biological contamination as pressure built in the system. Additionally, we wanted a way to control the flow rate of the system with code and a microcontroller rather than requiring manual adjustments.
Design:
While testing the air pump system, we realized that if we remove the air pump and keep one peristaltic pump at the last culture flask, the media would move through the system properly. We found the use of one pump created negative pressure throughout the system, causing liquid to move from stock to turbidostat to lagoon and lastly to waste.
Build:
Building this system was simple, as our other iterations had also used a normal peristaltic pump at the end of the system to control the flow rate. We simply removed all the unnecessary components to simplify the design so that it only used one pump.
Test:
We tested if using only one pump would result in the reliability that we needed, and found that, even after varying sizes of pipettes, altering the run times, and changing the liquid volumes in the flasks, the single pump worked well and maintained consistent liquid levels throughout our tests.
Learn:
We verified that the turbidostat can operate using one peristaltic pump. This was a critical finding because pumps were the most expensive component, allowing us to decrease costs by >25%, compact the design, and prevent safety hazards from positive pressure. Therefore, we ultimately opted for this fourth design which best fit our goals for the project.
Control Software
Design:
We designed our code as simply as possible, maintaining that the core functionality of each component was consistent and easily reproducible in subsequent experimental and assembling processes. All programs needed to run for hours to days without having an error, requiring a robust language with example applications. We chose the Arduino language over CircuitPython on the advice of graduate student Fernando Flores Mora for its robustness. Control software had to control each of these elements:
Pumps: Intermittent PWM control to a 12V motor control board.
Stir Motors: Continuous PWM control to a 5V motor control board
LEDs: Continuous independent PWM control of RGB and UV lights.
LCD and Rotary Interface: Display of active program and a selection knob to activate and choose between preprogrammed bioreactor protocols.
Build:
PWM controls: Pumps, motors and LEDs all used a standard PWM library in the Arduino programming language. We utilized the built in Arduino clock for timing of different actions and added new functionality in parallel with hardware development.
LCD Screen and Rotary Encoder User Interface: Developing the interface for our device required the integration of a hardware control system that detected turn on a dial and a display to return selected values to the user. The hardware coordinated the inputs from a rotary encoder with output on an LCD screen. We stored the clockwise or counterclockwise inputs of the rotary encoder into a counter variable, and utilized the modulus operator (%) to allow for continuous turning of the knob to give a limited number of inputs. We built a simple menu to test that the above code worked, which required the user to click the rotary encoder down once on a menu called “Run PACE,” to switch between menu options “Start” and “Back” by turning the rotary encoder, and then to return to the main menu by pressing the rotary encoder down on the “Back” option.
Test:
PWM controls: In our initial testing with the pump, since we had planned on utilizing gravity as a secondary source of ensuring proper flow, we planned on providing the pump with a lower voltage, however the pumps were more accurate when run for shorter time at max power. Conversely, stir motors at max power created an overly large vortex in the bottles, which inhibited the collection of optical density values in the photoresistor. At ¼ max power cultures were well homogenized whilst the view of the laser from the photoresistor remained clear.
LCD Screen and Rotary Encoder User Interface: By utilizing the simple “Run PACE” menu with options “Start” and “Back,” we were able to test whether the rotary encoder’s input coordinated with the LCD Screen’s overlay. To navigate to the two options, the user had to click the encoder down once. At this point, the “Start” option did not run functional PACE, so we only tested whether the “Back” option would return the user back to the Main Menu. Additionally, in our initial code design, we found that the rotary encoder’s counter could be negative if turned counterclockwise instead of clockwise, and this could have caused issues with the functionality of the user interface.
LEDs: We tested the functionality of the LEDs first by turning them white within the setup function, and then by creating separate functions for Red, Green, and UV states. We tested this first by implementing them independently and observing that they changed to the color they were supposed to, and implemented this test into the user interface on the LCD screen. Our first attempt at a turbidity experiment that was run without pumps and simply measured the OD of our bacterial culture over 13.7 hours. The average values were calculated every minute.
Learn:
PWM controls: We learned that the pump required full voltage to maintain proper flow and prevent bacterial stagnation within the tubing. Furthermore, we determined that in order to alter the flow rate of the system, it would be better to adjust how often the pump was run rather than at what voltage it was run, so we adjusted this in the next iteration. Stir motors should be run at low power to not disrupt optical density data collection while maintaining homogeneity. LCD Screen and Rotary Encoder User Interface: As we added more menu screens to our system, we had to refine the rotary encoder code allow for rotating in either direction, in the subsequent iteration, we opted to calculate the absolute value of the remainder for every menu option, which allows for positive and negative values as the encoder to be turned either way to scroll through menu options. LEDs: We found that there were problems with differentiating between the UV and the RGB LEDs, and also that we needed to implement the ability to turn the LEDs off into the user interface.
Design:
In our second iteration of the code, we implemented the changes that we learned from the previous DBTL process, ensured that hard coded functions worked, and focused on adding functionality to the LCD user interface in a new, LCD version of our code.
Build:
Running PACE and Calibration Function: In order for user-friendly PACE functionality, we initially coded the menu on the LCD screen to have the capability to simply Run PACE, and with subsequent design changes, to Run Calibration, which runs PACE without turning the pump on so that bacterial growth within the turbidostat can be measured on its own. Cleaning Function: We initially designed our cleaning function within the code to run the pump for the amount of time inputted by the user utilizing a delay function.
Test:
Running PACE and Calibration Function: When testing our “Run PACE” and “Calibration” submenus, we found that they turned the stir motors, photoresistor, and in the case of “Run PACE,” the pump on the way they were supposed to, but they could not be stopped from the interface, requiring the user to unplug the power sources from the system every time we wanted to stop them from running. Cleaning Function: We tested our initial cleaning menu with the delay function, which allowed for the user to input a set time for cleaning, and it worked extremely well, but we were unhappy with its usability in the context of the rest of the system. While it was called, the cleaning function halted everything else in the code until it was over, which allowed the cleaning function to work well for the set amount of time called for, but prevented it from being stopped within that time period.
Learn:
Running PACE and Calibration Function: In the subsequent design iteration, we opted to add the capability to run cleaning and calibration functions from the LCD screen. This subsequent design proved useful with further testing because it could be used to turn off the LEDs directly. Cleaning Function: To remedy the issue of not being able to stop the cleaning function before the inputted time was done, in subsequent iterations, we opted to replace the delay function with a clock timer. This let us set an arbitrary variable in seconds at the moment the cleaning function is called without pausing all other functions, allowing us to perform more complex tasks with our bioreactor.