TAGM OL: Target Aware Gradient-guided Molecule Generation

Vineeth Dorna * 1 2 D. Subhalingam * 1 Keshav Kolluru 1 Shreshth Tuli 1 3 Mrityunjay Singh 1 Saurabh Singal 1
N. M. Anoop Krishnan 4 Sayan Ranu 5

Abstract The advent of generative AI for molecules has accelerated

the rational drug-design—in contrast to the traditional Edis-
arXiv:2406.01650v1 [q-bio.BM] 3 Jun 2024

3D generative models have shown significant

promise in structure-based drug design (SBDD), onian trial-and-error approach—with the goal of creating
particularly in discovering ligands tailored to spe- drug-like molecules in 3D space that effectively bind to
cific target binding sites. Existing algorithms of- specific targets. Specifically, deep generative models, such
ten focus primarily on ligand-target binding, char- as those proposed by Luo et al. (2021); Liu et al. (2022);
acterized by binding affinity. Moreover, models Peng et al. (2022), autoregressively generate atoms and
trained solely on target-ligand distribution may bonds, while Zhang et al. (2023) generate motifs. Despite
fall short in addressing the broader objectives of their progress, the performance of autoregressive models
drug discovery, such as the development of novel heavily relies on the order of generation, as they condition
ligands with desired properties like drug-likeness, on previously generated atoms, which can lead to error prop-
and synthesizability, underscoring the multi- agation. Alternatively, diffusion models (Sohl-Dickstein
faceted nature of the drug design process. To over- et al., 2015; Ho et al., 2020) overcome this limitation by
come these challenges, we decouple the problem conditioning upon all the atoms simultaneously and there by
into molecular generation and property prediction. efficiently generating realistic molecules that demonstrate
The latter synergistically guides the diffusion sam- stronger binding affinities with their intended targets.
pling process, facilitating guided diffusion and The effectiveness of these generative models is heavily
resulting in the creation of meaningful molecules reliant on how well training datasets — consisting of
with the desired properties. We call this guided protein-ligand complexes — align with desired properties
molecular generation process as TAGM OL. such as binding affinity, drug-like topological features,
Through experiments on benchmark datasets, synthetic accessibility (ease of synthesis), to name a few.
TAGM OL demonstrates superior performance For instance, CrossDocked 2020 (Francoeur et al., 2020), a
compared to state-of-the-art baselines, achieving widely-used training dataset for SBDD tasks, predominantly
a 22% improvement in average Vina Score and includes complexes with moderate binding affinities. Con-
yielding favorable outcomes in essential auxiliary sequently, models trained solely on such datasets may yield
properties. This establishes TAGM OL as a com- sub-optimal molecules, intrinsically tying their success
prehensive framework for drug generation. closely to the dataset’s quality. Moreover, drug generation is
a multi-faceted process that encompasses not only binding
affinity but also a range of other desired properties. Com-
1. Introduction piling a comprehensive dataset that encompasses a broad
range of desired properties poses significant challenges,
The presence of molecular data featuring 3D spatial particularly associated with the high computational costs
information has paved the way for the complex field of of assessing a suite of properties. Additionally, refining
structure-based drug design (SBDD) (Anderson, 2003). datasets to meet specific quality constraints can drastically
* reduce the volume of usable training data. As the number of
Equal contribution 1 Molecule AI, New Delhi, India 2 Manning
College of Information and Computer Sciences, University of Mas- constraints grows, the likelihood of finding samples that sat-
sachusetts Amherst, Massachusetts, USA 3 Happening Technology isfy all these criteria becomes increasingly difficult, further
Ltd, London, England 4 Department of Civil Engineering, Indian exacerbating the situation. Consequently, generative models
Institute of Technology Delhi, New Delhi, India 5 Department trained on lower-quality, but high-volume, data may uninten-
of Computer Science and Engineering, Indian Institute of Tech- tionally capture suboptimal signals, leading to diminished
nology Delhi, New Delhi, India. Correspondence to: Vineeth
Dorna <[email protected]>, D. Subhalingam <sub- performance in the context of drug discovery. This situation
[email protected]>. prompts an exploration into how, during the denoising
phase of a generative model, we can effectively introduce

1
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

desired signals while ensuring meaningful reconstruction. 2. Preliminary

To address these challenges, we introduce TAGM OL– In this section, we introduce the preliminary concepts cen-
(Target-Aware Gradient-guided Molecule Generation), tral to our work and formulate the problem of gradient-
wherein we decouple molecular generation and property guided molecular generation. All notations introduced in
prediction. We start by training a time-dependent guide this section are summarized in Table 4 in the Appendix.
model that predicts properties from inputs with noise levels
similar to those in the base diffusion model. Crucially, we 2.1. Problem Statement
turn the challenge of using inferior quality data, i.e., the From the standpoint of generative models, SBDD task in-
property of interest is well spread with the inclusion of volves the creation of ligand molecules with the ability to
sub-optimal values in the dataset, to our strength for robust bind effectively to a specified protein binding site. A protein
guide training. Inspired by classifier guidance in diffusion binding site is characterized

NP by a set of atoms, denoted by
models (Dhariwal & Nichol, 2021), we use the gradient of P = { xP P
i , vi }i=1 , where NP represents the number of
guide to direct the latent space during the diffusion sampling protein atoms, xP 3
i ∈ R represents the 3D coordinates of
process, ensuring the reconstructed molecules possess the the protein atom, and viP ∈ RNf represents protein atom
targeted properties. In the sampling phase, we harness features such as element types and amino acid types, with
the strengths of both the generative model and the guide. Nf representing the number of such features. We aim to
This interactive dynamic enables us to explore regions jointly optimize binding affinity and desired pharmacologi-
with superior properties while simultaneously denoising cal properties (denoted as Y), by generating prospective lig-
M NM
and molecules M = { xM


the latent space to generate diverse molecules. While i , vi }i=1 , for a given protein
gradient guidance is a well-explored concept in the drug P. Here xM 3 M K
i ∈ R and vi ∈ R represents the atom co-
discovery domain, many existing methods overlook target ordinates and atom types of a ligand molecule, respectively,
awareness (Bao et al., 2022) or fail to integrate 3D structure with K representing the number of such features to repre-
(Eckmann et al., 2022; Stanton et al., 2022; Lee et al., sent atom types. The variable NM signifies the number of
2023). In contrast, our approach seeks to simultaneously atoms in the ligand molecule, which can be sampled during
optimize for both target-aware and molecular properties in inference utilizing either an empirical distribution (Hooge-
3D space. Our method marks a significant advancement boom et al., 2022; Guan et al., 2023a;b) or predicted through
in end-to-end molecular generation, reducing the reliance a neural network (Liu et al., 2022).
on post-optimization techniques. Furthermore, to address To simplify, in matrix representation the ligand molecule
real-world scenarios where multiple property constraints ex- is denoted as M = [XM , VM ], where XM ∈ RNM ×3
ist, we train separate guides for each property, subsequently and VM ∈ RNM ×K and the protein is denoted as P =
employing them to steer the diffusion process effectively. [XP , VP ], where XP ∈ RNP ×3 and VP ∈ RNP ×Nf .
Overall, the key contributions of our work are as follows. 2.2. Diffusion models for Target-Aware Generation
As delineated in previous works on diffusion-based target-
• Reformulation of the drug-discovery problem: We re-
aware molecular generations (Guan et al., 2023a;b), the
formulate the problem of drug generative modeling mov-
process involves two phases: noise injection, also termed as
ing beyond the myopic lens of optimizing binding activity.
forward diffusion, and denoising (backward diffusion).
The need to optimize other properties of interest, even
when these signals are not adequately present in the train Noise injection: This phase involves a gradual injection of
set, is explicitly coded into our problem formulation. Gaussian noise for co-ordinates and uniform noise for cat-
• Algorithm design: We design a novel generative process, egorical data through a Markov chain. This noise addition
called TAGM OL, which jointly leverages the signals is uniquely applied to the ligand molecule, excluding the
from two different components: an SE(3) equivariant protein in the diffusion process. In this context, the atom
graph diffusion model to mimic geometries of the train positions and types of the ligand molecule at time step t are
set, and a multi-objective guide model, empowered by represented as XM M
t and Vt respectively. The diffusion
an SE(3) invariant GNN, to steer the exploration region forward transition is defined by the following equations:
of diffusion sampling towards the property of interest by  
leveraging gradients. q (Mt |Mt−1 , P) = N XM 1 − βt XM
p
t ; t−1 , βt I
• Rigorous empirical evaluation: We demonstrate that   (1)
our model achieves 22% improvement in average Vina · C VtM | (1 − βt ) Vt−1
M
+ βt /K
Score, all the while being guided by considerations of
binding affinity and crucial pharmacological properties  √ 
q (Mt |M0 , P) = N XM t ; ᾱt XM
0 , (1 − ᾱt ) I
such as QED and SA.   (2)
· C VtM |ᾱt V0M + (1 − ᾱt ) /K

Preprint 2
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

a) Guide Training b) Guided Sampling

Reverse Process

Prior Guidance
Property
Predictor
Property
Predictor

Diffusion
Model
Loss
Denoising

Parameters Frozen Database

Figure 1. Overview of TAGM OL. (a) Training a property-oriented guide using existing data. (b) Utilizing the trained guide and diffusion
model to steer the diffusion sampling process towards the optimal regions of the property of interest.
where N and C denotes the normal and categorical distri- 1    
∥µ̃t XM M
− µ̃θ [XM M
(x) 2
Lt−1 = t , X0 t , Vt ], P, t ∥ + C
bution respectively while β1 , ..., βT represents the variance 2 β̃t

schedules. The corresponding posteriors are analytically = γt ∥XM bM 2

0 − X0|t ∥ + C
derived as follows: (5)
ᾱt−1 βt2
where γt = and C is a constant. As recom-
   
q (Mt−1 |M0 , Mt , P) = N XM M
t−1 ; µ̃t Xt , X0
M
, β˜t I 2σt2 (1−ᾱt )2
   mended by (Ho et al., 2020) and (Guan et al., 2023a), train-
M
· C Vt−1 |c˜t VtM , V0M ing the model using an unweighted Mean Squared Error
(3) (MSE) loss, by setting γt = 1, leads to enhanced perfor-
where,
√ᾱt−1 βt M √αt (1−ᾱt−1 ) M mance. Regarding the atom-type loss, the KL divergence
µ̃t XMt , X0
M
= 1−ᾱt X0 + 1−ᾱt Xt , of categorical distributions is computed in the following
1−ᾱt−1 t manner:
β̃t = 1−ᾱt βt , αt = 1 − βt , ᾱt = Πs=1 αs ,
K
c̃t VtM , V0M = c⋆ / k=1 c⋆k , and c⋆ VtM , V0M =

P
K
c VtM , V0M k


(v)
X
M M


[αt VtM + (1 − αt ) /K] ⊙ [ᾱt−1 V0M + (1 − ᾱt−1 ) /K]. Lt−1 = c Vt , V0 k log   (6)
k=1 c VtM , V bM
0|t
In practical applications, it is recognized that the schedules k

βt for coordinates and categories may differ. However, for

the sake of simplicity in this context, they are uniformly The final loss is a weighted sum of atom coordinate loss and
(x) (v)
represented. atom type loss, which is expressed as L = Lt−1 + λLt−1 .
Denoising phase: In the generative process, a neural net-
work parameterized by θ learns to recover M0 by iteratively 3. TAGM OL
denoising MT . During reverse process, M0 is approxi-
mated using θ and Mt−1 by predicting M c 0|t = [X b M, Vb M] In this work, we improve upon Conditional Diffusion
0|t 0|t
at time step t and Mt−1 is sampled as follows: models with a particular emphasis on utilizing protein
  pockets as a conditioning factor for generating ligand
pθ (Mt−1 |Mt , P) = q Mt−1 |M c 0|t , Mt , P molecules (Guan et al., 2023a;b). The effectiveness of
  such models in practical scenarios can be hindered if the
= N XM ˜
t−1 ; µ̃θ (Mt , P, t) , βt I conditioning signal is overlooked or weakened, an issue
that becomes more pronounced with datasets of inferior
 
M
· C Vt−1 |c˜θ (Mt , P, t) (4)
    quality. Models that are exclusively trained to maximize
= N XM M M
t−1 ; µ̃t Xt , X̂0|t , βt I
˜ the likelihood of protein-ligand complexes in such datasets

M
  naturally inherit the same quality issues. To counteract
· C Vt−1 |c˜t VtM , V̂0|t
M
this, TAGM OL employs a strategic approach to explicitly
integrate additional conditioning signals learned over a set
Training: In line with the principles outlined in the varia- of binding and desirable pharmacological properties Y (Re-
tional auto-encoder (Kingma & Welling, 2013), the model call § 2.1) during the model’s denoising phase. This tactic
undergoes training through the optimization of variational aligns with the method proposed by (Dhariwal & Nichol,
bound on the negative log-likelihood. Given that both 2021), which involves conditioning a pre-trained diffusion
q (Xt−1 |M0 , Mt , P) and pθ (Xt−1 |Mt , P) are Gaussian model using classifier gradients. In particular, for a property
distributions, the Kullback-Leibler (KL) divergence for the y ∈ Y, we train a regressor (or classifier as appropriate)
atom coordinates is expressed in a closed-form equation: pϕ (y|Mt , P, t) on noisy molecule Mt and then use the gra-

Preprint 3
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

dients ∇XMt
pϕ (y|Mt , P, t) to guide the diffusion sampling Once we get the final hidden states hLi,t , we predict the final
process towards the desirable properties encoded in y. property using an MLP layer as:
!
Fig 1 presents the pipeline of TAGM OL. TAGM OL con- f
X
L
ŷ = ϕy hi,t (8)
sists of two key components. The first component is Guide
i∈M
Training, introduced in § 3.1, where we present our SE(3) In-
variant GNN architecture and its modeling objectives, estab- While equivariance is essential for generative models to
lishing the foundation for our approach. The goal of this step maintain the consistency of the probability p(M0 |P) against
is to assess how well a molecule conforms to the desirable SE(3) transformations in protein-ligand complexes, it is also
properties Y. The second component, discussed in § 3.2, is vital that a scalar property predicted from guide remains
Guided Sampling. Here, we elaborate on how a trained guide independent of SE(3) transformations. Hence, the GNN in
model efficiently steers the diffusion sampling process to- guide is SE(3) invariant. This modeling approach introduces
wards specific regions of interest, emphasizing desired prop- a valuable inductive bias, which is particularly beneficial
erties. Our study goes further to demonstrate the concurrent in the generation of 3D objects, such as molecules (Köhler
optimization of multiple properties, as detailed in § 3.3. This et al., 2020; Satorras et al., 2021; Xu et al., 2022; Hooge-
multifaceted approach more holistically addresses the var- boom et al., 2022; Guan et al., 2023a;b).
ious aspects of molecular design, ultimately simplifying the
In our approach, diverging from the classifier-guided
efforts of chemists devote to the lead optimization process.
methodology (Dhariwal & Nichol, 2021) in diffusion pro-
3.1. Property Guide cesses where the classifier directly estimates probabilities,
We approach the input space as a 3D point cloud system we train our model to regress specific properties. The proba-
where we build a k-nearest neighbors (k-NN) graph G by bility modeling is articulated through a normal distribution
representing ligand and protein atoms as nodes, and each as:
atom is connected to its k-nearest neighbors. For properties pϕy (y|Mt , P) = N (y, ϕy (Mt , P, t), I) (9)
such as binding affinity, including protein atoms within the The training of these models is oriented toward minimiz-
graph is critical; however, the inclusion of protein atoms ing the Negative log-likelihood (N LL), which effectively
can be omitted for properties solely dependent on the ligand. simplifies the final objective to minimizing the Root Mean
We parameterize our guide model using an Invariant GNN Square Error (RMSE) between y and ϕy (Mt , P, t).
ϕy , which is trained to predict property y ∈ Y given a noisy
input Mt , protein P and time t. Later in the denoising T
X
phase, the gradients of ϕy are used to direct exploration N LL = −Ep(P,M0:T ) log(pϕy (y|Mt , P, t))
in regions of interest. Given the graph representation G at t=0
(10)
diffusion time step t, we define the GNN convolution layer T
X (y − ϕy (Mt , P, t))2
as follows: = Ep(P,M0:T )
2
di,j = D(∥xi,t − xj,t ∥ ) t=0
2
mi,j = ϕm l l
y (hi,t , hj,t , di,j , ai,j )
e Although we designed our guide for regression tasks, it is
ei,j = ϕy (mi,j )
X versatile enough to be adapted for classification tasks or
mi = ei,j mi,j other suitable applications.
j∈N (i)
(7)
3.2. Single Objective Guidance
hl+1
i,t = ϕhy (hli,t , mi )
( In this section, we first design the framework to obtain
ϕpy (viP ) if i ∈ P guidance from a single property. Subsequently, in § 3.3, we
h0i,t = M
ϕly ([vi,t , τ ]) if i ∈ M extend to multi-objective guidance.
Departing from the methodologies outlined in prior studies
where hli,t ∈ Rd represents the SE(3)-invariant hidden
(Guan et al., 2023a;b), our generative model distinctively
representation of protein and ligand atoms after l layers;
conditions on both the protein pocket and property to be
xi,t , xj,t ∈ R3 represents hidden representation of protein
guided. Remarkably, without retraining the diffusion model,
and ligand atoms. di,j represents the distance embedding
we guide the diffusion sampling process by shifting coor-
and ai,j represents the edge attributes in the graph.
dinates. This approach empowers the model to consistently
Invariance of hli,t stems from the fact that we take the
condition on optimal property values during the denoising
L2-distance between the atom representations. N (i) stands
phase. By steering the denoising process towards an optimal
for the set of neighbors for atom i. τ represents the time
property, it significantly enhances the model’s capability
embedding to make the model aware of noise at time step t.
to generate molecules with desired characteristics.
ϕm e h p l
y , ϕy , ϕy , ϕy and ϕy are Multi-Layer Perceptrons (MLP)
where as D is a distance encoder. As demonstrated in (Dhariwal & Nichol, 2021), efficiently

Preprint 4
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

sampling Mt−1 for each denoising transition can be ade- posterior distribution in Equation 12 as:
quately achieved by:
XM
t−1 ∼ N (µ̃θ (Mt , P, t) + δ, β̃I)
(15)
pθ,ϕy (Mt−1 |Mt , P, y) = Zpθ (Mt−1 |Mt , P) M
Vt−1 ∼ C (c˜θ (Mt , P, t))
· pϕy (y|Mt−1 , P, t − 1)
X
where δ= sy β̃∇XM log pϕy (y|Mt , P, t) (16)
(11) y∈Y
t

where Z is a normalizing constant. However, direct sam-

pling from this distribution is intractable. We approximate and sy represents the guidance strength for property y ∈ Y.
the sampling process via perturbation to a Gaussian dis-
tribution as per prior work (Sohl-Dickstein et al., 2015;
Dhariwal & Nichol, 2021). Thus, we use perturbation to 4. Experiments
pθ (Xt−1 |Mt , P) and sample XM t−1 as: In this section, we benchmark TAGM OL against state-
of-the-art molecular generative models and establish that
XM
t−1 ∼ N (µ̃θ (Mt , P, t) + sβ̃∇XM log pϕy (y|Mt , P, t), β̃I)
gradient guidance leads to superior performance across
t
M
an array of important metrics assessing binding and
Vt−1 ∼ C (c˜θ (Mt , P, t)) pharmacological properties.
(12)
In this context, the parameter s, denoting the guidance 4.1. Datasets
strength, plays a crucial role in prioritizing Mt with op- TAGM OL is trained and evaluated on the CrossDocked2020
timal property by sampling it from the updated

s distribution dataset (Francoeur et al., 2020), consistent with the ap-
∝ pθ (Mt−1 |Mt , P) pϕy (y|Mt−1 , P, t) . Fine-tuning proaches outlined in Luo et al. (2021); Peng et al. (2022);
of s is essential during the optimization process to improve Guan et al. (2023a;b). The dataset originally comprised of
property prediction while maintaining the effectiveness of 22.5 million docked binding complexes, which undergo a
denoising within the diffusion model. rigorous refinement process. This refinement narrows down
The incorporation of discrete variables such as atom types the dataset to better-quality docking poses, ensuring that the
VtM in the diffusion process poses challenges in directly root-mean-square deviation (RMSD) between the docked
applying explicit guidance through gradients. However, in pose and the ground truth is less than 1Å, and that the pro-
our approach, while we explicitly provide guidance for co- tein sequences exhibit less than 30% identity. From this
ordinates XM refined pool, 100,000 protein-ligand pairs are selected for
t , the guidance for discrete variables operates
implicitly. More specifically, at diffusion time step t, the the training set. For the evaluation, we employ a separate
denoised ligand atom types Vt−1 M
are influenced by both set of 100 proteins that are distinct from those in the train-
Xt and Vt . Therefore, by providing guidance for XM
M M ing dataset. To ensure a fair comparison with our baseline
t ,
M
the generative model is encouraged to denoise Vt−1 for the methods, we adhere to identical data splits for training our
optimized XM guide models and evaluating our overall method.
t , effectively making the guidance implicit
for the atom types. 4.2. Baselines
3.3. Multi Objective Guidance We benchmark against state-of-the-art baselines in the
realm of structure-based drug design (SBDD). This includes
We now generalize our objective to holistically enhance
liGAN (Ragoza et al., 2022), which leverages a conditional
various desired properties, collectively denoted as Y, in the
variational autoencoder (CVAE) and is trained on a grid
molecules we generate. At each denoising step t, we con-
representation of atomic densities in protein-ligand struc-
dition on Y and sample Mt−1 according to the probability
tures. Additionally, we consider AR (Luo et al., 2021),
distribution similar to Equation 11 as :
and Pocket2Mol (Peng et al., 2022), GNN-based methods
pθ,ϕY (Mt−1 |Mt , P, Y) = Zpθ (Mt−1 |Mt , P) both of which employ autoregressive frameworks to gen-
(13) erate 3D molecular atoms by conditioning on the protein
· pϕY (Y|Mt−1 , P, t)
pocket and previously generated atoms. Furthermore, our
comparison extends to recent diffusion-based approaches
Assuming all the properties y ∈ Y are conditionally inde- such as TargetDiff (Guan et al., 2023a) and DecompDiff
pendent given Mt−1 and P, Eq. 13 is factorized as follows: (Guan et al., 2023b), which have set new benchmarks in
pθ,ϕY (Mt−1 |Mt , P, Y) = Zpθ (Mt−1 |Mt , P) the non-autoregressive generation of atom coordinates and
·
Y
pϕy (y|Mt−1 , P, t) (14) types. DecompDiff enhances TargetDiff by integrating bond
y∈Y
considerations and introducing decomposed priors for the
ligand’s arms and scaffolds. For a comprehensive overview
Thus we train a set of models ϕY = {ϕy : ∀y ∈ Y} inde- of prior works and additional references, please refer to the
pendently and sample Mt−1 using a similar approximated § B of Appendix.

Preprint 5
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

4.3. Metrics 4.4. Experimental setup

To evaluate the quality of molecules generated by TAGM OL Generative Backbone. For our generative backbone
and the baselines, we adopt a multi-faceted assessment strat- model, we opted for TargetDiff (Guan et al., 2023a) over
egy encompassing molecular properties, their conformation, DecompDiff (Guan et al., 2023b). The decision was based
and binding affinity with the target. on TargetDiff’s self-contained nature, in contrast to Decom-
pDiff, which relies heavily on external computational tools.
4.3.1. M OLECULAR P ROPERTIES Specifically, DecompDiff utilizes AlphaSpace2 (Katigbak
et al., 2020) for extracting subpockets, which are poten-
QED (Quantitative Estimate of Druglikeness). This met-
tial protein binding sites. In fact, our study demonstrates
ric evaluates a molecule’s drug-likeness (Bickerton et al.,
how our guidance mechanism can effectively replace the
2012) by reflecting the typical distribution of molecular
computational tools employed in DecompDiff.
properties in successful drug candidates.
SA (Synthetic Accessibility). SA assesses the ease with Guide Training: In our experimental setup, as detailed
which a molecule can be synthesized (Ertl & Schuffenhauer, in § 3.1, we employed a 9-layer Invariant Graph Neural
2009), serving as a vital indicator of its practical manufac- Network (GNN) as the foundational model for construct-
turability in a laboratory or industrial setting. ing guides across various properties. Notably, for QED
Diversity. Diversity is measured as the average pairwise and SA — properties that are not dependent on the tar-
Tanimoto distances (Bajusz et al., 2015; Tanimoto, 1958) get — we adapted the k-Nearest Neighbors (k-NN) graph
among all ligands generated for a specific protein pocket. construction methodology, which omits protein atoms into
Bond Distance Distribution. We calculate the Jensen- consideration. The k value was specifically tailored to each
Shannon divergences (JSD) to assess the differences in bond property: set at 6 for both QED and SA and increased to
distance distributions between the reference molecules and 32 for the binding affinity property predictor to optimize
the generated molecules (Guan et al., 2023a;b). the model’s performance in different contexts. For training
our binding affinity guide, we used Autodock Vina scores
4.3.2. B INDING A FFINITY from the CrossDocked2020 dataset (Eberhardt et al., 2021;
Francoeur et al., 2020), and for QED and SA, we calculated
AutoDock Vina(Eberhardt et al., 2021) is employed to cal- scores using the RDKit package (Landrum, 2013). To effec-
culate the following metrics: tively guide the denoising phase, we train our guide using
Vina Score. Estimates binding affinity from the 3D pose the same noise as our backbone model, TargetDiff, which
of generated molecules, where a favorable score suggests applies Gaussian noise to coordinates and uniform noise
strong binding potential. to categories. Additional training and evaluation details of
Vina Min. By conducting a local structure minimization guide can be found in Appendix § D.
prior to affinity estimation, this metric presents a slightly
refined perspective on Vina Score. Guide Strengths: For the single objective guidance pro-
Vina Dock. Incorporating a re-docking process, Vina Dock cess, the optimal guide strength sopty for each property y
showcases the optimal binding affinity that can be achieved. is identified through a grid search on small set of 4 targets
High Affinity. This is a comparative metric evaluating the (see Appendix § E.1 for more details). The configuration
percentage of generated molecules that manifest better bind- that delivers the best value for the intended property in the
ing than a reference molecule for a given protein pocket. generated molecules is selected. For multi-objective guid-
opt
P sy are recalibrated
ance, the optimal guide strength values
4.3.3. H IT R ATE using a set of weights wy , where y∈Y wy = 1. These
modified guide strengths, wy sopt
y , are subsequently utilized
We report the percentage of molecules meeting hit criteria
to steer property optimization as described in Equation 15.
characterized as: QED ≥ 0.4, SA ≥ 0.5, and Vina Dock
In our approach, where all properties are considered equally
≤ -8.18 kcal/mol. These QED, SA thresholds are set in
important, we decided to assign equal weights and observed
line with the ranges observed in currently marketed drugs
the best hit rate. Further analyses on how performance
(Eckmann et al., 2022). The threshold for the Vina Dock
varies with respect to weights wy can be seen in Table 7 in
score is chosen to reflect a binding affinity less than 1µm.
Appendix.
This threshold is commonly used in medicinal chemistry
to ensure moderate biological activity (Yang et al., 2021; Evaluation Setup: Adhering to the setup outlined in our
Long et al., 2022; Guan et al., 2023b). Optimizing the hyper- baseline studies, we generated 100 molecules for each of
parameters in TAGM OL for hit rate allows us to generate the 100 test proteins and report the metrics detailed in § 4.3.
higher number of acceptable molecules as per our criteria.

Preprint 6
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

Table 1. Comparison of various properties between reference molecules and those generated by our model and other baselines. (↑) / (↓)
indicates whether a larger or smaller number is preferable. The first and second-place results are emphasized with bold and underlined
text, respectively. Refer to Appendix E.6 for evaluation on additional metrics suggested in (Harris et al., 2023).

Methods Vina Score (↓) Vina Min (↓) Vina Dock (↓) High Affinity (↑) QED (↑) SA (↑) Diversity (↑) Hit(↑)
Avg. Med. Avg. Med. Avg. Med. Avg. Med. Avg. Med. Avg. Med. Avg. Med. Rate %
Reference -6.36 -6.46 -6.71 -6.49 -7.45 -7.26 - - 0.48 0.47 0.73 0.74 - - 21
liGAN - - - - -6.33 -6.20 21.1% 11.1% 0.39 0.39 0.59 0.57 0.66 0.67 13.2
AR -5.75 -5.64 -6.18 -5.88 -6.75 -6.62 37.9% 31.0% 0.51 0.50 0.63 0.63 0.70 0.70 12.9
Pocket2Mol -5.14 -4.70 -6.42 -5.82 -7.15 -6.79 48.4% 51.0% 0.56 0.57 0.74 0.75 0.69 0.71 24.3
TargetDiff -5.47 -6.30 -6.64 -6.83 -7.80 -7.91 58.1% 59.1% 0.48 0.48 0.58 0.58 0.72 0.71 20.5
DecompDiff -4.85 -6.03 -6.76 -7.09 -8.48 -8.50 64.8% 78.6% 0.44 0.41 0.59 0.59 0.63 0.62 24.9
TAGM OL -7.02 -7.77 -7.95 -8.07 -8.59 -8.69 69.8% 76.4% 0.55 0.56 0.56 0.56 0.69 0.70 27.7

Table 2. Jensen-Shannon Divergence comparing bond distance dis-

tributions between reference molecules and generated molecules.
0.6
Lower values indicate better performance. ’-’ represents single Reference:
PocketMol: JSD(0.14)
bonds, ’=’ represents double bonds, and ’:’ represents aromatic 0.5 TargetDiff: JSD(0.09)
bonds. The first and second-place results are emphasized with bold DecompDiff: JSD(0.06)
0.4 Ours: JSD(0.09)
and underlined text, respectively.

Density
0.3
Bond liGAN AR Pocket2 Target Decomp TAGM OL
Mol Diff Diff 0.2
C-C 0.601 0.609 0.496 0.369 0.371 0.384
0.1
C=C 0.665 0.620 0.561 0.505 0.539 0.501
C-N 0.634 0.474 0.416 0.363 0.352 0.365 0.0
C=N 0.749 0.635 0.629 0.550 0.592 0.559 0 2 4 6 8 10 12
C-O 0.656 0.492 0.454 0.421 0.373 0.422
Distance of all atom pairs (Å)
C=O 0.661 0.558 0.516 0.461 0.381 0.430
C:C 0.497 0.451 0.416 0.263 0.258 0.269 Figure 2. Comparison of distance distributions between all-atom
C:N 0.638 0.551 0.487 0.235 0.273 0.252 distances of reference molecules in the test set (Reference) and
distances in model-generated molecules. The Jensen-Shannon
divergence (JSD) between these two distributions is calculated and
reported.
4.5. Results
4.5.1. P ERFORMANCE OF TAGM OL able performance, surpassing diffusion-based models in
QED by 14.6% and 22.2% compared to TargetDiff and De-
TAGM OL outperforms all baselines, including the refer-
compDiff, respectively, while maintaining similar diversity
ence molecules, in binding-related metrics and hit rate (see
levels. Although there’s a slight decrease in SA, the metric
Table 1). This achievement is especially noteworthy when
remains within a reasonable range, indicating satisfactory
compared to the state-of-the-art DecompDiff model, which
synthesizability. A detailed discussion of the challenges
relies on external computation for informed priors in the de-
in optimizing SA is provided in Appendix E.4. Moreover,
noising process. The success of our model, attained without
Appendix E.5 presents compelling evidence of statistically
requiring extra data, underscores the importance of effec-
significant changes in guided properties upon the incorpo-
tively learning useful signals from the existing training set
ration of guidance. Furthermore, as shown in Table 2, our
and skillfully guiding the diffusion denoising phase.
method closely maintains the bond distribution in align-
A key metric where TAGM OL excels is the Vina Score, ment with the backbone diffusion model, namely TargetDiff,
achieving a 22% improvement over the state of the art surpassing the non-diffusion baselines. This outcome em-
(AR). This highlights its proficiency not only in generating phasizes the efficient synergy between our guide model and
molecules that bind effectively with proteins but also in the diffusion model, markedly improving our ability to gen-
producing high-affinity poses. This is further substantiated erate molecules with targeted properties while preserving
by the fact that on average an impressive 69.8% of the molecular conformation.
molecules generated by our model exhibit superior binding
TAGM OL, which focuses on optimizing multiple properties,
affinity compared to reference molecules, surpassing all
achieves a state-of-the-art hit rate, surpassing the backbone
other baselines.
model—TargetDiff—by a considerable margin. This is ev-
In terms of molecular properties, TAGM OL shows remark- idenced by the higher fraction of molecules generated by

Preprint 7
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

Table 3. Ablation analysis assessing the properties of generated molecules under different property guidance scenarios. The first and
second-place results are emphasized with bold and underlined text, respectively.

Methods Vina Score (↓) Vina Min (↓) Vina Dock (↓) QED (↑) SA (↑) Hit (↑)
Avg. Med. Avg. Med. Avg. Med. Avg. Med. Avg. Med. Rate
backbone -5.47 -6.30 -6.64 -6.83 -7.80 -7.91 0.48 0.48 0.58 0.58 20.5
backbone + BA Opt -7.35 -8.18 -8.38 -8.46 -9.04 -9.04 0.49 0.50 0.53 0.53 22.6
backbone + QED Opt -5.48 -6.46 -6.77 -6.93 -7.93 -8.06 0.56 0.57 0.58 0.58 24.5
backbone + SA Opt -5.22 -6.03 -6.40 -6.57 -7.53 -7.73 0.47 0.48 0.61 0.60 19.4
TAGM OL -7.02 -7.77 -7.95 -8.07 -8.59 -8.69 0.55 0.56 0.56 0.56 27.7

the denoising model and the guide model, emphasizing their

combined power. Notably, a substantial enhancement is
observed when we guide for binding affinity. This improve-
ment is likely due to our approach of calculating binding
affinity through docking software (Trott & Olson, 2010),
which primarily depends on atomic distances. Our guide
model’s architecture, with its inherent inductive bias in mod-
eling atomic distances, effectively facilitates the explicit
guidance for coordinate generation, contributing to this ad-
vancement. Consequently, improvements in QED and SA
scores are comparatively less pronounced, as these prop-
erties exhibit a relatively lower dependence on molecule’s
geometric configuration. Lastly, when we provide guidance
for all properties simultaneously, we witness substantial
improvements across the majority of the properties, and
achieves the highest hit rate. However, a slight decrease in
SA scores is observed, which, nonetheless, lies well within
our hit criteria. We also present additional visualization
Figure 3. Visualization of reference molecules (left), alongside
plots in Appendix E.3.
molecules generated by our backbone, TargetDiff (middle), and
TAGM OL (right), for two targets: 5D7N and 5MGL.
5. Conclusions
our method that meet the hit criteria. The collective re- In this work, we redefined the problem of drug generative
sults underscore the formidable capabilities of TAGM OL modeling by extending our focus beyond merely optimizing
in the SBDD domain. These advancements in critical areas binding activity. We emphasize the importance of optimiz-
of molecular generation and binding affinity demonstrate ing additional properties and integrating these considera-
the model’s potential to meet the intricate and diverse chal- tions into our problem formulation, even when training data
lenges of drug design. Figure 3 presents examples of ligand is sparse. Secondly, we developed a optimized generative
molecules generated by our model, featuring valid structures method, TAGM OL, which effectively combines two key
and reasonable binding poses to the target. components: an SE(3) equivariant graph diffusion model
to accurately replicate the geometries found in the training
4.5.2. S INGLE & M ULTI O BJECTIVE G UIDANCE set and a multi-objective guidance driven by SE(3) invari-
ant GNN models. This innovative combination enables
In Table 3, we conduct an ablation study on our property our model to efficiently navigate the diffusion sampling
guidance mechanism for multiple properties separately and space, focusing on properties of interest through the use of
all of them in tandem, demonstrating its effectiveness across gradients. Lastly, our rigorous empirical evaluation demon-
various aspects. Specifically, we observe superior perfor- strates that TAGM OL notably enhances metric performance,
mance in each property when we provide respective guid- balancing the optimization of binding affinity with critical
ance, showcasing the robustness of our guidance mechanism. pharmacological properties such as QED and SA.
We observe that when we provide guidance for one property,
it does not result in a substantial deterioration in other prop- Limitations and Future Work. Although the present work
erties. This highlights the collaborative mechanism between shows promising results, we share some complementary

Preprint 8
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

directions worth investigating in the future. First, improved and Hopkins, A. L. Quantifying the chemical beauty of
representation of the system using additional inductive drugs. Nature chemistry, 4(2):90–98, 2012.
bias in terms of symmetry and coarse-graining could
furnish more accurate predictions of different properties Bjerrum, E. J. and Threlfall, R. Molecular generation with
that are structure dependent. Second, model compression recurrent neural networks (rnns). ArXiv, abs/1705.04612,
and parameter pruning could speed-up inference without 2017. URL https://api.semanticscholar.
compromising performance. Third, integrating constraints org/CorpusID:8280382.
on discrete attributes into gradient-based optimization may Chenthamarakshan, V., Das, P., Hoffman, S., Strobelt, H.,
enable the modeling of complementary properties, such as Padhi, I., Lim, K. W., Hoover, B., Manica, M., Born, J.,
enforcing hydrogen bonds between specific residues within Laino, T., et al. Cogmol: Target-specific and selective
the pocket, a capability our current algorithm lacks. Fourth, drug design for covid-19 using deep generative models.
exploring reduced noise sampling strategies or guidance Advances in Neural Information Processing Systems, 33:
mechanisms for generating more precise conformations 4320–4332, 2020.
could lead to reduced strain energy and the production of
stable molecules. Finally, it would be worth investigating Dhariwal, P. and Nichol, A. Diffusion models beat gans
physics-based techniques, such as molecular simulations, on image synthesis. Advances in neural information
to allow generalization to larger family of systems. processing systems, 34:8780–8794, 2021.
Eberhardt, J., Santos-Martins, D., Tillack, A. F., and Forli, S.
6. Broader Impact Autodock vina 1.2. 0: New docking methods, expanded
force field, and python bindings. Journal of chemical
Drug discovery is an area that can have broad impact on
information and modeling, 61(8):3891–3898, 2021.
humanity. Accelerating the discovery of novel drugs can en-
sure more inclusive healthcare worldwide and can have im- Eckmann, P., Sun, K., Zhao, B., Feng, M., Gilson, M. K.,
pact especially on reducing the disparity within and across and Yu, R. Limo: Latent inceptionism for targeted
countries. In the development of our methodology, we have molecule generation. arXiv preprint arXiv:2206.09010,
taken care to ensure its ethical application, fully acknowl- 2022.
edging that there exists inadvertent potential for misuse.
Specifically, there is a risk that our optimization algorithms Ertl, P. and Schuffenhauer, A. Estimation of synthetic acces-
could be repurposed for creating drugs with detrimental side sibility score of drug-like molecules based on molecular
effects. This underscores the importance of exercising cau- complexity and fragment contributions. Journal of chem-
tion and responsibility in the deployment of such algorithms informatics, 1:1–11, 2009.
in practical scenarios to prevent any unintended negative
Francoeur, P. G., Masuda, T., Sunseri, J., Jia, A., Iovanisci,
consequences from their use.
R. B., Snyder, I., and Koes, D. R. Three-dimensional
convolutional neural networks and a cross-docked data
References set for structure-based drug design. Journal of chemical
information and modeling, 60(9):4200–4215, 2020.
Alonso, H., Bliznyuk, A. A., and Gready, J. E. Com-
bining docking and molecular dynamic simulations Ghorbani, M., Gendelev, L., Beroza, P., and Keiser, M. Au-
in drug design. Medicinal Research Reviews, 26, toregressive fragment-based diffusion for pocket-aware
2006. URL https://api.semanticscholar. ligand design. In NeurIPS 2023 Generative AI and Biol-
org/CorpusID:16337213. ogy (GenBio) Workshop, 2023.
Anderson, A. C. The process of structure-based drug design. Gómez-Bombarelli, R., Wei, J. N., Duvenaud, D.,
Chemistry & biology, 10(9):787–797, 2003. Hernández-Lobato, J. M., Sánchez-Lengeling, B., She-
Bajusz, D., Rácz, A., and Héberger, K. Why is tanimoto berla, D., Aguilera-Iparraguirre, J., Hirzel, T. D., Adams,
index an appropriate choice for fingerprint-based similar- R. P., and Aspuru-Guzik, A. Automatic chemical de-
ity calculations? Journal of cheminformatics, 7(1):1–13, sign using a data-driven continuous representation of
2015. molecules. ACS central science, 4(2):268–276, 2018.

Bao, F., Zhao, M., Hao, Z., Li, P., Li, C., and Zhu, J. Equiv- Guan, J., Qian, W. W., Peng, X., Su, Y., Peng, J., and Ma,
ariant energy-guided sde for inverse molecular design. In J. 3d equivariant diffusion for target-aware molecule
The eleventh international conference on learning repre- generation and affinity prediction. In The Eleventh In-
sentations, 2022. ternational Conference on Learning Representations,
2023a. URL https://openreview.net/forum?
Bickerton, G. R., Paolini, G. V., Besnard, J., Muresan, S., id=kJqXEPXMsE0.

Preprint 9
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

Guan, J., Zhou, X., Yang, Y., Bao, Y., Peng, J., Ma, J., Liu, Liu, Q., Allamanis, M., Brockschmidt, M., and Gaunt, A. L.
Q., Wang, L., and Gu, Q. Decompdiff: Diffusion models Constrained graph variational autoencoders for molecule
with decomposed priors for structure-based drug design. design. In Neural Information Processing Systems,
2023b. 2018. URL https://api.semanticscholar.
org/CorpusID:43924638.
Harris, C., Didi, K., Jamasb, A., Joshi, C., Mathis, S., Lio,
P., and Blundell, T. Posecheck: Generative models for Liu, Z., Zhang, W., Xia, Y., Wu, L., Xie, S., Qin, T., Zhang,
3d structure-based drug design produce unrealistic poses. M., and Liu, T.-Y. MolXPT: Wrapping molecules with
In NeurIPS 2023 Generative AI and Biology (GenBio) text for generative pre-training. In Proceedings of the 61st
Workshop, 2023. Annual Meeting of the Association for Computational
Linguistics (Volume 2: Short Papers), Toronto, Canada,
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- July 2023. Association for Computational Linguistics.
bilistic models. Advances in Neural Information Process-
Long, S., Zhou, Y., Dai, X., and Zhou, H. Zero-shot 3d drug
ing Systems, 33:6840–6851, 2020.
design by sketching and generating. Advances in Neural
Hoogeboom, E., Satorras, V. G., Vignac, C., and Welling, M. Information Processing Systems, 35:23894–23907, 2022.
Equivariant diffusion for molecule generation in 3d. In Luo, S., Guan, J., Ma, J., and Peng, J. A 3d generative model
International conference on machine learning, pp. 8867– for structure-based drug design. Advances in Neural
8887. PMLR, 2022. Information Processing Systems, 34:6229–6239, 2021.

Huang, Z., Yang, L., Zhou, X., Zhang, Z., Zhang, W., Zheng, Olivecrona, M., Blaschke, T., Engkvist, O., and Chen, H.
X., Chen, J., Wang, Y., CUI, B., and Yang, W. Protein- Molecular de-novo design through deep reinforcement
ligand interaction prior for binding-aware 3d molecule learning. Journal of cheminformatics, 9(1):1–14, 2017.
diffusion models. In The Twelfth International Confer-
Peng, X., Luo, S., Guan, J., Xie, Q., Peng, J., and Ma, J.
ence on Learning Representations, 2024. URL https:
Pocket2mol: Efficient molecular sampling based on 3d
//openreview.net/forum?id=qH9nrMNTIW.
protein pockets. In International Conference on Machine
Jeon, W. and Kim, D. Autonomous molecule generation Learning, pp. 17644–17655. PMLR, 2022.
using reinforcement learning and docking to develop po- Peng, X., Guan, J., Liu, Q., and Ma, J. Moldiff: Addressing
tential novel inhibitors. Scientific reports, 10(1):22104, the atom-bond inconsistency problem in 3d molecule
2020. diffusion generation. arXiv preprint arXiv:2305.07508,
2023.
Katigbak, J., Li, H., Rooklin, D., and Zhang, Y. Alphaspace
2.0: representing concave biomolecular surfaces using β- Ragoza, M., Masuda, T., and Koes, D. R. Learning a
clusters. Journal of chemical information and modeling, continuous representation of 3d molecular structures
60(3):1494–1508, 2020. with deep generative models. ArXiv, abs/2010.08687,
2020. URL https://api.semanticscholar.
Kingma, D. P. and Welling, M. Auto-encoding variational org/CorpusID:224713572.
bayes. arXiv preprint arXiv:1312.6114, 2013.
Ragoza, M., Masuda, T., and Koes, D. R. Generating 3d
Köhler, J., Klein, L., and Noé, F. Equivariant flows: exact molecules conditional on receptor binding sites with deep
likelihood generative learning for symmetric densities. generative models. Chemical science, 13(9):2701–2713,
In International Conference on Machine Learning, pp. 2022.
5361–5370. PMLR, 2020. Satorras, V. G., Hoogeboom, E., Fuchs, F. B., Posner, I., and
Welling, M. E (n) equivariant normalizing flows. arXiv
Landrum, G. Rdkit documentation. Release, 1(1-79):4,
preprint arXiv:2105.09016, 2021.
2013.
Skalic, M., Jiménez, J., Sabbadin, D., and De Fabritiis,
Lee, S., Jo, J., and Hwang, S. J. Exploring chemical space G. Shape-based generative modeling for de novo drug
with score-based out-of-distribution generation. Proceed- design. Journal of chemical information and modeling,
ings of the 40th International Conference on Machine 59(3):1205–1214, 2019.
Learning, 2023.
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and
Liu, M., Luo, Y., Uchino, K., Maruhashi, K., and Ji, S. Ganguli, S. Deep unsupervised learning using nonequi-
Generating 3d molecules for target protein binding. In librium thermodynamics. In International conference on
International Conference on Machine Learning, 2022. machine learning, pp. 2256–2265. PMLR, 2015.

Preprint 10
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

Stanton, S., Maddox, W., Gruver, N., Maffettone, P., De- Zhou, X., Cheng, X., Yang, Y., Bao, Y., Wang, L., and
laney, E., Greenside, P., and Wilson, A. G. Accelerating Gu, Q. Decompopt: Controllable and decomposed diffu-
bayesian optimization for biological sequence design with sion models for structure-based molecular optimization.
denoising autoencoders. In International Conference on In The Twelfth International Conference on Learning
Machine Learning, pp. 20459–20478. PMLR, 2022. Representations, 2024. URL https://openreview.
net/forum?id=Y3BbxvAQS9.
Tanimoto, T. T. Elementary mathematical theory of classifi-
cation and prediction. 1958. Zhou, Z., Kearnes, S. M., Li, L., Zare, R. N., and Riley, P. F.
Optimization of molecules via deep reinforcement learn-
Trott, O. and Olson, A. J. Autodock vina: improving the ing. Scientific Reports, 9, 2018. URL https://api.
speed and accuracy of docking with a new scoring func- semanticscholar.org/CorpusID:53040150.
tion, efficient optimization, and multithreading. Journal
Zhou, Z., Kearnes, S., Li, L., Zare, R. N., and Riley, P. Op-
of computational chemistry, 31(2):455–461, 2010.
timization of molecules via deep reinforcement learning.
Scientific reports, 9(1):10752, 2019.
Xu, M., Yu, L., Song, Y., Shi, C., Ermon, S., and Tang, J.
Geodiff: A geometric diffusion model for molecular con-
formation generation. arXiv preprint arXiv:2203.02923,
2022.

Yang, K., Swanson, K., Jin, W., Coley, C. W., Eiden, P.,
Gao, H., Guzman-Perez, A., Hopper, T., Kelley, B. P.,
Mathea, M., Palmer, A., Settels, V., Jaakkola, T., Jensen,
K. F., and Barzilay, R. Analyzing learned molecu-
lar representations for property prediction. Journal of
Chemical Information and Modeling, 59:3370 – 3388,
2019. URL https://api.semanticscholar.
org/CorpusID:198986021.

Yang, Y., Ouyang, S., Dang, M., Zheng, M., Li, L., and
Zhou, H. Knowledge guided geometric editing for unsu-
pervised drug design. 2021.

You, J., Liu, B., Ying, Z., Pande, V., and Leskovec, J. Graph
convolutional policy network for goal-directed molecu-
lar graph generation. Advances in neural information
processing systems, 31, 2018.

Zhang, Z., Min, Y., Zheng, S., and Liu, Q. Molecule gen-
eration for target protein binding with structural motifs.
In The Eleventh International Conference on Learning
Representations, 2023. URL https://openreview.
net/forum?id=Rq13idF0F73.

Zhavoronkov, A., Ivanenkov, Y. A., Aliper, A., Veselov,

M. S., Aladinskiy, V. A., Aladinskaya, A. V., Terentiev,
V. A., Polykovskiy, D. A., Kuznetsov, M. D., Asadulaev,
A., et al. Deep learning enables rapid identification of po-
tent ddr1 kinase inhibitors. Nature biotechnology, 2019.

Zhou, G., Gao, Z., Ding, Q., Zheng, H., Xu, H., Wei, Z.,
Zhang, L., and Ke, G. Uni-mol: A universal 3d molecu-
lar representation learning framework. In The Eleventh
International Conference on Learning Representations,
2023. URL https://openreview.net/forum?
id=6K2RM6wVqKu.

Preprint 11
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

A. Notation Summary

Table 4. Notations

Notation Explanation Domain

NP Number of protein atoms ∈ R1
Nf Feature dimension of protein atom ∈ R1
xPi i-th protein atom coordinate ∈ R3
viP i-th protein atom feature ∈ RN f
NM Number of ligand atoms ∈ R1
K Number of ligand atom types ∈ R1
xMi i-th ligand atom coordinate ∈ R3
xMi,t i-th ligand atom coordinate at diffusion time step t ∈ R3
viM i-th ligand atom one-hot representation ∈ RK
M
vi,t i-th ligand atom one-hot representation at diffusion time step t ∈ RK
XM Matrix representation of all ligand atom coordinates ∈ RNM ×3
VM Matrix representation of all ligand atom types ∈ RNM ×K
XP Matrix representation of all protein atom coordinates ∈ RNP ×3
VP Matrix representation of all protein atom features ∈ RNP ×Nf
βt Variance Schedule for the diffusion model ∈ R1
q Diffusion noising transition function -
θ Parameters of generative model -
pθ Denoising diffusion transition function -
Y Set of properties to be optimized during generation -
y A property y ∈ Y -
ϕy Parameters of guide corresponding to y -
ϕY Set of guide parameters corresponding to ∀y ∈ Y -
sy Guide strength for property y ∈ R1
sopt
y Optimal Guide strength for property y ∈ R1

B. Related Work
Neural Drug Design Neural models have significantly changed the landscape for drug design that was previously
dominated by computational techniques such as MD simulations and Docking (Alonso et al., 2006). It has gone through
multiple paradigm shifts in the recent past - ranging from using only 1D representations (SMILES) (Bjerrum & Threlfall,
2017; Gómez-Bombarelli et al., 2018) to 2D representations (molecular graphs) (Liu et al., 2018; Zhou et al., 2018) to 3D
representations (Skalic et al., 2019; Ragoza et al., 2020) (molecular coordinates). Recent works have also showcased the
importance of using target-aware models for practical drug design. Advancements in target-aware drug design, particularly
in the realm of text and graph-based generative methods (Eckmann et al., 2022; Stanton et al., 2022; Chenthamarakshan
et al., 2020; Lee et al., 2023), have made significant strides. However, these methods often generate ligands without
considering the 3D structure of the target protein. TargetDiff (Guan et al., 2023a) and DecompDiff (Guan et al., 2023b)
are two diffusion-based models that consider the 3D structure of the target protein. DecompDiff extends TargetDiff by
decomposing the task into generating the arms and scaffold while explicitly considering bonds between the atoms during the
diffusion process which is ignored by TargetDiff. Fragment-based generation methods such as (Ghorbani et al., 2023) allow
for controlled generation, ensuring that only certain types of molecular fragments are present in the generated ligand. But
they still fall behind the performance of diffusion-based approaches in terms of their binding affinity.

Molecular Property Prediction Predicting the properties of molecules through physio-chemical experiments is an
expensive and time-consuming solution, that is unsuitable for getting intermediate feedback on AI-designed molecules.
Using neural models has allowed high-quality prediction of various molecular properties in an automated fashion. Neural
methods use different molecular representations to predict molecules, including SMILES representation with pre-trained
transformers (Liu et al., 2023) or molecular structures (Yang et al., 2019; Zhou et al., 2023) for predicting the properties of
interest.

Preprint 12
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

Molecular Property Optimization Prior methods for property optimization have focused on using Reinforcement learning
to generate molecules with desired properties (You et al., 2018; Zhou et al., 2019; Zhavoronkov et al., 2019; Jeon & Kim,
2020; Olivecrona et al., 2017). However, RL methods are often computationally expensive and challenging to optimize
due to the vast search space. LIMO (Eckmann et al., 2022) uses Variational Auto Encoders (VAEs) and gradient gudiance
on MLP-based property predictors but does not consider the 3D structure of the target protein or the generated ligand.
(Lee et al., 2023) introduced a novel graph-based diffusion model for out-of-distribution (OOD) generation, enhancing
explorability through OOD-controlled reverse-time diffusion and property-guided sampling, albeit focusing solely on 2D
molecular representations, unlike the target-aware 3D structure generation with multi-objective guidance featured in our
work.

C. Property Distribution in Training Data

Figure 4 illustrates the distribution of QED, SA, and Vina Scores within the training split. From the figure, it is evident that
a substantial proportion of molecules exhibit low binding and fall below the hit criteria for QED.

Histogram of QED Values Histogram of SA Values Histogram of Vina Score Values

17500
8000
12000
15000

6000 10000
12500
8000
10000
Frequency

4000
6000 7500

4000 5000
2000
2000 2500

0 0 0
0.0 0.2 0.4 0.6 0.8 0.2 0.4 0.6 0.8 1.0 17.5 15.0 12.5 10.0 7.5 5.0 2.5 0.0
QED SA Vina Score

Figure 4. Distribution of QED, SA, and Vina Score properties in the training split, with the mean values represented by dotted lines

D. Training and Evaluation of Guide

D.1. Training
The models are trained with a batch size of 4 to minimize the Root Mean Square Error (RMSE) loss, utilizing a gradient
descent algorithm. The initial learning rates were set at 0.001 for the binding affinity and QED guides, and at 5e-4 for the SA
guide. Additionally, to maintain the stability of the training process, the gradient norms were clipped at a maximum value
of 8. The learning rate underwent exponential decay by a factor of 0.6, with a lower bound set at 1e-5. This decay was
triggered in the absence of any improvement in the validation loss across 10 consecutive evaluations. Evaluations were
conducted every 2000 training steps, and convergence of all models was achieved within 200,000 steps. All experiments
were conducted using the NVIDIA A100 GPU.

D.2. Evaluation
The performance of each guide model was evaluated for their respective property predictions using the same test dataset. As
outlined in § 4.1, we employed the same split to evaluate our binding affinity guide model. While the train-test split ensures
the uniqueness of the protein-ligand complexes between the sets, 34 ligands are common to both, leading to potential train-
test leakage. Consequently, for QED and SA guides that depend solely on ligands, we have excluded these 34 overlapping
ligands from the test split (No overlaps). Table 5 presents the performance evaluation of each guide model, indicating high
R2 values on the original test dataset. After removing the overlapping ligands from the test set, the performance metrics
remained consistent, confirming the model’s effective generalization to unseen ligands. While we report the results on fair
test split, it’s crucial to note that, diverging from the traditional machine learning training and testing process, we train the
guide to direct the generated latent space of the diffusion model.

Preprint 13
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

Table 5. Evaluation of the guide model on the original test set with unique protein-ligand complexes, and on a modified test set (No
Overlaps) where 32 overlapping ligands have been removed.

Target Test Set (Original) Test Set (No Overlaps)

Property RMSE R2 RMSE R2
BA 0.642 0.953 0.652 0.951
QED 0.067 0.892 0.079 0.850
SA 0.054 0.848 0.065 0.821

E. Additonal Experiments
E.1. Search for optimal Guide Strength
To determine the optimal Guide Strength (s), a grid search was conducted over the set of values: {0, 1, 2, 5, 10, 20, 50}. Here,
s = 0 represents the scenario without guidance, included for comparative purposes to gauge guide-induced enhancements.
The generation process was limited to 50 molecules for a limited set of 4 targets, to accommodate computational constraints.
Initially, for each property, we find the optimal guide strength sopt
y over the grid. Once the optimal values for single objective
are tuned, for multi-objective guidance, we obtained optimal s values by recalibrating the sopt
y with wy , as detailed in §-4.4.

Table 6 presents the results of our tuning across different guide strengths (s) mentioned in the grid. It is evident that non-zero
s values yield improved property metrics, underscoring the efficacy of guidance. For Binding Affinity (BA) guidance,
optimal s values emerge as 1 and 2, based on average and median Vina Scores, respectively. Considering both Vina Min
and Vina Dock values, s = 2 is selected for superior outcomes. In the case of QED guidance, an s value of 20 distinctly
outperforms others in terms of both average and median QED scores. With SA guidance, although s = 20 and s = 50
appear to be optimal, the validity(Guan et al., 2023b) of generated molecules takes a low value of 67% and 46%, respectively.
Consequently, s = 5 is chosen, providing the second-best SA values, with 84% of molecules being valid and only a slight
decrease in average SA value by 0.02. In summary, the selected Guide Strengths are: sopt opt opt
ba = 2, sqed = 20, and ssa = 5.

Table 6. Search for optimal Guide Strength (s), conducted individually for each Guide Model (single-objective setting). To minimize
computational demands, the generation is limited to 50 samples across 4 targets. Only the metrics of the property being guided are
included. Symbols (↑) / (↓) denote preference for higher or lower values, respectively. Top results are highlighted in bold for first place
and underlined for second. Italicized figures indicate instances where over 25% of molecules generated under the specified guide strength
were not valid.

BA Guide QED Guide SA Guide

s Vina Score (↓) Vina Min (↓) Vina Dock (↓) QED (↑) SA (↑)
Avg. Med. Avg. Med. Avg. Med. Avg. Med. Avg. Med.
0 -5.37 -6.17 -6.31 -6.83 -7.65 -7.67 0.51 0.54 0.59 0.59
1 -7.82 -7.93 -8.16 -8.24 -8.62 -8.64 0.52 0.53 0.59 0.59
2 -7.69 -8.08 -8.33 -8.40 -8.86 -8.89 0.52 0.53 0.60 0.59
5 -7.55 -7.74 -8.02 -8.04 -8.67 -8.65 0.55 0.58 0.62 0.62
10 -7.11 -7.09 -7.53 -7.33 -8.42 -8.34 0.54 0.54 0.60 0.59
20 -6.85 -6.69 -7.03 -6.87 -8.16 -8.14 0.58 0.60 0.64 0.63
50 -6.11 -5.96 -5.68 -5.91 -7.73 -7.44 0.55 0.55 0.64 0.63

E.2. Ablation on Single and Multi-Objective Guidance

To assess the impact of guidance on various combinations of properties, we conducted a detailed ablation study. Initially,
we present results for single-objective guidance on different properties, then proceed to guidance for two properties at a
time, and ultimately, guidance incorporating all properties. Our results, highlighted in Table 7, indicate that single-objective
guidance maximizes improvement for the targeted property. When applying guidance to two properties, the enhancements
for each are moderate, illustrating a trade-off compared to separate guidance. This trade-off persists with guidance across all

Preprint 14
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

properties. However, given the multifaceted nature of drug discovery, where the goal is to produce molecules with high
binding affinity and desired properties like QED and SA, our approach significantly enhances hit rate and most properties
over no guidance. This underscores the effectiveness of guidance in navigating regions that are crucial in discovery.

Table 7. Extensive ablation analysis assessing the properties of generated molecules under different property guidance scenarios. The first
and second-place results are emphasized with bold and underlined text, respectively.

wba wqed wsa Vina Score (↓) Vina Min (↓) Vina Dock (↓) QED (↑) SA (↑) Hit (↑)
Avg. Med. Avg. Med. Avg. Med. Avg. Med. Avg. Med. Rate
0 0 0 -5.47 -6.30 -6.64 -6.83 -7.80 -7.91 0.48 0.48 0.58 0.58 20.5
1 0 0 -7.35 -8.18 -8.38 -8.46 -9.04 -9.04 0.49 0.5 0.53 0.53 22.6
0 1 0 -5.48 -6.46 -6.77 -6.93 -7.93 -8.06 0.56 0.57 0.58 0.58 24.5
0 0 1 -5.22 -6.03 -6.4 -6.57 -7.53 -7.73 0.47 0.48 0.61 0.6 19.4
0.5 0.5 0 -7.11 -7.96 -8.13 -8.21 -8.82 -8.85 0.56 0.57 0.55 0.55 26.1
0.5 0 0.5 -7.2 -7.95 -8.16 -8.26 -8.83 -8.84 0.5 0.51 0.55 0.55 24.7
0 0.5 0.5 -5.43 -6.34 -6.63 -6.86 -7.85 -7.97 0.55 0.56 0.59 0.59 24.1
0.34 0.33 0.33 -7.02 -7.77 -7.95 -8.07 -8.59 -8.69 0.55 0.56 0.56 0.56 27.7

E.3. Multi-Objective Rationale

No Opt (backbone only) Excluded Property on x-axis All Properties

2.00
0.12 3.5
1.75
3.0
0.10 1.50
2.5
0.08 1.25
Density

Density

Density

2.0
1.00
0.06
1.5
0.75
0.04
0.50 1.0

0.02 0.5
0.25

0.00 0.00 0.0

16 14 12 10 8 6 4 2 0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Vina Score ( ) QED ( ) SA ( )

Figure 5. Distribution of molecular properties in molecules generated by the backbone model without any guidance (No Opt), when
guided by two properties while excluding the one indicated on the x-axis (Excluded Property on x-axis), and when guided by all three
properties (All Properties). (↓) denote properties where lower values are preferred, while (↑) indicate properties where higher values are
desirable.

Figure 5 presents the distribution of molecular properties for 10,000 generated molecules (100 per target, across 100 targets).
These molecules are generated using three methods: (i) the backbone model without any guidance; (ii) guidance using only
two properties, leaving out the third one indicated on the x-axis; and (iii) guidance using all three properties. Optimizing all
the properties helps improve the Vina Score and QED compared to the ‘No Opt’ baseline. Although it doesn’t help improve
the SA, it is still helpful, as excluding the SA property in optimization worsens the SA scores. Thus, it is useful to provide
guidance for all three properties.

Preprint 15
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

E.4. Challenges in SA optimization

The SA score serves as a comprehensive metric for assessing synthetic feasibility, considering various non-standard structural
features such as large rings, unconventional ring fusions, stereocomplexity, and overall molecule size (Ertl & Schuffenhauer,
2009). However, diffusion-based generative models face a limitation in accurately positioning atoms, leading to unrealistic
molecular topologies, especially the formation of large rings, which can negatively impact SA scores (Peng et al., 2023).
As evident from Table 8, the diffusion-based models exhibit a subpar distribution of ring sizes, notably characterized by a
prevalence of larger rings. In TAGM OL, the provision of multiple guidance signals to coordinates increases the likelihood
of forming large rings. Consequently, this propensity towards larger ring formations contributes to either a drop in SA values
or minimal improvement despite the guidance provided.

Table 8. Distribution of ring sizes in reference and generated molecules, expressed as percentages

Ring Ref Pocket2 Target Decomp TAGM OL

Size Mol Diff Diff
3 1.7% 0.1% 0.0% 2.9% 0.0%
4 0.0% 0.0% 2.8% 3.7% 2.4%
5 30.2% 16.4% 30.8% 30.8% 26.9%
6 67.4% 80.4% 50.7% 45.6% 48.6%
7 0.7% 2.6% 12.1% 11.6% 15.0%
8 0.0% 0.3% 2.7% 2.3% 3.7%
≥9 0.0% 0.2% 0.9% 3.1% 3.4%

E.5. Statistical Significance of Guidance

To demonstrate that the integration of guidance contributes to statistically significant changes in the results, we employ
Paired t-test (for QED, SA, Vina Score, Vina Min and Vina Dock) and Chi-square test (for hit rate), comparing our model,
TAGM OL, against the backbone model, TargetDiff. For the Paired t-test, we group the generated samples by target protein
and compute the average scores for each property, yielding 100 pairs for comparison in our case. Our null hypothesis posits
that there is no difference in the average values of a given property produced by the two models across various protein
targets. The p-values for the guided properties QED, SA, and Vina Score were remarkably low, at 4.65E-33, 5.00E-13, and
7.05E-10, respectively. Similarly, for Vina Min and Vina Dock, the p-values were 2.89E-15 and 4.38E-06, respectively.
On the other hand, for the hit rate, we utilize the Chi-square test, since the t-test may not be the suitable test. Here, outcomes
for a protein target are categorized as either “a greater number of molecules generated by TAGM OL satisfied the hit criteria
compared to those by TargetDiff” or the reverse, with the null hypothesis being an equal likelihood of occurrence, implying
no difference between the models. The obtained p-value of 5.73E-07 further supports our findings.
Given that the p-values across all tests are substantially lower the than the conventional threshold of 0.05, we can confidently
reject the null hypothesis, affirming that the guidance incorporation leads to highly statistically significant differences in the
outcomes.

E.6. Benchmarking with PoseCheck

Expanding our evaluation, we utilized PoseCheck (Harris et al., 2023) to evaluate the 3D poses generated by the models.
Our evaluation is focused on steric clashes and strain energy. Steric clashes quantify instances where the pairwise distance
between a protein and ligand atom is below the sum of their van der Waals radii, with a clash tolerance of 0.5 Å. Meanwhile,
strain energy represents the internal energy accumulated within a ligand due to conformational changes upon binding.
In Table 9 and 10, we present the mean values of steric clashes and the median values of strain energy for the generated
molecules both before and after docking. In our discussion, we prioritize median values for strain energy as they offer
greater representativeness in this context, especially given the presence of significant outliers.
TAGM OL demonstrates superior performance compared to diffusion-based models TargetDiff and DecompDiff in terms
of clashes, highlighting the efficacy of our binding affinity guide. However, TAGM OL exhibits shortcomings concerning
strain energy. As elucidated in PoseCheck, diffusion-based methodologies often yield elevated strain energies, a trend

Preprint 16
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

also observed in TAGMol due to minor errors in atom placement resulting from coordinate guidance. We hypothesize that
integrating strain energy guidance during the generation phase could yield more stable molecules. We plan to investigate
this in our future work.

Table 9. Evaluation of poses of generated molecules on biophysical benchmarks, PoseCheck. (↑) / (↓) indicates whether a larger or smaller
value is preferable. The first-place results are emphasized in bold.

Methods Clashes (↓) Strain Energy (↓)

TargetDiff 9.19 1258.02
DecompDiff 12.53 539.89
TAGM OL 6.05 2143.03

Table 10. Evaluation of poses of generated molecules on biophysical benchmarks, PoseCheck, after redocking. (↑) / (↓) indicates whether
a larger or smaller value is preferable. The first-place results are emphasized in bold.

Methods Clashes (↓) Strain Energy (↓)

Reference 3.5 118.17
TargetDiff 5.9 602.00
DecompDiff 5.75 441.61
TAGM OL 5.36 709.35

E.7. Additional baselines

Enhancing our evaluation, we have incorporated two new methods, IPDiff1 (Huang et al., 2024) and DecompOPT1 (Zhou
et al., 2024). These methods, which have been recently proposed, introduce additional supervised signals to enhance the
generative process.
IPDiff first pre-trains a Graph Neural Network (GNN) using binding affinity signals as supervision. It then leverages this
pretrained network to integrate interactions between the target protein and the molecular ligand into the forward process.
On the other hand, DecompOPT adopts an optimization approach based on a controllable and decomposed diffusion model.
Initially, it generates a set of molecules using DecompDiff, which are subsequently broken down for additional local
optimization via evolutionary algorithms. Throughout the evolutionary process, supervised signals such as binding affinity,
QED, and SA for decomposed substructures are recalculated to retain the most effective substructures.
By including these new methods in our evaluation, we aim to provide a more comprehensive comparison that captures
the nuances of utilizing additional supervised signals, particularly binding affinity, QED and SA, in molecular property
optimization. We believe that this expanded evaluation enhances the robustness and depth of our study.
Table 11 presents a comparative analysis of the current state-of-the-art diffusion-based models, including two novel methods
described earlier. TAGM OL showcases superior performance across Vina Score, Vina Min, and QED, while maintaining
comparable diversity, highlighting the effectiveness of our guidance. TAGM OL’s significant outperformance of DecompOPT
across various metrics is particularly notable, especially considering that DecompOPT refines molecules generated by
DecompDiff. Moreover, TAGM OL surpasses IPDiff in all binding affinity-related metrics, despite IPDiff incorporating
binding affinity signals during training. In summary, TAGM OL’s success stands out when compared to models employing
additional supervised signals, highlighting the effectiveness of our guidance scheme.

F. Time Complexity
When executed on an NVIDIA A100 GPU, we observed a notable processing time trend for inference. Guidance for
properties like QED and SA led to significant improvements (refer to Table 7) with minimal time increases of 1.06x and
1
Since these are contemporary works that will appear in ICLR 24, a comprehensive comparison could not be conducted.

Preprint 17
 TAGM OL: Target-Aware Gradient-guided Molecule Generation

Table 11. Comparison of various properties between reference molecules and those generated by our model and other new baselines. (↑) /
(↓) indicates whether a larger or smaller number is preferable. The first and second-place results are emphasized with bold and underlined
text, respectively.

Methods Vina Score (↓) Vina Min (↓) Vina Dock (↓) High Affinity (↑) QED (↑) SA (↑) Diversity (↑)
Avg. Med. Avg. Med. Avg. Med. Avg. Med. Avg. Med. Avg. Med. Avg. Med.
Reference -6.36 -6.46 -6.71 -6.49 -7.45 -7.26 - - 0.48 0.47 0.73 0.74 - -
TargetDiff -5.47 -6.30 -6.64 -6.83 -7.80 -7.91 58.1% 59.1% 0.48 0.48 0.58 0.58 0.72 0.71
DecompDiff -4.85 -6.03 -6.76 -7.09 -8.48 -8.50 64.8% 78.6% 0.44 0.41 0.59 0.59 0.63 0.62
IPDiff -6.42 -7.01 -7.45 -7.48 -8.57 -8.51 69.5% 75.5% 0.52 0.52 0.59 0.59 0.74 0.73
DecompOPT -5.87 -6.81 -7.35 -7.78 -8.98 -9.01 73.5% 93.3% 0.48 0.45 0.65 0.65 0.60 0.60
TAGM OL -7.02 -7.77 -7.95 -8.07 -8.59 -8.69 69.8% 76.4% 0.55 0.56 0.56 0.56 0.68 0.69

1.07x, respectively, compared to the backbone. However, guiding for Binding Affinity (BA) led to a more substantial time
increase due to the inclusion of protein atoms, demanding extra computational effort compared to QED or SA guidance,
which involves only ligand atoms. Ultimately, TAGM OL recorded its longest processing time of 1.87x when applying
guidance for all properties. It is important to note that TAGM OL is still comparable to 1.72x of DecompDiff (Guan et al.,
2023a) sampling time.

Table 12. Inference time for different models

Method Time (s)

backbone 938
backbone + BA opt 1646
backbone + QED opt 995
backbone + SA opt 1007
TAGM OL 1755

G. Pseudo Code for TAGM OL

This section provides a summary of the overall sampling procedures employing multiple guidances.

Algorithm 1 Psuedo Code of TAGM OL

Require: The protein binding site P, generative model θ, Property Predictors ϕY .
Ensure: Generate ligand molecule M that binds to the protein pocket & optimized for properties Y
Sample NM atoms based on a prior distribution relative to the pocket size.
Move CoM of protein atoms to zero.
Sample coordinates XM M
T and atom types VT based on prior:
M
XT ∼ N (0, I)
VTM = one hot(arg maxi gi ), where g ∼ Gumbel(0, 1)
for t = T, T − 1, . . . , 1 do
b M, V
Calculate [X b M ] using θ([XM , VM ], t, P)
0|t 0|t t t
 
M M
Calculate µ̃t Xt , X̂0|t according to posterior in Equation 3
Calculate δ according to Equation 16
Sample Mt−1 according to the Equation 15
end for

Preprint 18