COMPUTATIONAL MRI:

COMPRESSED SENSING AND BEYOND

Mariya Doneva

Mathematical Models for Magnetic Resonance

Imaging Reconstruction
An overview of the approaches, problems, and future research areas

S
ince its inception in the early 1970s [1], magnetic resonance
imaging (MRI) has revolutionized radiology and medicine.
Apart from high-quality data acquisition, image reconstruc-
tion is an important step to guarantee high image quality in
MRI. Although the very first MR images were obtained from
data resembling radial projections of the imaged object by ap-
plying an iterative reconstruction algorithm [1], non-Cartesian
acquisition and iterative reconstruction techniques were not ad-
opted in clinical MRI for many years, and, even today, their use
is very limited. The reason for this is twofold. First, the underly-
ing assumption that the measured data are radial projections of
the imaged object fails in the presence of B 0 field inhomogene-
ity and/or gradient waveform imperfections. Second, the long
reconstruction times associated with iterative reconstruction
algorithms limit their practical application.
It was only after the introduction of spin-warp (Cartesian)
imaging [2], which allowed using the fast Fourier transform
(FFT) for image reconstruction, that MRI reconstruction
became practical and the image quality acceptable. For the next
two decades after the inception of Cartesian imaging, image
reconstruction in MRI was almost exclusively based on the
FFT, and the main effort involved tuning the scanner or correct-
ing the data to better fit the Fourier model. The k-space formal-
ism [3], [4] and the FFT made MRI reconstruction efficient, but
©ISTOCKPHOTO.COM/GOODLIFESTUDIO
this simple signal model does not tell the whole story.
The measured signal is a complex function of multiple tis-
sue and system properties as well as physical processes. The
signal is primarily determined by the tissue proton density
and relaxation constants (T1, T2, T2*). It also depends on the
system characteristics, such as the B 0 and B 1 fields [5], [6],
gradient performance [7], [8], and coil (array) used for data
acquisition [9], [10]. Motion at different scales, such as diffu-
sion [11], [12], perfusion, and cardiac or respiratory movement
[13], [14], will also affect the measurements. Other factors that
play a role include the tissue composition [15], [16], magne-
tization transfer, and magnetic susceptibility [17]. Finally,
specific sequence parameters like repetition time TR, echo
Digital Object Identifier 10.1109/MSP.2019.2936964
Date of current version: 15 January 2020 time TE, inversion time TI, flip angle, and k-space trajectory

24 licensed use limited to: The University of Toronto. Downloaded

Authorized IEEE SIGNAL PROCESSING MAGAZINE
on December | January
14,2023 2020 | UTC from IEEE Xplore. Restrictions
at 22:06:27 1053-5888/20©2020IEEE
apply.
 also influence the measured signal. By carefully adjusting the resonant frequency ~ L = cB 0 , known as the Larmor frequen-
sequence parameters, some of these tissue and system proper- cy. In a classical description, the spin magnetization vector is
ties and processes will be weighted more than others, but they precessing around the axis of the main magnetic field B 0 with
all alter the measured signal to some extent. This sensitivity to an angular frequency ~ L . The gradient system creates a spatial
multiple effects makes MRI scans very versatile but also prone variation of the magnetic field, which locally changes the Lar-
to artifacts. mor frequency. This is used for spatial encoding and will be
To obtain an artifact-free image or a quantitative map of a tis- discussed more in the next section. From a signal encoding and
sue or system property, the simple Fourier signal model needs to image reconstruction perspective, it is important that the B 0
be extended to include the complete physical description behind field is as homogeneous as possible and also that the gradient
the image formation [18]. This can be done by formulating the system is well characterized to be able to accurately describe
image reconstruction as an inverse problem and applying an the gradient encoding.
appropriate algorithm to solve it. Using such an extended signal A perturbation of the nuclear spin orientations can be
model provides a better description of the MR data acquisition achieved only when an oscillating magnetic field is applied with
but often poses a more complex reconstruction problem. Even a frequency that closely matches the Larmor frequency. This
with the conventional Fourier model, the inherently slow data oscillating magnetic field is called the B 1 field. For B 0 fields in
acquisition calls for advanced computational methods for image the range of a few Tesla, the Larmor frequency of 1 H nuclei is
reconstruction from undersampled data. These methods usu- in the RF spectrum. The RF system produces a B 1 field at this
ally include certain assumptions about image properties, such frequency for a short time (an RF pulse) to tip the magnetization
as sparsity or low rankedness, which are used as constraints in away from the B 0 direction, thus generating transverse magne-
the reconstruction [19]–[24]. With the recent success of machine tization that can be detected by an RF receive coil tuned to the
learning, neural networks have also begun to demonstrate same frequency. A spatially uniform B 1 field is important to
advantages in computational MRI [25], [26]. All of the afore- produce homogeneous excitation.
mentioned techniques have contributed significantly to improv-
ing the quality of MR images and reducing scan time. MRI data acquisition and the
This article gives an overview of the various models beyond forward mathematical model
the conventional Fourier approach that enable image recon- Data acquisition in MRI is performed by applying a series of RF
struction with reduced acquisition time and/or improved quali- pulses and magnetic field gradients, jointly described by a pulse
ty. It also highlights the underlying signal processing problems sequence (Figure 1). A pulse sequence is a set of instructions that
associated with MRI reconstruction, which provide research tell the MR scanner when and with what strength to apply RF
opportunities in this important area that can help maximize pulses, turn magnetic field gradients on and off, and switch on
the potential of MRI for clinical applications. the receivers to acquire the signal. Ignoring noise, the signal y (t)
measured in the receiver at time t can be described as
Overview of the MR system
and signal formation basics y (t) = # t(r) e -2 ik(t) $ r dr, (1)
r

To better understand the different effects that influence MRI

data acquisition, a brief overview of the MRI system compo- where t(r) is the image, or source density, at spatial location
nents and the basics of MR signal formation is provided here. r = (rx, ry, rz) and k (t) = (k x, k y, k z) is the k-space location at
A more detailed description can be found in [27]. time t, which is determined by the strength and duration of
MR scanners consist of three main components: a magnet, the applied magnetic field gradients G = (G x, G y, G z) in
a gradient system, and a radio-frequency (RF) system plus a each repetition:
computer system for user interaction, measurement control,
t
and signal processing. The magnet creates the main static
k (t) = c # G (x) dx. (2)
magnetic field B 0 . When placed in an external magnetic field,
0
nuclei with spin 1/2 have two energy states, with an energy dif-
ference proportional to the magnetic field TE = c&B 0 , where According to this model, the measured signal is the FT of
c is the gyromagnetic ratio (c/2r = 42.577 MHz/T for 1 H) the image (1). The signal encoding is therefore called Fourier
and & is the reduced Planck constant. This creates a small encoding, or gradient encoding, since it is performed by apply-
population bias favoring the lower energy state, resulting in a ing magnetic field gradients. In practice, only a finite number
net magnetization in the direction of the magnetic field B 0 that of k-space samples can be acquired, leading to the discretized
is proportional to the spin density in the imaged object. formulation of (1):
In human MRI, 1 H nuclei (or protons) are most common-
ly used for imaging because of their high abundance in the y (k i) = / t(r j) e -2rik i $ r j, (3)
human body. While nuclear magnetic resonance is a quantum j
mechanical phenomenon, a classical description is usually suf-
ficient to understand most of the effects that play a role in the where k i and r j are the discretized, possibly nonequispaced
signal formation. The spin m­ agnetization is characterized by a k-space and image space locations. Once a sufficient number

IEEE SIGNAL PROCESSING

Authorized licensed use limited to: The University of Toronto. Downloaded MAGAZINE
on December | January
14,2023 2020 | UTC from IEEE Xplore. Restrictions apply.
at 22:06:27 25
 of k-space samples is acquired, the image can be reconstructed The necessary extensions of the forward model in (1) to
by applying an inverse discrete FT (uniform or nonuniform). describe these effects is discussed in the next section. The
In Cartesian scans (Figure 1), k-space data are acquired on a continuous notation is kept, implying that these considerations
uniform grid, allowing the use of the FFT. According to the apply for different sampling trajectories and data dimensions
Nyquist–Shannon sampling theorem, the number of required (2D, 3D, dynamic data, and so forth).
k-space samples for an accurate reconstruction is determined
by the image field of view (FOV) and its spatial frequency Extended data acquisition models for MRI
bandwidth. The measured FOV is determined by the sampling
interval Tk = 1/ FOV, and the voxel size Tx is determined by Measuring data with a real system
the maximum sampled spatial frequency Tx = 1/ (2k max). A major assumption in the Fourier model is that the spatial
The FFT signal model has many desirable properties: the encoding is perfect, which is difficult to achieve with a real-life
image reconstruction is fast and simple, it gives a uniform reso- scanner. There are a number of effects that influence the sys-
lution and point spread function over the FOV, and the encod- tem behavior, including inhomogeneities of the static magnetic
ing and reconstruction operators are unitary. Because of these field, eddy currents, coupling between gradient coils, timing
attributes, the FFT model is widely accepted and much effort inaccuracies, concomitant fields, and gradient amplifier non-
has been made to adapt the MR system and the measurement linearities, which interfere with the spatial encoding and cause
process to better fit that model. artifacts if not accounted for. Non-Cartesian trajectories are
However, strictly speaking, this framework is accurate only especially prone to these effects, since data are acquired dur-
if we acquire a single k-space sample on a Cartesian grid in ing gradient field variations. However, Cartesian scans that put
each repetition of the pulse sequence using an ideal scanner, a high demand on the gradient system can also suffer from
the scanned object is band limited in its spatial frequencies, these deviations from the desired spatial encoding gradients.
and the object does not interfere with the magnetic field or Therefore, to obtain artifact-free images, one needs to be
change over time. This is a highly unrealistic scenario. Line- aware of these effects and either precompensate for them or
by-line Cartesian sampling with a short acquisition window, include them as part of the signal model. Most system charac-
as described in Figure 1, can still be well approximated by the terization approaches approximate the gradient chain as a lin-
FFT, but it is often quite time consuming. Acquiring multiple ear, time-invariant system. Such a system can be described by
lines (echoes) in each repetition of the pulse sequence or using a time-domain impulse response function, called the gradient
non-Cartesian sampling with a long acquisition window can impulse response function [7], [8]. The resulting gradient field
improve the sampling efficiency. However, the FFT cannot can be expanded in terms of spherical harmonics and used to
be directly applied to non-Cartesian sampling. Instead, the extend the data model [7]:
non-Cartesian samples are usually first interpolated to a Car-
tesian grid, which allows the use of the FFT, at the price of a y (t ) = # t(r) e iT
~ (r ) t
e -2ri /l k l (t)$ b l (r ) dr, (4)
small approximation error [28]. Furthermore, acquiring data
over long acquisition windows and with non-Cartesian trajec- where T~(r) is the resonance frequency offset because of
tories makes the Fourier encoding model even less accurate main field inhomogeneity TB 0 , b l (r) are spherical harmon-
because of the multiple effects related to the real-life scanner ics, and the coefficients k l (t) describe the corresponding
and patient. weights and temporal evolutions. In most clinical scanners,

Acquisition Reconstruction

Gslice
RF Pulse
Gphase
Acquired Signal
Greadout
Fixed Gradient RF
Variable Gradient Tacq
TE
TR
(a) (b) (c)

FIGURE 1. The image formation in MRI, exemplified for a gradient echo sequence. (a) A series of RF pulses and magnetic field gradients (pulse sequence)
is played on the scanner to generate and spatially encode the MR signal. A signal is measured for a short time Tacq (acquisition window) centered
around the echo time TE after the RF pulse. (b) In the simplest form of Cartesian acquisition, each repetition of the pulse sequence results in a line of the
so-called k-space matrix, corresponding to spatial frequencies of the imaged object. The sequence is repeated multiple times, with a repetition time TR .
In each repetition, the phase encoding gradient is varied, which shifts the acquired line in k-space. (c) Once k-space is fully sampled, the image can be
obtained by an inverse FT.

26 licensed use limited to: The University of Toronto. Downloaded

Authorized IEEE SIGNAL PROCESSING MAGAZINE
on December | January
14,2023 2020 | UTC from IEEE Xplore. Restrictions apply.
at 22:06:27
 (a) (b) (c)

FIGURE 2. A comparison of images acquired with a spiral trajectory and reconstructed using (a) the nominal trajectory sent to the gradient chain without
any compensation for the filtering effects of the gradient chain, (b) the standard trajectory calculated by the product software based on a simple analytical
model of the transfer function, and (c) the trajectory obtained by convolution of the nominal trajectory with the first-order gradient impulse response
functions. (Source: Jürgen Rahmer; used with permission.)

the gradient r­esponse is estimated by a simple analytical

model. A more accurate estimation can be done by a system
calibration measurement using the gradient system itself [8]
or using small field probes at different spatial positions [7].
Up to first-order effects can be interpreted as deviations of
ρ (r )c6(r)
the actual trajectory from the nominal trajectory sent to the
gradients and can be incorporated in the reconstruction by
using the corrected trajectory in the Fourier model. This is
ρ (r)c1(r ) ρ(r )c5(r)
illustrated in Figure 2 for a spiral trajectory. Higher-order ef-
fects require solving (4) for image reconstruction, which is
computationally challenging. ρ (r)

Multichannel acquisition
Today, MRI data are usually acquired with a coil array, con- ρ (r)c2(r ) ρ(r )c4(r)
sisting of multiple surface coil elements with nonuniform spa-
tial sensitivity over the FOV (Figure 3). To account for this, the
ρ (r )c3(r)
Fourier model has to be extended by including the coil sensi-
tivity maps of the individual coil elements. The signal y j (t) FIGURE 3. An example of multichannel acquisition. Data are acquired
measured with coil j becomes simultaneously with multiple surface coils that have a spatially varying
coil sensitivity. Applying the Fourier model results in shaded images for
different coil elements. The coil sensitivity can be used as an additional
y j (t) = # c j (r) t(r) e -2 ik(t)$r dr,
r
(5) encoding in the signal model, allowing scan acceleration in parallel
imaging.

where c j (r) is the coil sensitivity map, which is a receiver-spe-

cific complex factor for each spatial location r. ent encoding steps, which reduces the scan time. Therefore,
One could consider the coil sensitivities as part of the extending the signal model not only tells us how to combine
source density, first reconstructing images for each coil ele- the data from multiple receivers but also comes with the addi-
ment by applying an inverse FFT and combining them later in tional benefit of faster scanning.
a single image [9]. In parallel imaging [10], the coil sensitivities The downside is that the reconstruction is no longer a
are considered to be part of the encoding, introducing a hybrid simple FFT. It requires additional time and does not have the
encoding that consists of the Fourier and sensitivity encod- useful properties of a unitary transform, resulting in spatially
ing (SENSE). The expression (5) represents the signal model varying noise amplification, also known as g-factor [10]. It
used in a generalized SENSE reconstruction problem [10]. also requires knowledge of the coil sensitivities, which can
Simultaneously acquiring data with multiple coil elements be obtained by additional measurements or from (part of) the
allows image reconstruction from a reduced number of gradi- imaging information itself.

IEEE SIGNAL PROCESSING

Authorized licensed use limited to: The University of Toronto. Downloaded MAGAZINE
on December | January
14,2023 2020 | UTC from IEEE Xplore. Restrictions apply.
at 22:06:27 27
 Subject-induced off-resonance effects TB 0 effects and separate the water and fat in the image [15],
In practice, B 0 field inhomogeneity is always present to some [16]. To make this reconstruction possible, data need to be ac-
extent. Even in an ideal scanner with a perfectly homogeneous quired at multiple echo times.
magnetic field, the tissue magnetic susceptibility locally alters In addition to the static field inhomogeneity, temporal
the magnetic field, disrupting the relation between resonance TB 0 fluctuations can occur because of dynamic susceptibility
frequency and spatial position in the Fourier model. Ignoring effects caused by patient motion, breathing, or cardiac pulsa-
this effect leads to image artifacts, which may have different tion [17], [29]. These fluctuations are rather small and usu-
strengths and appearances depending on the k-space trajectory ally remain unnoticed, but in scans with long TE, such as T2*
and acquisition window. In spiral scans, B 0 inhomogeneity weighted scans, they can cause significant artifacts, if ignored.
causes blurring, as shown in Figure 4. Including the resonance The extended signal model includes a temporal variation of the
offset T~(r) due to B 0 inhomogeneity leads to a more accurate off-resonance frequency T~(r, t):
signal model [5], [6]:
y j (t) = # c j (r) t(r) e iT ~ (r, t) t
e -2rik (t)$ r dr . (8)
y j (t) = # c j (r) t(r) e iT~ (r) t -2rik (t ) $ r
e dr, (6)
The dynamic field variations can be measured using field
which enables the correction of the off-resonance-related ar- monitoring [29] or estimated from the measurements using the
tifacts. The TB 0 field distribution is typically measured in a self-consistency of the data [30].
separate calibration scan.
Another source of off-resonance is the chemical shift, due, Motion
for example, to the presence of fat. Hydrogen nuclei in fat mol- Motion is always an issue in MRI. Even in perfectly compli-
ecules have a different resonance frequency compared with ant patients, respiration, cardiac motion, and blood flow may
those in water, resulting in constructive or destructive interfer- cause inconsistencies in the k-space data, resulting in image ar-
ences at different echo times as well as different spatial encod- tifacts. Long acquisitions, such as high-resolution 3D scans, are
ing for the water and fat signals. To account for these effects, especially susceptible to motion, but 2D scans can also be af-
the source density can be represented as a sum of water and fat fected if the motion is fast compared with the acquisition time.
signals, with the corresponding spectral model: In some sequences, such as diffusion-weighted imaging, even
motion at a microscopic scale can influence the MRI signal.
y (t ) = # c t w (r) + / a m e iT ~m t
t f (r) m e
iT~ (r) t -2rik (t ) $ r
e dr. (7) In multishot diffusion-weighted MRI, data are acquired in
m multiple shots, or k-space segments. Each k-space segment p is
affected by a different spatially varying phase { p (r), which is
Here, a m and T~ m are the amplitudes and frequency shifts of caused by microscopic motion and amplified by the diffusion-
the individual peaks in the fat spectral model, which can be encoding gradients. Therefore, simply combining these data
determined in a separate calibration scan, and t w and t f are will lead to phase cancellation artifacts, as shown in Figure 5.
the source densities of water and fat, respectively. Using this An extended signal model including this motion-induced
extended signal model, one can simultaneously correct for the phase was first proposed in [11], together with an iterative
reconstruction method to solve for the corrected signal density:

y j, p (t) = # c j (r) t(r) e i {p (r)

e -2rik (t)$ r dr. (9)

The phase can be estimated from either an external navigator

scan or the data themselves [12].
Bulk motion can also be included as a part of the signal
model [13]. This can be done by including a time-dependent
spatial transformation T (t) (describing elastic motion) that is
applied to the source density, as in the generalized reconstruc-
tion by inversion of coupled systems method [13]:

(a) (b)
y (t) = # T (t) t(r) e -2 ik(t)$r dr.(10)
r

FIGURE 4. An illustration of off-resonance effects in spiral acquisition:

(a) the image reconstructed ignoring B 0 inhomogeneity, which causes
This formulation enables not only the reconstruction of mo-
errors in the spatial encoding, resulting in local pixel shifts (these shifts
manifest as spatially dependent blurring in images acquired with a spiral tion-free images but also obtaining motion information from
trajectory); (b) the image reconstructed using the extended signal model the data, which is the goal in dynamic imaging [13], [14]. The
of (6), which removes the TB 0- induced blurring. challenges in this approach are obtaining accurate motion

28 licensed use limited to: The University of Toronto. Downloaded

Authorized IEEE SIGNAL PROCESSING MAGAZINE
on December | January
14,2023 2020 | UTC from IEEE Xplore. Restrictions apply.
at 22:06:27
 information and the computational complexity of the problem. t

Sampling strategies like golden-angle radial trajectories can be

y j (t) = # c j (r) t(r) e - T2(r) e -2 ik(t)$r dr. (11)
r

helpful to estimate the motion from the imaging data them-

selves. Alternatively, image-based navigators can be inter- This extended signal model was used in the T2 shuffling
leaved with the imaging sequence, but this generally increases method [31] to reduce the blurring and produce images with
the acquisition time. different T2 weightings. This approach of using a generalized
extended signal model can be applied to any other property
Relaxation that influences the measured MR signal, giving a more com-
Relaxation effects cause signal variations over the course of plete and more accurate description of the acquisition process.
acquisition, resulting in different contrasts for different parts Accurate signal modeling is essential to allow the best possible
of k-space. Usually the measurement is performed in a way image quality.
that the desired contrast is achieved for the center of k-space
since the lower spatial frequencies carry most of the contrast Low-dimensional models for
information in an image, and the relaxation effects are ignored regularized image reconstruction
in the image reconstruction. However, these effects are not al- The image reconstruction problem using an extended signal
ways negligible. For example, in 3D fast spin echo scans with model is usually more complex and often requires an iterative
long echo trains and center-out acquisition, T2 relaxation can method to obtain the image [18]. However, considering MR im-
cause significant blurring of short T2 species as well as de- age reconstruction as a generalized inverse problem gives a lot of
viation from the desired contrast. This is one of the reasons flexibility to accurately describe the data acquisition process and
why 3D fast spin echo sequences are not often used in practice, to include prior knowledge in the reconstruction. In its simplest
even though they can provide improved signal-to-noise ratio form, image reconstruction can be formulated as minimizing the
and isotropic resolution. distance between the forward model and the measurements:
To correct for these effects, the T2 relaxation can be includ-
ed as a part of the signal model: xt = argmin < f ( x) - y<22 , (12)
x

Shot 1 Shot 2 Shot 3

k-Space
(a)

(b) (c)

FIGURE 5. An example of microscopic-motion-induced phase variations: (a) Each k-space segment (shot) has a different motion-induced phase, leading
to data inconsistency between different parts of k-space. (b) This data inconsistency causes artifacts using the Fourier model. (c) These artifacts can be
removed when using an extended forward model accounting for the phase variations. (Source: Malte Steinhoff; used with permission.)

IEEE SIGNAL PROCESSING

Authorized licensed use limited to: The University of Toronto. Downloaded MAGAZINE
on December | January
14,2023 2020 | UTC from IEEE Xplore. Restrictions apply.
at 22:06:27 29
 where f ( x) is the signal model (linear or nonlinear), x ! C N approaches can achieve additional scan time reduction without
is the image of interest, and y ! C M is the measurement vec- reducing the image quality compared with the parallel imag-
tor. Here, x could be a 2D or 3D image, a tissue or system ing baseline (Figure 6). While these generic transforms lead
parameter map, or a higher-dimensional image series. This re- to sparse representations of most images, data-adapted signal
construction problem is ill posed even with the simple Fourier representations (dictionaries) can achieve even higher sparsity
model and a fully sampled k-space because of the discretized [32]. Learning data-adapted signal representations is an exam-
and finite sampling and the presence of noise. Using an extend- ple of using machine learning in MR reconstruction while
ed signal model as discussed above and reducing the acquired keeping the overall model-based reconstruction framework.
data further complicates the problem, making the reconstruc-
tion problem nonlinear and/or nonconvex. Solving such prob- Low rank
lems may require careful initialization and selection of an In low-rank approaches, the recovery of a sparse vector is gen-
appropriate optimization algorithm. This section discusses eralized to the recovery of a low-rank matrix. The assumption
different signal models that can be used as a prior in the form is that the full data matrix (an image, image series, or their fully
of regularization R ( x) to obtain a stable solution: sampled k-space) can be represented by a matrix, which has
a rank that is much smaller than suggested by its dimensions.
xt = argmin < f ( x) - y<22 + m R( x). (13) Therefore, the data can be approximated by a linear projection to
x
a lower-dimensional subspace. The best low-rank approximation
Sparsity can be computed by a singular-value decomposition, followed
One particular type of regularization that has gained much at- by thresholding of the singular values. In the regularized inverse
tention in the last decade is the , 1- norm, which promotes sparse problem framework, the low-rank constraint can be included as
solutions and is used in the context of compressed sensing (CS) a regularization term, which is the nuclear norm of the matrix:
[19]. According to the CS theory, a signal can be accurately re-
constructed from a small number of incoherent measurements, xt = argmin < f ( x) - y<22 + m < x <* . (14)
x
provided it is sparse or compressible. In the context of MRI,
incoherent sampling can be performed by variable-density In MRI reconstruction, the low-rank constraint can be used
nonuniform sampling on a Cartesian grid or by non-Cartesian to exploit correlations between image patches or k-space data
sampling of k-space. This is of great practical value in clinical neighborhoods. However, it is most often applied in dynamic or
MRI, since reducing the number of measurements translates parametric imaging to incorporate spatiotemporal correla-
directly into shorter acquisition times. This improved scan ef- tions in the reconstruction. Dynamic data can be represented
ficiency increases patient throughput, reduces the susceptibil- as a sum of a small number of temporal functions weighted by
ity to motion, and enables acquisitions with higher resolution. spatial maps. This implies that the so-called Casorati matrix, in
CS introduces an interesting link between data acquisition which each column represents all spatial locations for a given
and the information content in the image: the sparser the image, time point and each row all time points for a given spatial loca-
the less information we need to acquire. Most MR images can tion, is low rank. The low-rank constraint can be applied glob-
be sparsified by applying an appropriate transform W. This ally on the entire FOV or locally, by restricting the Casorati
transform sparsity prior is incorporated in the reconstruction matrix to a local image region. This locally low-rank approach
by including the regularization term R ( x) = < Wx <1 . Common can be advantageous if there are strong correlations within the
choices for W in the MRI literature include wavelets and finite temporal evolutions of neighboring voxels, leading to a lower
differences [19]. In combination with parallel imaging, such rank of the local spatiotemporal data matrix [20], [24].

2:04 min 1:30 min 3:46 min 2:02 min

(a) (b) (c) (d)

FIGURE 6. A depiction of sparsity and CS. Additional scan time reduction can be achieved when combining parallel imaging with CS (incoherent sampling
and sparsity constraint), without sacrificing image quality. Examples are shown for (a) and (c) SENSE parallel imaging and (b) and (d) its combination
with CS for brain scans [(a) and (b)] and knee scans [(c) and (d)].

30 licensed use limited to: The University of Toronto. Downloaded

Authorized IEEE SIGNAL PROCESSING MAGAZINE
on December | January
14,2023 2020 | UTC from IEEE Xplore. Restrictions apply.
at 22:06:27
 In MR parameter mapping, multiple images are acquired and can be computed from the original data without explicitly
at different acquisition parameters and the tissue parameter of transforming the data to the higher-dimensional feature space.
interest (e.g., T1 or T2 relaxation) is obtained by voxel-wise
nonlinear fit. These measurements also fall into the low-rank Neural networks
framework since the temporal signals produced by different Recently, neural networks have also attracted significant inter-
tissues are correlated, and there are a limited number of tis- est as an approach for image reconstruction. The application
sues in the imaged volume. For this class of data, there is the of neural networks ranges from learning the regularization
additional advantage that the signal evolution over the different parameters and data-adaptive sparsity transform to using the
acquisition parameters can be simulated for each tissue param- networks to solve the entire inverse problem, including the FT
eter using Bloch simulations, and a low-dimensional subspace [25], [26]. In the variational neural network approach present-
can be learned from the simulated data [21]. ed in [25], the multichannel acquisition signal model (5) is uti-
lized, and a set of convolution kernels and nonlinear functions
Manifolds is learned from the data to perform denoising or data-adaptive
In low-rank approaches, the signal is approximated by a lin- regularization. In [26], a neural network is trained to perform
ear subspace. However, the image might not lie in a subspace the complete reconstruction, but the signal model is used to
of the original data space but rather on a nonlinear manifold. generate synthetic training data, and the network is trained to
For example, if we consider images in a free-breathing and un- essentially solve the inverse problem with an additional data-
gated cardiac data set, each image can be viewed as a point in driven regularization. Therefore, even in the era of neural
a high-dimensional space, which is a function of two param- networks, accurate information about the acquisition system
eters: the cardiac and the respiratory motion. This is a smooth behavior is essential for a successful reconstruction.
function, so the points will be lying on a smooth surface or a
manifold in the high-dimensional space. One can exploit the Outstanding signal processing issues
smoothness of the surface to recover the images from highly Though a great deal of progress has been achieved in develop-
undersampled measurements. This has been suggested in the ing reconstruction techniques using extended signal models,
Smoothness Regularization on Manifolds (SToRM) algorithm there are multiple challenges that provide research opportu-
[22], which is an extension of the temporal total variation that nities. Using a comprehensive signal model that accounts for
is often used in CS for dynamic MRI. In total variation regu- all processes that influence the image formation poses a very
larization, the differences between neighboring time frames complex and challenging nonlinear reconstruction problem.
are penalized, while in [22] all differences between all tem- This is why most works focus on a single aspect of extending
poral frames are considered, but these are weighted according the signal model and need to account for potential data incon-
to a similarity measure derived from the distance between the sistencies resulting from this approximation.
images on this manifold: The selection of an appropriate prior that captures useful infor-
mation about the image is also essential in reconstruction from
xt = argmin < f ( x) - y <22 + m //w i, j < x i - x j <2, undersampled data. Sparsity has been shown to be very useful in
x i j various applications to reduce the necessary data and thereby the

w i, j = exp c - m.
2
< zi - z j < acquisition time. Data-driven priors have the potential to push
2 (15)
v the acceleration limit even further. Data-adapted regularization
has been used in several different approaches, where the prior is
The manifold is learned by determining the weights w, which learned from a limited amount of navigator/calibration data that
are obtained from navigator data z i that are acquired for each is acquired during the imaging process. The new developments
frame i. in neural network approaches set the trend of learning regular-
Another approach to apply manifold learning in the recon- ization from multiple previously acquired scans. While learning
struction is through explicit lifting to a higher-dimensional from many examples could help to find a prior that generalizes
feature space, defined by a nonlinear transform U. The idea is over multiple data sets, learning from navigator data could be
that, while the original data lie on a nonlinear manifold, in the better adapted to the individual scan. These opposing directions
feature space they lie on a subspace. The reconstruction prob- of generalization versus specialization of data-driven priors have
lem can be formulated as a regularized least-squares problem, not been sufficiently investigated and will be an important area
with a regularization term, which is the nuclear norm of the with the increasing use of data-driven approaches.
data in the feature space: When applying regularization, a practical challenge is the
selection of the regularization parameters. Automatic opti-
xt = argmin < f ( x) - y <22 + m < Ux <* . (16) mization of parameters requires an image quality metric.
x
The mean-square error to a reference image has been tradi-
Applying explicit lifting to the high-dimensional feature tionally used, and several other metrics, such as structural
space is not practical for high-dimensional data. However, one similarity, have been developed, but none of the existing tech-
can solve this reconstruction problem more efficiently using the niques describes the subjectively perceived image quality of a
kernel function, which is the inner product in the feature space human observer sufficiently well. This is also important in the

IEEE SIGNAL PROCESSING

Authorized licensed use limited to: The University of Toronto. Downloaded MAGAZINE
on December | January
14,2023 2020 | UTC from IEEE Xplore. Restrictions apply.
at 22:06:27 31
 new neural-network-based reconstruction approaches, where [7] B. Wilm, C. Barmet, M. Pavan, and K. P. Pruessmann, “Higher order recon-
struction for MRI in the presence of spatiotemporal field perturbations,” Magn.
thousands of parameters often need to be estimated. Even Reson. Med., vol. 65, no. 6, pp. 1690–1701, 2011.
more important than the perceived image quality is that the [8] J. Rahmer, P. Mazurkewitz, P. Börner, and T. Nielsen, “Rapid acquisition of the
reconstruction maintains or improves the diagnostic accuracy. 3D MRI gradient impulse response,” Magn. Reson. Med., 2019. doi: 10.1002/
mrm.27902.
Proving this is much more difficult and is currently done by
[9] P. Roemer, W. Edelstein, C. Hayes, and S. Souza, “The NMR phased array,”
extensive testing in each individual institution whenever a new Magn. Reson. Med., vol. 16, no. 2, pp. 192–225, 1990.
technique is introduced. [10] K. Pruessmann, M. Weiger, M. Scheidegger, and P. Boesiger, “SENSE: Sensitivity
Although the focus of this article is on the data models and encoding for fast MRI,” Magn. Reson. Med., vol. 42, no. 5, pp. 952–962, 1999.
[11] C. Liu, M. Moseley, and R. Bammer, “Simultaneous phase correction and
not the reconstruction algorithms, the two go hand in hand. SENSE reconstruction for navigated multi-shot DWI with non-Cartesian k-space
Keeping the reconstruction time tractable is highly important sampling,” Magn. Reson. Med., vol. 54, no. 6, pp. 1412–1422, 2005.
for the clinical adoption of new MR reconstruction methods, [12] S. Guhaniyogi, M. Chu, H. Chang, A. Song, and N. Chen, “Motion immune
diffusion imaging using augmented MUSE (AMUSE) for high-resolution multi-shot
and developing fast algorithms is a necessary step that would EPI,” Magn. Reson. Med., vol. 75, no. 2, pp. 639–652, 2016.
facilitate the broader use of more accurate data models. [13] F. Odille, N. Cindea, D. Mandry, C. Pasquier, P.-A. Vuissoz, and J. Felblinger,
“Generalized MRI reconstruction including elastic physiological motion and coil
sensitivity encoding,” Magn. Reson. Med., vol. 59, no. 6, pp. 1401–1411, 2008.
Conclusions [14] L. Feng, L. Axel, H. Chandarana, K. Block, D. Sodickson, and R. Otazo,
MRI has come a long way in the last 45 years. With the in- “XD-GRASP: Golden-angle radial MRI with reconstruction of extra motion-state
creasing computing power and the development of novel re- dimensions using compressed sensing,” Magn. Reson. Med., vol. 75, no. 2, pp.
775–788, 2016.
construction techniques, we are now able to solve more com- [15] D. Hernando, Z.-P. Liang, and P. Kellman, “Chemical shift-based water/fat
plex problems in acceptable reconstruction times. While many separation: A comparison of signal models,” Magn. Reson. Med., vol. 64, no. 3,
pp. 811–822, 2010.
model-based reconstruction techniques have been developed
[16] D. Wang, N. Zwart, and J. Pipe, “Joint water–fat separation and deblurring for
to solve a specific problem, integrating a full description of the spiral imaging,” Magn. Reson. Med., vol. 79, no. 6, pp. 3218–3228, June 2018.
measurement process in the reconstruction is still a challenge [17] D. Raj, A. Anderson, and J. Gore, “Respiratory effects in human functional
needs to be addressed in future works. magnetic resonance imaging due to bulk susceptibility changes,” Phys. Med. Biol.,
vol. 46, no. 12, pp. 3331–3340, 2001.
[18] J. Fessler, “Model-based image reconstruction for MRI,” IEEE Signal Process.
Acknowledgments Mag., vol. 27, no. 4, pp. 81–89, 2010.
I would like to thank Holger Eggers, Jürgen Rahmer, Malte [19] M. Lustig and J. M. Pauly, and D. Donoho, “Sparse MRI: The application of
compressed sensing for rapid MR imaging,” Magn. Reson. Med., vol. 58, no. 6,
Steinhoff, and Peter Börnert for their helpful input. pp. 1182–1195, 2007.
[20] S. G. Lingala, Y. Hu, E. DiBella, and M. Jacob, “Accelerated dynamic MRI
Author exploiting sparsity and low-rank structure: k-t SLR,” IEEE Trans. Med. Imag.,
vol. 30, no. 5, pp. 1042–1054, 2011.
Mariya Doneva ([email protected]) received her
[21] M. Doneva, P. Börnert, H. Eggers, C. Stehning, J. Sénégas, and A. Mertins,
B.Sc. and M.Sc. degrees in physics from the University of “Compressed sensing reconstruction for magnetic resonance parameter mapping,”
Oldenburg, Germany, in 2006 and 2007, respectively, and her Magn. Reson. Med., vol. 64, no. 4, pp. 1114–1120, 2010.
Ph.D. degree in physics from the University of Lübeck, [22] S. Poddar and M. Jacob, “Dynamic MRI using SmooThness Regularization on
Manifolds (SToRM),” IEEE Trans. Med. Imag., vol. 35, no. 4, pp. 1106–1115, 2016.
Germany, in 2010. She is a senior scientist at Philips Research. [23] U. Nakarmi, Y. Wang, J. Lyu, D. Liang, and L. Ying, “A kernel-based low-
Her research interests include methods for efficient data acqui- rank (KLR) model for low-dimensional manifold recovery in highly accelerated
dynamic MRI,” IEEE Trans. Med. Imag., vol. 36, no. 11, pp. 2297– 2307, 2017.
sition, image reconstruction, and quantitative parameter map-
[24] T. Zhang, J. Pauly, and I. Levesque, “Accelerating parameter mapping with a
ping in the context of magnetic resonance imaging. She was a locally low rank constraint,” Magn. Reson. Med., vol. 73, no. 2, pp. 655–661, 2015.
research associate in the Department of Electrical Engineering [25] K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock,
and Computer Sciences at the University of California, and F. Knoll, “Learning a variational network for reconstruction of accelerated MRI
data,” Magn. Reson. Med., vol. 79, no. 6, pp. 3055–3071, 2017.
Berkeley, from 2015 to 2016. She is a recipient of the 2011
[26] B. Zhu, J. Liu, S. Cauley, B. Rosen, and M. Rosen, “Image reconstruction by
Junior Fellow Award of the International Society for Magnetic domain-transform manifold learning,” Nature, vol. 555, no. 7697, pp. 487–492,
Resonance in Medicine. 2018.
[27] Z.-P. Liang and P. Lauterbur, Principles of Magnetic Resonance Imaging: A
Signal Processing Perspective. Hoboken, NJ: Wiley-IEEE Press, 1999.
References [28] J. Fessler, “On NUFFT-based gridding for non-Cartesian MRI,” J. Magn.
[1] P. Lauterbur, “Image formation by induced local interactions: Examples employ- Reson., vol. 188, no. 2, pp. 191–195, 2007.
ing nuclear magnetic resonance,” Nature, vol. 242, no. 5394, pp. 190–191, 1973.
[29] S. Vannesjo, B. Wilm, Y. Duerst, S. Gross, D. Brunner, B. Dietrich,
[2] W. Edelstein, J. Hutchinson, G. Johnson, and T. Redpath, “Spin warp NMR T. Schmid, C. Barmet, et al., “Retrospective correction of physiological field fluctu-
imaging and applications to human whole body imaging,” Phys. Med. Biol., vol. 25, ations in high-field brain MRI using concurrent field monitoring,” Magn. Reson.
no. 4, pp. 751–756, 1980. Med., vol. 73, no. 5, pp. 1833–1843, 2015.
[3] S. Ljunggren, “A simple graphical representation of Fourier-based imaging [30] J. Meineke and T. Nielsen, “Data consistency-driven determination of
methods,” J. Magn. Reson., vol. 54, no. 2, pp. 338–343, 1983. B0-fluctuations in gradient-echo MRI,” Magn. Reson. Med., vol. 81, no. 5,
[4] D. Twieg, “The k-trajectory formulation of the NMR imaging process with appli- pp. 3046–3055, 2018.
cations in analysis and synthesis of imaging methods,” Med. Phys., vol. 10, no. 5, [31] J. Tamir, M. Uecker, W. Chen, P. Lai, M. Alley, S. Vasanawala, and M. Lustig,
pp. 610–621, 1983. “T2 shuffling: Sharp, multicontrast, volumetric fast spin echo imaging,” Magn.
[5] H. Eggers, T. Knopp, and D. Potts, “Field inhomogeneity correction based on Reson. Med., vol. 77, no. 1, pp. 180–195, 2017.
gridding reconstruction for magnetic resonance imaging,” IEEE Trans. Med. Imag., [32] S. Ravishankar and Y. Bressler, “MR image reconstruction from highly unders-
vol. 26, no. 3, pp. 374–384, 2007. ampled k-space data by dictionary learning,” IEEE Trans. Med. Imag., vol. 30,
[6] B. Sutton, D. Noll, and J. Fessler, “Fast, iterative image reconstruction for MRI no. 5, pp. 1028–1041, 2011.
in the presence of field inhomogeneities,” IEEE Trans. Med. Imag., vol. 22, no. 2, 
pp. 178–188, 2003. SP

32 licensed use limited to: The University of Toronto. Downloaded

Authorized IEEE SIGNAL PROCESSING MAGAZINE
on December | January
14,2023 2020 | UTC from IEEE Xplore. Restrictions apply.
at 22:06:27