2020, Vanherck, Tanh-Sinh Quadrature For Single and Multiple Integration Using Floating-Point Arithmetic
2020, Vanherck, Tanh-Sinh Quadrature For Single and Multiple Integration Using Floating-Point Arithmetic
Belgium
b IMEC, Kapeldreef 75, B-3001 Leuven, Belgium
c ESAT, KU Leuven, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium
Abstract
The problem of estimating single- and multi-dimensional integrals, with or
without end-point singularities, is prevalent in all fields of scientific research, and
in particular in physics. Although tanh-sinh quadrature is known to handle most
of these cases excellently, its use is not widely spread among physicists. Moreover,
while most calculations are limited by the use of finite-precision floating-point
arithmetic, similar considerations for tanh-sinh quadrature are mostly lacking
in literature, where infinite-precision floating-point numbers are often assumed.
Also little information is available on the application of tanh-sinh quadrature to
multiple integration.
We have investigated the risks and limitations associated with limited-preci-
sion floating-point numbers when using tanh-sinh quadrature for both single and
multiple integration, while obtaining excellent convergence rates. In addition,
this paper provides recommendations for a straightforward implementation using
limited-precision floating-point numbers and for avoiding numerical instabilities.
Keywords: double-exponential (DE) quadrature, tanh-sinh quadrature, Sinc
method, numerical integration, multiple integration, end-point singularity
1. Introduction
∗ Corresponding author
Email addresses: [email protected] (Joren Vanherck),
[email protected] (Bart Sorée), [email protected] (Wim Magnus)
2. Tanh-sinh quadrature
For integration over the entire real axis of exponentially decaying integrands as
|t| → ∞, the trapezoidal rule (also known as Sinc quadrature [2]) is the most
efficient [8–12] among quadratures with equidistant abscissae. Applying this rule
with a step size h between the evaluation points yields the approximate integral
+m
X
Ih = lim hΨ0 (ti )f Ψ(ti ) , (3)
m→∞
i=−m
1Any integral ab f (y) dy with a and b finite can be cast in this form by the linear substitution
R
2y = (b − a) x + (b + a).
2
thereby introducing a truncation error t , since the transformed integration
domain is now limited to the window [t−n , tn ]. The total error of the resulting
approximation as compared to the original integral is limited by I − Qnh 6
∆Qnh := |∆Ih | + |t |.
As to the mapping Ψ(t), Takahasi and Mori [1] proposed a tanh-sinh trans-
formation
Ψ(t) = tanh λ sinh(t) (6a)
λ cosh(t)
Ψ0 (t) = , (6b)
cosh2 λ sinh(t)
where λ = π/2. With this choice, which is illustrated in Fig. 1, the behaviour
of g(t) as |t| → ∞ is optimal in the sense that a faster decay leads to a higher
discretization error for a given
h, while a slower decay causes a larger truncation
error because Ψ0 (t)f Ψ(t) remains significant for large values of |t|. An extended
error analysis in the complex plane indeed shows that, for a large number of
evaluation points, the above transformation often yields an optimal error [1, 10]
as compared to other quadratures based on a variable transformation. This error
is of the order [1, 10–12]
πdN
|∆Qnh | = O exp − , with N = 2n + 1 (7)
ln(2dN )
when h is chosen optimally. The parameter d has to be chosen such that the
analytical continuation of the integrand g(z) = Ψ0 (z)f Ψ(z) in the complex
plane is regular in the strip around the real axis defined by |Im z| < d, meaning
that the integrand lies within the Hardy space.[10, 13] The choice of d and h
will be detailed in Sec. 3.3, the latter being of particular importance in avoiding
numerical instabilities.
The tanh-sinh quadrature scheme has several advantages worth pointing
out explicitly. (i) The error estimate (7) shows that doubling the number of
evaluation points also roughly doubles the number of significant digits, which
makes the quadrature especially suited for high precision calculations.[5, 6] (ii)
Even when not all conditions for the error estimate (7) are strictly fulfilled by
the integrand, convergence is still rather fast. (iii) For integrands with end-
point singularities tanh-sinh quadrature converges especially fast as compared to
other schemes, thanks to its double-exponential suppression of these divergences
(Fig. 1). (iv) The abscissae and weights can be directly extracted from the
transformation formulas (6a) and (6b), in contrast to some other quadrature
schemes leaning on iterative processes. (v) Finally, the transformed integral has
equidistant abscissae (trapezoidal rule, Fig. 1), which can straightforwardly be
reused together with the weights and the function evaluations when doubling
the quadrature order is required.
In the present paper we focus on the quadrature scheme as presented above
and its generalization to multiple integration, although several extensions are
possible. In general, the transformation function Ψ(t) should be chosen such
3
q
f (x) = 2
x+1
g(t) = Ψ0 (t)f (Ψ(t))
20
2
transform
10
1
0 x 0
−1 0 1 t−2 t−1 t0 t1 t2 t
Ψ(t) Ψ0 (t)
x2
x1
1 2
x0
1.5
x−
0 1
1
x−
2
0.5
−1
0
−2 0 2 t
Figure 1: Tanh-sinh transformation applied to the integral 01 √1x dx, brought to the standard
R
domain ]−1, 1[ (top left). The tanh-sinh transformation transforms the integration domain
from x ∈ ]−1, 1[ to t ∈ ]−∞, ∞[, while the integrands magnitude is strongly suppressed as
t → ∞ (e.g., g(t)|t=−2.5 ≈ 10−7 and g(t)|t=−4.5 ≈ 10−59 ). The trapezoidal rule is then
applied to this transformed integrand within the window t ∈ [−tn , tn ], for n = 2 (5 evaluation
points) as an illustration (top right). The abscissa xi of the original integration domain to
which the equidistant points ti in the transformed domain correspond, can be recovered as
xi = Ψ(ti ) (bottom right and dotted connection lines). Notice that most of the abscissae xi
end up very close to the domain boundaries. At those points, the effect of the original integrand
is strongly suppressed because it is multiplied by Ψ0 (t) (bottom right, grey). This results in
the excellent convergence properties of the tanh-sinh quadrature, especially for integrands with
singularities near the end-points.
4
number of evaluation points for t > 0 and t < 0 when truncating the series of the
trapezoid rule (3). The details, including error estimates with explicit constants,
have been worked out by Okayama et al. [2] and more information on asymmetric
ranges can be found in Chapter 2 of the book on Sinc methods by Lund and
Bowers.[14] Finally, specific modifications for handling oscillatory [15, 16] or
indefinite integrals are discussed in refs. [17, 18].
3. Discussion
The above mentioned favorable properties of tanh-sinh quadrature can only
be achieved if numerical instabilities are avoided, especially when dealing with
finite-precision floating-point numbers. Related caveats will be treated and
resolved in the following subsection, together with the extension of tanh-sinh
quadrature to multi-dimensional integration domains.
containing the distances from the respective abscissae xi to the closest interval
bound.
In the context of tanh-sinh quadrature, the most important cause of numeri-
cal instability is numerical underflow, occurring when numerical values become
smaller then the underflow level (UFL). This smallest positive normalized
floating-point number is Fmin = 2L , where L is the smallest exponent repre-
sentable in a given floating-point model. Specifically, we need to pay attention
to the weights, the abscissae and the function evaluations at these abscissae.
Both the smallest weight wn = min{wi } and the smallest stored abscissa value
yn = min{yi } are determined by window size tn , which bounds the transformed
integration window ti ∈ [−tn , tn ]. We will determine a window limit tmax and
choose tn 6 tmax to avoid numerical underflow in a given floating-point model.
For the weights, numerical underflow is avoided if wn = Ψ0 (tn ) > Fmin , such
that the corresponding window limit is
0
tw
max = max{t | Ψ (t) > Fmin } (11)
5
Similarly, the smallest stored abscissa should exceed the UFL, yn = 1 − xn =
1 − Ψ(tn ) > Fmin , such that we can define
txmax = max t t 6 Ψ−1 (1 − Fmin ) (12)
Table 1: This table shows the smallest representable exponent L, the corresponding underflow
level Fmin , the window limits tx w xw
max , tmax and tmax , and the corresponding intrinsic maximal
optimal order nxwmax for various floating-point (fp) models (IEEE 754-2008).
Besides the calculation of the abscissae and weights, also the integrand
evaluations need to be carefully inspected in view of the numerical stability,
which can be accomplished by a proper choice of another window limit tevalmax .
Examples will be discussed in section Sec. 4.
Finally, we take tmax = min{txw eval
max , tmax } as the window limit, imposing
the maximum on the transformed abscissae values ti as to avoid numerical
instabilities.
6
As an example in two dimensions, consider the integration of a function
f (x, y) over the domain −1 6 x, y 6 1, which is typically approximated as
Z 1 Z 1 +n1
X +n2
X
n ,n
f (x, y) dy dx ≈ Qh11,h22 = h1 h2 wi wj f (xi , xj ), (14)
−1 −1 i=−n1 j=−n2
since then wnD > Fmin . This ensures that each weight separately exceeds the
UFL. For higher dimensions, the product of all but one weight should also be
higher then the UFL in order that the corresponding term contribute to the sum.
In the latter case, one weight can be disregarded, because it can be compensated
by a large value from the function evaluation. Examples for one, two and three
dimensions are given in Table 1.
7
calculated. In practice, d = π/2 often leads to reasonable convergence rates.
The optimal transformed abscissae spacing then becomes
2
hopt (n) = W(πN ), with N = 2n + 1. (17)
N
While hopt (n) is theoretically optimal in an infinite-precision context, its
outstanding results can not always be achieved due to the limitations of floating
point types. The main problem is the window size tn = nhopt (n) that can
quickly reach beyond the window limit tmax , sometimes before full convergence
is reached. In order to avoid numerical instabilities, the optimal spacing should
never be used for orders higher than nmax = max{n | nhopt (n) 6 tmax } at which
this happens. This maximal order can depend on numerical difficulties in the
integrand evaluation through tmax and must therefore be determined on a per-
case basis. The maximal order itself is always limited by the intrinsic maximal
order nxw
max which is fully determined by the floating-point type and dimension
(Table 1).
As an alternative to the optimal spacing, we propose to use the maximal
spacing
t
h(n) = hmax (n) := max (18)
n
instead. Here, the window size tn is fixed, independent of the order n, to tmax so
as to avoid numerical underflow. Besides the obvious simplicity of the hmax (n),
also the reusability of function evaluations obtained from lower order estimates,
is an advantage when it comes to doubling the order index n.
4. Worked examples
In this section, we demonstrate the use of the tanh-sinh quadrature as
described above, specifically comparing results using optimal and maximal
spacing for different floating-point types. For each example integral, possible
pitfalls and noteworthy features are discussed, while showing the relative error
as a function of the quadrature order. This relative error is intrinsically limited
from below by the machine precision m , which is indicated by dashed lines for
each floating-point models. The results for optimal spacing can only be obtained
up to the maximum quadrature order nmax , which is shown as a larger data point
if it falls within the shown domain. Results obtained using double-precision
Gauss-Legendre quadrature with the same number of evaluation points are
always shown for comparison. We focus on integrands with a singularity near the
origin of the original integration domain, as may be relevant to solve numerous
problems in physics. For benchmarking purposes, it is not the intent to evaluate
the specific integral, but rather to demonstrate how the quadrature behaves for
different types of limiting behavior.
8
type is the integral Z 1
1
Iδ = dx = − ln δ, (19)
δ x
where δ is an arbitrarily small positive number. In spite of ln δ quickly diverging
when δ tends to zero, tanh-sinh quadrature quickly yields remarkably high
precision integrals when the integrand is sampled with many points near x = δ
(Fig. 2). This can, however, also be a source of errors: if the sampling points
are closer to x = δ than machine precision permits, the function evaluations will
be inaccurate. To prevent this, teval
max should be chosen such that the abscissae
x = x−n remains distinguishable from x = δ. For the errors shown in Fig. 2, we
chose teval
max such that (x−n − δ) /δ > a · m , with a = 100. Increasing the value
of a diminishes the fluctuations of the resulting error, but increases the achieved
minimal relative error.
Notice in Fig. 2 that nmax is very small and the corresponding error using
optimal spacing is still far above the achievable error. The use of the proposed
maximal spacing does allow for a steady and fast decrease of the relative error
up to full precision. Gauss-Legendre quadrature fails completely in this test. It
lacks the fine-grained sampling around x = δ.
Z 1
10−2 1
Iδ = dx with δ = 10−10
δ x
|(Iδ − Qn )/Iδ |
10−8 single
hmax nmax GL
hopt m
10−14
double
10−20 extended
Figure 2: The relative error decreases exponentially with quadrature order n. Both optimal
and maximal spacing yield similar convergence rates. However, whereas the maximal spacing
reaches full precision for all floating-point types, the optimal spacing can barely converge
because nmax is very small. The fluctuations as a function of n decrease upon increasing a.
No convergence can be distinguished when using Gauss-Legendre quadrature.
9
where the integrands are chosen to be
1
f1 (x) = √
x
1
f2 (x, y) = p
x2 + y 2
1
f3 (x, y, z) = 2 ,
x + y2 + z2
all containing a integrable singularity at r = 0. Integrals of the type ID are
typically encountered in solid state physics, where integration over the first
Brillouin zone is very common. For example, in a recent study of anisotropic
quantum Heisenberg ferromagnets [4], one of the integrands that determines the
Curie temperature falls of in the same way as f3 (r). The results2 for each of
these integrals are shown in Fig. 3. The convergence rate is especially large for
the one-dimensional integral, but also for higher dimensions the convergence is
fast and steady up to maximum precision. While no special numerical concerns
apply to f1 , numerical underflow in the calculation of the denominator
√ of f2 and
f3 needs to be avoided by choosing teval such that x−n > Fmin .
In the latter two cases, the maximal order nmax was reached in the single
precision optimal scheme before convergence up to full precision was achieved.
In the double-precision equivalent, full precision is reached at an order just below
nmax . All convergence rates seem to become lower when the dimensionality
of the integral is increased. This might be due to the fact that the choice of
d = π/2 in Eq. (16) is not optimal for the outer integrals. The scheme using
maximal spacing always reaches full precision. Even though its convergence
rate is somewhat slower than the optimal (especially for the extended precision
calculation), it still has a decent performance. In every case, Gauss-Legendre
quadrature converges, be it at a much lower rate than the tanh-sinh quadratures.
5. Conclusion
2 Elementary
√
integration of the function class
√
fD (r) leads to I1 = 2, I2 = 2 ln(1 + 2)
√
and I3 = 3 (Ti2 (3 − 2 2) − C) + 3π4
tanh−1 ( 2 3 2 ), where Ti2 (x) and C are respectively the
inverse tangent integral and the Catalan constant.
10
Z Z Z
1 1 1 3
I1 = √ dr I2 = √ d2 r I3 = d r
]0,1]1 r ]0,1]2 r2 ]0,1]3 r2
10−2
|(ID − Qn )/ID |
10−8 single
hmax m
hopt GL
nmax
10−14
double
10−20 extended
Figure 3: The relative error as compared to the exact result for one-dimensional quadrature
order n for functions with an integrable singularity at the origin in one, two and three dimensions.
The total number of sampling points in D dimensions is (2n + 1)D . The quadrature with
optimal spacing always converges slightly faster than the maximal spacing variant. However,
it does not reach full precision for the single-precision floating-point type in the two- and
three-dimensional examples, whereas the quadrature with hmax (n) does. While still being fast,
the convergence is more slow in higher dimensions. Gauss-Legendre quadrature converges, but
only very slowly.
straightaway by choosing a suitable window limit tmax . For the optimal spacing
hopt (n), this implies a maximum usable quadrature order nmax , sometimes
hampering convergence. The proposed maximal spacing rule hmax (n) alleviates
this limitation. Despite having slightly slower convergence rates, this rule achieves
full machine precision accuracy more consistently. Moreover, its implementation
is even more straightforward and it allows for the reuse of abscissae, weights
and function evaluations when it comes to recompute a particular integral for
increasing quadrature orders, as is often convenient to monitor convergence.
References
11
method, Annals of Physics 360 (2015) 520–538. doi:10.1016/j.aop.2015.
05.026.
[4] J. Vanherck, B. Sorée, W. Magnus, Anisotropic bulk and planar heisenberg
ferromagnets in uniform, arbitrarily oriented magnetic fields, Journal of
Physics: Condensed Matter 30 (2018) 275801. doi:10.1088/1361-648x/
aac65f.
[5] D. H. Bailey, K. Jeyabalan, X. S. Li, A comparison of three high-precision
quadrature schemes, Experimental Mathematics 14 (2005) 317–329. doi:10.
1080/10586458.2005.10128931.
[6] D. Bailey, J. Borwein, High-precision numerical integration: Progress
and challenges, Journal of Symbolic Computation 46 (2011) 741–754.
doi:10.1016/j.jsc.2010.08.010.
[7] R. M. Slevinsky, S. Olver, On the use of conformal maps for the
acceleration of convergence of the trapezoidal rule and sinc numerical
methods, SIAM Journal on Scientific Computing 37 (2015) A676–A700.
doi:10.1137/140978363.
R∞ 2
[8] E. T. Goodwin, The evaluation of integrals of the form −∞ f (x)e−x dx,
Mathematical Proceedings of the Cambridge Philosophical Society 45 (1949)
241–245. doi:10.1017/s0305004100024786.
[9] C. Schwartz, Numerical integration of analytic functions, Journal of Com-
putational Physics 4 (1969) 19–29. doi:10.1016/0021-9991(69)90037-0.
[10] M. Sugihara, Optimality of the double exponential formula - functional
analysis approach -, Numerische Mathematik 75 (1997) 379–395. doi:10.
1007/s002110050244.
[11] M. Mori, Discovery of the double exponential transformation and its
developments, Publications of the Research Institute for Mathematical
Sciences 41 (2005) 897–935. doi:10.2977/prims/1145474600.
[12] L. N. Trefethen, J. A. C. Weideman, The exponentially convergent trape-
zoidal rule, SIAM Review 56 (2014) 385–458. doi:10.1137/130932132.
[13] G. H. Hardy, The mean value of the modulus of an analytic function,
Proceedings of the London Mathematical Society s2 14 (1915) 269–277.
doi:10.1112/plms/s2_14.1.269.
[14] J. Lund, K. Bowers, Sinc Methods for Quadrature and Differential Equa-
tions, Other Titles in Applied Mathematics, Society for Industrial and
Applied Mathematics, 1992. URL: https://books.google.be/books?id=
Qfw91lUXUKoC.
[15] T. Ooura, M. Mori, The double exponential formula for oscillatory functions
over the half infinite interval, Journal of Computational and Applied
Mathematics 38 (1991) 353–360. doi:10.1016/0377-0427(91)90181-i.
12
[16] T. Ooura, M. Mori, A robust double exponential formula for fourier-type
integrals, Journal of Computational and Applied Mathematics 112 (1999)
229–241. doi:10.1016/s0377-0427(99)00223-x.
[17] F. Stenger, Numerical methods based on whittaker cardinal, or sinc func-
tions, SIAM Review 23 (1981) 165–224. doi:10.1137/1023037.
13