Scribd
Upload
4 views

FFT and Algebraic Computation

The document discusses the efficient multiplication of polynomials using the Fast Fourier Transform (FFT) algorithm, which improves upon the naive O(n^2) method by utilizing point-value representation. It explains the process of converting polynomials into point-value form, performing point-wise multiplication, and then using inverse FFT to retrieve the coefficient form of the product polynomial. Key concepts such as the Convolution Theorem, roots of unity, and the Discrete Fourier Transform are also covered to illustrate the efficiency of the FFT method.

Uploaded by

Vedant Kapoor
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
4 views

FFT and Algebraic Computation

The document discusses the efficient multiplication of polynomials using the Fast Fourier Transform (FFT) algorithm, which improves upon the naive O(n^2) method by utilizing point-value representation. It explains the process of converting polynomials into point-value form, performing point-wise multiplication, and then using inverse FFT to retrieve the coefficient form of the product polynomial. Key concepts such as the Convolution Theorem, roots of unity, and the Discrete Fourier Transform are also covered to illustrate the efficiency of the FFT method.

Uploaded by

Vedant Kapoor
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 29

Algorithm Analysis and Design

FFT and Algebraic Computation


Overview

• Given two polynomial A(x) and B(x), find the product C(x) =
A(x)*B(x).

• There is already an O(n2) naive approach to solve this problem.


This approach uses the coefficient form of the polynomial to
calculate the product.

• We can do better, if we represent the polynomial in another form


called point-value form, and we use FFT algorithm for the
conversion.
Polynomial Multiplication Using Co-
efficient Form
• A coefficient representation of a polynomial is
a = a0, a1, …, an-1.
• Example:
A(x) = 6x3 + 7x2 - 10x + 9, B(x) = -2x3 + 4x - 5
– `Coefficient representation of A(x) = (9, -10, 7, 6)
– `Coefficient representation of B(x) = (-5, 4, 0, -2)
 Input:
A[] = {9, -10, 7, 6}, B[] = {-5, 4, 0, -2}
 Output:
C(x) = a0*bi + a1*bi-1 + ... + ai*b0
= -12x6 - 14x5 + 44x4 - 20x3 - 75x2 + 86x - 45
Fourier Transform and Convolution
Theorm
• If we think of A and B as vectors, then C vector is called 'Convolution' of A
and B (represented as {A ⊗B}).
• Fourier analysis is the decomposition of any mathematical function into a
series of sine and cosine waves (sinusoids). We generally represent a
function in its Fourier Transform form, as it is easier to work with.
• The Convolution Theorm states that under suitable conditions the Fourier
transform of a convolution (denoted by ‘ ⊗’) is the point-wise product of
Fourier transforms.
– Let f and g be two functions with convolution {f ⊗ g}.
– Fourier transform be denoted by the operator ‘ζ’. So ζ(f) and ζ(g) are
the Fourier transforms of f and g, respectively.
– Therefore, ζ(f ⊗g) = ζ(f).ζ(g)
– The convolution operation is converted to point-wise multiplication (we
will discuss what point-wise multiplication means).
Point Value Representation
of Polynomials
• A polynomial of degree n-1 can be uniquely represe
nted by its values at at-least n different points.
– This is a result of the Fundamental Theorm of Algebra
that states: “A degree n-1 polynomial A(x) is uniquely
specified by its evaluation at n distinct values of x”.

• A polynomial A(x) =a0+a1x+a2x2+… anxn can be repr


esented as:
A set of n pairs {(x0, y0),(x1, y1),...,(xn, yn)} such
that:
 for all i != j, xi != xj
i.e., the points are unique
 for every k, yk = A(xk)
• In point-value form, multiplication C(x) = A(x)B(x) is given
by C(xk) = A(xk).B(xk) for any point (xk).

• If A and B are of degree-bound 'n', then C is of degree-


bound '2n'.
– Need to start with “extended” point-value forms for A and B
consisting of 2n point-value pairs each:
 A: (x0,y0), (x1,y1), ......, (x2n-1,y2n-1)
 B: (x0,y0'), (x1,y1'), ......, (x2n-1,y2n-1')
 C: (x0,y0y0'), (x1,y1y1'), ......, (x2n-1,y2n-1y2n-1')
• Example:
– A(x) = x3 − 2x + 1
– B(x) = x3 + x2 + 1
– xk = (−3, −2, −1, 0, 1, 2, 3), we need 7 coefficients!
– A: (−3,−20) ,( −2,−3) ,( −1,1) ,( 0, 1) ,( 1, 0) , (2, 5) , (3, 22)
– B: (−3,−17) ,(−2,−3) , (−1, 2) ,( 0, 1) ,( 1, 3) ,( 2, 13) ,( 3, 37)
– C: (−3,340) , (−2,9) , (−1,2) ,( 0, 1) ,( 1, 0) ,( 2, 65) ,( 3, 814)

• This is point-wise multiplication (Time Complexity: Θ(n)).


• The inverse of evaluation--determining the coefficient form of a polynomial
from a point-value representation is called interpolation.
• This is what the Convolution Theorm states with respect to Fourier
Transform of a function.
• Following diagram summarizes what we have discussed so
far:
• Now the efficiency of our algorithm depends on the efficiency
of conversion between the two representation:

• This is where we use the FFT algorithm.


• An Overview of the Algorithm:
Let m = 2n-1. [so degree of C is less than m]
1. Pick m points x_0, x_1, ..., x_{m-1} chosen cleverly.
2. Evaluate A at each of the points: A(x_0),..., A(x_{m-1}).
3. Same for B.
4. Now compute C(x_0),..., C(x_{m-1}), where C is A(x)*B(x)
5. Interpolate to get the coefficients of C.

• At a glance, it looks like points 2 and 3 from the Algorithm takes O(n2) time.
– However, the FFT will allow us to quickly move from coefficient
representation of polynomial to the point-value representation, and back,
for our cleverly chosen set of m points.
– Doesn’t work for arbitrary m points. The special points will turn out to be
the roots of unity.
Roots of Unity
• The roots of unity of order n are those numbers whic
h, when raised to the n th power, we get 1 ("unity").
They're also called "n th roots of unity".
– There are exactly n such numbers, one of which is alw
ays the number 1 itself.
– When n=2, they are 1 and −1 .

• To get any root of unity, other than 1, we need to de


lve into Complex Numbers.
• Visualising eight 8th roots of unity:
• Some Interesting Properties of Roots of Unity:

1. Principal nth Root of Unity


2. Cancellation Lemma
3. Halving Lemma
• As we will see, the fast Fourier transform algorithm cleverly
makes use of the following properties about ωn:
 ωn^n =1
 ωn^n+k = ωn^k
 ωn^n/2 = -1
 ωn^(n/2+k) = -ωn^k
Discrete Fourier Transform (DFT)
• Since we have broken up the problem, our goal now i
s to evaluate a given polynomial, A (x)(degree <m) a
t m points of our choosing in total time O(m log m).

 Let's asume m is a power of 2.


 Let’s first develop it through an example.
 Say m=8, so we have a polynomial:
A(x) = a_0 + a_1 x + a_2 x² + a_3 x³ + a_4 x⁴ + a_5
x⁵ + a_6x⁶ + a_7x⁷
(as a vector, A = [a_0, a_1, …, a_7])
• And we want to evaluate at eight points of our choosing. Here is an
idea. Split A into two pieces, but instead of left and right, have them
be even and odd.
 So, as vectors,
– A_even = [a_0, a_2, a_4, a_6]
– A_odd = [a_1, a_3, a_5, a_7]
 or, as polynomials:
– A_even(x) = a_0 + a_2 x + a_4 x² + a_6 x³
– A_odd(x) = a_1 + a_3 x + a_5 x² + a_7 x³.
• Each has degree < m/2. How can we write A(x) in terms of
A_even and A_odd?
– A(x) = A_even(x²) + x A_odd(x²).
– A(-x) = A_even(x²) — x A_odd(x²).
• Now, instead of calculating the value for 1 polynomial of degree m,
we now need to calculate 2 polynomials of degree (m/2) and then co
mbine them.

• Let T(m) = time to evaluate a degree-m polynomial at our special


set of m points. We’re doing this by evaluating two degree-m/2
polynomials at m/2 points each (the squares), and then doing O(m)
work to combine the results.
• The recurrence T(m) = 2T(m/2) + O(m) solves to O(m log m).

• What’s nice is that the effort spent computing A(x) will give us A(-x)
almost for free. So, we need to specially choose m points that will
have the property: the 2nd half of points are the negative of the 1st
half.
• We use complex numbers for this (roots of unity).
Fast Fourier Transform (FFT)
• The problem of evaluating A(x) at ωn^0 , ωn^1 , … , ωn^n−1 r
educes to
1. evaluating the degree-bound n/2 polynomials Aeven(x) and
Aodd(x) at the points (ωn^0)^2 ,(ωn^1)^2 , … , (ωn^n−1)
^2.
2. 2. combining the results by A(x) = Aeven(x2) + xAodd(x2).

• Why bother?
– The list (ωn^0)^2 ,(ωn^1)^2 , … , (ωn^n−1)^2 does not c
ontain n distinct values, but n/2 complex n/2-th roots of unit
y.
– Polynomials Aeven and Aodd are recursively evaluated at th
e n/2 complex n/2-th roots of unity.
– Subproblems have exactly the same form as the original pro
blem, but are half the size.
• Example:
• Algorithm:
• To apply this algorithm, ‘m’ should be a power of 2.We should pad extra
zeros to the polynomial to ensure that length of the array is a power of 2. This
algorithm returns an array of complex numbers which is then used for point-
wise multiplication.

• Below is the tree of input vectors to the recursive calls of the FFT procedure.
The initial invocation is for n = 8.
• Inverse Fourier Transform:
• Once we perform point-wise multiplication on the Fourier
Transforms of A and B and get C (an array of Complex
Numbers), we need to convert it back to coefficient form to get
the final answer.

• We use the Inverse Fourier Transform to get the coefficient


form of C (Don’t worry about the imaginary part of the complex
number.

• Once the Inverse Fourier Transform is performed, we get an


array of Complex Numbers with the imaginary part almost equal
to 0. Hence, it can be ignored.
• So, we have seen so far, computing A(x) at 1, ω,ω2, …,ω{m-
1} can be represented as:

• In matrix form:
• Let's use F to denote the matrix of primitive roots.

• In order to retrieve the coefficients, we need to get


F^-1(inverse of matrix F), and multiply it with the y-matrix.

• It turns out that F^-1 looks the same. In place of ω, the inverse
Fourier Matrix uses ω^-1.
– if ω = a+ib => ω^-1 = a-ib
– if ω = e^2Πi/k => ω^-1 = e^-2Πi/k
• So, F^-1:

• Based on the above observation, we can still apply FFT by


replacing a with y, y with a, ωn with ωn^−1 (that is, ωn^n-1 ),
and scaling the result by 1/n.
• So, the final algorithm is:

Let F_A = FFT(A, m, ω) // time O(n log n)


Let F_B = FFT(B, m, ω) // time O(n log n)
For i=1 to m, let F_C[i] = F_A[i]*F_B[i] // time O(n)
Output C = 1/m * FFT(F_C, m, ω-1). // time O(n log n)

You might also like