Scribd
Upload
1K views

Problem 1. Multilinear Algebra

CP decomposition is a special case of Tucker decomposition where the core tensor C is superdiagonal, with non-zero elements only along the main diagonal. In Tucker decomposition, C can have non-zero values off the main diagonal, allowing it to capture higher-order interactions between factors. When C is restricted to be superdiagonal, it reduces to CP decomposition.

Uploaded by

gau
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
1K views

Problem 1. Multilinear Algebra

CP decomposition is a special case of Tucker decomposition where the core tensor C is superdiagonal, with non-zero elements only along the main diagonal. In Tucker decomposition, C can have non-zero values off the main diagonal, allowing it to capture higher-order interactions between factors. When C is restricted to be superdiagonal, it reduces to CP decomposition.

Uploaded by

gau
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

ISYE 8803: High-Dimensional Data Analytics

Homework 3

Problem 1. Multilinear algebra

1 2 5 6
Suppose 𝐴 = [ ], 𝐵 = [ ]. Please Calculate
3 4 7 8

a) 𝐴⊗𝐵
b) 𝐴⊙𝐵
c) (𝐴 ⊗ 𝐵)−1
d) (𝐴−1 ⊗ 𝐵−1 )
e) 𝐴 ∗ 𝐵

Solution:

5 6 10 12
a) 𝐴 ⊗ 𝐵 = [ 7 8 14 16]
15 18 20 24
21 24 28 32
5 12
b) 𝐴 ⊙ 𝐵 = [ 7 16]
15 24
21 32
8 −6 −4 3
−1
c) (𝐴 ⊗ 𝐵) = [ −7 5 3.5 −2.5 ]
−6 4.5 2 −1.5
5.25 −3.75 −1.75 1.25
8 −6 −4 3
−1 −1
d) (𝐴 ⊗ 𝐵 ) = [ −7 5 3.5 −2.5 ]
−6 4.5 2 −1.5
5.25 −3.75 −1.75 1.25
5 12
e) 𝐴 ∗ 𝐵 = [ ]
21 32

Problem 2. CP and Tucker decomposition

a. To minimize the least square problem


p
. {𝐶, 𝑽1 , … , 𝑽𝑑 } = argmin‖𝑌 − ∑i=1 X i B‖

𝐵𝑗 = 𝐶𝑗 ×1 𝑈𝑗1 ×2 𝑈𝑗2 ×3 … ×𝑙𝑗 𝑈𝑗𝑙𝑗 ×𝑙𝑗+1 𝑉1 ×𝑙𝑗+2 … ×𝑙𝑗+𝑑 𝑉𝑑

𝑉𝑖𝑇 𝑉𝑖 = 𝐼𝑄̃𝑖

𝑃̃1 ×…×𝑃̃𝑙𝑗 ×𝑙𝑗+1 𝑄̃1 ×𝑙𝑗+2 …×𝑙𝑗+𝑑 𝑄̃𝑑


Where 𝑌 ∈ 𝑅 𝑄1 ×…×𝑄𝑑 and X i ∈ RPi1 ×…×𝑃𝑖𝑙𝑖 𝐶𝑗 ∈ 𝑅 is a core tensor with 𝑃̃𝑗𝑖 ≪

𝑃𝑗𝑖 and 𝑄̃𝑖 ≪ 𝑄𝑖 ; {𝑈𝑗𝑖 : 𝑗 = 1, … , 𝑝; 𝑖 = 1, … , 𝑙𝑗 } is a set of bases that spans the 𝑗 𝑡ℎ input space; and

{𝑉𝑖 : 𝑖 = 1, … , 𝑑} is a set of bases that spans the output space.


Prove the following theorem:

When 𝑈𝑗𝑖 , 𝑉𝑖 and 𝑅𝑗 are known, a reshape form of the core tensor 𝐶𝑗 can be estimated as
−1
𝐶̃𝑗 = 𝑅𝑗 ×1 (𝒁𝑗𝑇 𝒁𝑗 ) 𝒁𝑗𝑇 ×2 𝑽1𝑇 ×3 𝑽𝑇2 … ×𝑑+1 𝑽𝑇𝑑
𝑝
Where 𝑍𝑗 = 𝑋𝑗(1) (𝑼𝑗𝑙 ⊗ 𝑼𝑗𝑙−1 ⊗ … ⊗ 𝑼𝑗1 ) and 𝑅𝑗 = 𝑌 − ∑𝑖≠𝑗 𝐵𝑗 ∗ 𝑋𝑗 . Note that 𝐶̃𝑗 has fewer modes
(𝑑 + 1) than the original core tensor in (4), but it can be transformed into 𝐶 by a simple reshape operation.

This can be done by the following steps:


2 2
1) Prove argminC ‖𝑅𝑗(1) − 𝑋𝑗(1) 𝐵𝑗 ‖ = argminC ‖𝑣𝑒𝑐(𝑅𝑗(1) ) − (𝑉𝑑 ⊗ 𝑉𝑑−1 … ⊗ 𝑉1 ⊗ 𝑍𝑗 )𝑣𝑒𝑐(𝐶𝑗 )‖
𝐹 𝐹
where 𝑣𝑒𝑐(𝑋) stacks the columns of matrix 𝑋 on top of each other. 𝐻𝑖𝑛𝑡: (𝑣𝑒𝑐(𝐴𝐵𝐶 𝑇 ) =
(𝐶 ⊗ 𝐴)𝑣𝑒𝑐(𝐵))
Solution:
2 2
argminC ‖𝑅𝑗(1) − 𝑋𝑗(1) 𝐵𝑗 ‖ = ‖𝑅𝑗(1) − 𝑋𝑗(1) (𝑈𝑗𝑙 ⊗ 𝑈𝑗𝑙−1 ⊗ … ⊗ 𝑈𝑗1 )𝐶𝑗 (𝑉𝑑 ⊗ 𝑉𝑑−1 … ⊗ 𝑉1 )𝑇 ‖
𝐹 𝐹
2
= ‖𝑣𝑒𝑐(𝑅𝑗(1) ) − 𝑣𝑒𝑐(𝑋𝑗(1) (𝑈𝑗𝑙 ⊗ 𝑈𝑗𝑙−1 ⊗ … ⊗ 𝑈𝑗1 )𝐶𝑗 (𝑉𝑑 ⊗ 𝑉𝑑−1 … ⊗ 𝑉1 )𝑇 )‖
𝐹
2
= ‖𝑣𝑒𝑐(𝑅𝑗(1) ) − 𝑣𝑒𝑐(𝑍𝑗 𝐶𝑗 (𝑉𝑑 ⊗ 𝑉𝑑−1 … ⊗ 𝑉1 )‖ )𝑇
𝐹
2
= ‖𝑣𝑒𝑐(𝑅𝑗(1) ) − (𝑉𝑑 ⊗ 𝑉𝑑−1 … ⊗ 𝑉1 ⊗ 𝑍𝑗 )𝑣𝑒𝑐(𝐶𝑗 )‖
𝐹

2
2) Prove argminC ‖𝑣𝑒𝑐(𝑅𝑗(1) ) − (𝑉𝑑 ⊗ 𝑉𝑑−1 … ⊗ 𝑉1 ⊗ 𝑍𝑗 )𝑣𝑒𝑐(𝐶𝑗 )‖ has optimal solution:
𝐹

−1
𝑣𝑒𝑐(𝐶𝑗 ) = (𝑉𝑑𝑇 ⨂𝑉𝑑−1
𝑇
… . ⨂𝑉1𝑇 ⨂(𝑍𝑗𝑇 𝑍𝑗 ) 𝑍𝑗𝑇 ) 𝑣𝑒𝑐(𝑅𝑗(1) )

Solution:
−1
𝑇 𝑇
𝑣𝑒𝑐(𝐶𝑗 ) = ((𝑉𝑑 ⊗ 𝑉𝑑−1 … ⊗ 𝑉1 ⊗ 𝑍𝑗 ) (𝑉𝑑 ⊗ 𝑉𝑑−1 … ⊗ 𝑉1 ⊗ 𝑍𝑗 )) (𝑉𝑑 ⊗ 𝑉𝑑−1 … ⊗ 𝑉1 ⊗ 𝑍𝑗 ) 𝑣𝑒𝑐(𝑅𝑗(1) )

𝑇 −1 𝑇
= (𝐼𝑑 ⊗ 𝐼𝑑−1 … ⊗ 𝐼1 ⨂ (𝑍𝑗 𝑍𝑗 ) ) (𝑉𝑑 ⊗ 𝑉𝑑−1 … ⊗ 𝑉1 ⊗ 𝑍𝑗 ) 𝑣𝑒𝑐(𝑅𝑗(1) )

−1
= (𝑉𝑑𝑇 ⨂𝑉𝑑−1
𝑇
… . ⨂𝑉1𝑇 ⨂(𝑍𝑗𝑇 𝑍𝑗 ) 𝑍𝑗𝑇 ) 𝑣𝑒𝑐(𝑅𝑗(1) )

b. Show that CP decomposition is the special case of tucker decomposition

You might also like