Quantum information theory (MAT4430) Spring 2021

Lecture 10: Encoding classical information into quantum states

Lecturer: Alexander Müller-Hermes

In this lecture, we will study how classical information can be encoded into quantum
states. Specifically, we will consider a d-dimensional quantum system and define a classical
channel by first preparing it in some quantum state associated to a classical message, and
then measuring the system with some POVM. We will then prove Holevo’s theorem giving
an upper bound for the classical capacity of any such channel. It will turn out that this
capacity is always less or equal to log(d), and we conclude that a d-dimensional quantum
system cannot store more than log(d) classical bits of information reliably.

1 Pinsker’s inequality
The following theorem will be proved in the exercises:
Theorem 1.1 (Pinsker’s inequality). For any quantum states ρ, σ ∈ D(H) we have
1
D (ρkσ) > kρ − σk21 .
2 ln(2)
Pinsker’s inequality can be seen as a refinement of Klein’s inequality (see exercises), and
it shows that the relative entropy is faithful, i.e., D(ρkσ) = 0 holds if and only if ρ = σ.

2 The quantum mutual information

The quantum mutual information generalizes the classical mutual information:
Definition 2.1 (Quantum mutual information). For a quantum state ρAB ∈ D(HA ⊗ HB )
we define
I(A : B)ρAB = H(ρA ) + H(ρB ) − H(ρAB ).
Like its classical analogon, we can express the quantum mutual information in terms of
the relative entropy:
Lemma 2.2. For a quantum state ρAB ∈ D(HA ⊗ HB ) we have

I(A : B)ρAB = D(ρAB kρA ⊗ ρB ).

Proof. Exercise.

Consequences of the previous lemma are the following elementary properties of the quan-
tum mutual information: For any ρAB ∈ D(HA ⊗ HB ) we have
• I(A : B)ρAB > 0 with equality if and only if ρAB = ρA ⊗ ρB .
• I(A : B)(idA ⊗T )(ρAB ) 6 I(A : B)ρAB for any quantum channel T : B(HA ) → B(HC ).
We can reformulate the strong subadditivity inequality in the following way:
Theorem 2.3. For any quantum state ρABC ∈ D (HA ⊗ HB ⊗ HC ) we have

I(A : B)ρAB 6 I(A : (B, C))ρAB .

Proof. Exercises.

1
3 Convex structure of quantum measurements
Let H denote a complex Euclidean space and consider the set of all POVMs with at most
N
N ∈ outcomes, which we denote by
N
X
MN = {µ : {1, . . . , N } → B(H)+ : µ(n) = 1H }.
n=1

We will need a few basic properties of this set:

Lemma 3.1. The set MN is a compact and convex subset of B(H)N .
Proof. Clearly, MN is convex. To show that MN is compact, note that every µ ∈ MN gives
rise to a quantum state
 
µ(1)
 µ(2) 
1 
C

ρµ =
 µ(3)  ∈ D( N ⊗ H).

N

.. 
 . 
µ(N )

C
Since the set D( N ⊗ H) is compact and the restriction to the diagonal blocks is continuous,
we conclude that MN is compact as the image of a compact set under a continuous map.

Lemma 3.2. Let H denote a complex Euclidean space and µ : {1, . . . , N } → B(H)+ a
POVM. If µ ∈ Ext (MN ) is extremal, then we have
|{n ∈ {1, . . . , N } : µ(n) 6= 0}| 6 dim (H)2 .
Proof. Assume that
|{n ∈ {1, . . . , N } : µ(n) 6= 0}| > dim (H)2 .
Then, the operators {µ(1), . . . , µ(N )} are linearly dependent and there exist α1 , . . . , αN ∈ R,
not all of which are zero, such that
N
X
αn µ(n) = 0.
n=1

Next, we set αmax = max{|αn | : n ∈ {1, . . . , N }} and define µ0 , µ1 : {1, . . . , N } → B(H) by

1
µ0 (n) = µ(n) + αn µ(n)
αmax
and
1
µ1 (n) = µ(n) − αn µ(n).
αmax
Note that µ0 (n), µ1 (n) ∈ B(H)+ for every n ∈ {1, . . . , N } and that
N N N
X X 1 X
µ0 (n) = µ1 (n) = 1H ± αn µ(n) = 1H .
αmax
n=1 n=1 n=1

With this we conclude that µ0 , µ1 ∈ MN . Since

1
µ= (µ0 + µ1 ) ,
2
we find that µ is not an extreme point of MN .

2
4 The setting of Holevo’s theorem
To explain the setting of Holevo’s theorem, we consider a set of quantum states

{ρx : x ∈ ΣA } ⊂ D(H),

labeled by some alphabet ΣA . For any alphabet ΣB and any POVM µ : ΣB → B(H)+ , we
may define a classical communication channel Nµ : ΣA → P (ΣB ) by

Nµ (y|x) = hµ(y), ρx iHS .

Note that we use the notation N (y|x) for a classical channel N : ΣA → P (ΣB ) to denote the
probability of y ∈ ΣB with respect to the probability distribution N (x) (as in Lecture 1).
Recall the mutual information of a joint probability distribution pAB ∈ P (ΣA × ΣB ) given
by
I (A : B)pAB = H(pA ) + H(pB ) − H(pAB ),
where pA and pB denote the marginals of pAB . In Lecture 1, we stated Shannon’s channel
coding theorem:

Theorem 4.1 (Shannon’s channel coding theorem). For alphabets ΣA and ΣB let N : ΣA →
P(ΣB ) denote a communication channel. The capacity of N is given by

C(N ) = sup I(A : B)pN ,

AB
pA ∈P(ΣA )

where
pN
AB (x, y) = pA (x)N (y|x),

is a joint probability distribution on ΣA × ΣB defined for any probability distribution pA ∈

P (ΣA ). A rate R is achievable for communication via N if and only if

R < C(N ).

Motivated by this theorem, we define the following quantity:

Definition 4.2 (Accessible information). The accessible information of an ensemble {p(x), ρx }x∈ΣA ,
where p ∈ P(ΣA ) and ρx ∈ D(H) for all x ∈ ΣA is given by

Iacc ({p(x), ρx }) = sup I(A : B)pNµ ,

µ AB

where the supremum goes over all POVMs µ : ΣB → B(H)+ and all alphabets ΣB .

The accessible information quantifies the highes possible mutual information between
the input variable x ∈ ΣA and the measurement outcome y ∈ ΣB for any choice of the
measurement. For any given set of quantum states {ρx : x ∈ ΣA } ⊂ D(H) we conclude
that
sup C(Nµ ) = sup Iacc ({p(x), ρx }) .
µ pA ∈P(ΣA )

This quantity equals the supremum of achievable rates for information transmission through
the preparation-measurement process for an optimal choice of measurement operators. In
the following, we will analyze the accessible information in more detail:

3
5 Accessible information is attained
We will start by showing that the supremum in the definition of the accessible information
is always attained by some measurement with a finite number of measurement outcomes. To
show this, we will start by showing that the optimization in the accessible information Iacc
is over a convex function.
Lemma 5.1. Let {p(x), ρx }x∈ΣA denote an ensemble for p ∈ P(ΣA ) and quantum states
ρx ∈ D(H). For some finite alphabet ΣB , any pair of POVMs µ0 , µ1 : ΣB → B(H)+ , and
any λ ∈ [0, 1] we have

I(A : B)pNµ 6 (1 − λ)I(A : B) Nµ + λI(A : B) Nµ ,

AB pAB0 pAB1

where
µ = (1 − λ)µ0 + λµ1 .
Proof. Recall from the exercises that

I(A : B)pAB = D (pAB kpA × pB ) ,

and that the function N 7→ I(A : B)pN is convex (by joint convexity of the classical relative
AB
entropy). The statement of the lemma is an immediate consequence of this fact.

Now, we can show that the supremum in the definition of Iacc is always attained by an
extremal POVM.
Theorem 5.2. Let {p(x), ρx }x∈ΣA denote an ensemble for p ∈ P(ΣA ) and quantum states
ρx ∈ D(H). There exists an alphabet ΣB with |ΣB | 6 dim(H)2 and a POVM µ : ΣB →
B(H)+ such that
Iacc ({p(x), ρx }) = I(A : B)pNµ .
AB

Proof. Recall the set ML ⊂ B(H)L of POVMs with at most L outcomes. It will be conve-
nient to introduce the quantities

Iacc ({p(x), ρx }, L) = sup I(A : B)pNµ , (1)

µ∈ML AB

for every L ∈ N. Since the value of I(A : B)p Nµ

AB
for any µ : ΣB → B(H)+ only depends on
N
µ
the joint probability distribution pAB and not on any details of the alphabet ΣB , we may
identify any alphabet ΣB with the alphabet {1, . . . , |ΣB |}. This shows that

Iacc ({p(x), ρx }) = sup Iacc ({p(x), ρx }, L) .

L∈ N
For any L ∈ N the set ML of POVMs µ : {1, . . . , L} → B(H)+ is compact and convex by
Lemma 3.1. We conclude that for any L ∈ N
there exists an extreme point µopt ∈ ML
attaining the supremum in (1), i.e., such that

Iacc ({p(x), ρx }, L) = I(A : B) Nµ .

pABopt

By Lemma 3.2 we have that

|{y ∈ {1, . . . , L} : µopt (y) 6= 0}| 6 dim (H)2 .

By renaming the elements y ∈ {1, . . . , L} for which µopt (y) 6= 0, we can identify µopt with a
POVM in Mdim(H)2 . This shows that
 
I(A : B) Nµopt 6 Iacc {p(x), ρx }, dim (H)2 .
pAB

4
Combining the previous statements shows that
 
Iacc ({p(x), ρx }) = Iacc {p(x), ρx }, dim (H)2 = I(A : B) Nµ ,
pABopt

for some µopt ∈ Mdim(H)2 .

6 Holevo’s quantity and theorem

When studying how classical information can be encoded into quantum systems, it is very
useful to consider so-called classical-quantum states. A quantum state ρAB ∈ D(HA ⊗ HB )
is a classical-quantum state, if it is of the form
(cq)
X
ρAB = p(x)|xihx|A ⊗ σxB ,
x∈ΣA

for some probability distribution p ∈ P(ΣA ), quantum states σx ∈ D(HB ), and where
(cq)
{|xi}x∈ΣA denotes the computational basis. The physical interpretation of the state ρAB
is that system ‘A’ is in a classical state x with probability p(x). The classical state x is
represented by the pure state |xihx|A in the computational basis. Formally, the system ‘A’
is still a quantum system, but since the classical information is represented in a fixed basis
it can be accessed deterministically (by the PVM {|xihx|}x∈ΣA ).
To any ensemble {p(x), ρx }x∈ΣA with p ∈ P(ΣA ) and quantum states ρx ∈ D(H) we may
associate the classical-quantum state
(cq)
X
ρCA = p(x)|xihx|C ⊗ ρAx.
x∈ΣA

It is straightforward to compute the mutual information

 
X X
I(C : A)ρcq = H  px ρx  − px H(ρx ),
CA
x∈ΣA x∈ΣB

which is non-negative by concavity of the von Neumann entropy. Intuitively, one might
expect that this mutual information (which generalizes the classical mutual information)
could somehow quantify the information that the quantum system ‘A’ has about the classical
state x. This intuition is indeed correct, and as a result, the above quantity got its own name:
Definition 6.1 (Holevo information). For any ensemble {p(x), ρx }x∈ΣA with p ∈ P(ΣA )
and quantum states ρx ∈ D(H), we define
 
X X
χ ({p(x), ρx }) = H  px ρx  − px H(ρx ).
x∈ΣA x∈ΣA

We will now show the following theorem:

Theorem 6.2 (Holevo’s theorem). For any ensemble {p(x), ρx }x∈ΣA with p ∈ P(ΣA ) and
quantum states ρx ∈ D(H), we have

Iacc ({p(x), ρx }) 6 χ ({p(x), ρx }) .

Proof. Define the classical-quantum state

X
σCA = p(x)|xihx|C ⊗ ρA
x,
x∈ΣA

5
such that
χ ({p(x), ρx }) = I(C : A)σCA = D(σCA kσC ⊗ σA ).
Next, consider a POVM µ : {1, . . . , L} → B(H)+ and define a quantum channel Mµ :
C
B(H) → B( L ) by
L
X
Mµ (X) = hµ(y), XiHS |yihy|,
y=1

and note that

N
X X X X µ
(idA ⊗ Mµ ) (σCA ) = p(x)Nµ (y|x)|xihx| ⊗ |yihy| = pAB (x, y)|xihx| ⊗ |yihy|.
x∈ΣA y∈ΣB x∈ΣA y∈ΣB

Similarly, we note that

N
X X X
Mµ (σA ) = p(x)Nµ (y|x)|yihy| = pB µ (y)|yihy|,
x∈ΣA y∈ΣB y∈ΣB

and, by the data-processing inequality, we conclude that

I(A : B)pNµ = D ((idA ⊗ Mµ ) (σCA ) kσC ⊗ Mµ (σA ))

AB

6 D (σCA kσC ⊗ σA ) = χ ({p(x), ρx }) .

Since the POVM µ was arbitrary, the proof is finished.

Holevo’s theorem has a simple but important corollary:

Corollary 6.3. For any ensemble {p(x), ρx }x∈ΣA with p ∈ P(ΣA ) and quantum states ρx ∈
D(H), we have
Iacc ({p(x), ρx }) 6 log(dim(H)).

Proof. This follows by noting that

 
X X
Iacc ({p(x), ρx }) 6 χ ({p(x), ρx }) = H  px ρx  − px H(ρx ) 6 log (dim(H)) .
x∈ΣA x∈ΣB

Although any quantum system can be in a continuum of different quantum states, the
previous corollary shows that classical messages can only be send reliably at rates below
log (dim(H)) by the process of preparation and measurement. This can be interpreted as
saying that a qubit can only store 1 bit of information reliably.