Quantum Stud.: Math. Found.

(2020) 7:77–98 CHAPMAN INSTITUTE FOR

https://doi.org/10.1007/s40509-019-00201-w U N I V E R S I T Y QUANTUM STUDIES

REGULAR PAPER

From probabilistic mechanics to quantum theory

U. Klein

Received: 26 June 2019 / Accepted: 2 August 2019 / Published online: 29 August 2019
© The Author(s) 2019

Abstract We show that quantum theory (QT) is a substructure of classical probabilistic physics. The central
quantity of the classical theory is Hamilton’s function, which determines canonical equations, a corresponding flow,
and a Liouville equation for a probability density. We extend this theory in two respects: (1) The same structure is
defined for arbitrary observables. Thus, we have all of the above entities generated not only by Hamilton’s function
but also by every observable. (2) We introduce for each observable a phase space function representing the classical
action. This is a redundant quantity in a classical context but indispensable for the transition to QT. The basic
equations of the resulting theory take a “quantum-like” form, which allows for a simple derivation of QT by means
of a projection to configuration space reported previously [Quantum Stud Math Found 5:219–227, 2018]. We obtain
the most important relations of QT, namely the form of operators, Schrödinger’s equation, eigenvalue equations,
commutation relations, expectation values, and Born’s rule. Implications for the interpretation of QT are discussed,
as well as an alternative projection method allowing for a derivation of spin.

Keywords Quantum–classical relation · Ensemble theory · Quantization · Derivation of quantum theory

Mathematics Subject Classification 81P05 · 81S05 · 82C03 · 70H20

1 Introduction

General agreement regarding the meaning of the quantum-mechanical formalism has not been achieved so far.
This lack of clarity is closely related to a lack of clarity regarding the relation between quantum theory (QT) and
classical physics. The present-day ideas on the quantum–classical interface have been established more than 90
years ago. They have never undergone a critical reexamination, despite the fact that a wealth of new information,
both experimentally and theoretically, has been obtained since then.
Einstein’s claim that QT must be an ensemble theory, and not a theory about individual particles, is neither
generally accepted nor has it ever been refuted. Although not a majority view, it has been supported by many
outstanding physicists. An excellent review article about this “statistical interpretation” or “ensemble interpretation”

U. Klein (B)
Institute for Theoretical Physics, University of Linz, 4040 Linz, Austria
e-mail: [email protected]

123
78 U. Klein

is available [1]. Among the earliest, papers written in the spirit of Einstein’s interpretation are pioneering works by
VanVleck [2], Bopp [3,4], and Schiller [5].
Since then, several other papers sharing this interpretation, but using a variety of different methods and assump-
tions, have been published. An incomplete list includes attempts to understand QT in terms of ensembles either in
phase space [6–10] or in configuration space [11–17]. These works clarify several important aspects of QT. There
is, however, still considerable room for improvements, as regards the number and nature of the assumptions used
to derive QT.
The second main opinion, in particular promoted by Bohr in discussions with Einstein [18], claims that QT is a
“complete” theory for individual particles. This “individuality interpretation” is more common than the ensemble
interpretation despite the obvious fact that QT makes probabilistic predictions (this fundamental discrepancy is the
origin of all the ongoing discussions). The prevailing opinion seems to be that the question has already been decided
and one should not call into doubt the chosen path.
To understand QT means to clarify its relation to classical physics. But what exactly means “classical physics”
? If we accept Bohr’s individuality interpretation, we will identify “classical physics” with classical mechanics, the
classical theory of individual particles. On the other hand, if we prefer Einstein’s ensemble point of view, we will
identify “classical physics” with a probabilistic theory of classical particles. Our objects to study are then statistical
ensembles of particles rather then individual particles.
The present paper is the second in a series of works where Einstein’s point of view is taken and verified. In the
first paper [19], referred to as I, some essential relations of QT were derived using Koopman–von Neumann theory
as a starting point. In the present paper, the problem of the derivation of QT is attacked in a more systematic way. In
the following Sect. 2, we discuss the quantum–classical interface, taking into account all major theoretical results in
this field. This analysis leads to the conclusion that QT should be derivable from a probabilistic version of classical
mechanics, a conclusion to be verified in the remaining part of this paper.
Our starting point is the theory of ensembles of classical particles, formulated first (in Sect. 3) with particle
coordinates as independent variables, and rewritten then (in Sect. 4) in terms of the more familiar field-theoretic
formulation, where coordinates denote points in “space”. In our case, “space” means 2n-dimensional phase space,
with coordinates p and q (instead of the n-dimensional configuration space with coordinates q used for example
in QT). We introduce two basic dynamical variables, the probability density ρ(q, p, t), and the action S(q, p, t),
for the probabilistic description of particles moving in the course of time t. These variables obey two basic differ-
ential equations, the Liouville equation and an action equation. This structure is determined by the definition of a
single-phase space function, the Hamiltonian H (q, p). The solutions of the canonical equations define a family of
transformations, also referred to as flow, on phase space.
This construction may be formulated for arbitrary observables A(q, p), with corresponding independent param-
eter α; it is known that each A(q, p) generates a one-parameter Lie group, realized as a subgroup of canonical
transformations [20]. This extension is reported in Sect. 5. Thus, each A(q, p) defines a flow for varying α and two
dynamical variables ρ A (q, p, α), S A (q, p, α) obeying corresponding differential equations. We call the resulting
theory, where a multitude of fields occur and modern probability theory plays an important role the “Hamilton–
Liouville–Lie–Kolmogorov theory” (HLLK). Introducing for each A a complex-valued classical state variable,
one obtains, after some manipulations, a theory which shares many structural properties with QT (see Sects. 7–
10). Certain phase space operators L̂ A (introduced already in I), which are generalizations of Koopman–Neumann
operators, represent the HLLK counterpart of quantum operators Â.
This work is based on the fundamental assumption that all physical fields must be formulated in configuration
space; as a consequence, we have to perform a projection of phase space to configuration space. If the quantum-like
form of HLLK, reported in Sect. 8, is used, the transition from to QT becomes very simple. This transition, which is
performed in Sect. 11, creates Schrödinger’s equation, the form of the quantum-mechanical operators, commutator
relations, expectation values, and Born’s rule. Thus, all fundamental structural properties of QT may be derived
from HLLK. A general discussion of the present approach is given in Sect. 12, concluding remarks are made in the
final Sect. 13.

123
From probabilistic mechanics to quantum theory 79

2 The quantum–classical interface

Here, we compare the mathematical structures of QT and classical physics. More precisely, we compare the math-
ematical structures of QT and two classical theories, namely: the deterministic description of classical particles
(classical mechanics), and the probabilistic description of classical particles (probabilistic mechanics). We are
interested, in particular, in the question, which one of these three theories can be considered as a “covering the-
ory”, in the sense that it can be reduced to another theory. Correspondingly, an important mathematical tool of this
investigation is counting degrees of freedom.
Let us start with a comparison of the mathematical structures of classical mechanics and QT. We restrict ourselves
to mechanical systems which may be cast in the Hamiltonian form of classical dynamics. This formulation is also
best suited for comparison with QT. The canonical equations are given by the following:

∂ H (q, p) ∂ H (q, p)
q̇ = , ṗ = − , (1)
∂p ∂q

where H (q, p) is Hamiltons’s function (here assumed time-independent) and qk , pk (k = 1, . . . , n) are the com-
ponents of the generalized coordinates q and conjugate momenta p, respectively. The dot denotes a derivative with
respect to t. The state of the system, at each instant of time t, is given by a point in phase space determined by the
2n real numbers q, p. The basic law of QT is Schrödinger’s equation:

h̄ ∂
− ψ(q, t) = Ĥ ψ(q, t). (2)
ı ∂t

∂
Here, the Hamilton operator is defined by Ĥ = H (q, h̄ı ∂q ) and the dynamic variable ψ depends, at a particular time
t, on the n coordinates q1 , . . . , qn of the classical configuration space. The state of a system in QT, at a particular
time, is given by a point in Hilbert space, specified by the real and imaginary parts of the complex-valued function
ψ(q, t). The number of degrees of freedom required to specify a function is of course uncountable infinite. Let us
for comparison introduce a symbol for a large number, say I, to denote any finite approximation for the uncountable
infinite number of degrees of freedom associated with a real axis (at the times of Sophus Lie, we would use the
symbol ∞). A precise definition is not required; in particular, the precise mathematical definition of the cardinality
of the continuum is completely useless for our purpose. All we have to know is that I  n. Using this symbol, we
may say that a system described in classical mechanics by 2n numbers requires 2In numbers for its description in
QT.
Thus, there is a gigantic mismatch between classical mechanics and QT as regards the number of degrees of
freedom of one and the same physical system. Classical mechanics cannot, of course, be the covering theory of QT,
but the inverse may be true. In fact, according to the prevailing opinion, expressed by authorities [21], QT reduces
in the classical limit h̄ → 0 to classical mechanics. This is, however, not the case. It has been shown recently
that almost all states of QT do not reduce to corresponding solutions of classical mechanics [22]. There is still the
possibility left, that classical mechanics is the classical counterpart of QT in the sense that it can be used to derive
the form of all quantum-mechanical operators. This possibility of a “quantization”, in the sense a of a complete
structural similarity, must also be excluded, as Groenewold reported already many years ago his no-go theorem
for quantization, which says that a consistent map between the structures of classical and quantum-mechanical
observables does not exist [23].
Let us consider now the second possibility that the classical counterpart of QT is not classical mechanics, but
entails “only” the probabilistic description of classical particles. In this case, the fundamental dynamical variable
is given by the probability density ρ(q, p, t), which has to obey Liouville’s equation:

∂ρ ∂ρ ∂ H ∂ρ ∂ H
+ − = 0. (3)
∂t ∂q ∂ p ∂ p ∂q

123
80 U. Klein

The basic law takes now the form of a partial differential equation (as in QT), because an infinite number of degrees
of freedom is required to describe the behavior of particles. It is linear (as in QT), which means that almost all
Cauchy data will lead to well-behaved solutions.
The probabilistic theory described by Eq. (3) has a simple conceptual structure; the probabilistic element stems
from incomplete knowledge of initial values, while the deterministic behavior, realized by trajectories in phase
space, of classical particles remains intact. In statistical mechanics, the number of degrees of freedom is, in contrast
to probabilistic mechanics, much larger than the number of conserved quantities. As a consequence, statistical
mechanics does not make probabilistic predictions, as probabilistic mechanics does.
A state of this theory is now, in analogy to QT, a function on phase space (similar to a point in Koopman–Neumann
Hilbert space [24]). Using our above estimate, the number of degrees of freedom is 2I2n (the additional factor of
2 stems from the fact that a second dynamical variable will be introduced in Sect. 4). The set of all states contains
as a limiting case “pure states” described by Delta-function-like initial conditions. Despite certain similarities,
probabilistic mechanics and classical mechanics are completely different theories, in particular with regard to their
relation to QT.
If we compare now the number of degrees of freedom of probabilistic mechanics and QT, we find again a
gigantic mismatch, but this time the number of reals 2I2n required to describe a classical probabilistic state exceeds
the corresponding number 2In of QT by a vast amount. This result, which is at first sight surprising, implies that
QT does not reduce to probabilistic mechanics in the limit h̄ → 0. To discuss the mathematical relation of both
theories, the Wigner–Weyl formulation of QT is most appropriate [25,26]. The Wigner function, defined in terms
of ψ(q, t) and ψ(q  , t), is given by W (Q, P, t) = Wh̄ (Q, P, t), where
 
1 r   r   ı 
Ww (Q, P, t) = dr ψ  Q − , t ψ Q + , t exp − Pr , (4)
(2π w)n 2 2 w

and Q = q+q 
2 , r = −q + q . Note that the quantity h̄ is part of the definition of W (Q, P, t). The equation of
motion of Wigner’s function, first obtained by Moyal [27], differs from Liouville’s equation by an infinite number
of terms which all seem to vanish if h̄ → 0. However, the h̄−dependence of W (Q, P, t) must also be taken into
account. If this is done, it turns out that the time-evolution of W (Q, P, t), for almost all potentials (the exceptions
are the same as for the classical limit of Schrödinger’s Eq. [22]) in the limit h̄ → 0 does not follow Liouville’s
Eq. [7,28]. Thus, in agreement with our estimate, QT does not reduce to probabilistic mechanics in the classical
limit. The simple reason is that the dimension of configuration space is only half the dimension of phase space.
Consequently, Q, P are not independent variables and the “phase space of QT” is a “mock phase space” [29].
As QT does neither reduce to classical mechanics nor to probabilistic mechanics, we have to conclude that
probabilistic mechanics, or an appropriate extension of this theory, can somehow be reduced to QT. This conclusion
is astonishing at the first sight. We know that QT is the theory actually realized in nature and it is, at least in this
sense, certainly superior to any classical theory. One would, therefore, expect (according to our reductionistic way
of thinking) that QT embraces at least one of the classical theories and reduces to it in the classical limit. However,
this is not the case. Probabilistic mechanics is superior to QT in the sense that it contains a higher number of degrees
of freedom and it is this number which rules the mathematical relation between both theories, not the question
which theory is realized in nature. The old idea that the quantum constant h̄ plays the role of an accuracy limit in
phase space [4] is in perfect agreement with our present hierarchical ordering of theories in terms of degrees of
freedom. The considerations of this section may be summarized in the form of the following working hypothesis:
“QT is a configuration space version of the probabilistic description of classical particles”.

3 Lagrangian phase space ensembles defined by the Hamiltonian function

The systems that we want to study are not individual trajectories but ensembles of trajectories. It is important to
distinguish clearly between these two kinds of systems. We start our construction of HLLK by defining an ensemble

123
From probabilistic mechanics to quantum theory 81

as an infinite set of individual trajectories, which differ from each other by their states q0 , p0 at a fixed time t0 . We
study deterministic phase space ensembles, which means that each trajectory is completely known if a single point
on it is known. We start with the fundamental “Lagrangian” description of a classical ensemble, where the positions
of the individual members are determined by the trajectory label q0 , p0 and the time t [30]. The transition to the
more common “Eulerian” description will be performed in the next section.
The solutions of the equations of motion (1) and their inverses are written, suppressing the dependence on t0 , as
q = Q(q0 , p0 , t), p = P(q0 , p0 , t), (5)
q0 = Q 0 (q, p, t), p0 = P0 (q, p, t). (6)
These relations are subject to the constraints q0 = Q(q0 , p0 , t0 ), p0 = P(q0 , p0 , t0 ), which means that the
n−component quantities q0 , p0 are the particle coordinates and momenta at the initial time t0 . A (deterministic)
phase space ensemble [31] is the set of all possible trajectories:
 
E D P = q = Q(q0 , p0 , t), p = P(q0 , p0 , t) | (t, q0 , p0 ) ∈ R × Rn × Rn . (7)

This system possesses an infinite (continuous) number of degrees of freedom. The independent variables are
members of the set {t, q0 , p0 | (t, q0 , p0 ) ∈ R × Rn × Rn }. Correspondingly, the basic equations of our ensemble
theory are not given by (1) but by an infinite number of equations of the form (1), but each one with prescribed
initial values q0 , p0 . We may write


∂ H (q, p, t; q0 , q0 ) ∂ H (q, p, t; q0 , q0 )

q̇ = , ṗ = −
(q0 , p0 ) ∈ R × R ,
n n
(8)
∂p ∂q

to denote this infinite set. The independent variables q0 , p0 must be taken into account even if our basic equations do
not contain any derivatives with respect to these variables (It is for this reason that the present field theory does not
take the standard form of a partial differential equation but the “degenerate” form of an infinite number of ordinary
differential equations). A Lagrangian (here, we use the standard meaning of this term from classical mechanics) for
our basic equations, not written down here, contains an integral over all contributions from the initial values q0 , p0 .
In a classical probabilistic theory, particles move according to deterministic laws, but deterministic predictions
are nevertheless impossible, because the particle’s trajectories cannot be localized with certainty. This uncertainty
can be quantitatively described by introducing a Lagrangian probability density for the states of the particles at a
particular time t0 . Such probabilistic theories, where trajectories still exist, have been classified as “type 2 theories”
in a recent work of the present author [15].
Denoting the Lagrangian probability density at time t0 by ρ0 (q0 , p0 ), the infinitesimal quantity ρ0 (q0 , p0 ) dn q0
d p0 gives the probability to find a particle within an infinitesimal volume element located at q0 , p0 . The cor-
n

responding Lagrangian probability density for the same particle (labeled by q0 , p0 ) at a later time t is denoted
by ρ L (q0 , p0 , t). The probability to find this particle at time t within an infinitesimal volume element located at
q(t), p(t) is given by ρ L (q0 , p0 , t) dn q(t) dn p(t). The fact that trajectories remain intact in the course of time
implies the relation:

ρ L (q0 , p0 , t) dn q(t) dn p(t) = ρ0 (q0 , p0 ) dn q0 dn p0 . (9)

Note that q0 = q(t0 ), p0 = p(t0 ) and ρ0 (q0 , p0 ) = ρ L (q0 , p0 , t0 ). Eq. (9) indicates that the mapping between
q0 , p0 and q(t), p(t), defined by the solutions (5) of equations of motion (1), determines the relation between
ρ0 (q0 , p0 ) and ρ L (q0 , p0 , t). This mapping is denoted as Hamiltonian (or symplectic) flow. Indeed, performing a
transformation of variables, one finds that

ρ L (q0 , p0 , t)J (t) = ρ0 (q0 , p0 ), (10)

123
82 U. Klein

where the Jacobian J (t) is the determinant of the Jacobian matrix D associated with the mapping (5),

∂(Q 1 , . . . , Q n , P1 , . . . Pn )
J (t) = detD = . (11)
∂(q0,1 , . . . , q0,n , p0,1 , . . . p0,n )

A mathematical fact of fundamental importance is the uniqueness of the solutions of the first-order system (1) for
specified initial values q0 , p0 . This fact implies J (t) > 0 and the invertibility of the mapping (5). This is also a
necessary condition for the global validity of the solutions (5), which we assume to hold true. Actually, the stronger
statement J (t) = 1 holds true and this fact will be used below in a number of transformations.
We may now calculate the observable output of our Lagrangian ensemble theory. The expectation value Ā(t) of
the (Lagrangian) observable A0 (q0 , p0 , t) in the (Lagrangian) state ρ0 (q0 , p0 ) is given by integrating the product
A0 ρ0 over all phase space points q0 , p0 :

Ā(t) = A0 (q0 , p0 , t)ρ0 (q0 , p0 )dn q0 dn p0 . (12)

Note that in the Lagrangian formulation, observables depend on time, while states are time-independent.
The Lagrangian formulation outlined here provides the most natural framework for the definition of classical
deterministic ensembles. However, the use of particle labels as independent variables is inconvenient and a transition
to the more common “Eulerian” description, where points of “space” are used as independent variables, is useful—
this is all the more true as we are finally interested in QT, which is a Eulerian (configuration space) theory.

4 Eulerian phase space ensembles defined by the Hamiltonian function

The quantities q, p and q0 , p0 appearing in Eq. (5) denote particle properties. Today, physicists believe that it makes
sense to speak about abstract spaces which exist no matter whether particles are present or not. The points of these
abstract spaces may be obtained from the Lagrangian coordinates q0 , p0 by performing at each instant of time
a transformation from the initial values q0 , p0 to the final (at time t) values q, p of the particle properties. This
uncountable set of transformations:

ΦtH : Ω → Ω, (13)

is also referred to as flow. We introduced here the symbol Ω to denote 2n−dimensional phase space, Ω = Rqn × Rnp .
The flow ΦtH is defined by the solutions (5) of the equation of motion (1) and is in the present context denoted as
Lagrangian–Eulerian map [30,32]. The totality of all image points represents the same continuum at each instant
of time t. This property is responsible for the independent existence of the abstract space created this way. This
process replaces the independent coordinates q0 , p0 by new Eulerian coordinates, which are also denoted by q, p,
but characterize now points of (phase) space.
It is remarkable that the Lagrangian–Eulerian map defines a concept of space in terms of particle properties; a
careful analysis might possibly lead to the conclusion that it is unnecessary to introduce absolute space with the
help of extra axioms.
To perform the Lagrangian–Eulerian map for the basic equations of our ensemble theory, we start from the
solutions (5) and the associated inverse Eq. (6). For each Lagrangian function, say G L (q0 , p0 , t), a corresponding
Eulerian function G(q, p, t) is defined by the following:

G(q, p, t) = G L (Q 0 (q, p, t), P0 (q, p, t), t), (14)

123
From probabilistic mechanics to quantum theory 83

Using the fact that the initial values q0 , p0 (considered as a function both of time t and the time-dependent solutions
q, p) do not depend on t, a partial differential equation for the Eulerian observable G(q, p, t) may be derived by
total differentiation of (14) with respect to t. The result takes the form:

∂G ∂G ∂ H ∂G ∂ H ∂G L
+ − = . (15)
∂t ∂q ∂ p ∂ p ∂q ∂t

This is the general Eulerian partial differential equation which represents the most convenient way to study classical
ensembles; there is no way to describe individual particles anymore. To characterize a system, appropriate functions
on phase space, obtained from corresponding Lagrangian functions, have to be specified.
The most important Eulerian function is the probability density ρ(q, p, t), obtained from the time-independent
Lagrangian quantity ρ0 (q0 , p0 ). This field may be used to describe the “state” of our (Eulerian) probabilistic
ensemble. Then, the right-hand side of Eq. (15) vanishes and we obtain the Liouville equation:

∂ρ ∂ρ ∂ H ∂ρ ∂ H
+ − = 0. (16)
∂t ∂q ∂ p ∂ p ∂q

All time-independent Lagrangian functions lead to the same differential Eq. (16). As a consequence, this homoge-
neous linear partial differential equation has a huge manifold of solutions. The Liouville equation, together with
the Eulerian expression:


Ā(t) = A(q, p)ρ(q, p, t)dn q dn p, (17)

represent the basic building block of HLLK. The expectation value (17) is obtained from (12) by means of a
transformation of variables defined by (6). In contrast to the Lagrangian formulation, Eulerian states depend on
time, while Eulerian observables are time-independent (here, we find analogies with different formulations of
time-dependence in QT, see, e.g., [20]).
The above equations are sufficient to calculate most quantities of interest of HLLK, at least as far as the observable
H (q, p), ruling the time-dependence of observables, is concerned. There is, however, a certain arbitrariness in the
definition of a “state” of HLLK, even if we restrict ourselves to the single observable H (q, p) (see below). This
arbitrariness can be used to facilitate the transition to QT. We introduce a second Eulerian dynamical variable,
describing the purely deterministic part of the evolution in phase space. As an appropriate quantity, we consider the
classical action
 t 
S= dt  L q(t  ), q̇(t  ) , (18)
t0

where q(t  ) are the real paths, i.e., the solutions of the equations of motion. At these stationary points, the integrand
may be written in the form:

∂ H (q, p)
L (q, q̇) = L̄(q, p) = p − H (q, p), (19)
∂p

where q, p are solutions of the canonical Eq. (1). If we identify the 2n integration constants with the initial
coordinates and momenta q0 , p0 at t0 , then we have a Lagrangian function St0 (q0 , p0 , t) depending explicitly on
time t. The transition to the corresponding Eulerian differential equation may be performed with the help of Eq. (15),

123
84 U. Klein

as reported in detail in I. We find that our second Eulerian dynamical variable S(q, p, t) has to obey the equation:

∂S ∂S ∂H ∂S ∂H
+ − = L̄, (20)
∂t ∂q ∂ p ∂ p ∂q

where L̄ = L̄(q, p) is defined as in Eq. (19). Equations (16) and (20) for ρ(q, p, t) and S(q, p, t), together with
Eq. (17) for the expectation values Ā(t), represent an extended set of basic equations of HLLK.
The action equation represents the purely deterministic part of HLLK, as shown by the fact that ρ(q, p, t) does
not occur in (20). This kind of decoupling between S(q, p, t) and ρ(q, p, t) is the most important feature of classical
probabilistic physics. We introduced the new variable S(q, p, t), which might seem redundant at the first sight, to
formulate a theory where this decoupling can break down. In fact, in QT, this decoupling will break down and this
is the deeper reason why we introduced the field S(q, p, t).
We note in passing that Eq. (20) may be projected to configuration space by neglecting the p−dependence of S
and by replacing p by the gradient of S. In this way, one obtains the Hamilton–Jacobi equation in a very quick way.

5 Eulerian phase space ensembles defined by arbitrary observables

The above concepts were all defined in terms of a single-phase space function, the Hamiltonian H (q, p), which is
the most important observable. We required nowhere a special functional form of H (q, p). Thus, all these concepts
may also be defined for an arbitrary phase space function A(q, p). We incorporate these additional structures, for
arbitrary A(q, p), in the framework of HLLK, anticipating that it will be useful from a physical point of view.
Instead of defining canonical equations for a “Hamiltonian” A(q, p), let us start from the regions in phase space
where A(q, p) takes a constant value, say a. These regions are given by the level sets:

L A (a) = {(q, p) ∈ Ω|A(q, p) = a ∈ R A } , (21)

where the number a belongs to the image R A ⊆ R of A(q, p). Let us next consider a connected level set of the
form:

L A (a|q0 , p0 ) = {(q(α), p(α)) ∈ Ω|A(q(α), p(α)) = a ∈ R A , (q0 , p0 ) = (q(0), p(0))} , (22)

which defines a curve parameterized by the real parameter α and crossing the point (q0 , p0 ) at α = 0. As a
consequence of the fact that a is a constant, the points in L A (a|q0 , p0 ) obey the condition:

d ∂A ∂A
A(q(α), p(α)) = q̇k + ṗk = 0, (23)
dα ∂qk ∂ pk

where the dot denotes now derivation with respect to α. Condition (23) holds true if q(α), p(α) are solutions of the
ordinary differential equations:

∂ A(q, p) ∂ A(q, p)
q̇k = , ṗk = − , (24)
∂ pk ∂qk

with boundary conditions q(0) = q0 , p(0) = p0 . Obviously, Eq. (24) agree with (1) if A and α are identified with
the Hamiltonian H and the time t, respectively. The solutions of (24) and their inverses are denoted as follows:
q = Q A (q0 , p0 , α), p = P A (q0 , p0 , α), (25)
q0 = Q 0A (q, p, α), p0 = P0A (q, p, α). (26)

123
From probabilistic mechanics to quantum theory 85

Thus, an observable A(q, p) takes a constant value along an integral curve of the canonical Eq. (24) defined by
the same observable. The dimension of the independent variable α, which labels the points on the integral curves,
is the dimension of h̄ divided by the dimension of A; observables related this way are said to be conjugate, or
complementary, to each other.
The relations (24) play an important role in Lie’s theory of transformation groups and in the theory of Hamiltonian
vector fields. The solutions define again, in analogy to (13), a family of mappings:

ΦαA : Ω → Ω (27)

of phase space onto itself. The fact that such a mapping exists may be expressed, using the language of level sets,
by means of the relation: Ω = (Q A (q0 , p0 , α), P A (q0 , p0 , α))|(q0 , p0 ) ∈ Ω , which must be true for arbitrary
α ∈ R. The integral curves of all equations of the form (24) cover for each value of the parameters α the whole
of phase space. This statement is, of course, only true if the solutions of (24) are well defined throughout phase
space [32]).
Let us consider as an example the flows created by the fundamental phase space observables q1 , . . . , pn (other
examples may be found, e.g., in [33]). For A = qi , where i is a fixed integer from the set {1, . . . , n}, the parameter,
say π , labeling the integral curves of (24) has the dimension of momentum. The solutions do not depend on π with
the exception of pi (π ) = −π + pi0 . Thus, the integral curves are lines parallel to the pi −axis; the observable qi
creates translations, with negative sign, in the direction of the corresponding component pi of the momentum. For
A = pi the parameter, say χ , has the dimension of position. The solutions do not depend on χ with the exception
of qi (χ ) = χ + qi0 ; the observable pi creates translations in the direction of qi . More generally, the observables
A(q, p) play the role of generators of one-parameter groups of (canonical) transformations on phase space [20].
We may associate, in analogy to Sect. 4, two Eulerian phase space functions with each observable A(q, p), a
probability density ρ A (q, p, α) and an action function S A (q, p, α), which obey the differential equations:
∂ρ A ∂ρ A ∂ A ∂ρ A ∂ A
+ − = 0, (28)
∂α ∂q ∂ p ∂ p ∂q
∂ SA ∂ SA ∂ A ∂ SA ∂ A
+ − = L̄ A . (29)
∂α ∂q ∂ p ∂ p ∂q
The inhomogeneous term in (29) is the Lagrange function L̄ A associated with A, which is defined by the following:

∂ A(q, p)
L̄ A (q, p) = p − A(q, p), (30)
∂p

and the action expressed in Lagrangian coordinates, which is used to derive (29), is given by the following:
  

α   ∂ Q(q0 , p0 , α )  
S A (q0 , p0 , α) = dα P(q0 , p0 , α )  − A(Q(q0 , p0 , α ), P(q0 , p0 , α )) . (31)
α0 ∂α

The basic equations of the probabilistic theory defined by A(q, p) are mathematically equivalent to the relations
reported in the last section. On the other hand, the physical meaning of the various theories, obtained for different
A(q, p), is quite different.

6 Comparison with standard probability theory

In this section, we discuss the relation between HLLK and modern probability theory; To do this, we have to use
ρ as state variable, leaving a possible role of S aside. We start by recalling the basic features of time-independent

123
86 U. Klein

(Kolmogorovian) probability theory. Then, we give a short review of the extension to time-dependent phenomena.
Finally, we discuss how HLLK fits into the time-dependent framework.

6.1 Time-independent probability theory

The result of a single experiment, called “trial”, is denoted as “outcome” (or elementary event) and is mathematically
represented by a point ω ∈ Ω. An “event” is a subset E ⊆ Ω. The set F of admissible subsets must be a σ -algebra:
this means, among other things, that F is closed under formation of complements and countable unions [34]. The
remaining essential element is a universal probability measure P(E), a function defined on F and normalized
according to P(Ω) = 1. The measures take the form of integrals over E ∈ F:

P(E) = dω ρ(ω), (32)
E

assigning weight to the different points of E according to a probability density ρ(ω) ≥ 0. A classical probability
space is given by the triple (Ω, F, P). Classical observables are arbitrary (sufficiently smooth) functions A : Ω →
R, ω → A(ω), which are in a probabilistic context referred to as random variables.
The most important number characterizing the statistical behavior of an observable A(ω) is its expectation value
Āρ , in the state ρ, which is defined by the following:

Āρ = dω A(ω)ρ(ω). (33)

Most experimental predictions of a statistical theory may be written in the form (33). For example, P(E) as given
by (32) takes the form of (33) if A(ω) is identified with the characteristic function I E (ω) of the set E, defined by
the following:

1 :ω∈E
I E (ω) =
0 :ω∈ / E.
Similarly, the average of an observable A with respect to a subset E is given by (33) if A(ω) is replaced by
A(ω)I E (ω).
The mathematical structure (Ω, F, P) can be interpreted as theoretical image of a physical experiment, which
is of a purely probabilistic (time-independent) nature. The space Ω and the functional form of ρ(ω) have to be
chosen in such a way that the experimental setup is properly described. This framework, where no free parameter
like the time t appears, is of course not appropriate for a description of HLLK. It can, however, be used to describe
HLLK at a single instant of time, the initial time t0 , when the functional form of ρ(ω) is completely under our
control. Thus, we may identify ρ(ω) with the initial distribution ρ0 (ω) (and Ω with the phase space). This means,
we start at t0 from a Lagrangian description of particle motion; the coordinates ω describe properties of particles
and not abstract space points. The expectation value (33) agrees, after an appropriate change in notation, with the
Lagrangian expression (12) taken at t = t0 .

6.2 Time-dependent probability theory

To take the time-dependence (replacing here for definiteness the general parameter α by the familiar time t)
of real systems into account, the probability space (Ω, F, P) must be supplemented by the continuous set of
transformations Φt : Ω ⇒ Ω, ω0 → ω = φt (ω0 ), ω0 , ω ∈ Ω, which is provided by the solutions (25) of
the equations of motion. This set, say Φ, of transformations on Ω represents a dynamical system. The extended
structure (Ω, P, Φ) (we omit here the symbol F for brevity) may either be interpreted as a dynamical probability

123
From probabilistic mechanics to quantum theory 87

space, or as a probabilistic dynamical system (in the mathematical literature, it is denoted as measure-theoretic
dynamical system). From a physical point of view, a dynamical system (Ω, Φ) alone, representing the dynamics
of an uncountable number of individual systems, does not make sense; it must be supplemented by a (probability)
density describing the distribution of individual systems in phase space.
Using this extended framework, the expectation value (33) must be replaced by the time-dependent Lagrangian
expression (12), which, in the present notation, takes the form:

Āρ (t) = dω0 A0 (ω0 , t)ρ0 (ω0 ). (34)

Performing a change of variables from ω0 to ω, as given by the dynamical map ω = φt (ω0 ), we obtain the
corresponding Eulerian expression:

Āρ (t) = dω A(ω)ρ(ω, t), (35)

which represents the most common way to write the expectation value. To define a measure in the time-dependent
setting, we introduce the symbol E t to denote a particular subset of Ω chosen at time t. The time-dependent measure
is then defined by the following:

P(E t , t) = dω ρ(ω, t). (36)
Et

This expression may again be derived from the time-independent Lagrangian expression (32) by means of a trans-

formation ω = φt (ω0 ). This derivation shows also that P(E t , t) = P(E t  , t ), i.e., the measure defined by (36) is
invariant under the dynamical map; the term “measure-preserving transformation” is frequently used.
The mathematical structure (Ω, P, Φ) represents the theoretical image of an experiment associated with A
which shows both probabilistic and deterministic aspects. The probabilistic aspect is due to uncertainty at a initial
time t0 , the deterministic aspect belongs to the subsequent movement of the ensemble in phase space. We are
interested in dynamical systems of Hamiltonian type, with Hamiltonian A. Then, the probability density remains
constant on curves (solutions of the canonical equations defined by A) of constant A.

6.3 Comparison of time-dependent probability theory with HLLK

The mathematical structure, (Ω, P, Φ), where the dynamical system Φ is of Hamiltonian type, represents the
basic building block of HLLK, but does not exhaust it. Comparing both structures, we note the following points:
– HLLK may contain several versions of this building block; each observable A(ω) may be used to define
a structure (Ω, PA , Φ A ). Thus, HLLK contains several components which describe independent experi-
ments, but share the same sample space Ω. Using the standard notation, HLLK may be written in the form
(Ω; PA , Φ A ; PB , Φ B , . . .).
– Each component of HLLK is implicitly defined by two functions which may be chosen freely, namely the
observable A(ω) and the initial value ρ 0A (ω) of the probability density. Indeed, the set of Hamiltonian trans-
formations Φ A is implicitly defined by the form of A(ω); the same is true for the time-dependence of PA and
ρ A (ω, t). However, the initial value ρ 0A (ω) of ρ A (ω, t) may be chosen freely. If we denote the pair A, ρ 0A by
the symbol E A , HLLK may also be written in the form H L L K = [Ω, E A , E B , . . .].
– Each component E A belonging to HLLK is associated not only with an observable A(ω) but also with a
corresponding independent variable α. Of course, we assume that Hamilton’s function H (ω), with its associated
parameter t, is always contained in the set of relevant observables. Comparing with standard time-dependent

123
88 U. Klein

probability theory, we note that HLLK, in its present general form, has a multi-parameter structure which calls
for a reduction.
In its general form, HLLK describes several (dynamical) experimental arrangements each one associated with a
particular observable (a reduction of the dynamics will be performed in Sect. 8). More precisely, we assume that the
experimental arrangement defined by A is constructed in such a way that sharp values of the state variable ρ A may
be obtained. The various experimental arrangements are in principle independent from each other, as regards the
measurement of the various probability densities. However, there may be an interplay between different observables,
in the sense that expectation values of arbitrary observables (random variables) may be measured in each one of
these experiments. The question arises if experimental arrangements exist, where state variables ρ A , ρ B belonging
to different observables, may be measured simultaneously. A related question will be studied in Sect. 9.

7 Introducing a complex state variable

The relations (28) and (29) represent the basic equations of our general theory, defined by an arbitrary observable
A(q, p). The first of these describes the deterministic evolution, for varying α, of the probabilistic quantity ρ A ; the
second describes the evolution of the deterministic quantity S A . There is a certain freedom as regards the choice of
a “state variable” for our system. The quantity ρ A (or a function thereof) is certainly the first choice; in this case, we
have the single basic law (28), and the second basic Eq. (29) plays no role. On the other hand, a proper combination
of ρ A and S A is also allowed; the action S A describes the same physics as ρ A , as least far as the deterministic
evolution in phase space is concerned. In this section, we introduce a combination of ρ A and S A which will be
useful for the transition to QT.
Let us first introduce a more compact notation. Points q1 , . . . , qn , p1 , . . . , pn in Ω will be denoted by ω if no
need for distinction between q and p arises. The Poisson bracket of two phase space functions A(ω), B(ω) is
defined by the following:

n 
 
∂A ∂B ∂A ∂B
{A, B} = − . (37)
∂qk ∂ pk ∂ pk ∂qk
k=1

Besides the obvious properties of bilinearity and antisymmetry, the Poisson bracket obeys the relations:
{AB, C} = A {B, C} + {A, C} B, (38)
{{A, B} , C} + {{B, C} , A} + {{C, A} , B} = 0, (39)
which are referred to as product rule and Jacobi identity [20]. Two observables with vanishing Poisson bracket are
said to be in involution.
Using this notion, the basic equations (24), (28), and (29) take the form:
q̇k = {qk , A} , ṗk = { pk , A} , (40)
√
∂ ρ A √ 
+ ρ A , A = 0, (41)
∂α
∂ SA
+ {S A , A} = L̄ A . (42)
∂α
Denoting the points of phase space by ω the solutions of (40) may be written in the compact form ω = φα (ω0 ),
with the inverse given by ω0 = φα−1 (ω). This uncountable set of transformations Ω ⇒ Ω, with Jacobian equal to
unity, represents a one-parameter Lie group with unit element φα0 .
Given two arbitrary phase space functions F(ω), A(ω), the condition {F, A} = 0 is equivalent to the statement
that F is a constant of motion with regard to the canonical equations (40) defined by A; this statement remains true

123
From probabilistic mechanics to quantum theory 89

if the roles of F and A are exchanged. The Poisson bracket {A, B} may also be written in the form D̂ A B, where
D̂ A is a linear differential operator defined by the following:

n 
 
∂ A ∂· ∂ A ∂·
D̂ A · = {A, ·} = − . (43)
∂qk ∂ pk ∂ pk ∂qk
k=1

It is referred to as Lie derivative with respect to A. Application of D̂ A means differentiation along the integral curves
defined by the observable A. As D̂ A B = − D̂ B A holds, a vanishing Poisson bracket between two observables means
that each one is invariant under the flow generated by the other observable.
Using the Lie derivative, the partial differential Eqs. (41) and (42) take the form:
 
∂ √
− D̂ A ρ A = 0, (44)
∂α
 
∂
− D̂ A S A = L̄ A . (45)
∂α
√
In (41) and (44), we replaced the dynamical variable ρ A by ρ A ; this is allowed, since an arbitrary function of a
solution of the Liouville equation is again a solution. Thus, the state of the ensemble associated with the observable
√
A is defined to be ρ A . This definition is preferable as far as the transition to QT, reported in Sect. 11, is concerned.
√
In a purely classical context, both state definitions, ρ A and ρ A , are equivalent and the simpler term ρ A is sometimes
more convenient.
If we want to construct a new state variable, which is a function of both ρ A and S A and obeys a single (linear)
evolution equation, we are more or less automatically led to the range of complex numbers. It is easy to see that the
two decoupled Eqs. (44) and (45) may be written in the form:

 
h̄ ∂ h̄
+ L̂ A φ A = 0, L̂ A = − D̂ A − L̄ A , (46)
ı ∂α ı

if a new complex-valued state variable φ A , defined by the following:

 ı
φ A (ω, α) = ρ A (ω, α) exp S A (ω, α), (47)
h̄

is introduced. Thus, the basic equations of HLLK may be written in a form resembling Schrödinger’s equation.
There are certain conditions to be fulfilled [35], to guarantee the uniqueness of the “classical wave function” φ A ,
which are not important in the present context.
A natural expression for an inner product is obtained by extending the standard definition of QT from configuration
space to phase space:


(φ, ψ) = dωφ ∗ (ω)ψ(ω). (48)

The operator L̂ A is self-adjoint with regard to this inner product. Equation (46) is a generalization of the Koopman–
von Neumann equation (which is obtained for L̄ A = 0) as discussed in more detail in I. These changes in notation
make the equations look slightly more “quantum-like”, but are irrelevant as far as the physical content of the present
classical theory is concerned. However, they will turn out to be very useful later, when the transition from HLLK
to QT is performed.

123
90 U. Klein

8 Reduction to a single-parameter theory

In its present general form, HLLK has a multi-observable (multi-parameter) structure. Besides the Hamiltonian H ,
and its associated independent parameter t, other observables A, with associated parameters α, appear in the theory.
The physical meaning of these “non-time-like” components of HLLK is still unclear.
To clarify this point, let us consider, as an example, a single particle (set n = 3) and identify the observable A
with the z−component of the angular momentum, A(q, p) = L 3 = q1 p2 −q2 p1 . The parameter α is dimensionless
and can be identified with an angle of rotation. The solutions of the canonical Eqs. (40) are given by the following:
q1 = q10 cos α − q20 sin α, q2 = q10 sin α + q10 cos α, q3 = q30 (49)
p1 = p10 cos α − p20 sin α, p2 = p10 sin α + p10 cos α, p3 = p30 . (50)
These relations may be interpreted as kinematical relations in R6 , describing the change of the coordinates of a
point under a rotation with an angle α. They may also be interpreted as a family of mapping R6 → R6 . Let us
consider next the state function ρ L 3 (q, p, α) corresponding to this flow. If the initial values at α = 0 are distributed
according to a probability density ρ L0 3 (q 0 , p 0 ), and if the transformation inverse to Eqs. (49) and (50) is denoted
by qi0 = Q i0 (q, p, α), pi0 = Pi0 (q, p, α), then the probability density ρ L 3 (q, p, α) is given by the following:

ρ L 3 (q, p, α) = ρ L0 3 (Q i0 (q, p, α), Pi0 (q, p, α)).

This is the solution of Eq. (41) which agrees with ρ L0 3 (q 0 , p 0 ) at α = 0. For arbitrary initial values ρ L0 3 (q 0 , p 0 ),
the Eulerian function ρ L 3 (q, p, α) will generally depend on α. However, if ρ L0 3 (q 0 , p 0 ) is chosen to be invariant
under the mapping defined by A, then ρ L 3 (q, p, α) cannot depend on α. This happens if ρ L0 3 (q 0 , p 0 ) depends only
on L 3 or on observables in involution with L 3 . For example, if ρ L0 3 depends on the invariants (q10 )2 + (q20 )2 , q30 and

( p10 )2 + ( p20 )2 , p30 , then ρ(q, p, α) = ρ 0 q12 + q22 , q3 , p12 + p22 , p3 is also invariant under the mapping defined
by A, does not depend on α, and its Poisson bracket with A vanishes according to (41).
Let us come back now, with this example in mind, to our question concerning the physical meaning of the
parameters α and the fields ρ A (ω, α). A “multi-parameter theory”, taking the dependence of all densities ρ A (ω, α)
on the corresponding parameters α into account, does probably not correspond to anything realized in nature.
Therefore, we restrict ourselves to the common “single-parameter structure”, where the dynamics is only taken
into account for A = H and α = t. For brevity, the letter H in ρ H (ω, t) will be omitted and the term “Liouville
equation”, without any addition, will refer to the Liouville equation defined by H . For all other observables A = H
and α = t only the “stationary” solutions of (41), not depending on α, will be taken into account. This seems to
be the only natural way to reduce the general form of HLLK to a standard dynamical theory (actually of the same
type as QT) while taking into account the individual role of observables to the greatest possible extent.
√
Using the notation, where the state variable is given by ρ A , the basic equations for these “stationary” fields,
say ρ̄ A (q, p), are given by the following:
 
ρ̄ A , A = 0. (51)

The vanishing of the left-hand side means that ρ̄ A is a constant of motion with regard to the canonical equations (40).
There is a particularly close connection between ρ̄ A and A; both the values of ρ̄ A and of A are constant along the
solutions of (40). In an analogous way, one obtains α−independent action fields S̄ A (q, p) as solutions of:
 
−a + S̄ A , A = L̄ A , (52)

where a is a real number of the same dimension as A. The relation between S A (q, p, α) and S̄ A (q, p) is given by
S A (q, p, α) = −aα + S̄ A (q, p); a corresponding decomposition for ρ A (q, p, α) is forbidden by the normalization
condition of probability.

123
From probabilistic mechanics to quantum theory 91

The two basic stationary Eqs. (51) and (52) may again, in close analogy to Sect. 7, be rewritten as a single relation
if a proper complex state variable φ̄ A , not depending on α, is introduced. The result takes the form of an eigenvalue
equation:

L̂ A φ̄ A = a φ̄ A . (53)

The eigenvalues a must be real, because the operators L̂ A are self-adjoint. In terms of φ̄ A and S̄ A , the stationary
state variable φ̄ A is given by the following:
 ı
φ̄ A (ω) = ρ̄ A (ω) exp S̄ A (ω). (54)
h̄

The eigenvalue equation (53) may also be derived from the α−dependent Schrödinger Eq. (46), using the method of
separation of variables. To obtain “quantum-like” relations like (53), the original formalism had to be extended, by
introducing a new variable S A and defining a new state variable. While the physical content of the extended theory
is essentially the same as before, it offers new mathematical methods to solve the classical equations of motion.
After this reduction process, we have now a version of HLLK which contains two classes of observables. The
first class is that of dynamical observables and it contains only a single element, namely Hamilton’s function.
Its associated parameter is the time t, and its associated fields, state variables, and operators will be denoted by
ρ(q, p, t), S(q, p, t), φ, L̂, omitting the index H . These fields are obtained by solving the classical time-dependent
Schrödinger equation (this notation will be justified later):
 
h̄ ∂
+ L̂ φ = 0. (55)
ı ∂t

The second class is that of “stationary” observables. It contains all other observables A, B, .. with parameters
α, β, . . ., and fields ρ̄ A (q, p), S̄ A (q, p), φ̄ A ; ρ̄ B (q, p), S̄ B (q, p), φ̄ B ; etc, which are determined by eigenvalue
equations:

L̂ A φ̄ A = a φ̄ A , L̂ B φ̄ B = bφ̄ B , etc, (56)

not depending on α, β, . . .. Of course, a stationary equation of the type (56) exists also for the Hamiltonian H ; it is
obtained by solving (55) by the method of separation of variables.
After this reduction of our multi-parameter problem, we have now a situation analogous to QT. In fact, a
Schrödinger equation exists in QT only for a single operator, the Hamiltonian Ĥ , while all other operators Â, B̂, . . .
are characterized by eigenvalue equations not depending on the corresponding parameters α, β, . . ..

9 Simultaneous measurements of several observables

Only the case of an experimental arrangement associated with a single variable A was discussed up to now. We
may ask under which circumstances the simultaneous measurement of states ρ̄ A , S̄ A (or φ̄ A ) and ρ̄ B , S̄ B (or
φ̄ B ), corresponding to two different observables A and B, may be performed. The theoretical description of such
simultaneous measurements requires that phase space functions ρ̄ AB , S̄ AB exist, which obey the conditions:
   
ρ̄ AB , A = 0, −a + S̄ AB , A = L̄ A (57)
   
ρ̄ AB , B = 0, −b + S̄ AB , B = L̄ B . (58)
If solutions of these equations exist, they represent the theoretical image of an experimental arrangement which
allows for the simultaneous measurement of the states defined by A and B.

123
92 U. Klein

√
The corresponding complex-valued state variable φ̄ AB (ω) = ρ̄ AB (ω) exp h̄ı S̄ AB (ω) must be a simultaneous
solution of the two eigenvalue equations:

L̂ A φ̄ AB = a φ̄ AB , L̂ B φ̄ AB = bφ̄ AB . (59)

This condition leads to mathematical questions which belong to the standard repertoire of QT [36]. Eigenfunctions
belonging to two different operators can only exist if these operators commute with each other. The commutation
relations of the operators L̂ A are given by the following: [37]:

h̄
[ L̂ A , L̂ B ] = − L̂ {A,B} , (60)
ı

where the bracket, for general operators Û , V̂ , is defined by [Û , V̂ ] = Û V̂ − V̂ Û . The assignment A ⇒ L̂ A , with
L̂ A defined by (46), preserves the Poisson-bracket structure of the phase space observables, as discussed in detail
in I. Let us recall [see Sect. 7] that a vanishing Poisson bracket {A, B} = 0 means that A is invariant under the flow
defined by B and vice versa. The commutation relations (60) show that this invariance property is responsible for
the commutability of the corresponding phase space operators.
The relations (46), (53), (60) show already a astonishing close structural similarity between classical relations,
ruling the behavior of flows in phase space, and operator relations of QT. Of course, the reasoning performed here
for two observables may be extended to more than two (commuting) observables, but we do not want to go into
details here. Let us just mention that there is an analogy between Liouville’s integrability theorem for a complete
set of observables in involution [38], and the concept of a simultaneous eigenstate of a complete set of commuting
operators in QT [36].

10 The classical counterpart of Born’s rule

Let us consider the expectation value (35), where the time-dependent state ρ(ω, t) is associated with the Hamiltonian;
states associated with other “stationary” observables may be used as well, but this is the most important case.
The expression for the probability density describing the measurement of a value a of the observable A(ω) in
the state ρ at time t , say Wρ (a, t), can easily be deduced from the definition (35). Let R A ⊆ R denote the range
of possible values of the observable A(ω). In each trial, A(ω) takes a certain value a ∈ R A corresponding to the
observed outcome ω. The set of all ω mapped to a is the pre-image:

E A (a) = {ω ∈ Ω|A(ω) = a} (61)

of a. These level sets provide a partition of Ω. Using this fact, Eq. (35) may be written in the form:

Āρ (t) = da a Wρ (a, t), (62)

where the probability density Wρ (a, t), associated with the observable A, is given by the following:

Wρ (a, t) = dω ρ(ω, t). (63)
E A (a)

(m)
In general, the subset E A (a) will not be connected, but will consist of several disjoint connected sets E A (a),
which are referred to as connected components of E A (a). We will neglect this possibility just as we will neglect
degenerate states, both in HLLK and QT.

123
From probabilistic mechanics to quantum theory 93

It is interesting to rewrite the expression (35) in still another way, in terms of the operator L̂ A . As a first step, we
use the definition (47), for A = H and α = t, to rewrite (35) in the slightly more “quantum-like” form:


Āρ (t) = dω φ ∗ (ω, t)A(ω)φ(ω, t). (64)

Note that this classical expectation value does not depend on S; the two terms containing S cancel each other. Next,
we use the definitions (46), (43), and (30) of the quantities L̂ A , D̄ A , and L̄ A , to express the observable A in the
following way as a sum of operators and functions:

 
h̄ ∂ A ∂ ∂A h̄ ∂
A = L̂ A + − − pk . (65)
ı ∂qk ∂ pk ∂ pk ı ∂qk

Replacing A in (64) with the help of this formula, the expectation value (35) takes the form:


Āρ (t) = dω φ ∗ (ω, t) L̂ A φ(ω, t) + Δ Āρ (t). (66)

The first term is now in perfect quantum-like form, while the second terms stems from the difference between A
and L̂ A :

   
∗ h̄ ∂ A ∂ ∂A h̄ ∂
Δ Āρ (t) = dq d p φ (q, p, t) − − pk φ(q, p, t). (67)
ı ∂qk ∂ pk ∂ pk ı ∂qk

The set of eigenfunctions of the self-adjoint operator L̂ A constitutes a basis in the present Koopman–von Neumann
Hilbert space of square-integrable phase space functions. We assume for simplicity that the operator L̂ A has a
discrete spectrum; its eigenvalues and orthonormalized eigenfunctions are denoted by ak and φ̄ A,k (q, p). Expanding
φ(q, p, t) and φ ∗ (q, p, t) in this basis and performing some well-known steps [36], we obtain the representation:



2
Āρ (t) = ak
φ, φ̄ A,k
+ Δ Āρ (t). (68)
k

In deriving this result, we neglected again the possibility of degenerate states; appropriate generalizations can
usually be carried out without problems.
We obtained two quite different representations, Eqs. (62) and (68), for the fundamental expression (35). The first
expression (62) is much simpler than the second, and allows for an obvious interpretation in terms of measurement
results a and corresponding probabilities. Such an interpretation does not exist for the operator representation (68);
the numbers ak do not belong to the experimentally verifiable output of our classical theory and a probability density
describing the measurement of these numbers cannot be read off from (68).
There is a certain arbitrariness in the choice of the relevant operator, as mentioned already and as discussed in
more detail in I. For example, the Koopman–von Neumann operator D̂ A and the real-valued state variable ρ have
been used to derive a classical Born-like representation for the probabilities [39]. The present classical operator
representation, using L̂ A and complex-valued state variables, does not take the form of Born’s rule [because of the
term Δ Āρ (t)] . It represents, as shown below, the classical counterpart of Born’s rule, in the sense that the true
quantum Born rule can be derived from it.

123
94 U. Klein

11 The transition to quantum theory

Our set of basic classical equations in operator form contains the time-dependent evolution Eq. (55), the eigenvalue
Eq. (56), the commutation relations (60), and the operator form (68) for the expectation value of an observable. All
these relations are purely classical, but show a remarkable structural similarity with the most important relations of
QT. Such similarities may also be found in the original Koopman–von Neumann theory, where S is a solution of
the Liouville equation; see [40–44]. However, the present replacement of the Koopman–von Neumann operator D̂
by the operator L̂ makes this structural similarity much stronger.
Which modifications will lead us from these quantum-like relations to quantum relations? All fields of physics,
including the complex-valued wave functions of QT, are functions of q, t only and do not depend on the generalized
momenta p. This seems intuitively clear, because the variables q, t are directly measurable coordinates of our every
day’s “real” space–time environment. The p  s, on the other hand, are derived quantities which cannot be measured
directly. Thus, the coordinates q, t are most fundamental (they are of course not unique, but all possible coordinate
systems must be derivable from them). We may consider this as a principle of nature.
Starting from a classical ensemble of trajectories, we derived in this work a field theory with independent
variables q, p, t, which is in conflict with this principle of nature. To restore this principle, we have to get rid of the
coordinates p. A systematic way to do this, to be reported elsewhere, is to replace p by a momentum field depending
on q, t. However, our present quantum-like form of HLLK allows a simpler way to perform the transition, a kind
of short-cut derivation of QT.
Let us rederive for completeness the simple “quantization rules”, reported already in I, by considering the
evolution Eq. (55) in explicit form:
   
h̄ ∂ h̄ ∂ H ∂ ∂H h̄ ∂
− + − pk + H φ = 0. (69)
ı ∂t ı ∂qk ∂ pk ∂ pk ı ∂qk

The elimination of the variables pk in this equation may be performed by assuming that φ depends not on pk , and
by implementing the quantization rules:

∂ h̄ ∂
= 0, pk = (70)
∂ pk ı ∂qk

in the operator acting on φ(q, t). This step eliminates the second and third terms in the bracket of (69) and replaces
the observable H (q, p) by an operator Ĥ , according to the general rule:
 
h̄ ∂
A  Â = A q, . (71)
ı ∂qk

As a result, Eq. (69) becomes Schrödinger’s equation. Considering the evolution equation in the form (55), the
quantization process may be described in the even more compact form L̂  Ĥ . The corresponding general rule for
arbitrary observables is given by the following:

L̂ A  Â. (72)

We are using here the terms “quantization” and “quantization rule”, despite the fact that the present derivation
differs considerably from the common quantization procedure.
If the same quantization rule [(70) or (72)] is applied to the eigenvalue Eq. (56) of HLLK, one obtains the
eigenvalue equations of QT:

Âφ̄ A,k = ak φ̄ A,k , B̂ φ̄ B,k = bk φ̄ B,k , etc, (73)

123
From probabilistic mechanics to quantum theory 95

where the quantum operators Â, B̂, . . . are defined according to (71), and the eigenfunctions φ̄ A,k , φ̄ B,k , . . .,
associated with the eigenvalues ak , bk , . . . of the operators Â, B̂, . . ., depend only on q.
One might ask if this simple derivation of the most fundamental differential equations of QT is an accident, but
this question must be answered in the negative. As mentioned already, the transition to QT consists of two steps: A
replacement of the momenta by fields, and a linearization. The reason why the simple quantization rule (70) works
is the fact that the evolution Eq. (55) and the eigenvalue Eqs. (56) are already linear.
Let us turn now to the problem of several observables; it is sufficient to consider two observables. In HLLK,
the condition for the simultaneous measurement of two ensemble states, defined by the observables A and B [see
Eq. (59)], is the vanishing of the commutator of the operators L̂ A and L̂ B . This vanishing is a consequence of A
and B being in involution. Application of the quantization rule (72),

{A, B} = 0 ⇒ [ L̂ A , L̂ B ] = 0  [ Â, B̂] = 0, (74)

leads to the well-known condition [ Â, B̂] = 0 of QT, for the existence of common eigenfunctions of the operators Â
and B̂. As noted already in I, the complete structural similarity between Poisson brackets and commutator brackets,
which still holds true for the transition A(q, p) ⇒ L̂ A , breaks down under the transition (72) from phase space
to configuration space. Thus, the transition A  Â does, in general, not preserve the Lie bracket structure [23].
This is no surprise considering the enormous reduction in the number of degrees of freedom brought about by this
transition. Likewise, it is no surprise considering the fact that a single particle is different from an ensemble of
particles. It is however a surprise, and an inconsistency, if one believes that the classical counterpart of a “quantum
particle” is a single classical particle [45].
Let us finally turn to the quantization of the expectation value Āρ (t) represented either in the form (66) or (68).
In addition to the quantization rules (70), we have to assume now that the range of integration in all integrals is
restricted to configuration space. Implementing in Eq. (66) this additional assumption as well as the quantization
rules (70), we see that the term Δ Āρ (t) vanishes, and Āρ (t) reduces to the quantum-mechanical expectation value
of an operator  in a state φ:

  
 h̄ ∂
Āρ (t)  φ, Âφ = dq φ ∗ (q, t)A q, φ(q, t). (75)
ı ∂qk

The quantization of Āρ (t) in the form (68) leads in an analogous way to the quantum-mechanical representation of
the expectation value of  as a weighted sum over the eigenvalues of Â:



2
Āρ (t)  ak
φ, φ̄ A,k
. (76)
k

Here, ak and φ̄ A,k are to be understood as eigenvalues and eigenstates of the operator  and the inner product
is defined as a integral over configuration space. Both representations of Āρ (t) are equivalent; Eq. (76) may also
be obtained from (75) performing appropriate calculations in configuration space. From the representation (76),


2
Born’s rule, which says that
φ, φ̄ A,k
is the probability to obtain a measurement result ak of the observable  if
the considered system is in the state φ, can be immediately read off. The present derivation indicates that the terms
“system”, “observable”, and “measurement result” should be interpreted in the sense of statistical ensembles and
not in the sense of individual particles.
The derivation of these formulas completes the transformation from phase space to configuration space of the
most important formulas of HLLK, creating thereby the most important formulas of QT. These results verify the
conclusion obtained in Sect. 1, that QT must be a substructure of a probabilistic description of classical particles.
The present approach provides an explanation of QT which is of unusual simplicity.

123
96 U. Klein

12 Discussion

Quantum theory in its present form was invented during the first decades of the last century. The process of invention
was a combined effort of theory and experiment, adapting the theory in such a way that empirical requirements
were met whenever they arose. This way of proceeding led to a number of empirically verified formal results, but
gave no generally accepted interpretation, or derivation, of these results. A list of empirically verified results of QT
includes the following main points:
1. Schrödinger’s equation as fundamental dynamical law and eigenvalues as observable numbers.
2. The nonstandard probabilistic structure of QT—in particular non-commuting observables.
3. Born’s rule—the law which tells us how to extract probabilistic predictions from the theory.
4. The minimal-coupling rule—the way interactions are formulated in QT.
5. The existence of spin—a particularly mysterious phenomenon believed to belong to QT exclusively
6. The anomalous value of the magnetic moment of the electron—a spin-related phenomenon
7. The spin–statistics connection—a spin-related multi-particle phenomenon
All these results, as well as their associated problems of interpretation, belong to the realm of non-relativistic
few-particle QT. Taking into account a relativistic space–time structure or many degrees of freedom leads to more
precise predictions, but does not solve the associated problems of interpretation.
The fundamental assumption underlying the present work is the old idea—put forward by Einstein [46], Born [47],
and others—that the classical counterpart of QT might be a probabilistic theory. This offers—by means of the
simple counting of degrees of freedom performed in Sect. 2—a unexpected possibility, namely to understand QT
as a substructure of classical probabilistic physics. Planck’s constant h̄ was introduced as a consequence of the
projection of HLLK from phase space to configuration space. Prior to this projection, h̄ did not appear in any
prediction of HLLK; the dependence on h̄ was spurious. The appearance of a new fundamental constant h̄ may be
interpreted in different ways. According to a common “deterministic” metaphysics, shared by Einstein and others,
it is an indication of some deeper physics. From the “indeterministic” metaphysical point of view, promoted for
example by Popper [48], it realizes just an accuracy limit—indicating the elimination of the unrealistic assumption
of arbitrary high accuracy.
In the present work, we showed, by deriving QT from HLLK, that QT is a substructure of an extended version
of probabilistic classical physics. The very success of such a calculation represents, of course, a strong argument in
favor of Einstein’s ensemble interpretation of QT [1,46,49]. As regards questions of interpretation, we do not want
to go into details here; let us just note that our results do not support concepts like “completeness” or “nonlocality”
of QT [50]. Summarizing, we could say “Einstein was right”, quoting the title of a recent careful analysis [51] of
Bell’s inequality.
The present derivation is based on a few general and reasonable assumptions, such as the superiority of a
probabilistic description as compared to a deterministic description, and the superiority of the space–time coordinates
as compared to the space–momentum–time coordinates. We were able to derive some crucial features of QT, namely
those corresponding to points 1–3 in the above list. While there are still open points, it seems that such a coherent
derivation of fundamental concepts of QT has never been obtained before. We expect, therefore, that a complete
derivation of QT, including all points in the above list, is possible. However, it is clear that such a complete derivation
of QT requires a more detailed analysis of the steps leading to QT than given here.

13 Concluding remarks

One of the most fundamental conclusions of the present work is the following: not classical physics “emerges”
from QT in the limit h̄ → 0, but the inverse is true; QT emerges from the classical theory HLLK. The new quantum
constant h̄ appears as a consequence of a projection of HLLK to configuration space. Using a more philosophical
terminology, we might say that the otherwise powerful principle of reductionism is not useful for clarifying the
relation between classical physics and QT.

123
From probabilistic mechanics to quantum theory 97

The question why Schrödinger’s equation is complex is sometimes considered as crucial for the understanding
of QT. From the present point of view, the complexity is a consequence of the requirement of linearity of the basic
equation. All fundamental field equations of physics (equations not containing any additional material parameters)
must be linear. Otherwise, they would be unable to describe a large number of individual events—simply because
almost all solutions of nonlinear equations become singular after a sufficient long time. A linear theory containing
two real-valued dynamical variables can only be constructed if a complex-valued state variable is introduced.
In the present theory with two real variables, we derived as a special case, for N = 1, Schrödinger’s equa-
tion describing single particle ensembles of “massive spinless particles”. According to present-day’s experimental
evidence, such objects do not exist; all structure-less massive particles found in nature have spin one-half. This
indicates that the present derivation contains an error. This error is implicitly contained in the present somewhat
crude projection to configuration space. A more careful projection contains as a first step a replacement of the
momenta by fields depending on q, t. If this replacement is performed in a correct way, one obtains for a single
particle (ensemble) a theory with four real variables, which means that a single particle has spin one-half. Details
will be reported in forthcoming work.

Acknowledgements Open access funding provided by Johannes Kepler University Linz.

Compliance with ethical standards

Conflicts of interest The author states that there is no conflict of interest.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://
creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you
give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes
were made.

References

1. Ballentine, L.E.: The statistical interpretation of quantum mechanics. Rev. Mod. Phys. 42, 358 (1970)
2. Van Vleck, J.H.: The correspondence principle in the statistical interpretation of quantum mechanics. Proc. Natl. Acad. Sci. U.S.
14, 178 (1928)
3. Bopp, R.: La mecanique quantique est-elle une mecanique statistique classique particuliere? Ann. Inst. H. Poincare 15, 81 (1956)
4. Bopp, F.: The principles of the statistical equations of motion on quantum theory. In: Körner, S. (ed.) Observation and Interpretation,
p. 189. Butterworths Scientific Publications, London (1957)
5. Schiller, R.: Quasi-classical theory of the nonspinning electron. Phys. Rev. 125(3), 1100 (1962)
6. Hayakawa, S.: Atomism and cosmology. Suppl. Progr. Theor. Phys. 34, 533 (1965)
7. Takahashi, K.: Distribution functions in classical and quantum physics. Progr. Theor. Phys. Suppl. 98, 109 (1989)
8. Olavo, L.S.F.: Foundations of quantum mechanics: non-relativistic theory. Phys. A 262, 197 (1999)
9. Bolivar, A.O.: The Wigner representation of classical mechanics, quantization and classical limit. Phys. A 301, 219 (2001)
10. Wetterich, C.: Quantum particles from classical probabilities in phase space. Int. J. Theor. Phys. 51, 3236 (2012)
11. Reginatto, M.: Derivation of the equations of nonrelativistic quantum mechanics using the principle of minimum Fisher information.
Phys. Rev. A 58, 1775 (1998)
12. Hall, M.J., Reginatto, M.: Schrödinger equation from an exact uncertainty principle. J. Phys. A 35, 3289 (2002)
13. Klein, U.: Schrödinger’s equation with gauge coupling derived from a continuity equation. Found. Phys. 39, 964 (2009)
14. Skala, L., Cizek, J., Kapsa, V.: Quantum mechanics as applied mathematical statistics. Ann. Phys. 326, 1174 (2011)
15. Klein, U.: The statistical origins of quantum mechanics. Phys. Res. Int. 2010, 808424 (2010). https://doi.org/10.1155/2010/808424
16. Klein, U.: A statistical derivation of non-relativistic quantum theory. In: Measurements in Quantum Mechanics, ed. by M.R.
Pahlavani (ISBN:978-953-51-0058-4, 2012), pp. 141–174. See also arXiv:1109.6244 [quant-ph]
17. Raedt, H.D., Katnelson, M.I., Michielsen, K.: Quantum theory as the most robust description of reproducible experiments. Ann.
Phys. 347, 45 (2014)
18. Bohr, N.: Niels Bohr Collected Works Volume 6, Foundations of Quantum Physics I (1926-1932) (North-Holland, Amsterdam,
1985). See p. 99
19. Klein, U.: From Koopman-von Neumann theory to quantum theory. Quantum Stud. Math. Found. 5, 219 (2018)
20. Sudarshan, E.C.G., Mukunda, N.: Classsical Dynamics: A Modern Perspective. Wiley, New York (1974)
21. Dirac, P.A.M.: The Principles of Quantum Mechanics. Oxford University Press, Oxford (1947)

123
98 U. Klein

22. Klein, U.: What is the limit h̄ → 0 of quantum theory? Am. J. Phys. 80, 1009 (2012)
23. Groenewold, H.J.: On the principles of elementary quantum mechanics. Physica 12, 405 (1946)
24. Koopman, B.O.: Hamiltonian Systems and Transformations in Hilbert Space. Proc. Natl. Acad. Sci. U.S.A. 17, 315 (1931)
25. Weyl, H.: Quantenmechanik und Gruppentheorie. Z. Phys. 46, 1 (1927)
26. Wigner, E.P.: On the quantum correction for thermodynamics equilibrium. Phys. Rev. 40, 749 (1932)
27. Moyal, J.E.: Quantum mechanics as a statistical theory. Proc. Camb. Philos. Soc. 45, 99 (1949)
28. Ballentine, L.E.: Quantum Mechanics. Prentice Hall, Englewood Cliffs, NJ (1989). See chapter 15
29. Balazs, N.L., Jennings, B.K.: Wigner’s function and other distribution functions in mock phase spaces. Phys. Rep. 104, 347 (1984)
30. Bennett, A.: Lagrangian fluid dynamics. Cambridge University Press, Cambridge (2006)
31. Rylov, Y.A.: Pauli’s electron as a dynamic system. Found. Phys. 25, 1055 (1995)
32. Caratheodory, C.: Calculus of Variations and Partial Differential Equations of the First Order, Part I. Holden-Day Inc, San Francisco
(1965)
33. Hall, B.C.: Quantum Theory for Mathematicians. Springer, New York (2013)
34. Ash, R.: Basic Probability Theory. Dover Publications, Mineola, New York (2008)
35. Wallstrom, T.C.: Inequivalence between the Schrödinger equation and the Madelung hydrodynamic equations. Phys. Rev. A 49(3),
1613 (1993)
36. Cohen-Tanoudji, C., Diu, B., Laoë, F.: Quantum Mechanics. Wiley, Hoboken (1998)
37. Wollenberg, L.S.: Derivations of the Lie algebra of polynomials under Poisson bracket. Proc. Am. Math. Soc. 20, 315 (1969)
38. Fasano, A., Marmi, S.: Analytical Mechanics—An Introduction. Oxford University Press, New York (2006)
39. Brumer, P., Gong, J.: Born rule in quantum and classical mechanics. Phys. Rev. A 73, 052109 (2006)
40. Prigogine, I.: Non-equilibrium Statistical Mechanics. Interscience, New York (1962)
41. Jaffe, C., Brumer, P.: Classical Liouville mechanics and intramolekular relaxation dynamics. J. Phys. Chem. 88, 4829 (1984)
42. Wilkie, J., Brumer, P.: Quantum-classical correspondence via Liouville dynamics. I Integrable systems and the chaotic spectral
decomposition. Phys. Rev. A 55, 27 (1997)
43. Mauro, D.: On Koopman–von Neumann waves. Int. J. Mod. Phys. A 17, 1301 (2002)
44. Gozzi, E., Mauro, D.: On Koopman–von Neumann waves II. Int. J. Mod. Phys. A 19, 1475 (2004)
45. Dirac, P.A.M.: The Principles of Quantum Mechanics, 4th edn, p. 88. Oxford University Press, Oxford (1958)
46. Einstein, A.: In: Schilpp, P.A. (ed.) Albert Einstein: Philosopher-Scientist, p. 665. Harper and Row, New York (1949)
47. Born, M.: Vorhersagbarkeit in der Klassischen Mechanik. Zeitschrift für Physik 153, 372 (1958)
48. Popper, K.R.: The Open Universe—An Argument for Indeterminism. Rowman and Littlefield, Totowa (1982)
49. Klein, U.: Is the individuality interpretation of quantum theory wrong ? ArXiv:1207.6215 [quant-ph], see also http://statintquant.
net
50. Perlman, H.S.: Quantum mechanics is incomplete but is consistent with locality. Found. Phys. 47, 1309 (2017). https://doi.org/10.
1007/s10701-017-0111-6
51. Hess, K.: Einstein was right. CRC Press, 6000 Broken Sound Parkway NW (2015)

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

123