Towards Flight Control Verification Using Automate
Towards Flight Control Verification Using Automate
net/publication/221413036
CITATIONS READS
12 380
4 authors, including:
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Luis Rodrigues on 27 March 2014.
1 Introduction
Modern commercial passenger aircraft are extremely complex systems and their
designs must meet strict design and safety requirements. The Federal Aviation
Administration (FAA) specifies that the catastrophic failure rate of a passen-
ger aircraft digital flight-control system must be extremely improbable (less than
10−9 faults per hour) [1]. However, the system must be built using embedded
computers, sensors, actuators and control components each with individual fail-
ure rates several orders of magnitude higher than that of the level set by the
FAA. A combination of redundancy and fault tolerance must therefore be used
to achieve this strict reliability requirement.
In general, aircraft are verified using simulation methods. A mathematical
model based on the physical equations of flight is constructed and then simulated.
An extensive analysis of the experimental results is necessary to ensure a robust
result. There are several graphical aids such as Nyquist diagrams and Nichols
plots [9] that are commonly used to simplify this task. These techniques provide
easily identifiable zones for which the plot should not pass near or enter, clearly
indicating the control system’s margin of stability [11]. However these graphical
methods still require visual analysis to process the information.
M. Bobaru et al. (Eds.): NFM 2011, LNCS 6617, pp. 89–100, 2011.
c Springer-Verlag Berlin Heidelberg 2011
90 W. Denman et al.
Even though there are over 78,000 flights without incident per day [10], we
cannot assume that the current verification methods are perfectly sound. The
first issue with this conclusion is that with simulation alone it is not possible
to give 100% safety assurance due to the great number of variations of the
model components and parameters. There will always be the possibility of a
catastrophic failure due to design errors. Second, to achieve the FAA’s failure rate
a complex multi-domain, labour intensive and costly process must be undertaken.
It is therefore quite important to investigate methods that will reduce the effort
and cost of the verification process while ensuring the reliability of the results.
Formal verification is a method where logical reasoning can be used to prove
that the implementation of a system correctly matches its design specification.
Unlike simulation, a formal proof is valid regardless of the input test cases.
There have been several breakthroughs in formal analysis of discrete systems.
Systems of large orders of magnitude can now be verified. The tools and methods
available for the formal verification of continuous and hybrid-systems cannot
handle systems at the same level of complexity. This is one major hurdle that has
limited the application of formal methods to the physical portion of aeronautical
models.
MetiTarski [2] is an automatic theorem prover for real-valued analytical func-
tions, including trigonometric and exponential functions. It works by a combi-
nation of resolution inference and algebraic simplification, invoking a decision
procedure (QEPCAD) [5] to prove polynomial inequalities over the real closed
filed (RCF). The output of MetiTarski is a complete proof that contains alge-
braic simplification and decision procedure calls that can be verified using other
tools.
This paper illustrates a methodology for ensuring the stability of a flight con-
trol system by performing a formal analysis of a Nichols plot using the MetiTarski
automated theorem prover. A Nichols plot is a transfer function’s gain plotted
versus its phase. Information about the stability of a system can be deduced
from a visual inspection of the plot. The formal analysis we present removes
the need for drawing and checking the Nichols plot visually. We present our
investigations on verifying the lateral autopilot of a Model 24 Learjet subsonic
business jet (SBJ) [4]. The control system model was implemented in Simulink
and the goal of our proposed verification methodology is to supplement design
work-flows that depend on the Matlab/Simulink Control Systems Toolbox [15].
The rest of the paper is organized as follows, we first discuss related work
in Sect. 2. A description of MetiTarski and its syntax is presented in Sect. 3.
Details of the proposed methodology are given in Sect. 4. This is followed by the
case study in Sect. 5, before concluding the paper with Sect. 6.
2 Related Work
The bulk of the work on formal verification for aeronautical systems has been on
the software components of flight control. Nevertheless, there have been several
interesting advancements on the verification of hybrid systems [16]. From those
Towards Flight Control Verification Using Automated Theorem Proving 91
latest results and experiments, it is obvious that they will ultimately play a
strong role in the complete formal verification of aircraft autopilots.
Hardy [7] developed and implemented a decision procedure to reason about
functions that have a finite number of inflection points. This decision procedure
was implemented in the Nichols plot Requirements Verifier (NRV) to perform an
automated formal Nichols plot analysis. The tool was developed using the com-
puter algebra system Maple, the formal theorem prover PVS and the quantifier
elimination system QEPCAD [5]. NRV was successfully applied to two classic
control system examples: an inverted pendulum and a disk drive reader. Our
work is closely related to that of Akbarpour and Paulson [3] who successfully
formally verified these two examples using MetiTarski. Our main contribution
is to remove the required inflection point analysis. We prove over all frequency
values that the exclusion region is not entered, not just at single points. This is
particularly important when dealing with exclusion regions that are not bounded
by linear constraints. In particular, in the analysis of ellipsoid exclusion regions
Hardy’s [7] inflection point analysis does not hold.
SOSTOOLS [14] is a Matlab toolbox that can convert difficult optimiza-
tion problems into a sum of squares formulation that can then be analyzed
by a convex optimization technique known as semi-definite programming. It has
widespread use in the nonlinear control field. In particular, it can be used to
search for a Lyapunov function that can be used to verify the stability of dy-
namical systems. For a particular equilibrium to be stable, it is required that
the candidate Lyapunov function V be positive definite and its derivative with
respect to time be negative semi-definite [8]. SOSTOOLS can be used to prove
the un-satisfiability of systems of non-linear polynomial equations and inequal-
ities over the real numbers [12]. For many problems, SOSTOOLS could replace
QEPCAD as the polynomial reasoning engine under MetiTarski. This would not
be trivial to implement effectively. Nevertheless, improvements to the theory
behind SOSTOOLS would have the potential to enhance MetiTarski.
for instance the code in Fig. 1. The “fof ” keyword indicates to MetiTarski that
the logic language used is a first-order formula. It is then followed by a label
of the proof as well as the keyword “conjecture” indicating that the following
formula is to be proved with the included axioms. The conjecture is read as
follows: For all (!) X between 0 and 2.39 × 10−9 the formula is always less than
0.03. For a syntax guide see Table 1.
fof(
example1,conjecture, ! [X] :
(
(0 <= X & X <= 2.39*10^(-9)) =>
-0.0059 - 0.000016*exp(-2.55*10^8*X) + 0.031*exp(-5.49*10^7*X)
< 0.03
)
).
include(’Axioms/general.ax’).
include(’Axioms/exp-upper.ax’).
include(’Axioms/exp-lower.ax’).
3.2 Axioms
In addition to the problem definition, the required axioms must be chosen using
the ’include’ keyword. It is critical that only axioms files for functions in the
problem definition are included. Each additional set of axioms can greatly in-
crease the time taken by MetiTarski to complete the proof. For example, there
are two sets of axiom declarations for the exponential function. One for regular
bounds and one for extended bounds. The extended bounds are used in cases
where a higher level of precision is needed.
There have been cases where including the extended bounds will make the
inequality test run until manually stopped. In that specific example, removing
Towards Flight Control Verification Using Automated Theorem Proving 93
the extended axioms allowed MetiTarski to complete the proof in seconds. The
inverse can also happen, if for instance the TPTP description contains trigono-
metric functions and those axioms are not included, then MetiTarski will never
terminate. To mitigate this situation when running MetiTarski on a set of prob-
lems, as is done in the case study investigated in this paper, a CPU time limit
can be set. Deeper analysis is then required to choose the correct axioms for
those problems that were not proved.
There are automated scripts included in the MetiTarski distribution that can
insert the axioms directly into the TPTP file description. This enables a low level
analysis of the problem where specific axioms can be isolated and removed. This
axiom weeding out procedure is currently manual, but by doing so has led to
proofs for functions with extremely large arguments, such as arctan(1025 ×X 16 ).
4 Proposed Methodology
An important verification property is to ensure that a system under design is
stable. Negative feedback is commonly used to achieve this. In this configuration,
the difference between the system’s current output and what is required is used
to steer the output to the correct value. Time delays around the feedback loop
can still cause the system to remain unstable. An in-depth stability analysis of
the feedback system is thus quite essential in the design process.
Classic control theory provides several graphical methods to assess the stabil-
ity of feedback systems: the Bode diagram, the Nyquist plot and the Nichols plot.
The idea behind these graphical methods is to show visually how much margin
the system has against instability [9]. Note that it is the analysis of the open-loop
response that reveals information on the stability of the closed-loop system. The
feedback loop must be “broken” to analyze how the signal is processed along the
signal loop path.
In this paper, we are concerned with the analysis of a Nichols plot. This type
of plot is commonly used in the analysis of flight control laws [6] and requires
repeated visual inspection. Our goal is to automate this analysis and provide a
formal proof guaranteeing the results.
A Nichols plot is constructed by plotting the gain (in decibels) on the x-axis
and the phase shift (in radians) on the y-axis of a Cartesian plane. If the system
is described using the transfer function G(jw) then the following equations are
used to construct the Nichols plot.
Im(G(jw))
x = arctan (1)
Re(G(jw))
Gain (dB)
10
-5
-10
The conditions to remain outside of the edges of the exclusion region are
defined as
12 5
y> π + 18 from (− 4 π, 3) to (−π, 6)
y< π − 18 from (− 4 π, −3) to (−π, −6)
− 12 5
y> − 12
π − 6 from (− 4 π, 3) to (−π, 6)
3
the bounds can be even more tightly chosen to determine the quality of the
flight control in terms of handling and response to pilot commands. This will be
discussed in more detail below.
The following step is the conversion of the bounds of the exclusion region (in
terms of decibels and radians) into inequalities described in terms of the transfer
function (frequency domain) using Maple. MetiTarski is first used to verify the
results that Maple produces. The resulting expressions for each boundary of the
exclusion region are then processed by MetiTarski which automatically generates
a proof if it can determine that the inequality holds. This resulting proof indicates
that the Nichols plot curve never enters the defined exclusion region.
If MetiTarski is successful, it delivers a proof and we are done. If unsuccessful,
it will run until terminated by the user. In the most recent version of MetiTarski
(v1.8) it is possible for the user to specify a CPU time limit on the proof. In the
event of the CPU limit being reached, we must consider modifying the exclusion
region. This has the effect of reducing the required stability margins. A relaxation
of the exclusion region can be performed automatically when the CPU limit is
reached.
The benefit of this method compared to other aeronautical verification meth-
ods is two-fold. First, there is no need to visually inspect any of the plots. If
MetiTarski returns that the proof is true then we can be sure that the specifica-
tion is met. MetiTarski also operates automatically on the continuous range of
variables.
96 W. Denman et al.
The input to linmod is the Simulink design where an input port and an output
port have been explicitly defined. The output is a state space model, ẋ = Ax +
1 0.1 da
p
In1
Hphi
phi 1
dr phi
Out1
psi
SBJ4
Washout3
s
s+1
Bu, y = Cx+Du. Then using the Matlab command ss2tf , the state space model
is converted into a transfer function G(s) and G(jw) is obtained by replacing
instances of the variable s with jw.
The gain and phase of the system with the transfer function G(jw) are com-
puted as described in (1) and (2), see Sect. 4.
The next step is to select the exclusion region of the Nichols plot as described
before. At the most basic level, we can choose a hexagonal region that is centered
around the point (-π,0) which is shown in Fig. 5.
Gain (dB)
10
-5
-10
Now that the Nichols exclusion region has been defined, Maple is used to solve
for the frequencies where the Nichols plot passes through the endpoints of the
exclusion region. We use MetiTarski to ensure that Maple’s computations are in
fact correct.
The interval [−3, 3] of the gain (y-axis of Nichols plot), corresponds to the
interval w ∈ [23080/32333, 75843/46168] in the frequency domain. MetiTarski is
used to show that outside this frequency interval, we have (y ≥ 3) ∨ (y ≤ −3).
Then to show that the exclusion region is never entered from the right middle
segment, MetiTarski proves that
The interval [−π, −3π/4] of the phase (x-axis of the Nichols plot) corresponds
to the interval w ∈ [42049/14953, 978208/3695] in the frequency domain. Meti-
Tarski is used to show that outside this frequency interval, we have (x ≥ −3π/4)∨
(x ≤ −π). Then to show that the exclusion region is never entered from the bot-
tom right segment, MetiTarski proves that
12
∀w. w < 978208/3695 ∧ w > 42049/14953 ⇒ y < − x+6
π
98 W. Denman et al.
From the results obtained from Maple and MetiTarski, we can infer that the
Nichols plot does not pass through any other points of the exclusion region and
thus does not pass through any of the other four boundaries. A snapshot of the
code used to prove this fact is shown in Fig. 6.
The experimental results are shown in Table 2. For the “right-middle” ex-
periments, U and L indicate the upper and lower points at which the transfer
function could possibly enter the exclusion region. When an experiment is split
into multiple sub-experiments (1,2,3,4), this indicates that the phase function is
taking on different values due to arctan being defined only over (−π/2, π/2). The
“check” experiments are verifying Maple’s output. The “exclusion” experiments
are verifying that the transfer function does not enter the exclusion region. The
runtimes were measured on a 2.8 GHz Dual Quad-Core Mac Pro, with 4GB of
RAM. The middle boundary proofs completed faster because they are defined
using only the ln function, about which MetiTarski can reason very efficiently.
The right bottom boundary is defined using a combination of both the arctan
and ln functions, which is more difficult to reason about primarily because of
the extremely large values that their arguments take. The difference between
proof times is not problematic because the final positive result is eventually ob-
tained. Further improvements to the axioms used by MetiTarski, will ultimately
improve the proof times.
fof(Nichols-Exclusion,! [X] :
((X > 0.9582 & X < 2.86) =>
10/ln(10)*ln(0.25*10^(-24)*
(3862622500*X^20+0.3566432250*10^41*X^18+
... + 0.8478030764*10^17*X^8)))
< -6+(12/pi)*arctan(0.2*10^(-3)*(-6100459+
...+0.246*10^25*X^16)))
)).
Fig. 6. MetiTarski Input for Proving Lower Right Edge of the Exclusion Region
Towards Flight Control Verification Using Automated Theorem Proving 99
Acknowledgments
We would like to thank the following people: Kyungjae Baik from Concordia
University for providing us with the Simulink control system design we used for
the case study. Dr. Lawrence Paulson from the University of Cambridge for his
help with MetiTarski.
References
1. Advisory Circular: System design and analysis. Tech. rep., Federal Aviation Ad-
minisration (1988)
2. Akbarpour, B., Paulson, L.C.: MetiTarski: An automatic prover for the elementary
functions. In: Autexier, S., Campbell, J., Rubio, J., Sorge, V., Suzuki, M., Wiedijk,
F. (eds.) AISC 2008, Calculemus 2008, and MKM 2008. LNCS (LNAI), vol. 5144,
pp. 217–231. Springer, Heidelberg (2008)
100 W. Denman et al.