Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Automated Specification Analysis Using an Interactive Theorem Prover

Harsh Raju Chamarthi and Pete Manolios
Northeastern University

October 31, 2011

1 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Motivation
Teaching freshmen how to reason about programs using ACL2s .

2 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Motivation
Teaching freshmen how to reason about programs using ACL2s Success of QuickCheck .

C .

rexa ounte

mple

s!!

3 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Motivation
Teaching freshmen how to reason about programs using ACL2s Success of QuickCheck Combining testing and theorem-proving (ACL2 2011 Workshop) .

C .

rexa ounte

mple

s!!

4 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Motivation
Teaching freshmen how to reason about programs using ACL2s Success of QuickCheck Combining testing and theorem-proving (ACL2 2011 Workshop) Can we do even better? .

C .

rexa ounte

mple

s!!

5 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Motivation
Teaching freshmen how to reason about programs using ACL2s Success of QuickCheck Combining testing and theorem-proving (ACL2 2011 Workshop) Can we do even better? Apply technology behind ACL2 to help the regular programmer .

C .

rexa ounte

mple

s!!

6 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Motivation
Teaching freshmen how to reason about programs using ACL2s Success of QuickCheck Combining testing and theorem-proving (ACL2 2011 Workshop) Can we do even better? Apply technology behind ACL2 to help the regular programmer .

C .

rexa ounte

mple

s!!

.
7 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Overview
Goal
Analyse specifications - Find counterexamples!

8 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Overview
Goal
Analyse specifications - Find counterexamples!

The problem
What to do when the Search Procedure doesnt return an answer?

9 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Overview
Goal
Analyse specifications - Find counterexamples!
QuickCheck .

The problem
What to do when the Search Procedure doesnt return an answer?

10 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Overview
Goal
Analyse specifications - Find counterexamples!
QuickCheck .

The problem
What to do when the Search Procedure doesnt return an answer?
Decision Procedure .

Constraint Solver .

11 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Overview
Goal
Analyse specifications - Find counterexamples!
QuickCheck .

The problem
What to do when the Search Procedure doesnt return an answer?
Decision Procedure .

The main idea

Reduce the search space and guide the procedure towards a counterexample .

Constraint Solver .

12 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Search Proc .

Property P .

Is P false? .

13 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Search Proc .

Property P .

Is P false? .

Yes .

14 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Search Proc .

. Yes .

Property P .

Is P false? .

15 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Search Proc .

Property P .

Is P false? .

Dont Know .

16 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Search Proc .

. Dont Know .

Property P .

Is P false? .

17 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Search Procedure .

Property P .

Is P false? .

18 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Guide . Property P .

19 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Guide . Property P . P . .
x1 , x2 , . . . , xn .

Is P false? .

20 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Guide . Property P . P . .
x1 , x2 , . . . , xn .

Is P. . Sfalse? elect var

21 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Guide . Property P . P . .
x1 , 3, . . . , xn .

Is P. . Sfalse? elect var Assign value .

22 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Guide . Property P . P . . x1 , . . . , xn . . Is P. . Sfalse? elect var Assign value . Simplify using ITP .

23 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Guide . Property P . P . . x1 , . . . , xn . . Is P. . Sfalse? elect var Assign value . Simplify using ITP .

Inconsistent?

24 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Guide . Property P . P . .
x1 , x2 , . . . , xn .

Is P false? . Backtrack!! .

25 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Guide . Property P . P . . x1 , . . . , xn . .
Search .

Is P false? .

Is P' false? .

26 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

ITP .

Is P true? .

P .

27 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

ITP .

Yes .

Is P true? .

P .

28 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

. Yes .
ITP .

Is P true? .

P .

29 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

ITP .

Dont Know .

Is P true? .

P .

30 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

. Dont Know .
ITP .

Is P true? .

P .

31 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

ITP .

Is P true? .

P .

. p1 . p2 . . . . pn .
32 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The Main Idea

Search . ITP .

Is P false? .

Is P true? .

P .

. p1 . p2 . . . . pn .
33 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Assumptions
Specification language L
Multi-sorted first-order logic Extensible Executable

34 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Assumptions
Specification language L
Multi-sorted first-order logic Extensible Executable

35 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Assumptions
Specification language L
Multi-sorted first-order logic Extensible -- can introduce new function and predicate symbols using well-founded recursive definitions Executable

36 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Assumptions
Specification language L
Multi-sorted first-order logic Extensible Executable

37 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Assumptions
Specification language L
Multi-sorted first-order logic Extensible Executable

Properties of form hyp1 hypn concl

No nested quantifiers Implicitly universally quantified

38 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Assumptions
Specification language L
Multi-sorted first-order logic Extensible Executable

Properties of form hyp1 hypn concl

No nested quantifiers Implicitly universally quantified

39 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Assumptions
Specification language L
Multi-sorted first-order logic Extensible Executable

Properties of form hyp1 hypn concl

No nested quantifiers Implicitly universally quantified

An Interactive Theorem Prover (ITP) that can reason about specifications in L.

Smash Simplify

40 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Assumptions
Specification language L
Multi-sorted first-order logic Extensible Executable

Properties of form hyp1 hypn concl

No nested quantifiers Implicitly universally quantified

An Interactive Theorem Prover (ITP) that can reason about specifications in L.

Smash takes as input a goal, a formula written in L, and returns a list of equi-valid subgoals. Simplify

41 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Assumptions
Specification language L
Multi-sorted first-order logic Extensible Executable

Properties of form hyp1 hypn concl

No nested quantifiers Implicitly universally quantified

An Interactive Theorem Prover (ITP) that can reason about specifications in L.

Smash takes as input a goal, a formula written in L, and returns a list of equi-valid subgoals. Simplify takes as input an L-formula, c, and a list of formulas, H, and returns a simplified formula that is equivalent to c assuming H are true.

42 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Example
Select ? .

P .

Property x = hash(y) y = hash(z)

. x + y = v2
z > 10 w < min(x, y) w < z

A . .

43 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Example
Select ? .
X .

P .

W .

Y .

Property x = hash(y) y = hash(z)

V . Z .

. x + y = v2
z > 10 w < min(x, y) w < z

Equality dependency Graph .

A . .

44 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Example
Select ? .
X .

P .

W .

Y .

W . V .

Property x = hash(y) y = hash(z)

V . Z . Z .

. x + y = v2
z > 10 w < min(x, y) w < z

Equality dependency Graph .

Rest dependency G .

A . .

45 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Example
Select z .
X .

P .

W .

Y .

W . V .

Property x = hash(y) y = hash(z)

V . Z . Z .

. x + y = v2
z > 10 w < min(x, y) w < z

Equality dependency Graph .

Rest dependency G .

A . .

46 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Example
Assign ? .

X .

P .

W .

Y .

W . V .

Property x = hash(y) y = hash(z)

V . Z . Z .

. x + y = v2
z > 10 w < min(x, y) w < z

Equality dependency Graph .

Rest dependency G .

A . .

47 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Example
Assign z = 34 (decision) .

. P' .
Property x = hash(y) y = hash(34) x + y = v2 w < min(x, y)

A . .

w < 34
48 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Example
Propagate with Simplify .

. P' .
Property x = hash(y) y = hash(34) x + y = v2 w < min(x, y)

A . .

w < 34
49 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Example
Propagate with Simplify .

P' .

Property x = 3623878690 y = 268959709

2 . x + 268959709 = v

if(x < 268959709) w<x w < 268959709 w < 34

A . .

50 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Example
Inconsistent? .

P' .

Property x = 3623878690 y = 268959709

2 . x + 268959709 = v

if(x < 268959709) w<x w < 268959709 w < 34

A . .

51 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Example
Update Assignment .

P' .

Property x = 3623878690 y = 268959709

2 . x + 268959709 = v

if(x < 268959709) w<x w < 268959709 w < 34

z = 34 A . .

52 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Top-level Analyze algorithm

Input: Property P, Summary summary Output: Status, Summary of the analysis of P if P is closed then return AnalyzeConst(P) initialize . Search till ... if StopCond(summary) then return (done, summary) Decompose P into subgoals using ITP If progress then Recurse on each subgoal . return (not-done, summary)

53 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Top-level Analyze algorithm

Input: Property P, Summary summary Output: Status, Summary of the analysis of P if P is closed then return AnalyzeConst(P) initialize . Search till ... if StopCond(summary) then return (done, summary) Decompose P into subgoals using ITP If progress then Recurse on each subgoal . return (not-done, summary)

54 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Top-level Analyze algorithm

Input: Property P, Summary summary Output: Status, Summary of the analysis of P if P is closed then return AnalyzeConst(P) n, status := 0, not-done . Search till ... if StopCond(summary) then return (done, summary) Decompose P into subgoals using ITP If progress then Recurse on each subgoal . return (not-done, summary)

55 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Top-level Analyze algorithm

Input: Property P, Summary summary Output: Status, Summary of the analysis of P if P is closed then return AnalyzeConst(P) initialize . Search till ... if StopCond(summary) then return (done, summary) Decompose P into subgoals using ITP If progress then Recurse on each subgoal . return (not-done, summary)

56 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Top-level Analyze algorithm

Input: Property P, Summary summary Output: Status, Summary of the analysis of P if P is closed then return AnalyzeConst(P) initialize while StopCond(summary) n slimit do A, n := Search(P), n + 1 . summary := updateA(summary, P, A) if StopCond(summary) then return (done, summary) Decompose P into subgoals using ITP If progress then Recurse on each subgoal . return (not-done, summary)

57 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Top-level Analyze algorithm

Input: Property P, Summary summary Output: Status, Summary of the analysis of P if P is closed then return AnalyzeConst(P) initialize . Search till ... if StopCond(summary) then return (done, summary) Decompose P into subgoals using ITP If progress then Recurse on each subgoal . return (not-done, summary)

58 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Top-level Analyze algorithm

Input: Property P, Summary summary Output: Status, Summary of the analysis of P if P is closed then return AnalyzeConst(P) initialize . Search till ... if StopCond(summary) then return (done, summary) Decompose P into subgoals using ITP If progress then Recurse on each subgoal . return (not-done, summary)

59 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Top-level Analyze algorithm

Input: Property P, Summary summary Output: Status, Summary of the analysis of P if P is closed then return AnalyzeConst(P) initialize Search till ... . if StopCond(summary) then return (done, summary) S := Smash(P) summary := updateS(summary, P, S) If progress then Recurse on each subgoal . return (not-done, summary)

60 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Top-level Analyze algorithm

Input: Property P, Summary summary Output: Status, Summary of the analysis of P if P is closed then return AnalyzeConst(P) initialize . Search till ... if StopCond(summary) then return (done, summary) Decompose P into subgoals ... If progress then Recurse on each subgoal . return (not-done, summary)

61 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Top-level Analyze algorithm

Input: Property P, Summary summary Output: Status, Summary of the analysis of P if P is closed then return AnalyzeConst(P) initialize Search till ... if StopCond(summary) then return (done, summary) . Decompose P into subgoals ... if S = {P} then for all p S do status, summary := Analyze(p, summary) if status = done then return (done, summary) return (not-done, summary) .

62 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Top-level Analyze algorithm

Input: Property P, Summary summary Output: Status, Summary of the analysis of P if P is closed then return AnalyzeConst(P) initialize . Search till ... if StopCond(summary) then return (done, summary) Decompose P into subgoals ... If progress then Recurse on each subgoal ... . return (not-done, summary)

63 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) Initialize and Select first variable Iteratively construct counterexample or fail

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) Initialize and Select first variable Iteratively construct counterexample or fail

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) A, i, x := [], 0, Select(P) Iteratively construct counterexample or fail

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) A, i, x := [], 0, Select(P) Iteratively construct counterexample or fail

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) A, i, x := [], 0, Select(P) while true do v, t := Assign(x, P) . P := Propagate(x, v, P) if inconsistent(P ) then Extend A, continue search if not done else if A = [] then backtrack fail if ... .

68 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) A, i, x := [], 0, Select(P) while true do v, t := Assign(x, P) . P := Propagate(x, v, P) if inconsistent(P ) then Extend A, continue search if not done else if A = [] then backtrack fail if ... .

69 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) A, i, x := [], 0, Select(P) while true do v, t := Assign(x, P) . P := Propagate(x, v, P) if inconsistent(P ) then Extend A, continue search if not done else if A = [] then backtrack fail if ... .

70 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) A, i, x := [], 0, Select(P) while true do v, t := Assign(x, P) . P := Propagate(x, v, P) if inconsistent(P ) then Extend A, continue search if not done else if A = [] then backtrack fail if ... .

71 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) A, i, x := [], 0, Select(P) while true do v, t := Assign(x, P) P := Propagate(x, v, P) if inconsistent(P ) then . if t = ``decision" then i := i + 1 A := push((x, v, i, t, P), A) if A is complete then return A i, P, x := 0, P , Select(P ) else if A = [] then backtrack fail if ... .
72 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) A, i, x := [], 0, Select(P) while true do v, t := Assign(x, P) . P := Propagate(x, v, P) if inconsistent(P ) then Extend A, continue search if not done else if A = [] then backtrack fail if ... .

73 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) A, i, x := [], 0, Select(P) while true do v, t := Assign(x, P) P := Propagate(x, v, P) if inconsistent(P ) then . Extend A, continue search if not done else if A = [] then repeat (x, , i, t, P) := head(A) A := pop(A) until (t = ``decision" i blimit) A = [] fail if ... .
74 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) A, i, x := [], 0, Select(P) while true do v, t := Assign(x, P) . P := Propagate(x, v, P) if inconsistent(P ) then Extend A, continue search if not done else if A = [] then backtrack fail if ... .

75 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Search Algorithm
Input: Property P with at least one free variable Output: A counterexample (assignment) or fail local Stack A of (var, val, # assigns, type, property) A, i, x := [], 0, Select(P) while true do v, t := Assign(x, P) P := Propagate(x, v, P) . if inconsistent(P ) then Extend A, continue search if not done else if A = [] then backtrack if A = [] (t = ``implied" i > blimit) then return fail .

76 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Select Algorithm
Input: Property P with at least one free variable Output: A free variable in P . . if h hyps(P) of form x = c then return x

77 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Select Algorithm
Input: Property P with at least one free variable Output: A free variable in P . . if h hyps(P) of form x = c then return x

78 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Select Algorithm
Input: Property P with at least one free variable Output: A free variable in P . if h hyps(P) of form x = c then return x . G= := EqualityDependencyGraph(P, vars(P))

1. Case: x = y. Add x y.

2. Case: x = fterm y freeVars(fterm) and x / freeVars(fterm) add x y

79 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Select Algorithm
Input: Property P with at least one free variable Output: A free variable in P if h hyps(P) of form x = c then return x . G= := EqualityDependencyGraph(P, vars(P)) Do SCC on G= , collect the leaf components in L leaves = := pick x from each l L .

80 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Select Algorithm
Input: Property P with at least one free variable Output: A free variable in P if h hyps(P) of form x = c then return x . G= := EqualityDependencyGraph(P, vars(P)) SCC on G= . G := RestDependencyGraph(P, leaves= )

1. x

y where

{<, , >, }: No edge

2. x fterm such that is a binary relation, y freeVars(fterm) and x / freeVars(fterm): Add x y

. 3. R(term1 , term2 , . . ., termn ), such that

x freeVars(termi ), y freeVars(termj ), i = j, n 2 and R is an arbitrary n-ary relation: Add x y.

81 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Select Algorithm
Input: Property P with at least one free variable Output: A free variable in P if h hyps(P) of form x = c then return x . G= := EqualityDependencyGraph(P, vars(P)) SCC on G= G := RestDependencyGraph(P, leaves= ) . Do SCC on G to get dag D

82 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Select Algorithm
Input: Property P with at least one free variable Output: A free variable in P if h hyps(P) of form x = c then return x G= := EqualityDependencyGraph(P, vars(P)) . SCC on G= G := RestDependencyGraph(P, leaves= ) Do SCC on G to get dag D X := the leaf in D with maximum i= value . return X

i= (x) denotes number of nodes that can reach it in G= . i= (X) denotes max value of i= among nodes X

83 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Hardware: Finding hazards in a pipeline

Analysing a 3-stage Pipeline
1. Fetch 2. Read 3. Execute/Write-back

84 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Hardware: Finding hazards in a pipeline

Analysing a 3-stage Pipeline
1. Fetch 2. Read 3. Execute/Write-back

Primary Concern
Avoid resource conflicts (Data/Control hazards)

85 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Hardware: Finding hazards in a pipeline

Analysing a 3-stage Pipeline
1. Fetch 2. Read 3. Execute/Write-back

Primary Concern
Avoid resource conflicts (Data/Control hazards)

Correctness
Show all behaviors of MA are observationally equivalent to behaviors of ISA

86 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Hardware: Finding hazards in a pipeline

Primary Concern
Avoid resource conflicts (Data/Control hazards)

Correctness
Show all behaviors of MA are observationally equivalent to behaviors of ISA

Can we find design errors that lead to hazards?

1. Assuming designer has modelled both ISA and MA 2. Formalize above correctness condition 3. Analyze it using our method (demo)

87 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Hardware: Finding hazards in a pipeline

Primary Concern
Avoid resource conflicts (Data/Control hazards)

Correctness
Show all behaviors of MA are observationally equivalent to behaviors of ISA

Can we find design errors that lead to hazards? Observations

1. No assertions were written 2. No lemmas were specified 3. No manual tests or test driver given.
88 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Software: Comparison with Alloy

Alloy
Alloy is a declarative modeling language based on sets and relations (relational logic with transitive closure) Used for describing and analyzing high-level specifications and designs.

Automatic Analysis
Given a bound on # model elements, called scope, Alloy models (and its specifications) translated into Boolean formulas and shipped to off-the-shelf SAT solvers.

89 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Software: Comparison with Alloy

Property delUndoesAdd addIdempotent addLocal lookupYields writeRead writeIdempotent hidePreservesInv cutPaste pasteCut pasteAffectsHidden markSweepSound markSweepComplete Alloy Analyzer Scope Time Result 25 26.41 -25 37.76 -3 0.08 CE 3 0.05 CE 34 99.69 -33 44.13 -61 24.91 -3 0.20 CE 3 0.20 CE 27 117.63 -8 47.34 -7 58.12 -Our method Time Result 0.07 QED 0.19 QED 1.35 CE 0.83 CE 0.02 QED 0.01 QED 0.26 QED 0.49 CE 1.38 CE 0.42 QED 0.28 QED 0.34 QED

Table: Comparison with Alloy Analyzer (AA)

90 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Software: Comparison with Alloy

Property delUndoesAdd addIdempotent addLocal lookupYields writeRead writeIdempotent hidePreservesInv cutPaste pasteCut pasteAffectsHidden markSweepSound markSweepComplete Alloy Analyzer Scope Time Result 25 26.41 -25 37.76 -3 0.08 CE 3 0.05 CE 34 99.69 -33 44.13 -61 24.91 -3 0.20 CE 3 0.20 CE 27 117.63 -8 47.34 -7 58.12 -Our method Time Result 0.07 QED 0.19 QED 1.35 CE 0.83 CE 0.02 QED 0.01 QED 0.26 QED 0.49 CE 1.38 CE 0.42 QED 0.28 QED 0.34 QED

Table: Comparison with Alloy Analyzer (AA)

1
1 Ghazi and Taghdiri.

Relational Reasoning by SMT Solving. In FM 2011

91 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Software: Comparison with Alloy

Methodology
Modeled above examples in ACL2, mimicking original formulation in Alloy. Used set types and map types i.e., binary relations, provided by our data definition framework.

92 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Software: Comparison with Alloy

Observations
1. The ordered sets and records library in ACL2 distribution, powerful enough to prove all the properties that Alloy posits are true 2. No intermediate lemmas provided, no hint or guidance offered to the theorem prover 3. Highlights effectiveness of powerful libraries by the tool-writer put to use by the choice of right abstractions by the programmer

93 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Discussion on the advantages of using ITP

Prune away huge subspaces Extensible Domain-specific lemma libraries powerful domain-specific reasoning User can also help formalize facts/insight

.
94 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Discussion on the advantages of using ITP

Prune away huge subspaces Extensible Domain-specific lemma libraries powerful domain-specific reasoning User can also help formalize facts/insight

.
95 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Discussion on the advantages of using ITP

Prune away huge subspaces Extensible Domain-specific lemma libraries powerful domain-specific reasoning User can also help formalize facts/insight

.
96 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Discussion on the advantages of using ITP

Prune away huge subspaces Extensible Domain-specific lemma libraries powerful domain-specific reasoning User can also help formalize facts/insight

.
97 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Discussion on the advantages of using ITP

Prune away huge subspaces Extensible Domain-specific lemma libraries powerful domain-specific reasoning User can also help formalize facts/insight

.
98 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

Conclusions

Automatically analyze properties, interleaving ITP and testing in a fine-grained fashion Search algorithm guides testing when it is stuck (Decision Procedures can also benefit) Select algorithm can be used as a starting point by concolic testing Combining automated methods with ITP technology results in a more powerful, yet automated method. Better interactive theorem proving experience

99 / 101

Introduction

Algorithm

Experimental Evaluation

Discussion and Conclusions

The End

Thank you

100 / 101