A Generic Explainability Framework for Function Circuits

Sylvain Hallé & Hugo Tremblay
Sylvain Hallé
Hugo Tremblay
Université du Québec à Chicoutimi
CANADA
A Generic Explainability Framework
for Function Circuits
CRSNG
NSERC

+
Functions
Basic processing unit: function

+
Functions
input pin(s)
output pin(s)

+
Functions
Given input arguments, a function produces output
values

+
Functions
Given input arguments, a function produces output
values
3
2
5

A function can be "anything"; it can be
parameterized by another function
Functions
α
2

Functions
α
2
apply on each element square

Functions
α
2
[4,2,5] [16,4,25]

>
+
×
>
>
3
Function circuits
Functions can be composed to form circuits

Outputs of "upstream" functions become inputs of
"downstream" functions
>
+
×
>
>
3
Function circuits

2
6
3
>
+
×
>
>
3
Function circuits

2
6
3
6
8
3
18
6
6
>
+
×
>
>
3
Function circuits

2
6
3
6
8
3
18
6
6
6
33
8
18
>
+
×
>
>
3
Function circuits

2
6
3
6
8
3
18
6
6
6
33
8
18
⊥
⊤
>
+
×
>
>
3
Function circuits

2
6
3
6
8
3
18
6
6
6
33
8
18
⊥
⊤
⊤
⊥
>
+
×
>
>
3
Function circuits

2
6
3
6
8
3
18
6
6
6
33
8
18
⊥
⊤
⊤
⊥
⊤
>
+
×
>
>
3
Function circuits

A circuit can be encapsulated into a function
>
+
×
>
>
3
Function circuits

A circuit can be encapsulated into a function
x
y
z
t
(x + y > y × z) ∨ (y > 3)
>
+
×
>
>
3
Function circuits

α
>
3
Parameterized functions result in multiple sub-
evaluations of a circuit
All together now

α
>
3
All together now
[4,2,5]

α
>
3
All together now
[4,2,5]
>
3
4
44
3
T
T3
T[ ]

α
>
3
All together now
[4,2,5]
>
3
4
44
3
T
T3
T[ ]
>
3
2
2
3
T
T
3
T

α
>
3
All together now
[4,2,5]
>
3
4
44
3
T
T3
T[ ]
>
3
2
2
3
T
T
3
T
>
3
5
5
3
T
T3
T

Part of
Most provenance models implicitly involve the
concept of "part of"
Straightforward in the relational world...

Part of
a relation R

Part of
a relation R
a part of R
⊆

Part of
a relation R
a part of R
⊆
a tuple t
a=v
a=v
a=v
a=v
a=v
a=v
a=v

Part of
a relation R
a part of R
⊆
a tuple t
a=v
a=v
a=v
a=v
a=v
a=v
a=v
a part of t
⊆

Part of
a relation R
a part of R
⊆
a tuple t
a=v
a=v
a=v
a=v
a=v
a=v
a=v
a part of t
⊆
What if the universe of discourse is not only made
of sets?

Our proposed model is based on an abstract
topology of objects
Topology of objects
𝖀 is a set of objects
⊑ is a partial order between objects ("part of")
□ is the only object that is part of all others
We suppose that ⊑ follows the intuition for common
objects:
scalars have no parts but themselves
a part of a set is a subset or a single element
a part of a string (list) is a sub-string (-list)

A designator is any function that takes an object
and returns a part of this object. Examples:
Designator
[i] the i-th item of a list
[i : j] the substring between characters i and j
the contents of some ﬁle
the i-th input pin of a function
the i-th output pin of a function
↑i
↓i

Designators can be composed:
↑2 [3] [1:5]◦ ◦
Designator
↑i
↓i

Designators can be composed:
↑2 [3] [1:5]◦ ◦
"The ﬁrst ﬁve characters of the third
item of the second argument of the
function"
Designator
↑i
↓i

Designation graph
∧
↓ , f1
< <
↑ , f1< <
∧
[1],↑ , f2< < [1],↑ , f3< <
A designation graph is an and-or directed acyclic
graph (DAG) where:
the endpoints are made of a designator d and a
function f
the head of d points to an input or an output of f
It expresses an "and-or"
relationship between
function inputs and
function outputs in a
circuit.

Given a function f with m inputs and n outputs, a
derivation operator is a function ∆f:
Derivation
∆f : 𝕯 × 𝖀m
× 𝖀n
→ 𝕿

Derivation
∆f : 𝕯 × 𝖀m
× 𝖀n
→ 𝕿
designator function
input
function
output
designation
graph

Derivation
∆f : 𝕯 × 𝖀m
× 𝖀n
→ 𝕿
The graph must be such that:
its root is the designator d given to ∆f
if d's head points to an input of f, the leaves
point to an output of f (and vice versa)
designator function
input
function
output
designation
graph

Given a circuit an input/output
pair and a designator, one can
reconstruct the derivation for the
circuit by connecting the ends of
the derivation graph for each
individual function.
Derivation for circuits
>
+
×
>
>
3

>
+
×
>
>
3
⊤
2
6
3

>
+
×
>
>
3
⊤
2
6
3
∧
↓ ,1
< <
>
↑ ,1< <
>
∧
↓ ,1
< < >
∧
↑ ,1< < > ↑ ,2< < >
∧
↓ ,2
< <
∧
↓ ,2
< < 3

>
+
×
>
>
3
⊤
2
6
3
∧
↓ ,1
< <
>
↑ ,1< <
>
∧
↓ ,1
< < >
∧
↑ ,1< < > ↑ ,2< < >
∧
↓ ,2
< <
∧
↓ ,2
< < 3
6 3

/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example

/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
split ﬁle
into lines

/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
on each
line...

/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
split on
commas

/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
take
second
element

/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
convert to
number

/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
on each
window of
3 values...

/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
take the
average

/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
on each
average...

/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
check that is
is greater than 3

/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
assert they
are all so

⊤the,2,penny
fool,7,lane
on,18,come
the,2,together
hill,-80,i
strawberry,7,am
fields,1,the
forever,10,walrus
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example

⊤the,2,penny
fool,7,lane
on,18,come
the,2,together
hill,-80,i
strawberry,7,am
fields,1,the
forever,10,walrus
∧
↓ , G1< <
∧ ∧∧
[4:5], [3] ,< < [5:5], [4] ,< < [6:8], [5] ,< < [8:8], [7] ,< <[ : ], [6] ,< <12 12
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example

⊤the,2,penny
fool,7,lane
on,18,come
the,2,together
hill,-80,i
strawberry,7,am
fields,1,the
forever,10,walrus
∧
↓ , G1< <
∧ ∧∧
[4:5], [3] ,< < [5:5], [4] ,< < [6:8], [5] ,< < [8:8], [7] ,< <[ : ], [6] ,< <12 12
∧
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example

Observations
Pros
generic computation model (circuits)
not limited to tuples and relations
modular (define ∆f separately for each f)
support for alternate lineage ("or" nodes)
fine granularity (parts of objects)
same algorithm, different lineage depending on
definition of ∆f for each function f
Cons
who defines ∆f ? according to what requirements ?
circuits: a way to program, or a model to reason
about a program ?

Future work
Provide formal definitions of ∆ that correspond
to various (existing) provenance relationships
Implement and test a proof of concept
(under way: https://github.com/liflab/petitpoucet)
Retrofit this framework into existing software
(BeepBeep, Cornipickle)
Study circuits according to the designation
graphs they induce
Thank you!

A Generic Explainability Framework for Function Circuits

Recommended

More Related Content

What's hot (20)

Similar to A Generic Explainability Framework for Function Circuits (20)

More from Sylvain Hallé (20)

Recently uploaded (20)

A Generic Explainability Framework for Function Circuits