CMPT383 Comparative Programming Languages

Lecture 11: Type Inference

Yuepeng Wang
Spring 2025

1
Overview
• Review of type checking

• Introduction to type inference

• Simple examples of type inference

• Hindley-Milner type inference

• Constraint-based typing

• Unification

• Type inference in the FUN language

2
Strategies of Typing
• Type checking
• Ask the programmer to provide declarations (mostly for variables)
• Verify the types are compatible with certain constraints (programmer's
declaration, allowed operations, etc)

• Type inference
• Do not ask the programmer for type annotations
• Automatically find an appropriate type of every expression in the
program and report errors if types are incompatible

3
Motivation of Type Inference
• Providing type annotations is good because it facilitates type checking

• Providing type annotations is annoying sometimes

• Haskell: ((Int -> Int) -> (Int -> Int)) -> (Int -> Int) -> (Int -> Int)

• Java: HashMap<String, HashMap<String, ArrayList<String>>>

• C++: std::vector<std::vector<int>>::iterator

• Type inference can help programmers get benefits from type checking
without writing complicated type annotations

4
Review of Type Checking in FUN
• Consider the FUN language with type annotations

e ::= c | b | x | '(' e ')'

Γ[x ◃ T1] ⊢ e : T2
| e '+' e | e '-' e | ... (T-Abs)
| e '==' e | ... Γ ⊢ lambda x : T1 . e : T1 → T2
| 'if' e 'then' e 'else' e
| 'lambda' x ':' t '.' e
| 'app' e e Γ[x ◃ T1] ⊢ e1 : T1
| 'let' x ':' t '=' e 'in' e
Γ[x ◃ T1] ⊢ e2 : T2
t ::= 'Int' | 'Bool' | '(' t ')'
(T-Let)
| t '->' t Γ ⊢ let x : T1 = e1 in e2 : T2

5
Type Inference Example 1
• Now remove the type annotations and consider the FUN program:
let x = 1 in x

• It has a variable. But do we really need type annotations to know the type
of this expression?

• We know that 1 is an integer constant in FUN

• The type of x must be Int

• The type of "let x = 1 in x" must also be Int

6
Type Inference Example 2
• Consider the following FUN program:
let f = lambda x. x + 1 in f

• We know function f adds one to its argument

• We know + is only defined on integers in FUN

• Since x is one operand of +, x must be of type Int

• Thus, the type of f must be Int → Int

7
Type Inference Example 3
• Consider the following FUN program:
let f = lambda x. lambda y. x + y in f

• We know function f adds its two arguments

• We know + is only defined on integers in FUN

• Since x and y are operands of +, x and y must be of type Int

• Thus, the type of f must be Int → Int → Int

8
Type Inference Example 4
• Consider the following FUN program:
let f = lambda x. lambda y. x + 1 in f
• We know function f adds one to its first argument

• We can easily know the type of x is Int

• But we don't know much about y, i.e., f works for any type of y

• No matter what is the type of y, x+1 must have type Int

• Thus, the type of f must be Int → X → Int
• where X is a type variable, which should be understood as "for any X"

9
Type Inference Example 5
• Consider the following FUN program:
let f = lambda g. app g 1 in f

• Function f applies its argument g to 1

• We know g must be a function, because it can be applied to 1

• In addition, the type of g must be Int → X

• "app g 1" has type X

• Thus, the type of f must be (Int → X) → X

10
Type Inference Example 6
• Consider the following FUN program:
let f = lambda g. app g (app g 1) in f
• Function f applies its argument g to 1 twice

• From "app g 1", the type of g is Int→ X, and the type of "app g 1" is X
• From "app g (app g 1)", the type of g must be X → Y

• Since the type of g is both Int → X and X → Y. What does it mean?

• X = Y = Int
• Thus, the type of f must be (Int → Int) → Int
11
Example of Type Checking
• Show the proof/derivation process of checking "lambda x: Int. 1 + x" has
type Int → Int with annotations

Int 1 Ident x Γ[x ◃ Int](x) = Int

(T-Int) (T-Ident)
Γ[x ◃ Int] ⊢ 1 : Int Γ[x ◃ Int] ⊢ x : Int
(T-Plus)
Γ[x ◃ Int] ⊢ 1 + x : Int
(T-Abs)
Γ ⊢ lambda x: Int. 1 + x : Int → Int

12
High-level Idea of Type Inference
• Type inference on "lambda x. 1 + x"

• Assume x has an annotated type variable X

• Perform type checking for program "lambda x: X. 1 + x" and verify its
type is X → Y

• Whenever there is a requirement on the type variables X, Y, write the

requirement as a constraint

• Solve all the constraints to get the solution of X and Y

13
Example of Type Inference
• Type inference on "lambda x. 1 + x", conceptually

Int 1 X = Int Ident x Γ[x ◃ X](x) = Int

(T-Int) (T-Ident)
Y = Int Γ[x ◃ X] ⊢ 1 : Int Γ[x ◃ X] ⊢ x : Int
(T-Plus)
Γ[x ◃ X] ⊢ 1 + x : Y
(T-Abs)
Γ ⊢ lambda x: X . 1 + x : X → Y

** Constraints are shown in blue boxes

14
High-level Idea of Type Inference
• This idea also works for other programs!

• Introduce a type variable for each type annotation

• Treat the type variable as a concrete type

• Perform type checking to collect constraints

• Solve the constraints to obtain concrete types for (some) type

variables and obtain the type of each expression in the program

15
Hindley-Milner Type Inference
• The Hindley-Milner (H-M) type inference uses this idea

• Develop an algorithm that computes the most general type of an

expression without type annotations

• Named after two scientists

• Roger Hindley

• Robin Milner (1991 Turing Award winner)

16
The FUN Language
• Consider the FUN language without type annotations

e ::= c | b | x | '(' e ')'

| e '+' e | e '-' e | ... (arithmetic op)
| e '==' e | ... (logical op)
| 'if' e 'then' e 'else' e (conditional)
| 'lambda' x '.' e (fun abstraction)
| 'app' e e (fun application)
| 'let' x '=' e 'in' e (let binding)

• where c is an Int constant, b is a Bool constant, and x is an Identifier

17
Types in the FUN Language
• Types of expressions in the FUN language are defined as follows

t ::= 'Int' | 'Bool' | X | '(' t ')'

| t '→' t (fun type)

• Int and Bool are primitive types

• T1 → T2 represents a function type, where T1 and T2 can be primitive or
function types
• Note that a function type is different from the return type of a function
• → is right-associative
• X is a type variable. It can be instantiated with primitive types or function types
18
Constraint-Based Typing
• Type inference is divided into two main phases

• Constraint-based typing

• Given a typing environment and a program/expression, it computes a

set of constraints (in the form of equalities) that must be satisfied by
the solution of type variables

• Constraint solving

• Given a set of equality constraints, solve the types of type variables

19
Judgments for Constraint-Based Typing
• Judgments for constraint-based typing are of the form
Γ⊢e:T|C

• where Γ is the typing environment, e is the expression, T is the type, and

C is a set of collected constraints

• It means "under the typing environment Γ, the type of expression e is T,

and the typing constraints are C"

• If we consider the judgment as a function, it takes the environment Γ and

expression e as input and produces a type T and constraints C as output
20
Basic Constraint-Based Typing Rules
e ::= c | b | x | '(' e ')'
| e '+' e | e '-' e | ...
Int c Bool b
(CT-Int) (CT-Bool)
| e '==' e | ... Γ ⊢ c : Int | {} Γ ⊢ b : Bool | {}
| 'if' e 'then' e 'else' e
| 'lambda' x '.' e
Ident x Γ(x) = T
| 'app' e e (CT-Ident)
| 'let' x '=' e 'in' e Γ ⊢ x : T | {}

• For integer constant c, its type is Int and the typing constraints are empty
• For boolean constant b, its type is Bool and the typing constraints are empty
• For identifier (variable) x, its type is determined by the typing environment Γ,
which may get stuck if there is no entry for x in Γ, i.e. x ∉ dom(Γ). The
typing constraints are also empty
21
Alternative Rules for Identifiers
e ::= c | b | x | '(' e ')'
| e '+' e | e '-' e | ... Ident x x ∈ dom(Γ) Γ(x) = T
(CT-Ident1)
| e '==' e | ... Γ ⊢ x : T | {}
| 'if' e 'then' e 'else' e
| 'lambda' x '.' e
Ident x x ∉ dom(Γ)
| 'app' e e (CT-Ident2)
| 'let' x '=' e 'in' e Γ ⊢ x : Terr | {Cerr}

• There is a way to make the constraint-based typing rules never get stuck
• We can always check if the identifier x ∈ dom(Γ). If yes, the type is Γ(x)
• If not, we generate a constraint that is always unsatisfiable (e.g., a special
constraint Cerr or Int = Bool). The type error is deferred to be revealed
until the constraint solving phase
22
The Rule for Plus
e ::= c | b | x | '(' e ')'
| e '+' e | e '-' e | ... Γ ⊢ e1 : T1 | C1
| e '==' e | ...
Γ ⊢ e2 : T2 | C2
| 'if' e 'then' e 'else' e
| 'lambda' x '.' e C = C1 ∪ C2 ∪ {T1 = Int, T2 = Int}
(CT-Plus)
| 'app' e e
Γ ⊢ e1 + e2 : Int | C
| 'let' x '=' e 'in' e

• In the FUN language, the + operator only works for integers, so we know
the result of e1 + e2 must be of type Int

• In addition, the type of e1 is Int and the type of e2 is Int

23
The Rule for Minus
e ::= c | b | x | '(' e ')'
| e '+' e | e '-' e | ... Γ ⊢ e1 : T1 | C1
| e '==' e | ...
Γ ⊢ e2 : T2 | C2
| 'if' e 'then' e 'else' e
| 'lambda' x '.' e C = C1 ∪ C2 ∪ {T1 = Int, T2 = Int}
(CT-Minus)
| 'app' e e
Γ ⊢ e1 − e2 : Int | C
| 'let' x '=' e 'in' e

• CT-Minus is very similar to CT-Plus, because the - operator in FUN only

works for integers

24
The Rule for Equal
e ::= c | b | x | '(' e ')'
| e '+' e | e '-' e | ... Γ ⊢ e1 : T1 | C1
| e '==' e | ...
Γ ⊢ e2 : T2 | C2
| 'if' e 'then' e 'else' e
| 'lambda' x '.' e C = C1 ∪ C2 ∪ {T1 = T2}
(CT-Eq)
| 'app' e e
Γ ⊢ e1 == e2 : Bool | C
| 'let' x '=' e 'in' e

• Unlike T-Eq for type checking, CT-Eq does not restrict the operands of ==
to be only primitive types. It also works for function types
• Otherwise, there would be disjunctions (logical or) in the constraints,
e.g., T1 = Int ∨ T1 = Bool, which makes constraint solving harder

• CT-Eq requires the two operands of == have the same type

25
Example of Constraint-Based Typing
• Constraint-base typing for "3 == 1 + 2" (+ has higher precedence than ==)

Int 1 Int 2
(CT-Int) (CT-Int)
Int 3 Γ ⊢ 1 : Int | {} Γ ⊢ 2 : Int | {}
(CT-Int) (CT-Plus)
Γ ⊢ 3 : Int | {} Γ ⊢ 1 + 2 : Int | {Int = Int}
(CT-Eq)
Γ ⊢ 3 == 1 + 2 : Bool | {Int = Int}

** Duplicated elements in sets are removed

26
The Rule for If-Then-Else
e ::= c | b | x | '(' e ')'
Γ ⊢ e1 : T1 | C1
| e '+' e | e '-' e | ...
| e '==' e | ...
Γ ⊢ e2 : T2 | C2
| 'if' e 'then' e 'else' e Γ ⊢ e3 : T3 | C3
| 'lambda' x '.' e
C = C1 ∪ C2 ∪ C3 ∪ {T1 = Bool, T2 = T3}
| 'app' e e (CT-ITE)
| 'let' x '=' e 'in' e
Γ ⊢ if e1 then e2 else e3 : T2 | C

• CT-ITE requires that the condition expression has Bool type

• It also requires that two branches have the same type

• The type of the if-then-else expression is the same as the type of both
branches
27
The Rule for Function Abstractions
e ::= c | b | x | '(' e ')'
| e '+' e | e '-' e | ...
| e '==' e | ...
fresh X
| 'if' e 'then' e 'else' e Γ[x ◃ X] ⊢ e : T | C
(CT-Abs)
| 'lambda' x '.' e
| 'app' e e
Γ ⊢ lambda x . e : X → T | C
| 'let' x '=' e 'in' e

• CT-Abs assumes the type of formal parameter x is a type variable X

• X must be a "fresh" type variable, meaning it is a new type variable that does not occur
anywhere else in the derivation
• If we can determine the type of e is T and the typing constraints are C under the
environment where x has type X, then "lambda x . e" has type X → T and the typing
constraints are also C
28
Freshness
fresh X
Γ[x ◃ X] ⊢ e : T | C
(CT-Abs)
Γ ⊢ lambda x . e : X → T | C
• The freshness requirement on the type variable X is very important
• Intuitively, we assign variable x to a new type X that is never used
elsewhere in the derivation, because we know nothing about the type X
• Then we can use X to derive other types and collect constraints
• The rule is not correct without the freshness requirement
• The typical way to generate a fresh type variable is to associate the
variable name with some global counter, e.g., X1, X2, X3, …
29
Example of Constraint-Based Typing
• Constraint-based typing for "lambda x. lambda y. x"

Ident x x ∈ dom(Γ[x ◃ X1][y ◃ X2])

Γ[x ◃ X1][y ◃ X2](x) = X1
(CT-Ident1)
fresh X2 Γ[x ◃ X1][y ◃ X2] ⊢ x : X1 | {}
(CT-Abs)
fresh X1 Γ[x ◃ X1] ⊢ lambda y . x : X2 → X1 | {}
(CT-Abs)
Γ ⊢ lambda x . lambda y . x : X1 → X2 → X1 | {}

30
The Rule for Function Applications
e ::= c | b | x | '(' e ')' fresh X1, X2
| e '+' e | e '-' e | ...
Γ ⊢ e1 : T1 | C1
| e '==' e | ...
| 'if' e 'then' e 'else' e Γ ⊢ e2 : T2 | C2
| 'lambda' x '.' e
C = C1 ∪ C2 ∪ {T1 = X1 → X2, T2 = X1}
| 'app' e e (CT-App)
| 'let' x '=' e 'in' e Γ ⊢ app e1 e2 : X2 | C

• CT-App needs to use two fresh type variables X1, X2

• It requires that e1 has a function type X1 → X2, i.e., e1 must be a function

• It requires that the type of e2 is the same as the input type X1 of e1

• The type of "app e1 e2" is the same as the return type X2 of e1

31
The Rule for Let Bindings
e ::= c | b | x | '(' e ')' fresh X
| e '+' e | e '-' e | ... Γ[x ◃ X] ⊢ e1 : T1 | C1
| e '==' e | ...
| 'if' e 'then' e 'else' e Γ[x ◃ X] ⊢ e2 : T2 | C2
| 'lambda' x '.' e C = C1 ∪ C2 ∪ {X = T1}
| 'app' e e (CT-Let)
| 'let' x '=' e 'in' e Γ ⊢ let x = e1 in e2 : T2 | C

• CT-Let introduces a fresh type variable X for variable x

• It can infer types for recursive functions because Γ[x ◃ X] is used for typing both
e1 and e2
• It requires that the type of e1 is also X
• If e2 has type T2 then the expression "let x = e1 in e2" also has type T2
32
 Example of Constraint-Based Typing
• Constraint-based typing for "let x = 1 in 2 + x"

Ident x x ∈ dom(Γ[x ◃ X1])

Int 2 Γ[x ◃ X1](x) = X1
(CT-Int) (CT-Ident1)
Γ[x ◃ X1] ⊢ 2 : Int | {} Γ[x ◃ X1] ⊢ x : X1 | {}
Int 1
(CT-Int) (CT-Plus)
fresh X1 Γ[x ◃ X1] ⊢ 1 :Int | {} Γ[x ◃ X1] ⊢ 2 + x : Int | {Int = Int, X1 = Int}
(CT-Let)
Γ ⊢ let x = 1 in 2 + x : Int | {Int = Int, X1 = Int}

33
 Example of Constraint-Based Typing
• Constraint-based typing for "let x = True in 2 + x"

Ident x x ∈ dom(Γ[x ◃ X1])

Int 2 Γ[x ◃ X1](x) = X1
(CT-Int) (CT-Ident1)
Γ[x ◃ X1] ⊢ 2 : Int | {} Γ[x ◃ X1] ⊢ x : X1 | {}
Bool True
(CT-Bool) (CT-Plus)
fresh X1 Γ[x ◃ X1] ⊢ True : Bool | {} Γ[x ◃ X1] ⊢ 2 + x : Int | {Int = Int, X1 = Int}
(CT-Let)
Γ ⊢ let x = True in 2 + x : Int | {Int = Int, X1 = Int, X1 = Bool}

The constraints cannot be satis ed, so there is a type error

34
fi
Constraint Solving
• After obtaining the set of constraints from the typing rules, we need to
solve the constraints and get types for the type variables
• A solution (called a unifier) to a set of type constraints is a substitution σ
that maps type variables to types (concrete types or type variables) such
that all type constraints are satisfied
• For each equality constraint, the LHS and RHS must be syntactically
identical (i.e., have identical tree representations) after applying σ

• Example: type constraints {X1 = X2 → X3, X2 = Int}

• One solution: σ1 = [X1 ↦ Int → Int, X2 ↦ Int, X3 ↦ Int]
• Another solution: σ2 = [X1 ↦ Int → X3, X2 ↦ Int]
35
Applying Substitutions
• Applying a substitution σ to a type T is denoted by Tσ

• For FUN types

• Base case: Xσ = T if X ↦ T in the substitution σ; Xσ = X if X is not
in the domain of σ. Here, X is a type variable
• Base case: (Int)σ = Int, (Bool)σ = Bool
• Recursive case: (T1 → T2)σ = T1σ → T2σ
• Example:
((X1 → Int) → X2)[X1 ↦ Bool, X2 ↦ Int] = (Bool → Int) → Int

36
Composition of Substitutions
• A substitution σ1 can be composed with another substitution σ2, denoted
by σ1 ∘ σ2 to a form a new substitution

σ1 ∘ σ2 = [ ]
X ↦ Tσ1 for each (X ↦ T) ∈ σ2 and
X↦T for each (X ↦ T) ∈ σ1 with X ∉ dom(σ2)

• σ1 ∘ σ2 contains
• (1) a mapping X ↦ Tσ1 for each X ↦ T in σ2
• (2) all mappings X ↦ T in σ1 where X is not in the domain of σ2
• Property: T(σ1 ∘ σ2) = (Tσ2)σ1, i.e. applying σ1 ∘ σ2 to T is the same as
first applying σ2 to T and then applying σ1 to the result
37
Example of Composition
• Given σ1 = [X0 ↦ Bool → X1, X2 ↦ Int] and σ2 = [X1 ↦ X2], what is
σ1 ∘ σ2 ?

• σ1 ∘ σ2 = [X0 ↦ Bool → X1, X1 ↦ Int, X2 ↦ Int]

• Given σ1 = [X0 ↦ Bool → X1] and σ2 = [X1 ↦ X2], what is σ1 ∘ σ2 ?

• σ1 ∘ σ2 = [X0 ↦ Bool → X1, X1 ↦ X2]

38
Generality of Unifiers
• Some unifier is more general than others. We say σ2 is more general than (at
least as general as) σ1 if there exists a substitution σ such that σ1 = σ ∘ σ2

• Example: σ1 = [X1 ↦ Int → Int, X2 ↦ Int, X3 ↦ Int]

and σ2 = [X1 ↦ Int → X3, X2 ↦ Int]

• σ2 is more general than σ1 because if we substitute X3 with Int in σ2, we

can obtain σ1, i.e., σ1 = [X3 ↦ Int] ∘ σ2
• Not vice versa. We cannot substitute concrete types by definition

• The goal of constraint solving is to find a most general unifier

39
 Unification
• The most general unifier of type constraints can be found by unification
procedure unify(C)
The unifier is an empty map if there is no constraint
if C = {} then [ ]
else assert C = {S = T} ∪ C′ Take one constraint in the form of S = T from all constraints
if S and T are identical
then unify(C′) "Occur check" to avoid generating a cyclic substitution like
else if S is variable X and X ∉ TVars(T ) X ↦ X → X. TVars(T ) returns all type variables in T
then unify(C′[X ↦ T]) ∘ [X ↦ T]
else if T is variable X and X ∉ TVars(S)
C′[X ↦ T] means apply the substitution [X ↦ T] to the
then unify(C′[X ↦ S]) ∘ [X ↦ S]
remaining constraints C′
else if S is S1 → S2 and T is T1 → T2
then unify(C′ ∪ {S1 = T1, S2 = T2})
If both S and T are function types, we add equalities for
else
their input types S1 = T1 and return types S2 = T2 into the
fail
constraints

40







Example of Unification
• Consider constraints C = {X1 = Bool, Int → X2 = X3 → X1}
unify(C)
= unify({Int → X2 = X3 → X1}[X1 ↦ Bool]) ∘ [X1 ↦ Bool]
= unify({Int → X2 = X3 → Bool}) ∘ [X1 ↦ Bool]
= unify({Int = X3, X2 = Bool}) ∘ [X1 ↦ Bool]
= (unify({X2 = Bool}[X3 ↦ Int]) ∘ [X3 ↦ Int]) ∘ [X1 ↦ Bool]
= (unify({X2 = Bool}) ∘ [X3 ↦ Int]) ∘ [X1 ↦ Bool]
=…
= ([X2 ↦ Bool] ∘ [X3 ↦ Int]) ∘ [X1 ↦ Bool]
= [X2 ↦ Bool, X3 ↦ Int] ∘ [X1 ↦ Bool]
= [X2 ↦ Bool, X3 ↦ Int, X1 ↦ Bool]
41
Type Inference Result
• Given an expression e, perform constraint-based typing Γ ⊢e:T|C
with Γ being an empty map

• If constraints C cannot be satisfied, expression e has a type error

• If constraints C are satisfied with a most general unifier σ, then the type of
expression e is Tσ

42
Fresh in Haskell
• We can use the State monad ("State s" monad) to implement fresh
functions in Haskell

• Because the fresh functions need to have an internal state to avoid

generating same elements twice

• Haskell has a library for the State monad: Control.Monad.State.Lazy

• Example: implement a function called getFreshInt, which returns a

monadic value that yields a different integer every time it is evaluated

43
The State Monad
• State is a type constructor of kind * -> * -> *
• "State s a" is a type where s is the type of the internal state and a is
the type of the result
• "State s" is a monad
• Common functions
(>>=) :: State s a -> (a -> State s b) -> State s b
return :: a -> State s a
get :: State s s -- sets the result to be the state
put :: s -> State s () -- sets the result to (), sets the state
evalState :: State s a -> s -> a -- takes a state monad and the initial state
-- and returns the final result
runState :: State s a -> s -> (a, s) -- similar to evalState, also returns next state

44
The State Monad
• Suppose we use Int as the internal state
type InferState a = State Int a

• InferState is a monad

(>>=) :: InferState a -> (a -> InferState b) -> InferState b

return :: a -> InferState a
get :: InferState Int -- sets the result to be the state
put :: Int -> InferState () -- sets the result to (), sets the state
evalState :: InferState a -> Int -> a -- takes a state monad and the initial state
-- and returns the final result
runState :: InferState a -> Int -> (a, Int) -- similar to evalState, also returns next state

45
Fresh Functions
• Implement getFreshInt
getFreshInt :: InferState Int
getFreshInt = do n <- get
put (n + 1)
return n

bar :: InferState (Int, Int, Int)

foo :: InferState Int bar = do x <- getFreshInt
foo = do x <- getFreshInt y <- getFreshInt
y <- getFreshInt z <- getFreshInt
return y return (x, y, z)

ghci> evalState foo 1 ghci> evalState bar 1

2 (1,2,3)
ghci> runState foo 1 ghci> runState bar 1
(2,3) ((1,2,3),4)

46