Advanced Verification

Bob Oden
UVM Field Specialist, Mentor
Instructor, ECE Department, NCSU
November 27, 2017
 Agenda - Monday
n  The circumstances: Electronic design
n  The problem: Project risk
n  The solution: Systematic verification
n  The industry: Trends in verification technology
n  SystemVerilog: Standard language for verification
n  UVM: Standard methodology for verification

© Mentor Graphics Corp. Company Confidential

2 rdo, Advanced Verification, November 2017

 Agenda - Wednesday
n  Block level environments
n  Golden models
n  Chip level environments
n  Emulation
n  Reuse – Horizontal, vertical, platform
n  Simulation and emulation in regression testing
n  Verification management for closing coverage

© Mentor Graphics Corp. Company Confidential

3 rdo, Advanced Verification, November 2017

 The Circumstances
n  Ever increasing design size and complexity
—  Cell phone in your pocket
—  Electronics in your car
—  Connected devices in your home

© Mentor Graphics Corp. Company Confidential

4 rdo, Advanced Verification, November 2017

 The Problem

n  Project risk is directly proportional to

the gap between design capability and Project
verification capability Risk

Design
Capability

Verification
Capability

© Mentor Graphics Corp. Company Confidential

5 rdo, Advanced Verification, November 2017

 The Solution
n  Systematic verification planning and execution
—  parallels design planning and execution
n  Abstract verification language
Module A Environment A

Module B Environment B

Module C Environment C

Module D Module E Environment D Environment E

© Mentor Graphics Corp. Company Confidential

6 rdo, Advanced Verification, November 2017

INDUSTRY TRENDS
IN VERIFICATION

© Mentor Graphics Corp. Company Confidential

 Industry Trends - Flaws

© Mentor Graphics Corp. Company Confidential

8 rdo, Advanced Verification, November 2017

 Industry Trends – Verification Languages

© Mentor Graphics Corp. Company Confidential

9 rdo, Advanced Verification, November 2017

 Industry Trends – Verification Methodologies

© Mentor Graphics Corp. Company Confidential

10 rdo, Advanced Verification, November 2017

VERIFICATION PLANNING

© Mentor Graphics Corp. Company Confidential

 Verification Architecture Document
n  Identifies testing goals at each simulation level
—  Feature verification at block level
—  Integration and performance at top level
—  Traffic patterns to be tested
n  Identifies environment(s) to be developed
—  Block level environments required
—  Upper level environments required
—  Which block level environments are required in upper level simulations
—  Interface pacakges required
n  Documents architecture for each environment
—  Required prediction models identified

© Mentor Graphics Corp. Company Confidential

12 rdo, Advanced Verification, November 2017

 Verification Architecture Document
n  Resource and schedule planning
—  Engineering resources estimated
– People, tools, expertise,
—  Milestones determined
– Verification requirements identified
– Environment architectures defined
– Order of environment development
– Verification performed without environments
– Delivery of prediction models
– First simulation dates
– RTL drop dates
– Coverage closure

© Mentor Graphics Corp. Company Confidential

13 rdo, Advanced Verification, November 2017

 The Universal Questions of Verification
n  “How do we know when were done?”
n  “How do we get there?”
n  “Are we there yet?”

© Mentor Graphics Corp. Company Confidential

14 rdo, Advanced Verification, November 2017

 Verification Test Plan – What is tested
n  “How do we know when were done?”
– What needs to be tested?
– Everything can’t be tested
– Manage risk with informed decisions

n  Test Plan
– Identifies what needs to be tested
– Start by identifying what without consideration of how

© Mentor Graphics Corp. Company Confidential

15 rdo, Advanced Verification, November 2017

 Verification Test Plan – How it’s tested
n  “How do we get there?”
– How do we complete each item in the Test Plan?
– Identify best way of achieving each item
n  Verification Test Plan
—  Identifies what needs to be tested
—  Identifies how each Test Plan item will be completed
– Code coverage
– Functional coverage
– Assertion coverage
– Directed test
– Prediction

© Mentor Graphics Corp. Company Confidential

16 rdo, Advanced Verification, November 2017

 Determining Verification Completion?
n  “Are we done?”
– If not, when will we be done?

n  Verification loop
1.  Add test scenarios
2.  Collect coverage during test regression
3.  Merge test coverage results
4.  Rank test/seed pairs
5.  Generate custom coverage reports
6.  Identify coverage holes
7.  Identify coverage closure trends

© Mentor Graphics Corp. Company Confidential

17 rdo, Advanced Verification, November 2017

Test Plan Example

© Mentor Graphics Corp. Company Confidential

Slide 18
ECE 792-036 – Advanced Verification with UVM
SYSTEMVERILOG

© Mentor Graphics Corp. Company Confidential

 SystemVerilog - Overview

SystemVerilog IEEE 1800

Native Assertions
Classes with
SVA
methods &
Functional inheritance,
Coverage interfaces, Direct
Constrained
structs, unions, Programming
random
enums, arrays Interface
generators

IEEE 1364 Verilog 2001

Verilog 95

© Mentor Graphics Corp. Company Confidential

20 rdo, Advanced Verification, November 2017

 SystemVerilog Assertions
A concise description of [un]desired behavior
0 1 2 3 4 5
req

ack
Example intended behavior

property single_req;
After clk)
@(posedge the request signal
disable iff is asserted, the
(rst)
acknowledge
($rose(req))signal must&&
|=> ((!ack come 1 to 3 cycles
req)[*0:2] later
##1 ack);
endproperty

© Mentor Graphics Corp. Company Confidential

21 rdo, Advanced Verification, November 2017

 SystemVerilog Constrained Random
n  Declare a set of random variables – X,
Y, Z
n  Declare a set of constraints – Y < 42, X
Z
≤Y≤Z
n  Find the set of values that meet the
given constraints
n  Randomly pick solutions Y

© Mentor Graphics Corp. Company Confidential

22 rdo, Advanced Verification, November 2017

 Class Based Randomization
n  Randomization within SV is
object based
class Bus;
rand bit [15:0] addr; // Randomized
—  All SV data types randomizable randc int data; // Randomized
employee busDriver; // Not randomized
—  Variables preceeded with rand
constraint word_align { addr[1:0]==2 b0;}
or randc keywords randomized
endclass : Bus
—  Objects instantiated within the
object are not automatically …
Bus bus = new;
randomized assert(bus.randomize());
assert(bus.data.randomize() ); // Only randomize data
—  Constraints applied when …

variables randomized // Overlay constraint on constraints inside class

assert(bus.randomize() with { addr[2] == 1 b0;});

© Mentor Graphics Corp. Company Confidential

23 rdo, Advanced Verification, November 2017

 Randomization Distribution
n  rand: Uniform distribution of random values with possible repeating
n  randc: Uniform cyclic values cover all possible values before
repeating
n  Applies to
—  SV data type variables
—  All elements of arrays
—  All elements and size of dynamic and associative arrays

© Mentor Graphics Corp. Company Confidential

24 rdo, Advanced Verification, November 2017

 Constraint Examples

© Mentor Graphics Corp. Company Confidential

25 rdo, Advanced Verification, November 2017

 Declarative vs. Procedural
n  Contained in any class
n  pre_randomize(): class packet;
Automatically called before …
object randomization function void pre_randomize();
length = data.size();
—  Override to perform endfunction : pre_randomize
initialization or set function void post_randomize();
preconditions, etc dataSum = data.sum;
endfunction : post_randomize
n  post_randomize(): endclass : packet
Automatically called after
object randomization
—  Override to cleanup, report, set
post conditions, etc.

© Mentor Graphics Corp. Company Confidential

26 rdo, Advanced Verification, November 2017

 Coverage – Improving Your Understanding
n  Increasing angles of observation increases understanding
—  Each coverage type is a different view of verification effort
—  Each coverage type is complimentary and provides sanity against each
other

© Mentor Graphics Corp. Company Confidential

27 rdo, Advanced Verification, November 2017

 Coverage Options
n  Code Coverage
—  Code was executed
n  Functional Coverage
—  Data and scenario
n  Assertion Coverage
—  Signaling

n  Note on coverage measurement during simulation run

—  Test should not run until coverage achieved
—  Use test ranking to optimize

© Mentor Graphics Corp. Company Confidential

28 rdo, Advanced Verification, November 2017

 Code Coverage
n  Statement coverage — counts the execution of each statement on a line
individually, even if there are multiple statements in a line.
n  Branch coverage — counts the execution of each conditional “if/then/else” and
“case” statement and indicates when a true or false condition has not executed.
n  Condition coverage — analyzes the decision made in “if” and ternary statements
and can be considered as an extension to branch coverage.
n  Expression coverage — analyzes the expressions on the right hand side of
assignment statements, and is similar to condition coverage.
n  Toggle coverage — counts each time a logic node transitions from one state to
another.
n  FSM coverage — counts the states, transitions, and paths within a finite state
machine.

© Mentor Graphics Corp. Company Confidential

29 rdo, Advanced Verification, November 2017

 Functional Coverage
n  Covergroup - encapsulates the specification of a coverage model
and may include: clocking event, coverpoints, bins, cross coverage,
transition coverage, coverage options.
n  Coverpoint – specifies an integral expression to be covered.
Values of variables within the class. Can be divided and described
using bins.
n  Bins – separate collections of variable values from among all
possible variable values
n  Cross – coincident values of one or more coverpoints
n  Transition – Tracks sequential values or value groups
© Mentor Graphics Corp. Company Confidential

30 rdo, Advanced Verification, November 2017

 Assertion Coverage
n  Cover directive
—  Identifies observed signal relationships
—  Applied to properties or sequences
—  Reports attempt and success count
—  Useful for identifying vacuous passes
—  Signal sequences can trigger other operations
–  Coverage sampling
–  Event notification

© Mentor Graphics Corp. Company Confidential

31 rdo, Advanced Verification, November 2017

 UNIVERSAL
VERIFICATION
METHODOLOGY

© Mentor Graphics Corp. Company Confidential

 © Mentor Graphics Corp. Company Confidential

33 rdo, Advanced Verification, November 2017

 UVM Reuse Structure

Test Bench ■  Complete Home: DUT Specific

Bench and Stimulus
■  Completed Rooms: Interface
Reusable IP and Environment Packages
■  Pre-Fabricated Rooms: UVM Use
UVM Framework Model and Reuse Methodology

UVM ■  Building Supplies: Common

Verification Building Blocks

SystemVerilog ■  Raw Materials: Abstract

Programming Language
© Mentor Graphics Corp. Company Confidential

34 rdo, Advanced Verification, November 2017

 Common TB Features
n  Core capabilities of every verification environment
—  Component hierarchy
—  Moving data between components
—  Managing test flow
—  Generating messages
—  Synchronizing activities
—  Sharing resources
—  Generating stimulus
—  Checking results
—  Creating test cases

All simulation benches do the same things…differently.

© Mentor Graphics Corp. Company Confidential

35 rdo, Advanced Verification, November 2017

 UVM Features
n  Core capabilities of every verification environment
—  Component hierarchy
—  Moving data between components
—  Managing test flow
—  Generating messages
—  Synchronizing activities
—  Sharing resources
—  Generating stimulus
—  Checking results
—  Creating test cases

UVM provides a standardized implementation…Freedom from choice!

© Mentor Graphics Corp. Company Confidential

36 rdo, Advanced Verification, November 2017

 UVM Components

n  UVM_component
—  Encapsulates common
data and functionality
of all components
–  Hierarchy
–  Phasing
–  Objections
–  Factory
–  Recording

n  Basis for environment

hierarchy

© Mentor Graphics Corp. Company Confidential

37 rdo, Advanced Verification, November 2017

 UVM Stimulus
n  Transaction object sent
to environment through
the sequencer

n  Represents bus
operation

n  Untimed

n  Contains sequence ID
and sequence item ID
© Mentor Graphics Corp. Company Confidential

38 rdo, Advanced Verification, November 2017

 UVM Reporting

© Mentor Graphics Corp. Company Confidential

39 rdo, Advanced Verification, November 2017

 UVM Common Phases
•  Only components participate
in phases
•  Phase order
•  As shown in table
•  Function phases
•  Build, connect,
end_of_elaboration,
start_of_simulation, extract, •  Components participate in phase if
check, report task/function of phase name exists
•  Task phases (time consuming) •  Automatically executed
•  run

© Mentor Graphics Corp. Company Confidential

40 rdo, Advanced Verification, November 2017

 Additional Time Consuming Phases in UVM

n  Run-Time phases
executed in
parallel with run
phase.

n  Do Not Use
—  Run –Time phases
—  User defined
phases
—  Phase jumping

© Mentor Graphics Corp. Company Confidential

41 rdo, Advanced Verification, November 2017

 UVM Factory – Constructor Proxy
n  UVM Class Reference
—  “The uvm_factory is used to manufacture (create) UVM objects and
components. Only one instance of the factory is present in a given
simulation”
n  Used to determine the class type of an object being constructed
n  Used to change the class type of an object being constructed

© Mentor Graphics Corp. Company Confidential

42 rdo, Advanced Verification, November 2017

 UVM Factory - Registration
Factory Registry
n  Register Classes with the Factory
—  Objects Requested Returned
–  `uvm_object_utils Type Type
–  `uvm_object_param_utils ahb2wb_predi ahb2wb_predi
—  Components ctor ctor
–  `uvm_component_utils
–  `uvm_component_param_utils wb2ahb_predi wb2ahb_predi
ctor ctor
n  Change Registry Table
—  set_type_override()
—  set_instance_override()
n  Construct Classes with the Factory
—  Create() instead of new()
© Mentor Graphics Corp. Company Confidential

43 rdo, Advanced Verification, November 2017

 UVM Factory – Overrides
Factory Registry Before
Requested Type Returned Type
ahb2wb_predictor ahb2wb_predictor

wb2ahb_predictor wb2ahb_predictor

ab2wb_predictor::set_type_override(ahb2wb_dpi_predictor::get_type())

Factory Registry After

Requested Type Returned Type Set overrides BEFORE
object construction
ahb2wb_predictor ahb2wb_dpi_predictor Inside test class or using UVM CLI

wb2ahb_predictor wb2ahb_predictor
© Mentor Graphics Corp. Company Confidential

44 rdo, Advanced Verification, November 2017

 Resource Sharing Within UVM
n  Resources typically shared within a simulation
—  Configuration objects
—  Virtual interface handles
—  Sequencer handles

n  Resource visibility limited by type, scope, name

© Mentor Graphics Corp. Company Confidential

45 rdo, Advanced Verification, November 2017

 UVM Reuse Structure

Test Bench ■  Complete Home: DUT Specific

Bench and Stimulus
■  Completed Rooms: Interface
Reusable IP and Environment Packages
■  Pre-Fabricated Rooms: UVM Use
UVM Framework Model and Reuse Methodology

UVM ■  Building Supplies: Common

Verification Building Blocks

SystemVerilog ■  Raw Materials: Abstract

Programming Language
© Mentor Graphics Corp. Company Confidential

46 rdo, Advanced Verification, November 2017

© Mentor Graphics Corp.
www.mentor.com
Company Confidential
Advanced Verification
Bob Oden
UVM Field Specialist, Mentor
Instructor, ECE Department, NCSU
November 29, 2017
 Agenda - Wednesday
n  Block level environments
n  Golden models
n  Chip level environments
n  Emulation
n  Reuse
n  Simulation and emulation in regression testing
n  Verification management for closing coverage

© Mentor Graphics Corp. Company Confidential

49 rdo, Advanced Verification, November 2017

 BLOCK LEVEL
ENVIRONMENTS

© Mentor Graphics Corp. Company Confidential

 Block Level Environment

n  Contents
—  Configuration Env Configuration Environment

—  Agents UVM Register

Model
Predictor Scoreboard

—  Predictors Agent A Coverage

–  Modeling options Config

—  Scoreboards Agent B
Agent B
–  Verifying DUT output Config Agent A

–  End of Test checking

—  Coverage
–  Objective measurement of
test results
—  Stimulus vs. Analysis
–  Independent for reuse

© Mentor Graphics Corp. Company Confidential

51 rdo, Advanced Verification, November 2017

 Agent
n  Provides connection point between BFM
signal level activity and environment
transaction level activity Agent

n  Active agent Coverage

—  Provides data to driver BFM for signal

activity
—  Receives data from monitor BFM of signal Monitor

activity
n  Passive agent
—  Receives data from monitor BFM of signal Sequencer Driver

activity
n  Broadcasts transactions to other
components within environment
n  No DUT specific operations
© Mentor Graphics Corp. Company Confidential

52 rdo, Advanced Verification, November 2017

 Predictor
n  Models all or some of DUT operation
—  Single, all inclusive, golden model of DUT Predictor
—  Distributed golden model of DUT
—  Based on data flow through the design
—  Creates expected DUT output transactions Predictor

n  Untimed
n  Receives transactions from agents through
analysis_export(s)
—  Creates expected DUT output transaction from Predictor
–  Input transaction
–  Prior transaction(s)
–  Current configuration or state
© Mentor Graphics Corp. Company Confidential

53 rdo, Advanced Verification, November 2017

 Predictor
n  Sends transactions to scoreboards or other
predictors through analysis_port(s)
n  Any combination of analysis_exports and
analysis_ports possible
—  Dependent on DUT data flow and environment
architecture Scoreboard A

n  Can provide output checking in some cases

—  Memory checking
—  State flow Predictor A Predictor B
—  Etc.

© Mentor Graphics Corp. Company Confidential

54 rdo, Advanced Verification, November 2017

 Prediction – Modeling Behavior

n  Language options for modeling DUT behavior

—  SystemVerilog – Native
—  C/C++ - Use Direct Programming Interface, DPI, to call C/C++
functions from SystemVerilog and SystemVerilog from C/C++
functions
—  SystemC – Use UVM Connect to pass data from SystemVerilog
and SystemC and from SystemC to SystemVerilog

© Mentor Graphics Corp. Company Confidential

55 rdo, Advanced Verification, November 2017

 Scoreboard
n  Verifies DUT output
—  Compared against predicted value
—  Uses compare function in transaction class
Scoreboard
n  Transaction handles discarded after
comparison
—  Allows for memory to be reclaimed through
garbage collection
n  Performs end of test operations
—  Ensure expected transaction storage is empty
—  Delay test completion until storage is empty
—  Ensure scoreboard received expected
transactions
—  Output summary of scoreboard activity

© Mentor Graphics Corp. Company Confidential

56 rdo, Advanced Verification, November 2017

 Scoreboard – Storing Predicted Transactions
n  Two basic mechanisms to handle two basic data flows
—  In order
–  DUT output transaction order predictable and guaranteed to be in order
in relation to DUT input transactions
–  Use uvm_tlm_analysis_fifo to store expected transactions
—  Out of order
–  DUT output transaction order not predictable or ordered
–  Use SystemVerilog associative array to store expected transactions

© Mentor Graphics Corp. Company Confidential

57 rdo, Advanced Verification, November 2017

 Scoreboard – EOT Checking and Reporting

n  Checks
—  Empty check
—  Activity check
—  Wait for drain
n  Reports
—  Transaction counts
—  Remaining transaction display
—  summary

© Mentor Graphics Corp. Company Confidential

58 rdo, Advanced Verification, November 2017

 Coverage
n  Gathers and records data for functional
coverage Coverage
—  Transactions
—  Configuration settings
—  Current state
—  Transitions Coverage

n  Only receives transactions

—  Does not broadcast
n  Contains
—  Covergroups Coverage
—  Coverpoints
—  Cross Coverage
—  Etc.

© Mentor Graphics Corp. Company Confidential

59 rdo, Advanced Verification, November 2017

 Block Level Environment Components
Environment
Env Configuration
Predictor Scoreboard
UVM Register
Model

In Agent Coverage
Config

Out Agent
In Agent Out Agent
Config

© Mentor Graphics Corp. Company Confidential

60 rdo, Advanced Verification, November 2017

 Block Level Test Component
Test top
Environment
Env Configuration
Predictor Scoreboard
UVM Register
Model

In Agent Coverage
Config

Out Agent
In Agent Out Agent
Config

Seq Seq Seq

Seq

© Mentor Graphics Corp. Company Confidential

61 rdo, Advanced Verification, November 2017

 Block Level Test Bench Modules
Test top Hdl_top
Environment
Env Configuration
Predictor Scoreboard
UVM Register BFM
Model
BFM
In Agent Coverage
Config DUT

Out Agent
In Agent Out Agent
Config
Hvl_top

Seq Seq Seq

Seq

© Mentor Graphics Corp. Company Confidential

62 rdo, Advanced Verification, November 2017

 Agent and Bus Functional Models

Agent Signal Bundle DUT

Coverage Net type

Monitor BFM Protocol
Signals
Task and Function
API’s to Interface
Monitor With agent monitor

Driver BFM
Sequencer Driver Task and Function
API’s to Interface
With agent driver

© Mentor Graphics Corp. Company Confidential

63 rdo, Advanced Verification, November 2017

 Agent and BFM – Data Flow
Transaction Signal
Variables Values
Agent Passed Passed Signal Bundle DUT

Coverage Net type

Monitor BFM Protocol
Signals
Task and Function
API’s to Interface
Transactions Monitor With agent monitor
Passed

Driver BFM
Sequencer Driver Task and Function
API’s to Interface
With agent driver

© Mentor Graphics Corp. Company Confidential

64 rdo, Advanced Verification, November 2017

 CHIP LEVEL
ENVIRONMENTS

© Mentor Graphics Corp. Company Confidential

 Chip Level Environment Components
Test top
Environment
Env Configuration
Sub A environment Sub B environment
UVM Register
Model

Sub A env
Configuration

Sub B env
Configuration

© Mentor Graphics Corp. Company Confidential

66 rdo, Advanced Verification, November 2017

 Chip Level Environment Components
Test top
Environment
Env Configuration
Sub A environment Sub B environment
UVM Register
Model Predictor Scoreboard Predictor Scoreboard

Sub A env Coverage Coverage

Configuration
In Agent Out Agent In Agent Out Agent

Sub B env
Configuration

© Mentor Graphics Corp. Company Confidential

67 rdo, Advanced Verification, November 2017

 Chip Level Test Components
Test top
Environment
Env Configuration
Sub A environment Sub B environment
UVM Register
Model

Sub A env
Configuration

Sub B env
Configuration

Seq Seq Seq

Seq

© Mentor Graphics Corp. Company Confidential

68 rdo, Advanced Verification, November 2017

 Chip Level Test Bench Modules
Test top Hdl_top
Environment
Env Configuration BFM
Sub A environment Sub B environment
UVM Register BFM
Model
BFM
Sub A env
Configuration
DUT

Sub B env
Configuration
Hvl_top

Seq Seq Seq

Seq

© Mentor Graphics Corp. Company Confidential

69 rdo, Advanced Verification, November 2017

UVM & CO-EMULATION

© Mentor Graphics Corp. Company Confidential

 Testbench-Driven Verification Productivity
SystemVerilog, UVM, Block-to-Top Reuse, Platform Portability

n  Electronics systems companies need large

improvements in (simulation-based) verification
productivity
n  Adoption of SystemVerilog & UVM for increased
productivity
—  Faster to create reusable verification components,
testbenches and tests
—  Horizontal and vertical reuse:
–  Components, modules, libraries across projects
–  Block to sub-system to system level within a single project

n  Veloce enables a 3rd dimension of reuse for accelerating

SV/UVM
—  Platform portability:
–  Testbenches, ABV, CDV, VIP, etc. across engines/tools

© Mentor Graphics Corp. Company Confidential

71 UVM & Emulation, MDH

 Testbench is Pivotal to Acceleration Speed-Up

Testbench Design
Simulation

Design Acceleration Testbench D

(cycle-based, i.e. co-simulation) 1 – 10 X

Testbench Acceleration T D
(transaction-based, i.e. co-emulation) 10 – 1000 X

© Mentor Graphics Corp. Company Confidential

72 UVM & Emulation,72MDH

 UVM Layered Testbench

Scoreboard

Much of UVM Domain

Test
Controller
Coverage is Naturally Untimed

Monitor Monitor

Stimulus Driver DUT Responder Slave

RTL Transactor Layer Testbench

Layer (Abstraction Bridge) Layer

Timed Domain Untimed transactions

© Mentor Graphics Corp. Company Confidential

73 UVM & Emulation, MDH

 Unified UVM Simulation/Emulation Testbench
SV Classes

Scoreboard SV Interfaces

SV Modules
Test Tasks/Functions
Coverage
Controller
Pin Wiggles

Monitor
Transactions
Monitor
Proxy Proxy
Timed Domain

Untimed Domain
Monitor Monitor
BFM BFM

Responder
Driver Driver Proxy
Stimulus Proxy BFM DUT Slave
Responder
BFM

RTL Transactor Testbench

Layer Layer Layer

© Mentor Graphics Corp. Company Confidential

74 UVM & Emulation, MDH

 Transactors – Co-Emulation Building Blocks
HDL BFM + HVL Proxy + HVL-HDL Channel

Host (Transaction Level) Emulator (Pin Level)

Transactions DUT written in RTL

T/F
call Inbound communication

HVL HDL
RTL
Testbench proxy BFM
API
class mod/if DUT

T/F
Outbound communication call

Higher-level TB
(SV/UVM, or C/C++/SC)
Transactor proxy
Transactor Signal or port
connections
is-a uvm_driver or
uvm_monitor BFM written in “XRTL”

© Mentor Graphics Corp. Company Confidential

75 UVM & Emulation, MDH

 Agent and BFM – Data Flow
Transaction Signal
Variables Values
Agent Passed Passed Signal Bundle DUT

Coverage Net type

Monitor BFM Protocol
Signals
Task and Function
API’s to Interface
Transactions Monitor With agent monitor
Passed

Driver BFM
Sequencer Driver Task and Function
API’s to Interface
With agent driver

© Mentor Graphics Corp. Company Confidential

76 rdo, Advanced Verification, November 2017

 Block Level Test Bench Modules
Test top Hdl_top
Environment
Env Configuration
Predictor Scoreboard
UVM Register BFM
Model
BFM
In Agent Coverage
Config DUT

Out Agent
In Agent Out Agent
Config
Hvl_top

Seq Seq Seq

Seq

© Mentor Graphics Corp. Company Confidential

77 rdo, Advanced Verification, November 2017

 Block Level Test Bench Modules
Test top Hdl_top
Environment
Env Configuration
Predictor Scoreboard
UVM Register BFM
Model
BFM
In Agent Coverage
Config DUT

Out Agent
In Agent Out Agent
Config
Hvl_top

Seq Seq Seq

Seq

© Mentor Graphics Corp. Company Confidential

78 rdo, Advanced Verification, November 2017

 Chip Level Test Bench Modules
Test top Hdl_top
Environment
Env Configuration BFM
Sub A environment Sub B environment
UVM Register BFM
Model
BFM
Sub A env
Configuration
DUT

Sub B env
Configuration
Hvl_top

Seq Seq Seq

Seq

© Mentor Graphics Corp. Company Confidential

79 rdo, Advanced Verification, November 2017

 Chip Level Test Bench Modules
Test top Hdl_top
Environment
Env Configuration BFM
Sub A environment Sub B environment
UVM Register BFM
Model
BFM
Sub A env
Configuration
DUT

Sub B env
Configuration
Hvl_top

Seq Seq Seq

Seq

© Mentor Graphics Corp. Company Confidential

80 rdo, Advanced Verification, November 2017

REUSE

© Mentor Graphics Corp. Company Confidential

 Characteristics of Reusable Components
n  Consistent construction
—  Common constructor arguments
n  Consistent initialization
—  Required information passed down through configuration hierarchy
n  Recursive construction
—  Build their own sub-components
n  Self-containment
—  Contain all required configuration and variables within self or sub-
components
n  Functionality compartmentalization
—  Group related functionality for reuse
© Mentor Graphics Corp. Company Confidential

82 rdo, Advanced Verification, November 2017

 Types of Reuse
n  Horizontal
—  Components and sequences across projects
—  Interface packages, utility packages
n  Vertical
—  Components and sequences from block to top
—  Environment packages, utility packages
n  Platform
—  Simulation and emulation
—  Same structure flow and stimulus

© Mentor Graphics Corp. Company Confidential

83 rdo, Advanced Verification, November 2017

 Block Level Test Bench Modules
Test top Hdl_top
Environment
Env Configuration
Predictor Scoreboard
UVM Register BFM
Model
BFM
In Agent Coverage
Config DUT

Out Agent
In Agent Out Agent
Config
Hvl_top

Seq Seq Seq

Seq

© Mentor Graphics Corp. Company Confidential

84 rdo, Advanced Verification, November 2017

 Chip Level Test Bench Modules
Test top Hdl_top
Environment
Env Configuration BFM
Sub A environment Sub B environment
UVM Register BFM
Model
BFM
Sub A env
Configuration
DUT

Sub B env
Configuration
Hvl_top

Seq Seq Seq

Seq

© Mentor Graphics Corp. Company Confidential

85 rdo, Advanced Verification, November 2017

REGRESSION TESTING

© Mentor Graphics Corp. Company Confidential

 Simulation and emulation in regression testing
n  Each technology has strengths and weaknesses
—  Formal, CDC, simulation, emulation, etc

n  Over reliance on any technology

—  Yields risk exposure due to its weakness
—  Increases project schedule

n  Technologies are compilmentary

—  The strength of one covers the weakness of others

n  Simulation and emulation are complimentary technologies

—  Speed, visibility, cost

© Mentor Graphics Corp. Company Confidential

87 rdo, Advanced Verification, November 2017

 Block Level Testing for Feature Verification
n  Simulation strength
—  Smaller design than full chip
—  Increased controlability of stimulus
—  Full visibility for debug
—  Native functional coverage support Environment

Predictor Scoreboard

n  Transition to emulation
—  Development transitions from Coverage
debug to regression

In Agent Out Agent

© Mentor Graphics Corp. Company Confidential

88 rdo, Advanced Verification, November 2017

 Chip Level Testing for Integration Verification
n  Emulation strength
—  Large design leverages emulation
concurrency
—  Broad stimulus
Environment
–  Traffic patterns testing sub-block
integration Sub A environment Sub B environment
—  Only interconnect visibility required

n  Transition to simulation
—  Reproduce errors discovered in sub
blocks

© Mentor Graphics Corp. Company Confidential

89 rdo, Advanced Verification, November 2017

 Chip Level Testing for Performance Verification
n  Emulation strength
—  Emulation concurrency models
design concurrency
—  Broad stimulus
Environment
–  Traffic patterns testing performance
–  Identify bottlenecks Sub A environment Sub B environment
—  Only interconnect visibility required

n  Transition to simulation
—  Reproduce performance
bottlenecks discovered in sub
blocks

© Mentor Graphics Corp. Company Confidential

90 rdo, Advanced Verification, November 2017

 Chip Level Testing for SW Integration Verification
One user at a time
JTAG
1 MHz

Codelink
100 MHz ~10 Simultaneous users

CODELINK
TRACE FILE

© Mentor Graphics Corp. Company Confidential

91 rdo, Advanced Verification, November 2017

VERIFICATION
MANAGEMENT

© Mentor Graphics Corp. Company Confidential

 Verification management for closing coverage
XML Verification Plan
Import
Plan into UCDB Test Plan Items Linked
Merged during merge
UCDB

Ranking identifies
redundant Tests

Test UCDB’s/simulation allows Test Analyze - Add / Re-run Tests

Triage
© Mentor Graphics Corp. Company Confidential

93 rdo, Advanced Verification, November 2017

© Mentor Graphics Corp.
www.mentor.com
Company Confidential