Introducing Obsidian Software

and RAVEN-GCS for PowerPC

Obsidian Software
Founded in 1997 by 3 ex-AMDers
Located in Austin, TX
Bootstrapped Private Company
Focus: Processors & Verification
Made Inc.500s List of Fastest Growing Companies

Understanding Your Verification Methodology

Describe your processor development activities.
Type/schedule

What is your methodology for stimulus generation?

Are you using any verification languages?
e
Vera

Understanding Your Verification Methodology

Do your current test generation tools have any
limitations?
Limits on randomness?
Coverage holes?
Ease of use?

How many processor verification engineers will be on

the project?
Level of experience
4

Obsidians Mission
Random Test Generators
Standard Architectures
Custom Architectures
Simple Processors

Verification Services

Resource Expenditure
Task

Tool/Solution

Required Effort

Correctness Checking

ISS

Low (Reuse ISS from other

projects)

Testbench Development

Memory Model

Low

Coverage Measurements

Architectural

Moderate

Implementation

Moderate

Code

Moderate

Directed Tests

High

Random Test Generators

High

Engineers Looking at
Waveforms

High

Stimulus Generation
Debug Effort

Common Customer Challenges

Scaling existing test generators for future designs
Challenge of staffing and maintaining in-house expertise
Test generators often need rewrites for new architecture features
Test generation development is often in the critical path
Focus should be on processor, not tool generation
Understanding the cost of generating an internal test generator
The last 10% is the most difficult and highest risk

What do Random Test Generators Do?

Reduce the Need for Directed Tests
Tests Full Range of the Architecture
Generate Directed-Random Tests
Allow Verification Engineers to Focus on
Failing Tests

Model of Operation

The RAVEN Solution

RAVEN is not:
A test bench language (e, vera, perl) to write a Random Test
Generator
Used to wiggle pins on buses

RAVEN is:
A tool to intelligently generate assembly instructions to provide
test stimulus for a microprocessor
A full featured processor specific random test generator
Under complete user control: intelligently direct tests into
difficult to reach areas
Extensible users can add new instructions
10

Architecture
Architecture Description
Extracted from Headers or
Processor Documentation

XML

GUI

XML

ISS
Dynamically linked
shared object

C API

RAVEN Core

XML

C++ libraries
C++ API
(function overrides)

Template File
generated by
GUI or text editor

Legend
Customer

Architectural Layer
Linked in

11

Customer or Obsidian
Obsidian
Generated

Business Model
Standard Processors

Proprietary Processors

ARMv(4-7), MIPS 32/64, X86

CISC, RISC, VLIW, SIMD, DSP

Full featured generators for above cores

Obsidian develops the architecture layer and

licensed as run time licenses

Obsidian maintains architecture layer
Obsidian provides technical support thru
tape-out including RAVEN bug fixes and
work-arounds

delivers it as source code to customer

Core technology licensed to customer as run
time licenses
On-going support of architecture layer can be
done by customer or Obsidian
Obsidian provides technical support for both
RAVEN core and development effort.

Verification Services : Obsidian can provide verification services at additional cost to

increase verification productivity and reduce overall project risk. These services may
include, training, test creation services, RAVEN customization, coverage services, and other
RAVEN related activities.

12

Pricing Considerations
Product or Service

Description

RAVEN licensing

One year time based licensing

Quantity affects total price
Standard software support included in license price

Development services

Engineering resource billed as a result of RAVEN

customization
Engineering resource billed as a result of custom
architecture layer development in RAVEN-GCS

Verification Services

Test creation services, training, coverage related

services, methodology integration, test planning

Intellectual Property

Instruction set simulators

Can be sold as time based license or source code

13

Solution
Structure

Relationship
Trusted
Advisor

Strategic
Partner

Engagement Activities

Impact on Success

Same as below plus:

Onsite RAVEN verification
engineer
High level Methodology
consultant

Same as below plus:

Work closely with team onsite to
maximize productivity
Early stage verification
planning to identify areas of focus
for you and Obsidian

Best
Proactive services insure
success due to identification of
key areas of focus and
dedication to maximum
verification productivity

Same as below plus:

Full time dedicated
development engineer for
ongoing enhancement requests
and feature development

Same as below plus:

Interactive development support
for continued RAVEN
enhancement in synch with your
architecture changes and RAVEN
productivity enhancement
requests

Better
Enhanced development
support to bolster productivity
at all phases in the design cycle

RAVEN Licensing
CPU Arch layer
development
Std. Software Support
RAVEN training

Arch layer development

ISS Integration
Phased product delivery
Arch layer and RAVEN core
support

Good
High-end solution focused
specifically on your processor
feature set

Technology
Provider

14

Engagement Model
Customer and Obsidian agree in principle to move
forward together
Obsidian provides on-site 1-2 day tutorial for detailed |
technical evaluation with following objectives:

Provide in depth technical view of RAVEN

Determine architecture challenges
Gather requirements for test generator

Customer and Obsidian agree on proposal and SOW

15

Engagement Model
Phased delivery (3-9 months)
Integrate core ISS from customer
Deliver architecture layer in multiple phases

Provide upgrades and maintenance for core test

generator as well as guidance for customer
architecture development
Provide sustaining or peak development help for
customer architecture layer as an option

16

In Summary
Collett International Research 2002 Data:
39% First SI success
60% of initial failures caused by logical / functional problems
82% - Design errors
47% - Specification errors
14% - Imported IP problems
50% - 70% of resources consumed in verification

17

In Summary
Verification is Usually on the Critical Path to SI
Failure Costs Ranges From Bad (lots of money)
to Very Bad (lost market)

18

Spins
A few months time to market
Delay of next project
Loss of confidence in the market

RTG Methodology

19

Overview of Methodology
Use random or mostly random templates to
generate initial coverage
Define coverage points based on design knowledge
Write directed-random templates to hit missed
coverage points
Review all templates and relax restrictions where
possible

20

Step 1: Random Templates

Theres random and then theres random
In an immature design some features will be disabled
Some features have enables rather than sliders
Enable exceptions
Coprocessor present
Some control register bits need to be configured at reset
Initial test may be single CPU/thread
RAVEN architectural layer may need enhancement to configure
some behaviors

21

Coverage Using Random Only

Verification Space

Covered
behaviors

Expect about
80% coverage
22

Step 2: Define Coverage Points

Black-box
Coverage defined based on architectural definition
Coverage based on architectural specification from
block designers
Coverage of block interfaces
Should not be sensitive to implementation
Points are defined by verification engineers

White-box
Areas of greatest change or risk
Can be sensitive to implementation
Points are defined by designers
23

Distribution of Coverage Points

Design
knowledge and
risk assessment
determines
placement
and density
of coverage
points

24

Coverage Points

Intersection of Coverage Points and Pure Random

Random
template
coverage is
combined
with
directed
test
coverage

25

Step 4: Review Templates

An engineer should examine all existing templates
Find any areas where the template is artificially restricted and
remove restrictions
Increase the number of instructions per template where
appropriate
Make sure MP/MT behavior is enabled where appropriate
Allow the template to be used as a random macro where
appropriate

Engineer should not be the same as the person who

originally created the template, if possible
26

RAVEN-GCS Introduction

27

RAVEN-GCS Features
Random test generator
Fully-random, semi-directed, and fully-directed
Generator knows full state of the processor at each instruction
boundary
Test files contain all register and memory updates
Full definition of the initial register and memory image
Full report of the final register and memory image
Never generates an invalid test

28

RAVEN-GCS Features
Easy to use

29

Intuitive GUI for template creation

Templates can be run from the GUI or the command line
User and developer manuals
Simple and well-documented interface to simulator
Can automatically place comments in test output
Supports user-defined test sequences that can be used as macros

RAVEN-GCS Features
Expandable to all architectures

30

VLIW any packet size

Multi-processing and/or multi-threading
Paging
Exceptions
CISC or RISC
Cache support
Floating point
Predicated instruction execution
Branch-not-taken code insertion

RAVEN-GCS Features
Biases or constraints for all supported features

31

Instruction
Operands
Addresses
Number generation
Operating modes
Register values

RAVEN-GCS Features
Data files are used to define architecture
Much of the architecture can be modified without recompiling
Features can be added as they become available in the RTL
Easy to work around known RTL or simulator bugs

Test output in XML format

Easy to parse
Source for converter for test output to customer specific format
provided
Example provided for conversion to user-readable form

32

RAVEN ISS interface

Template
File
Test
Output
File

Register Write

RAVEN
Test
Generator

Memory Write
Register Update
Memory Update

Instruction
Set
Simulator

Step Simulator

RAVEN generator reads template file

Populate internal event queue with requested behaviors

Generator picks instruction or event from top of event

queue

For example if the queue element is InstrTree, a specific instruction such as an

ADD register immediate is picked

Generator picks operands/data values/memory

addresses/etc based on current environment (settings and
overrides)
33

ISS-Interface
Generator writes values to uninitialized registers and
data memory
Generator determines exact opcode of instruction and
writes it to the simulators memory
Generator calls simulator Step()
Simulator returns memory and register updates caused by instruction
execution

Repeats steps 2 through 7 until end of event queue

Generator writes test output file including final memory
and register image

34

RAVEN-GCS GUI

35

RAVEN-MP GUI

36

Instruction Overrides

Values for instruction fields may

be generated randomly or may be
overridden with specific values.
Double-clicking on a canvas
element invokes the appropriate
override pop-up window.
37

Test Output Files

Test output files are in XML format
Consist of the following parts:
Initial state
memory and registers
completely defines the state of the processor before any instructions
in the test have generated
register state can be used for hotloading if supported by the testbench
Initialization sequence
used if hotloading is not supported
instructions from reset to create the state needed to execute the first
instruction in the test
38

Test Output Files

Consist of the following parts
Instructions
sequence created from the canvas
all memory and registers updated as a result of instruction execution
are reported
the full state of the system is known at every instruction boundary
Final state
fully defines the state of memory and registers after completion of the
test

39

Architecture definition

40

Architecture defintion
Most of the architecture can be specified in data files
these are all xml files
many can be parsed directly from architecture definition files such as
manuals or simulator header files

Types of files
definition of instructions and instruction trees
definition of operands and operand groups
definition of special conditions
exceptions
pipelining issues
packetization rules for VLIW
display information for the GUI

41

Instruction tree definition

42

Instruction definition

43

Instruction definition
Operand class is AddressingModeOperand

Formula specifies how to determine address

Formula: NIA+(LI << 2)

NIA = next instruction address
LI = value of the LI operand (an immediate)
&lt; &lt; = << syntax due to xml restrictions
=> New address = next instruction address + (immediate << 2)

44

Handling non-determinism

45

Using RAVEN Register Output

RAVEN test output files

contain a complete list of
expected values for all
registers that affect the
behavior of the test.
This example shows a
simple case with no
unpredictable register fields

46

GP1

GP2

CR1

Initial State

0x01234567

0xabcdef0

0x35742278

Instr. 1

0x55327a8b

---

---

Instr. 4

---

---

0x35743378

Instr. 22

---

0xabcd7a1f

---

Instr. 55

0x00000000

0x55327a8f

---

Instr. 56

---

---

0x25743378

Instr. 121

0x00000001

---

---

Instr. 139

0x00000002

0x55327a90

---

Final State

0x00000002

0x55327a90

0x25743378

Basic Register Checking

List of expected
values from test
output file

0x00000002
0x00000001
0x00000000
0x55327a8b
0x01234567

Actual values
from RTL

GP1
Initial State

0x01234567

Instr. 1

0x55327a8b

Instr. 4

---

Instr. 22

---

Instr. 55

0x00000000

Instr. 56

---

Instr. 121

0x00000001

Instr. 139

0x00000002

Final State

0x00000002

0x01234567

=
pass

0x55327a8b
pass

0x00000000
pass

=
fail

47

0x000000a1

Register Checking
With Unpredictable Fields
With each reported register
update, RAVEN also reports a
valid mask
Compare logic needs to examine
only those bits whose
corresponding valid mask bit is
set
The valid mask may change over
the course of the test to reflect
processor state

48

CR1 value

CR1 mask

Initial State

0x35742278

0xffffffff

Instr. 1

---

---

Instr. 4

0x35743378

0xfffffcff

Instr. 22

---

---

Instr. 55

---

---

Instr. 56

0x25743378

0xfffffcff

Instr. 121

---

---

Instr. 139

---

---

Final State

0x25743378

0xfffffcff

Using RAVEN Memory Output

in an MP/MT Test

Each CPU reports all

memory writes
Between zone semaphores,
multiple writes to the same
address results in nondeterministic memory
The memory final state is
reported as bytes with nondeterministic bytes shown
as Xs

49

Writes to Addr X

CPU1

Initial State

0x01234567

Instr. 1

0x01327a8b

---

Instr. 22

---

0xabcd7a1f

Instr. 55

0x00000000

0x39327a8f

Instr. 56

---

---

Instr. 121

0x55abf790

---

Instr. 139
Final State

CPU2

0x55327a90
0x55,
0xXX (0xab/0x32),
0xXX (0xf7/0x7a),
0x90

Memory Checking in an MP/MT Test

CPU1
0x55abf790
0x00000000
0x01327a8b
0x01234567
0x55abf790
0x00000000
0x01327a8b

0x55abf790
0x00000000

0x55abf790

Always compare against

the last unmatched write
for each CPU

CPU2
0x55327a90
0xabcd7a1f
0x01234567

0x1234567

=
0x55327a90
0xabcd7a1f

0x1327a8b or
0xabcd7a1f

0x01234567
pass

0x01327a8b

0x00000000 or
0xabcd7a1f

0x55327a90
0xabcd7a1f

pass

=
0x55327a90
0xabcd7a1f

fail

0x00000000

pass

0x55abf790 or
0xabcd7a1f

=
50

Actual values
from RTL

pass

0xabcd7a1f

Using Zone Semaphores

Zone semaphores divide instruction execution
into known time fragments
A shared memory location is used as a counter
No CPU that shares the semaphore can advance to the next zone
before the counter reaches zero
Execution of all instructions within the zone must complete before
advancing to the next zone
This may require synchronizing instructions
Does not imply cache flush

Not all CPUs need to use all semaphores

51

Constraining non-determinism using Semaphores

CPU1

CPU2

CPU3

CPU4

0xaabbccdd

0xaabbccdd

0xaabbccdd

0xaabbccdd

0x9867a430

0x20080325

0x03195725

0x0f152b35

0xa9fc20d4

0xa9441901

0x2298afc7

0xffffff34

= zone 3

0x00001003

0x70073447

0x7e9a98e0

0x99a9c0d8

= zone 4

0x109000ff

0x70073448

0xacde3f54

0xa5aa0977

0xdcb9444a

0x70073449

0x00000000

0x00000000

0x8987564a

0x7007344a

0x7007344c

0x00000001

0xf0cd7aad

0x7007344b

0x10001000

0x32457699

0x7007344c

0xefffefff

0x75913052

0x7007344d

0x7091304d

= initial state
= zone 1
= zone 2

CPU1 implements all 4 semaphores

CPU2 implements all 4 semaphores
CPU3 implements 2 &4 only
CPU4 implements 1, 2 & 4 only
CPU3 does no writes to this address
in zone 4

All 4 CPUS read and write the same semaphore here

52

Constraining Non-determinism using

semaphores
At the end of zone 1:
9 possible values
At the end of zone 2:
0x8987564a or
0x7007344b or
0x7007344c or
0x00000000
At the end of zone 3:
6 possible values
At the end of zone 4:
0x75913052 or
0x7007344d or
0x7091304d
Final memory reported as:
0x75 or 0x70
0x91 or 0x07
0x30 or 0x34
0x52 or 0x4d or 0x4c

53

CPU1

CPU2

CPU3

CPU4

0xaabbccdd

0xaabbccdd

0xaabbccdd

0xaabbccdd

0x9867a430

0x20080325

0x03195725

0x0f152b35

0xa9fc20d4

0xa9441901

0x2298afc7

0xffffff34

0x00001003

0x70073447

0x7e9a98e0

0x99a9c0d8

0x109000ff

0x70073448

0xacde3f54

0xa5aa0977

0xdcb9444a

0x70073449

0x00000000

0x00000000

0x8987564a

0x7007344a

0x7007344c

0x00000001

0xf0cd7aad

0x7007344b

0x10001000

0x32457699

0x7007344c

0xefffefff

0x75913052

0x7007344d

0x7091304c