ECE 354

Lab 2: Capturing and Displaying

Digital Image

ECE Department: University of Massachusetts, Amherst

Big Picture Introduction

Understand the existing SOPC builder setup with new
modification
Capturing image using camera
Storing the captured image in the frame buffer
Display
p y the image
g on a CRT monitor with the use of a VGA
controller
Perform simple image processing
Primarily written in C code

ECE 354

Additional Hardware
TRDB_D5M Digital Camera
Serial
S i lC
Connector
t and
d
Cable
CRT Monitor

ECE 354

Skills to learn

Integrate code with preexisting code

Understanding previously written code
Connect to a device that you did not design
Writing a communication protocol to somebody elses
specifications

ECE 354

Steps taken to complete project

Begin by looking over camera documentation
Understand SOPC builder setup given
Write a C program to capture data into FIFO buffer
C program to transfer data from FIFO to flash memory
Gain knowledge off how the DE2_NIOS_HOST_MOUSE_VGA
project displays an image to the CRT monitor
Use that knowledge to display your image
Finally have fun performing image processing

ECE 354

Lab2 Setup

[1]
[1] TRDB_D5M_Userguide
http://www.terasic.com.tw/attachment/archive/281/TRDB_D5M_UserGuide.pdf
p
_
_
p
ECE 354

Overview of DE2_NIOS_HOST_MOUSE_VGA
DE2 NIOS HOST MOUSE VGA project
Implements a monochrome display, with a preloaded image,
where the user can draw on it with a mouse
USB mouse should be connected USB HOST port
CRT monitor should be connected to the VGA port
You will integrate your code so that the project displays the
image received from digital camera

ECE 354

SOPC builder setup:

Based on DE2_NIOS_HOST_MOUSE_VGA
The
e program
p og a code is
s stored
sto ed in SRAM
S
memory
e o y
Parallel Input/output interface(PIO) for LED, switches
Camera Avalon Interface(Camera_IF) helps in transferring
image data
The dedicated SDRAM is used as a frame buffer that holds
a single frame of video at the time.
The SDRAM is controlled and accessed by using the fourfour
port controller hardware
Controller can be used to read or write to the SDRAM
through its four FIFO buffers.
buffers
Flash controller is used to control the data transfer to flash
memory
VGA controller
t ll di
displaying
l i
d
data
t on tto VGA

ECE 354

Block diagram of Terasic camera system

TRDB_D5M_Userguide
http://www.terasic.com.tw/attachment/archive/281/TRDB_D5M_UserGui
de.pdf

ECE 354

Hardware Modules for this Lab

SDRAM-Multiport Controller:
Thi module
This
d l iis a generic
i 4 portt SDRAM controller.
t ll
It uses two FIFO buers for inputs (writing) and
two FIFO buers for outputs. The FIFO buers
are made using Altera MegaFunctions and are 16
bit wide and can hold up to 512 words of this
size In this system the SDRAM controller is used
size.
as a frame buer to hold a single picture frame
Flash Controller
VGA controller
Camera Avalon Interface module
ECE 354

10

To get started with the lab

Run the DE2_NOIS_MOUSE_VGA_project present in the
project folder given to you
Run SOPC builder and understand each of the components
added and configuration present
Hardware component
p
files are p
present under IP folder

It contains SDRAM_4 port controller components

Camera Avalon Interface

Software components are present under the software folder

Hello_led_0 has all the required software component files
Some part of code is given in this project which is present in
file
e hello
e o_led.c
ed c

ECE 354

11

Information for C program

C code to capture camera data and store it in frame buffer
Picture taken at 640x480 has 3 data per pixel times 8 bits
(at least) each, this would mean a total of 7,372800Mbit or
921,6 Kbyte
So efficient storage design required
Recommended C code function
Camera Capture
Copying RGB(pixels)
(
l )d
data ffrom FIFO to Flash
l h memory
Using flash memory to store the pixel data before
transferring to VGA output
Program space memory is less (SRAM).Avoid unnecessary
buffers. Write programs as efficient as possible

ECE 354

12

Readout modes: D5M camera

The Terasic D5M camera supports a function
called :
1. binning -This function reduces the resolution
of the image
g by
y averaging
g g pixels
p
together
g
2. Skipping - reduces the output resolution
without
ith t aecting
ti
th
the field-of-view.
fi ld f i
It does
d
this
thi by
b
not sampling entire rows and columns of pixels.
p 2X mode skips
p one of p
pixels for every
y pair
p
A skip
of output.

ECE 354

13

Displaying image through VGA controller

Look through VGA.h for
VGA functions
Write C code similar to
that shown below that
calls functions from VGA.h
to display each pixel
It is your job to come up
with an equation to check
each pixel in your receive
buffer

ECE 354

14

Additional information
The image should be 2-bit gray scale with a resolution of
640x480
Uncompressed image
I recommend starting with the smallest image resolution
and work from there

ECE 354

15

Image Processing
It is required that you implement two forms of
image processing
Recommendations include:

ECE 354

Add timestamp
p onto image
g
Counter to keep track of number of pictures taken
Rotate, mirror, invert image
Simple edge detection (challenging)
Detect changes in images (challenging)

16

References for software and hardware design in Lab2

I recommend looking over section II of the NIOS II Software
Developers Handbook
g p
project
j
titled- Embedded Demonstrator for Video
Go through
Presentation and Manipulation by Cato Marwell Jonasse (Google
it) to get clear insight on hardware description for this project
((You can g
go through
g chapter
p
9 specifically)
p
y)
DE2_NIOS_HOST_MOUSE_VGA project
NIOS II Software Developers Handbook
http://www.altera.com/literature/lit-nio2.jsp
ECE 354

17

Where does this project lead?

Next lab will be on the topic sending data over a network
Sending image between DE2 boards
Dont worry if you have not taken computer networking

ECE 354

18

Questions and Comments

ECE 354

19

Back up

ECE 354

20