Optika-Team Technex 2012 PDF

Team Optika
IT-BHU
VARANASI
IMAGE PROCESSING BASED ROBOTICS
robotics.cca@itbhu.ac.in | www.technex.in
Team Optika
Getting Started
Image Processing comprises of any form of signal processing for which the input is an image. Not very surprising! How do we use image processing in Robotics? The Robot we would be using has an eye. Now where is this eye located? Its generally a webcam that may be permanently attached at some specific location, or attached to the Robot itself. This webcam inputs images, which are sent to the computer via the webcam connecting wire. The concept of Image processing now comes in, and the acquired image is processed using MATLAB programming (introduced later in the tutorial). Now we need to inform the Robot about the processed details. For this, we use a port of the computer, which sends information from the computer to the Robot. The Robot contains a microcontroller which would decode the information send via the port. This information is worked upon, and signals are sent to the motors. We would work on understanding these aspects STEP by STEP as follows 1. Image acquisition Image taken in from webcam to be brought into the environment (MATLAB) where it will be processed 2. Image processing Processing of the image using the environment (MATLAB) 3. Using the port Interfacing using Serial ports 4. Motor control Controlling DC motors and Stepper motors
2
Robotics Club, IT-BHU
robotics.cca@itbhu.ac.in
Team Optika
This diagram roughly depicts the connections.
MATLAB
MATLAB stands for Matrix Laboratory. It is a numerical computation environment and 4th generation programming language that uses matrices to do hell lot of things! As I mentioned we would be using MATLAB, it makes sense that the work of processing the Image would be done in MATLAB. MATLAB uses matrices to store multimedia entities. Hence, our images would also be stored as matrices.
Team Optika
MATLAB environment
We would be using MATLAB 7.8.0 (R2009a) for the tutorial. A typical MATLAB 2009 window would look like the image above. It is divided into four major sections Command Window We write all the commands in this section Current Directory Shows us the contents of the current directory Workspace Displays the details of variables being used Command History Shows the list of commands recently used The toolbar gives option for setting a custom current directory. You were informed that MATLAB does hell lotta stuff using matrices. You can yourself find the tutorials to all that using the Help button
4
Team Optika The Help menu looks like this
We would be restricting ourselves to the Image Acquisition tool and the Image Processing tool for this tutorial. Now lets start working with MATLAB. All the commands would be written on the >> symbol you can see in the Command Window. Type k = 9 and Press ENTER
Team Optika
You can see that MATLAB automatically stores k as a matrix of size 1x1 with value 9. This action also reflects in the Command History. 1D and 2D matrices can be stored as follows
We can use the ; operator after the statement to avoid display.
To access any element in the matrix, we can use indices corresponding to that location. Indices in MATLAB start from 1. For instance
The value stored at second row and first column is 5 We can do simple mathematical operations using the matrix variables -
Team Optika
ans is the default variable of MATLAB, and is used if the assignment operator is not present. We can assign the result of this operation using the = and a variable on the left.
Result will be stored in c instead of ans
Trigonometric functions
sin(1) value in radians is passed sind(45) value in degrees is passed asin(1) inverse sin in radians asind(1) inverse sin in degrees Similarly, we can use cos(), cosd(), tan(), tand(), etc General commands clc Clear the Command Window clear Reset the Workspace and delete all the variables stored
7
Team Optika The colon : operator
The colon operator is used for specifying throughout the range.
Some illustrations of the use of colon operator are given above.
Relational Operators in MATLAB
Control statements in MATLAB Conditions statement if Iteration statement for, while

8
Team Optika Syntax if <condition> statements else statements end while <condition> statements end for <initialization and condition> statements end
zeros(k) : Returns matrix of dimension k-by-k filled with zeros zeros(p,q) : Returns matrix of dimension p-by-q filled with zeros ones(k) : Returns matrix of dimension k-by-k filled with ones ones(p,q) : Returns matrix of dimension p-by-q filled with ones
dot(A,B) : Returns dot product of two vectors A and B
Team Optika Working with Images We can read images in MATLAB as follows
imread() function
Now the variable img stores the image cameraman.tif as a matrix. One would expect an error message on typing this directly, but MATLAB has some default images present, among which cameraman.tif is one. If one wonders the value of the matrix formed, try NOT using the semicolon once!
The image can be displayed using the imshow() function as shown.

10
Team Optika MATLAB supports three types of images
Binary Image (Black and White) Consisting of only two colours
Grayscale Black, White and shades of grey
Colour Contains RED, GREEN and BLUE colours in varied intensities
11
Team Optika RGB value
For each pixel in a colour image, we have a value of the RGB colour triplet. This triplet indicates the intensity of Red, Green and Blue on a scale of 0 to 255 (256 colours). RGB value of an image can be very easily obtained using the Data Cursor tool For instance, in the toolbox of the Figure 1 window of MATLAB. has RGB value as 255, 128 and 0.
You can verify this by using the data cursor tool, which shows -
Making m-files
It sounds quite sensible when I say that we do not write the complete code in Command Window. MATLAB codes are stored in special files called m files. You can use the icon for opening the Editor
12
Team Optika Type in the code inside this editor
Save the m file as sample.m
For running the m file, write onto the Command Window, the name of the m file without the extension
And the output (display of image in our case) will appear
13
Team Optika FUNCTIONS in Matlab
Functions in MATLAB are also stored in an m-file, with an exception that the first executable statement of the file must be of the form function [out1, out2, ...] = <function_name>(in1, in2, ...) The out1, out2, etc are the return values of the function. Unlike C,C++, etc, a MATLAB function can return multiple values. in1, in2, etc are the input parameters required by the function. The function is written in the m-file as follows
Save the m-file with the name of the function stat.m in our case
The function can be executed from the Command Window in the following manner
The vector a is passed to the function stat as parameter, and return values m and std are obtained.
14
Team Optika
Coming on to the aspects of Image processing we would be primarily using 1. Obtaining the Image 2. Isolating the area of Interest 3. Obtaining the desired outputs
IMAGE ACQUISITION
Hardware Information For obtaining an image, we would obviously be having an image acquisition device (like a webcam) onto our PC/laptop. MATLAB has software called adaptors for accessing these devices. You can obtain this information by using this command >> imaqhwinfo Can be visualized as im-aq-hw-info (image-acquisition-hardwareinformation) It produces the following output on my system
The adapter that MATLAB indicates on my system is winvideo as shown above. We can now use this adapter to gallop into further details! For getting some more information onto the default colour format, device name, etc, use the following command 15
Team Optika
In case you have multiple image acquisition devices, you may use further DeviceIDs like 2, 3, etc in the statement
Now we would see the formats that our adapter supports
These are different formats of capturing the video. The 1024x768 indicates the resolution of the video in that particular mode. Defining Video Input Object
winvideo is the adapter name, 1 is the DeviceID of the device we would be using, and YUY2_320x240 is one of the supported format.
16
Team Optika
On pressing ENTER, one would see the following output
This gives all details of the Video Input Object. Now, how to switch the PREVIEW ON?
And the Video Preview window opens with the resolution you specified in the object creation statement -
Preview can also be viewed using different formats (Resolutions) from the Supported Formats list.
17
Team Optika Capturing Image
We use the getshapshot() function for capturing an image from our image input object -
The object appears so weird, you think either your webcam has gone berserk, or its some evil leprechaun!
Nothing to worry, guys! This happened because we took picture in YCbCr family of colour spaces, which is YUY2_... format. You can grab some more details about YCbCr here - http://en.wikipedia.org/wiki/YCbCr We would have to convert this to RGB as follows -
18
Team Optika
The new window shows the snapshot in RGB colour space.
Saving an Image file Similar to imread() command, we have the imwrite() for writing an image variable onto the disk in form of an image file.
Image file stored IMPORTANT Getting a snapshot requires multiple steps switching ON the camera, sufficient light entering in it, capturing taking place. So there should be a pause of some duration between both the statements, when the code is written in m-file.
19
Team Optika
vid=videoinput('winvideo',1, 'YUY2_160x120'); preview(vid); pause(3); % Provides the required pause while(1) img=getsnapshot(vid); % Do all image processing and analysis here end
We now have an idea about how to acquire an image. Let us work on processing the acquired image now!
IMAGE PROCESSING
As the image is now with us as input, our main task is to work on it. Typical Image processing problems include locating a particular object with some colour, or counting objects with the same colour, etc. Let us see a simple example Suppose we have an image shapes.tif
We can store this image in a variable img using the imread() function, and check for the RGB values at different positions. For the image taken, the RGB at Red region is (255, 0, 0), at Green region is (0, 255, 0), and at Blue region is (0, 0, 255).
20
Team Optika We make an m-file as follows
Save this file as BB.m Running the program in the Command Window
Figure Window
21
Team Optika
What we did in the program was to declare a zero matrix with dimensions same as that of the test image. For every pixel location, we test if the regions RED value is 255 and GREEN and BLUE values are 0. If yes, we mark that place as 1 (white), otherwise it remains 0 (black) in the resultant binary image. A piece of cake! But real time problems consist of not a uniform colour, but a distribution of intensities! Let us take another example Suppose we have an image flower.tif in our MATLAB directory
22
Team Optika
The image is processed using the above program. Lets name the file BB2.m On executing the program
We have approximately localized the violet colour region from the real world image. This choice of intensity selection has to be done manually and with great care, as the final binary image will be computed accordingly. For if(im(i,j,1)>50 && im(i,j,2)>50 && im(i,j,3)>120);
23
Team Optika Noise Removal
As we saw in the first binary picture of the flower, the resultant image is distorted, with presence of holes, irregularities, etc. This can be treated using the imclose() function. But for using the imclose() function, we need to define a structuring element first, using the strel command as follows
Combining this with the program
Modified Output
24
Team Optika
Another important function used for filling all holes in a binary or grayscale image is imfill() function Using the imfill() function along with the program
Resultant binary image that is produced does not contain any hole
Similar to imclose() function, we have the imopen() function, which performs morphological opening of the grayscale or binary image. Example
I = imread('snowflakes.png'); imshow(I)
25
Team Optika
se = strel('disk',5); I_opened = imopen(I,se); imshow(I_opened)
What actually happens here is that all snowflakes with radius less than 5 pixels were removed by using the structuring element of type disk with size 5 pixel, which is used. Image Noise removal is very essential for making the data to be computed error free, and hence, our program need to be efficient at it.
26
Team Optika
Some more important commands used for Image Processing are regionprops() STATS = regionprops(BW, properties) It measures a set of properties for each connected component (object) in the binary image, BW. The image BW is a logical array; it can have any dimension.
For example, if you want to locate centroid of any object in a picture you can use the following command s = regionprops(BW, 'centroid'); Web Link: http://www.mathworks.in/help/toolbox/images/ref/regionprops.html
bwlabel() [L, num] = bwlabel(BW, n) It returns a matrix L, of the same size as BW, containing labels for the connected objects in BW. The variable n can have a value of either 4 or 8, where 4 specifies 4-connected objects and 8 specifies 8connected objects. If the argument is omitted, it defaults to 8.
27
Team Optika [L, num] = bwlabel(BW, n)
It returns in num the number of connected objects found in BW. Web Link: http://www.mathworks.in/help/toolbox/images/ref/bwlabel.html
imcrop() I = imcrop creates an interactive Crop Image tool associated with the image displayed in the current figure, called the target image. The Crop Image tool is a moveable, resizable rectangle that you can position interactively using the mouse. I2 = imcrop(I) displays the image I in a figure window and creates a cropping tool associated with that image. I can be a grayscale image, a truecolor image, or a logical array. The cropped image returned, I2, is of the same type as I. Web Link: http://www.mathworks.in/help/toolbox/images/ref/imcrop.html
bwboundaries() B = bwboundaries(BW) traces the exterior boundaries of objects, as well as boundaries of holes inside these objects, in the binary image BW. B = bwboundaries(BW,conn) specifies the connectivity to use when tracing parent and child boundaries. conn can have either of the following scalar values.
Web Link: http://www.mathworks.in/help/toolbox/images/ref/bwboundaries.html

28
Team Optika
USING THE PORT

Now we have a scenario where we have to interface the computer with the microcontroller. When an image is processed, we have a signal for performing some kind of work. This signal needs to reach the uC (short for microcontroller), which is done by the port. We will demonstrate Serial and Parallel ports in our tutorial. PARALLEL PORT SERIAL PORT
In laptops/computers of today, we generally do not have parallel ports, and some modern laptops (like mine!) do not have Serial port also. But they do have multiple USB (Universal Serial BUS) ports. So we use USB to Parallel, or USB to Serial port converters USB to Serial Port USB to Parallel Port
29
Team Optika Parallel Port
The parallel port has 25 pins, which are numbered as follows
The pinouts for a Parallel port are
DB25 pinout
30
Team Optika
Before you being working with parallel port, do check if the parallel port in your system is operational, by following these steps My Computer>System Properties>Device Manager>Ports You will find the name of your parallel port from here. Let us assume it is LPT1 >> parport = digitalio (parallel, LPT1); We have now defined an object named parport. Port address may be obtained using these statements >> get (parport, PortAddress) >> daqhwinfo (parallel); Now we need to define PINS 2-9 as OUTPUT pins using the addline() function >> addline(parport, 0:7, out) Now, depending upon the motion of the robot desired, you can put the data in a matrix using logical() function >> dout = logical([1 0 0 1 0 0 0 1]); Using the putvalue() function, we can output this value >>putvalue(parport, dout); We can also send the decimal (or octal/hexadecimal) value directly >>dat = 259; >>putvalue(parport, dat); While using the parallel port, we do not need any microcontroller, as outputs can be directly received by the motors from the parallel port using H-bridge (L293D or L298 IC Motor driver ICs) for driving the motor in both directions clockwise and anticlockwise.
31
Team Optika Serial Port
A Serial port has 9 pins, which are numbered as follows
Serial transfer means that if you want to transfer a byte of information, then Serial port will transfer the 8 bits bit-by-bit (one bit at a time). As seen from the above diagram, the transmission happens from PIN3 (Tx), and receiving happens at PIN2 (Rx). Now, we would require a microcontroller with UART (Universal Asynchronous Receiver and Transmitter) to be present. Most of the uC that we use in Robotics (like ATMEGA 16L, PIC, etc) have UART, which needs to be initialized for Serial data transfer. The standard used for Serial communication is RS-232 (Recommended Standard 232). This defines the voltage values that would be equivalent to logic ZERO and logic ONE. Valid signals are positive and negative 3 to 15 volts.
32
Team Optika
Now the problem that arises is that our microcontroller works on TTL standards, and hence would not work on RS-232. So, here we introduce IC MAX 232, which would help us in this situation. The connection diagram of MAX 232 with Serial port, for our purpose is given is follows
PINs 11 and 12 can now be used for communication via pins of uC meant for Serial transfer of data. Before you start using the Serial Port, check in the Device Manager (Port section), whether you can see COM ports available. If yes, then your Serial port is ready for some action! Let us assume your Serial port device name to be COM1 We can create a MATLAB object using the following commands >> ser = serial (COM1, BaudRate, 9600); with Baud Rate = 9600 bps (bits per second)
33
Team Optika
Baud Rate - In telecommunications and electronics, baud is synonymous to symbols per second or pulses per second. While setting this value in the above command, we must make sure that the value is the same as set while configuring the microcontroller on-board the robot. The Serial data format includes 1 Start bit 5 to 8 Data bits 1 Stop bit Parity bit (may be there) Additional Stop bit (may be there)
Asynchronous transmission using the UART character So the data that we are sending must be of 8bits. Let us suppose that we keep codes in hexadecimal A Forward B Backward C Turn left D Turn right No we open the Serial port and start sending data through it >> fopen (ser); >> fwrite (ser, A); >> fwrite (ser, C); Signal for moving the robot one unit FORWARD, then turning LEFT.
34
Team Optika
This data has to be sent to the microcontroller. So now we need to program the microcontroller to listen to the COM1 port. Let us take ATMEGA16L as our microcontroller. We would generate the program to be burnt, using CodeVision AVR software. A typical window of CVAVR looks like this
We need to go to File>New This appears
Select Project and click on OK. From the new menu, select the first option, and click on OK
35
Team Optika
Now comes the most important part configuring the microcontroller for use. The window that opens shows several tabs on the left
Select ATmega16L (or any chip you are using) from the Chip tab. We now need to configure the microcontroller for using its USART in asynchronous mode, and along with that set the PORTs of the uC which would be used for signaling the motors.
36
Team Optika
Select the USART tab from the same window
Make the necessary settings in this submenu putting a tick on the Transmitter option, and Receiver also if it is being used in your case. The Baud Rate is by default set to 9600. Notice that this is the same as the rate set by us while defining the Serial port object in MATLAB. Decide and set the communication parameters, and set the Mode as asynchronous. PORTs can be set using the Ports tab
37
Team Optika Once, all settings are done, Click on Program>Generate, Save and Exit
Write the same names in the Save windows that come. The program now appears in front of you with all configuration settings done.
Browse down the window with the program code to find this
You can now write your code here for accepting data from Serial port, checking it across a set of operations (like moving left, forward, blinking LED, etc), and perform the required action using the respective PORTs that you already have set as input/output. Output from here would now drive the motors.
38
Team Optika
MOTOR CONTROL
Now all the data acquisition and processing is complete. All we need to do is to RESPOND to the instructions that the microcontroller gives. We would do this by interfacing the Output PORT of the microcontroller with the motors. The problem that arises is the output potential of the port of microcontroller is not enough to drive a motor, hence, we need to use a Motor driver IC (L293D, L298) for doing the same. We would use these with both DC motors and Stepper motors as given below.
DC Motors
As already mentioned above, the DC motor would require an IC for working with the microcontroller output. We can use L293D for this purpose
39
Team Optika
Pin diagram of L293D Each L293D has four Input-Output pairs. For making a motor run both directions, we need to use two Input-Output pairs per motor. Hence, we can use ONE L293D for controlling two DC motors
40
Team Optika
Vs-pin stands for Motor Voltage and is the voltage with which the motors would be driven. We can use a 12V DC connection on this pin. Enable pins must be set HIGH for enabling the working of both motors. These pins can also be used to DEACTIVATE the motor functionality under certain circumstances as directed by the algorithm. For various INPUT conditions at IN1, IN2 and IN3, IN4, we see the following pattern per motor -
DC motors are simple, but they lack accuracy. In case we want to use DC motors, we have to use an efficient feedback mechanism to overcome and compensate the loss of accuracy. Stepper Motors
It is a brushless, electric motor that can divide a full rotation into a large number of steps. The motor's position can be controlled precisely without any feedback mechanism.
41
Team Optika How it works?
II
III
IV
Hence, a complete rotation happens after four such steps. The angle which the Stepper motor rotates with after completion of any step above is called the Resolution of the Stepper motor. Steppers are available in many Resolutions, ranging from 30 to 1.8 degrees per step. Stepper motor can be either Unipolar or Bipolar. As you can see, each stepper has four coils. Hence we have four connections per stepper motor. So, one L293D can drive one bipolar 2 phase stepper motor. The connection diagram of the same is given as follows
42
Team Optika
We have the four Controller Pins going to the OUTPUT port of the microcontroller. And the data coming through these pins will be controlling the operation of the Stepper. We saw in a diagram above that there is a specific sequence in which coils should get activated, so as to ensure proper completion of a cycle. The pulse sequence has to be in the following order Step Step Step Step 1 2 3 4 Coil 1A High Low Low High Coil 2A High High Low Low Coil 1B Low High High Low Coil 2B Low Low High High
*For reversing the direction, follow the steps in opposite order.
Unipolar Stepper Motors A Unipolar stepper motor has one winding with center tap per phase. The special thing about Unipolar Steppers is that we can reverse the direction of rotation of the motor without changing the direction of current.
43
Team Optika
Thank you for going through the tutorial. If you have any queries or suggestions that you wish to share, feel free to contact me at karanbajaj23@gmail.com karan.bajaj.cse09@itbhu.ac.in
For Unipolar Stepper, we use IC ULN2003/2004 instead of L293D, as it also has a COM port for serving as the Common connection of the coils.
44
Team Optika
45

Optika-Team Technex 2012 PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Optika-Team Technex 2012 PDF

Uploaded by

Copyright:

Available Formats

Team Optika

IMAGE PROCESSING BASED ROBOTICS

IMAGE PROCESSING BASED ROBOTICS

Robotics Club, IT-BHU

IMAGE PROCESSING BASED ROBOTICS

This diagram roughly depicts the connections.

Robotics Club, IT-BHU

IMAGE PROCESSING BASED ROBOTICS

Robotics Club, IT-BHU

Team Optika The Help menu looks like this

IMAGE PROCESSING BASED ROBOTICS

Robotics Club, IT-BHU

IMAGE PROCESSING BASED ROBOTICS

We can use the ; operator after the statement to avoid display.

Robotics Club, IT-BHU

IMAGE PROCESSING BASED ROBOTICS

Result will be stored in c instead of ans

Robotics Club, IT-BHU

Team Optika The colon : operator

IMAGE PROCESSING BASED ROBOTICS

The colon operator is used for specifying throughout the range.

Some illustrations of the use of colon operator are given above.

Relational Operators in MATLAB

Control statements in MATLAB Conditions statement if Iteration statement for, while

Robotics Club, IT-BHU

IMAGE PROCESSING BASED ROBOTICS

dot(A,B) : Returns dot product of two vectors A and B

Robotics Club, IT-BHU

IMAGE PROCESSING BASED ROBOTICS

The image can be displayed using the imshow() function as shown.

Robotics Club, IT-BHU

Team Optika MATLAB supports three types of images

IMAGE PROCESSING BASED ROBOTICS

Binary Image (Black and White) Consisting of only two colours

Grayscale Black, White and shades of grey

Colour Contains RED, GREEN and BLUE colours in varied intensities

Robotics Club, IT-BHU

Team Optika RGB value

IMAGE PROCESSING BASED ROBOTICS

Robotics Club, IT-BHU

Team Optika Type in the code inside this editor

IMAGE PROCESSING BASED ROBOTICS

Save the m file as sample.m

And the output (display of image in our case) will appear

Robotics Club, IT-BHU

Team Optika FUNCTIONS in Matlab

IMAGE PROCESSING BASED ROBOTICS

Robotics Club, IT-BHU

IMAGE PROCESSING BASED ROBOTICS

Robotics Club, IT-BHU

IMAGE PROCESSING BASED ROBOTICS

Now we would see the formats that our adapter supports

Robotics Club, IT-BHU

IMAGE PROCESSING BASED ROBOTICS

On pressing ENTER, one would see the following output

Robotics Club, IT-BHU

Team Optika Capturing Image

IMAGE PROCESSING BASED ROBOTICS

Robotics Club, IT-BHU

IMAGE PROCESSING BASED ROBOTICS

The new window shows the snapshot in RGB colour space.

Robotics Club, IT-BHU