You are on page 1of 40

A Publication

W
PR dow
BE CTIC & .N
in
A s
ST ES ET
:
VOLUME 5 • ISSUE 10 • OCTOBER 2008 • $8.95 • www.stpmag.com

Make Code Correlation


Work For Your Business

Stitch Security and QA Together page 14

Why Automation Projec t s Fail:


How to Keep Your s Toge ther

Who’s Afraid of The


Big, Bad SCM Tool?
4AKE THE
HANDCUFFS OFF
QUALITY ASSURANCE
Empirix gives you the freedom to test your way.
Tired of being held captive by proprietary scripting? Empirix offers a suite of
testing solutions that allow you to take your QA initiatives wherever you like.
Download our white paper, “Lowering Switching Costs for Load Testing
Software,” and let Empirix set you free.

www.empirix.com/freedom
VOLUME 5 • ISSUE 10 • OCTOBER 2008

Contents A Publication

14
C OV E R S T ORY
‘Till Hacks Do Us Part’; Keep
Security and Quality Together
Minimize the risk of security vulnerabilities with security testing throughout
development–from grave-digging to lightning strike. By Danny Allan

18 Find Your Copy-


Cats With SCC
Source code correlation, a process for
finding similarities in application code,
can shed new light on the activities in
your company. By Bob Zeidman

Depar t ments
29
Picking Out
7 • Editorial
If only a tester’s conference were like a soft-
ware application.

Your SCM 8 • Contributors


Get to know this month’s experts and the
Software configuration management best practices they preach.
tools can simplify even the most com-
plex development project, if you steer 9 • Feedback
in the right direction. It’s your chance to tell us where to go.
By Vinny Vallarine 11 • Out of the Box
News and products for testers.

24 Why Automation
Projects Fail
How to keep clear of the obstacles that
33
Two Cents
13 • ST&Pedia
Industry lingo that gets you up to speed.

can cause your automation efforts to On Dev & 36 • Best Practices


fall down, in the words of someone .NET demands new ways of testing.
who’s been there. Part one of three. Testing By Joel Shore
By Elfriede Dustin Why the activities
38 • Future Test
are two sides of the As a security measure, static analysis will fail.
same coin. By Wayne Ariola
By Prakash Sodhani
OCTOBER 2008 www.stpmag.com • 5
A BZ Media Event

OCTOBER 28–30, 2008 • HYATT REGENCY RESTON • RESTON, VA

REGISTER ONLINE TODAY! KEYNOTES


WEDNESDAY, OCTOBER 29
EclipseWorld: the first and only technology conference 8:30 am – 9:15 am
Opening Keynote
dedicated 100% to Java development using Eclipse. Mike Milinkovich
Executive Director The Eclipse Foundation

OUR WORLD-CLASS FACULTY IS READY TO TEACH YOU


WEDNESDAY, OCTOBER 29
AND YOUR DEVELOPERS HOW TO BECOME ECLIPSE MASTERS! 4:45 pm – 5:30 pm

NACI DAI NATE OSTER Afternoon Keynote


307 Developing SOA-Ready Java Web 604 Using the Process Framework Composer to Ivar Jacobson
Applications, Part 1 Create Process Content Co-Creator of UML and the Rational Unified Process

PAT HUFF ANNAS (ANDY) MALEH


507 Looking Good! Polishing Rich Client Apps
406 Managing Eclipse Adoption in Your Enterprise
805 Practical Design Patterns for Rich Client
REGISTER ONLINE FOR
DOUG CLARKE
503 Java Persistence Freedom Through XML
Binding
Development

CHRISTOPHER JUDD
FREE ADMISSION
DWIGHT DEUGO W-4 Develop Better Java EE Applications With
Eclipse Web Tools Platform
TO THE EXHIBIT HALL
W1 Eclipse 101 for Java Developers Visit the EclipseWorld Exhibit Hall for a floor full of demos!
106 Beat Those Java Dependencies: Extend the Find out the latest plug-ins, who is offering what, and
101 First Steps for Building and Deploying Web Tools Platform With Facets
become a better Java developer.
Eclipse RCP Applications
206 Interacting With Relational Databases
EXHIBIT HALL HOURS:
OVER 60 CLASSES TO CHOOSE FROM! Wednesday, October 29 2:30 pm – 7:00 pm
Thursday, October 30 10:00 am – 1:15 pm
DOWNLOAD THE COURSE CATALOG TODAY!

www.eclipseworld.net
PLATINUM SPONSORS SILVER SPONSOR MEDIA SPONSORS PRODUCED BY

www.code-magazine.com

GOLD SPONSORS
Ed Notes

Did I See You In


VOLUME 5 • ISSUE 10 • OCTOBER 2008
EDITORIAL
Editor Editorial Director
Edward J. Correia Alan Zeichick
+1-631-421-4158 x100 +1-650-359-4763
ecorreia@bzmedia.com

Copy Desk
Adam LoBelia
Diana Scheben
alan@bzmedia.com

Contributing Editors
Matt Heusser
Chris McMahon
Joel Shore
Booooston?
ART & PRODUCTION As I write this, BZ Media’s have the ability to test
Art Director conference group and I your code from the very
LuAnn T. Palazzo
lpalazzo@bzmedia.com
are gearing up for the first character of the very
Software Test & Perform- first method (not that you
SALES & MARKETING ance Conference, which is would, of course).
Publisher taking place Sept. 24-26 at As a tester, you could
Ted Bahr the Marriott Copley Place even be involved in the
+1-631-421-4158 x101
ted@bzmedia.com
in Boston. I’m excited to requirements, design and
have the conference right initial development phas-
Associate Publisher in the city rather than in es. And you should. Be-
David Karp
+1-631-421-4158 x102
Cambridge, as in prior cause that’s when many of
years. Not that there’s any- Edward J. Correia the tester’s concerns can
dkarp@bzmedia.com
thing with Cambridge; it’s a beautiful best be addressed. Particularly those of
Advertising Traffic Reprints
Liz Franklin Lisa Abelson
place and our conference venue was security.
+1-631-421-4158 x103 +1-516-379-7097 nestled along the Charles River. Very And that’s the subject of our lead fea-
lfranklin@bzmedia.com labelson@bzmedia.com scenic and tranquil. But being right ture by security expert Danny Allan, direc-
inside the city, I think, will give the tor of security research at IBM Rational.
List Services Accounting
Lisa Fiske Viena Ludewig conference a joie de Danny provides real-
+1-631-479-2977 +1-631-421-4158 x110 vivre that was absent world methods of inte-


lfiske@bzmedia.com vludewig@bzmedia.com across the river. grating the testing
As we prepare to effort with every aspect
READER SERVICE
Director of Circulation Customer Service/
produce the confer- of development, from
Agnes Vanek Subscriptions ence in a new venue, requirements through
+1-631-443-4158
avanek@bzmedia.com
+1-847-763-9692
stpmag@halldata.com
it occurs to me that As a member of a to post-deployment
there’s no real way to maintenance.
Cover Photograph Courtesy of Sideshow Collectibles,
test any of the hotel’s development team, you By the time you
Thousand Oaks, CA facilities in advance read this, the confer-
—to test our confer-
ence “software” and
have the ability to test ence will be over, so I
can’t invite you to sit
processes before de-
ployment, as it were.
your code from the in on a class by Danny
Allan. He’s scheduled
Sure, we can tour the to teach two one-hour
facilities, see and
very first character of sessions on security
President BZ Media LLC
Ted Bahr 7 High Street, Suite 407 hear from the host, vulnerabilities and
Executive Vice President Huntington, NY 11743
talk about what they
the very first method. Web 2.0 concerns.
Alan Zeichick +1-631-421-4158
fax +1-631-421-4130 offer in terms of Perhaps you hap-
www.bzmedia.com
info@bzmedia.com

Software Test & Performance (ISSN- #1548-3460) is


published monthly by BZ Media LLC, 7 High Street,
space, equipment,
infrastructure, cater-
ing and security. We
can sometimes even
• pened to end up in
one of those. If so,
turn to page 14 to
gain more from his
Suite 407, Huntington, NY, 11743. Periodicals postage hear the opinions of experience.
paid at Huntington, NY and additional offices.
other conference organizers that have
Software Test & Performance is a registered trade-
mark of BZ Media LLC. All contents copyrighted used the facility. But I would see that About the Cover
2008 BZ Media LLC. All rights reserved. The price as a red herring; the hotel isn’t likely Special thanks go to Heidi Haynes at
of a one year subscription is US $49.95, $69.95 in
Canada, $99.95 elsewhere. to provide negative references, after Sideshow Collectables (www.sideshowtoy
POSTMASTER: Send changes of address to Software all. .com), of Thousand Oaks, Calif. Heidi
Test & Performance, PO Box 2169, Skokie, IL 60076.
Software Test & Performance Subscribers Services Fortunately, technical conferences was kind enough to send high resolution
may be reached at stpmag@halldata.com or by

um format figures (both are sold out). ý


calling 1-847-763-9692.
are not like software applications. As a photos of two of its most popular premi-
member of a development team, you

OCTOBER 2008 www.stpmag.com • 7


Contributors
DANNY ALLAN is director of security
research at IBM Rational. Prior to its
acquisition of security tools maker
Watchfire, Danny spent seven years
with that company, serving in positions
of team lead of consulting services and
engineer.
In Security and Quality: A Union
of Equals, which begins on page 14,
Danny explores ways to minimize the
impact of security vulnerabilities by performing security testing at
all phases of development.

BOB ZEIDMAN is president of


research and development con-
tractor Zeidman Consulting. He has
created a mathematical framework
for expanding source code correla-
tion for different requirements and
presents it here, beginning on page
18, and explains how to make it
work for you.

ELFRIEDE DUSTIN, author of


“Effective Software Testing” (Add-
ison-Wesley, 2002) and lead author
of numerous other works, joins us
again to grace our pages. This time
she presents a three-part series on
software automation testing, begin-
ning with “Why Automation Pro-
jects Fail,” on page 24.

Software configuration management


activities track software objects in
development. VINNY VALLARINE,
senior software engineer with defense
contractor IDT, has set up and been
using SCM systems for nearly a
decade. Beginning on page 29, Vinny
tells you what’s important in an SCM
tool and how to choose the best one
for your organization.

Development teams are often given


all the time they need, and testers
have to cope with what’s left. So says
career test automator PRAKASH
SODHANI, who explains on page 33
why he thinks testing and develop-
ment teams should be treated as two
sides of the same coin.
Prakash is a quality control spe-
cialist with a global IT organization.

TO CONTACT AN AUTHOR, please send e-mail to feedback@bzmedia.com.

8 • Software Test & Performance OCTOBER 2008


Feedback

WAVING AT OPEN SOURCE


Regarding “Functional Testing Tools, the PAT ON THE BACK, BUT...
Open-Source Wave,” Test & QA Report,
I’d like to let you know that you are doing a great job by educating SQA community
July 22, 2008, (www.sdtimes.com/link and engineering managers in modern testing techniques. I highly recommend Software
/32583): You missed a big one--Selenium Test & Performance to my colleagues and students. This is the only magazine, in my
from OpenQA.org (selenium.openqa opinion, that addresses real practical issues in software testing. I like the topics the
.org). There hasn't been much announced magazine presents as well as its “look and feel:” clear diagrams, graphs, and glossy
about it yet, but Twist also looks to be an paper.
exciting new "keyword" tool that will
be based on the Selenium engine To make it even better, I propose introducing permanent rubrics,
(studios.thoughtworks.com/twist). such as Testing Methodology,Test Documentation,Test Automation,
Brian Dall Quality Metrics and Defect Tracking.

I was surprised you did not include I’m always looking forward to new interesting issues of Software
the test automation tool Watir in Test & Performance.
your article. Watir has been down-
loaded more than 85,000 times and Vladimir Belorusets, PhD
is easily one of the most widely used SQA Manager, Xerox
tools for automating functional tests Palo Alto, CA
of browser-based applications.
Pete Dignan to write documents and not take offense. Maybe you can even
and e-mails for address in a future issue whether we
Thanks for the information on the various clients that were easy to understand and should draw the line between QA, test-
tools. Providing the original web site for reduced confusion," she says. ing, and good old-fashioned copy editing
the source of your searching is helpful. I Of course, as anyone with a basic grasp and proofreading (and no, I do not think
found your brief description of the various of grammar should know, it would be even that running Spell-Check suffices,
tools useful enough for me to consider nicer if there were a way to write documents although it’s better than nothing).
investigating some of them. My company and e- mails for clients that were easy to Anne Simoncelli
has adopted TestLink as a test case repos- understand and reduced confusion. Relex Software, Greensberg, PA
itory and execution tracking tool. We have Ah, the subjunctive - For most people,
adapted to the quirks of the tool and I have it would seem, it's an inflammation of Regarding “Open-Source Tools: Asked
found it to be useful but it could use some the clear lymphoid tissue membranes that and Answered,” Test & QA Report, Aug.
more enhancements that are currently in cover the white part of the eyes and line 5, 2008, (www.sdtimes.com/link /32664):
the queue waiting for the funding that is the inside of the eyelids. As someone mentioned WatiR we should
necessary to make the changes. I now have Reg Blank now also mention WatIN (watin.source-
another web site to add to my list of useful forge.net), which is a Web application
places to visit. "And good written communication skills testing tool for .NET languages, and of
Thanks again. start with a command of the language." course it is free. It works similar as
Gretchen Henrich Is that a sentence? Good written? Funny RANOREX (www.ranorex.com) which
article.... the article is about writing well, was once also free but now they detected
You missed the eValid solution, which but is poorly written. Good one. they've built something that is worth some
goes them all one better by putting ALL Randall Karl money. WatiN also provides a recorder.
of the test functions inside a browser. Even though we all know that cap-
www.e-valid.com CATCHING UP ture/replay has little to do with mature
Edward Miller, President Regarding “Playing Catch-Up With This test automation, recorders are still appre-
www.e-valid.co, San Francisco, CA Week's Report,” Test & QA Report, August ciated and useful if you use it to help you
26, 2008, (www.sdtimes.com/link/32764): identify how objects are addressed.
ME RITE GUD, GET JOB,YAH? Maybe it’s not too late to edit your article Cheers.
Regarding “Good Communication Starts – it’s “pore” over, not “pour” over. (That is, T.J. Zelger
with Good English,” Test & QA Report, unless you are actually talking about spilling Audatex Systems, Switzerland
Sept. 2, 2008, (www.sdtimes.com/link/327 your energy drink/beer/wine /whatever
97): So, the woman who wants to explain over your technical manuscripts…) FEEDBACK: Letters should include the writer’s
the importance of the use of correct Call me fussy. I like to think that prop- name, city, state, company affiliation, e-mail
grammar to all and sundry doesn't know er word choice, spelling, grammar, and address and daytime phone number. Send your
the difference between a conditional and punctuation fall under the Testing head- thoughts to feedback@bzmedia.com. Letters
become the property of BZ Media and may be
a hypothetical, eh? ing, and so they’re fair game. Hopefully
edited for space and style.
"It would be nice if there was a way you will find my nitpicking instructive,

OCTOBER 2008 www.stpmag.com • 9


Out of the Box

Arxan Keeps Apps


Ahead of Pirates
Piracy prevention tools maker Arxan in cryptography key
recent weeks began shipping Trans- representation,”
formIT, a tool for adding cryptograph- which the company
ic key protection to your applications. describes as algo-
The company also released a new ver- rithmically strong
sion of GuardIT for Microsoft .NET, its enough to resist so-
automated obfuscation tool that now called break-once,
protects applications based on run-everywhere
Silverlight. (ORE) attacks. Keys
With the addition of protection for are always hidden,
Microsoft’s multimedia platform, even in memory
GuardIT for Microsoft .NET now pro- dumps and memo- The latest version of Arxan’s GuardIT for Microsoft .NET Frame-
vides anti-tamper protection to appli- ry remains. An API work protects applications based on Silverlight.
cations written in C/C++, C#, VB.NET facilitates integra-
and Silverlight, as well as applications tion with other applications and tweak- separation of software from hardware.
that combine managed and native code. ing of the depth and breadth of key The tool also helps prevent phishing,
GuardIT for Microsoft .NET is available graphs, to fine-tune security strength. spoofing and code tampering attacks,
now. Arxan offers similar tools for Linux TransformIT also permits applica- the company says. “We’re pleased to
and Windows in 32- and 64-bit editions. tions to tie into machine constraints, bring the latest in key hiding and pro-
Released in August, TransformIT hardware IDs and other node-locking tection technology to the market,” said
features something called “white-box anchors to decrease the likelihood of Arxan CEO Mike Dager.

uTest Community: More Testers For


Projects, More Income for Testers
Testing software-as-a-service function, load, perform-
company uTest in August ance and usability testing.
launched the uTester Testers can sign up for free.
Community (www.utest.com Organizations that want
/community.htm). to use the service select a
The new service puts team and provide a link to
organizations lacking in- their software. Teams can
house testing staff in touch be selected based on test-
with software testers in need ing experience, knowledge
of work. of programming and spo-
The Massachusetts-based ken language(s), geograph-
start up positions the service ic region or other factors.
as a kind of social network, The platform links with
which companies can tap into an organizations existing
periodically during peak Bugzilla, Jira or other bug
times in their development tracking system for in-house
cycle or continuously as part monitoring and mainte-
of a process. nance.
uTest provides on-line bug-tracking and a project status screen that is acces-
A counter at the uTest Web sible to all project members and ties into a company’s existing bug tracker. Communications between
site currently shows nearly a company and its contract
10,000 testers, which are mostly in the scription and on-demand pricing to testers is provided by the uTest platform.
U.S. and India. match your business pattern. Spot pric- There's also a thriving blogger com-
The company offers annual sub- ing starts at US$2,000; services include munity (blog.utest.com).

OCTOBER 2008 www.stpmag.com • 11


formatted in the Automatic Test
Skytap API Brings Markup Language used extensively by
data profiling and data quality in an
open source suite.”
Cloud Down to Earth the military, supports USB interfaces I’ll go out on a limb and guess that
Skytap on Sept. 8 took the wraps off an and has batch-build capability. claim is accurate. But why did data quali-
API that’s like a conduit between its Usability has been improved, the com- ty get top billing? “Companies in every
cloud-based virtual systems and your pany said, of its back and forward his- industry know too well the costs of poor-
company’s terrestrial ones. The compa- tory navigation and auto-recover fea- quality data, including lost sales, wasted
ny earlier this year released Virtual Lab, tures. It also supports files formatted as employee hours, unnecessary mailing
Web-based infrastructure that virtually MIME HTML, the proposed standard costs and damaged reputations,” said
provisions hardware, software, network- for Web page archiving that binds page Bertrand Diard, CEO and co-founder of
ing and storage in which to run and test links with HTML code in the same Talend in a statement that seemed to
applications. file,(often with an .mht extension). answer my rhetorical question. “Talend
With its new API, development teams Data Quality ‘cleanses’ dirty data until it’s
accurate, complete and consistent, so
can create what the company character-
izes as a hybrid model whereby physical
Fanfare Trumpets companies can regain control of one of
systems can be extended to make use of iTest 3.2 their most valuable assets—their data.”
those in the cloud, transparently to the Dirty data might be defined as nick-
Development and test tool maker Fanfare
applications and systems involved. names such as “Meg” or “Peggy” for
recently began shipping iTest 3.2, the lat-
Making this possible, the company Margaret, duplicate records and incor-
est version of its test automation tool for
explains, is a REST-based Web service rectly shortened street addresses, the lat-
testing devices and multi-unit systems.
interface that enables programmatic ter of which are repaired using reference
According to the company, iTest 3.2
control of cloud resources with public, data supplied by such sources as the U.S.
now has the ability to perform “In-Grid
static IP addresses providing access to Postal Service and those of other coun-
Analysis” of test pass/failure and improves
Skytap environments. IT environments tries, for example. The suite also
transferability of tests with a test bed
are linked via VPN, all of which is auto- includes the ability to enrich data with
rollover feature. Also new is support for
mated and GUI-driven. An organiza- census data, credit scores, GPS location
Ixia’s IxNetwork and IxLoad test equip-
tion’s existing virtual machines can be and other information.
ment software, applications written in Java
uploaded without modification to the Talend just this summer introduced
Swing, as well as Syslog and Wireshark
Virtual Lab and controlled from the Open Profiler, a GUI-based tool for
protocol analyzers.
ground as usual. The infrastructure sup- Linux, Unix and Windows that allows
At its core, iTest is a recording solu-
ports hypervisors from Citrix and development and test teams to peek
tion. It captures a tester’s actions, com-
VMware; support for Microsoft Hyper-V inside data sources they’re about to work
mands and executed tests, be they exe-
is planned. Linux, Solaris and Windows with to verify that the data adheres to
cuted at the command line or an SNMP,
operating systems are supported. quality goals or metrics.
Web or CMD-shell app. After changes are
“The advantages of cloud comput- Open Profiler comes with a metadata
made to the application under test,
ing introduce an entirely new model repository for storing results of its file
devices and test actions can be recalled
for IT,” said Skytap CEO Scott Roza, and data introspections. This metadata
and reproduced. The tool also generates
“where organizations can leverage can then be used by developers and data
all documentation automatically, chron-
their existing virtualization invest- analysts to create metrics and indicators
icling each command, action and device
ments, increase business agility and and track improvement or degradation
response.
reduce costs by transitioning … envi- of data quality over time. These indica-
Recorded test scenarios, or test scripts,
ronments into the cloud.” tors are statistics such as groups of data
can be sent to remote testers, developers
with certain numbers of rows, null val-
or automation teams for reference, edit-
ues, distinct or unique values, and dupli-
ing and reuse through iTest’s drag-and-
Simplify Measurement drop interface. iTest works with Linux,
cates or blank fields.
Other indicator values can be mini-
With ATEasy Solaris and Windows.
mum, maximum and average lengths
Geotest-Marvin Test Systems in early of text in fields; computation of
September released ATEasy 7.0, an
An Open-Source numerical summary values such as
update to its test and measurement Data Quality Suite mean, average, inner quartile and
application generator that can now cre- range definitions; and mode, frequen-
ate DLLs for use with external lan- With Talend cy tables and other advanced statistics.
guages or by its own drivers. The com- Talend, which makes open-source data The tool also can render those statis-
pany also has added multi-threading integration tools, was scheduled in tics as tables and graphs.
savvy to its debugger and support for September to begin shipping Data
Send product announcements to
external executables. Quality Suite, claiming it to be “the first
stpnews@bzmedia.com
ATEasy 7.0 can now read test results product to combine data integration,

12 • Software Test & Performance OCTOBER 2008


ST&Pedia
Translating the jargon of testing into plain English

Application Security
Neither of us is a professional security tester, birth, or social security number. To do this,
but we’re both enthusiastic amateur security the criminal sets up a realistic-looking website,
testers. Security testing is a field both broad similar to a legitimate site, then attempts to
and deep, so this issue of STPedia we focus on lure prey to the site with e-mail “alerts.” The
the security of computers exposed to the gen- victim sees the alert, clicks and logs into the
eral Internet in three ways –possible exploits, fake site using their personal information The
Web technologies and the security techniques criminal saves that information and can then
used to prevent them. Matt Heusser and gain access to the real site to transfer balances,
Chris McMahon arrange loans, and so on.
Possible Exploits
BLACK HAT SCAN (PORT SCAN)
Person responsible for malicious attacks on
computers and networks. Also known as a
Q: What would your
answers be?
The use of a script to quickly identify all open
ports on a system. Many intrusion detection
‘Cracker.’ Did you exhaustively test systems are sensitive to port scans. Many black
this? Are we doing an SVT hat tools are designed to avoid intrusion
CRACK after our BVT? Does the detection systems.
Software that tries to log in using dictionary performance testing pass?
entries. The popularity of such software in the What are your equivalence SQL INJECTION
1980s gave rise to password security policies classes? Which heuristics
SQL is a database language; SQL injection is
requiring capitalization, numbers, and non- an attempt to gain access to a Website by mod-
are you using?
word characters. ifying submit fields. The most common
method is to change the username and pass-
CROSS SITE SCRIPTING (XSS)
Some Web sites, such as forums, allow users to
A: ST&Pedia will help
you answer questions
like these and earn the
word fields to something like:

write text for others to view. A black hat might respect you deserve. uid’ or ‘hello=hello
include some JavaScript in a forum post that
takes control of your browser to download a Upcoming topics: If the SQL is generated dynamically, it may
virus, re-direct to a specific website, or be used as return TRUE and log the user in, despite hav-
a pawn for a denial of service attack. (See below) November ing an invalid password.
Testers Choice Awards
DENIAL OF SERVICE (DOS) Web Technologies
December
To make a large number of independent serv-
Test Automation
PORT
ice requests simultaneously in an attempt to Internet protocols agree to communicate,
disrupt the service for legitimate users. A January 2009 with each element of the communication
black hat may attempt to gain control of mul- Change Management, ALM assigned a number between 0 and 65535. That
tiple independent systems and do a synchro- number is a port. Some ports are reserved for
nized DOS attack; we refer to this as a distrib- February certain uses, such as port 80 for HTTP. The
uted denial of service, or DDOS. Tuning SOA Perfomance SQL Slammer worm exploited an error in
Microsoft SQL Server communicating over
MAN IN THE MIDDLE (MITM) March port 1434. Exposing one’s database to the
Web Perf. Management
One type of attack is to intercept communica- Internet is in itself a security risk (see firewall.)
tion - such as a username and password – April
between a sender and a recipient. Unlike Security & Vuln. Testing ROOT
phishing, this attack does not seek to imper- The root user on Unix-like systems controls
sonate either sender or recipient, but seeks the entire machine. The phrase “to get root”
Matt Heusser and Chris McMahon
only to capture and exploit private informa- is the goal of a class of security attacks regard-
are career software developers,
tion. testers and bloggers. They’re col- less of operating system.
leagues at Socialtext, where they
PHISHING perform testing and quality assur- Prevention
A type of fraud, phishing is a process by which ance for the company’s Web- CERTIFICATE
a criminal attempts to obtain sensitive infor- based collaboration software. An independent, third-party guarantee that
mation such as a user login, password, date of continued on page 37 >

OCTOBER 2008 www.stpmag.com • 13


By Danny Allan

eb applications, a long-ago niche, have


W become a key focus for companies deliv-
ering critical business functionality. Growing consumer
demand for online access to products and services
once found only in brick and mortar facilities has
forced organizations to expand their businesses and
thus, their technology requirements. The quality of
those applications—their functionality and perform-
ance—is a reflection of the company brand.
There’s no bigger reflection of application quality
than that of the security of your data or that of your
customer and the applications that house it. Building
security into the software development lifecycle has
become a business imperative.
Of the 90 percent of all externally accessible appli-
cations that are Web-enabled, two-thirds have
exploitable vulnerabilities, says Gartner research.
Applications with significant security risks continue
to be released because security coverage within the
development and quality assurance cycle is often an
undocumented and informal process. Security needs to
be considered in the early stages of software develop-
ment, not at the end of development or after deploy-
ment.
Incorporating security into the development stages
doesn’t mean developers need to fully understand how
to exploit buffer overflows, cross-site scripting or other
security issues – this remains the domain of IT and the
security team. Developers and testers must ensure,
however, that all input is properly validated, that they

Danny Allan is director of security research at IBM and a fre-


quent conference speaker.

OCTOBER 2008
understand the fundamentals of building functional,
secure and compliant software and then be responsible If Hackers
for regular testing in the quality assurance process. Any
emergent issues can be considered software bugs and
defects, and require testing by the quality assurance
Come At Your
team.

Understanding Common Vulnerabilities


Website Like
The first step in incorporating security into the SDLC is
understanding what the threats are and where they’re
coming from. Since 2004, the Open Web Application
Monsters In
Security Project (OWASP) community has maintained a
list of the top 10 vulnerabilities. The top three offend- The Night,
ers on this list include cross-site scripting (XSS), injec-
tion flaws and malicious file execution. While it may be
possible to train developers, quality assurance profes-
Will Your Apps
sionals and the development community about the top
ten vulnerabilities, it is unreasonable and prohibitively
expensive to ensure that this community understands
And Data Be
the techniques and nuances of all Web application vul-
nerabilities.
A more successful approach is to focus on the most
Protected?
common vulnerabilities and attacks, and then to ensure
that the development teams understand the causes and
the potential outcomes of these issues.
For example, developers should understand that not
only can cross-site scripting result in identity theft and
sensitive information leakage, but a single XSS vulnera-
bility within the application means it can no longer
trust that any user interaction was intended by the user,
and the user can no longer trust that anything they view
on the site is legitimate. It could also lead to a classic
man-in-the-middle attack where a remote individual
could monitor the user environment including all

OCTOBER 2008 www.stpmag.com • 15


MONSTER MARRIAGE

and the ways they can be addressed, it for the most common and risky securi-
TABLE 1: FIRST STEPS
can start to think about how to effec- ty issues, it frees them to focus on the
Phase Action tively introduce security into its inter- more obscure while maintaining an
1 Employ external security consult-
nal software development lifecycle. audit of the common issues.
ants to do security assessment
There are two key concepts when The second concept to consider is
2 Security team implements pre- implementing this process: increment iteration. It is not enough to roll out
deployment application vulnerabili- and iteration. these tasks as a one-time action within
ty scanning tool and begins com- Security awareness and testing can be the SDLC. The most powerful mecha-
municating results
effectively addressed through an incre- nism for increasing security is to
Security team begins logging secu- mental introduction to the security ensure that these actions are consis-
3
rity defects into defect tracking process and testing strategy. The phased tently repeated within the process. By
system approach follows the five steps in Table 1. enabling security testing in the func-
As the organization reaches the tional or performance test plans of the
4 Quality assurance and security later phases, it should continue to quality assurance team, the repetitive
team collaborate to run automated employ external security consultants nature of this work ensures that appli-
security tests within scheduled
for security assessments and penetra- cations are repeatedly tested for secu-
regressions
tion testing of more sensitive and reg- rity issues before being deployed.
5 Quality assurance team configures ulated applications. It might also It is also important to consider some
and runs application security tests, choose to alter the role of the security essential foundations that enable this
passing security issues and fix rec- team to perform random audits of the incremental and iterative security initia-
ommendations into the defect applications in the pipeline to ensure tive to be rolled out together: education,
tracking system
that the process is working as planned. corroboration, consolidation and automa-
The overall goal is to use tion. Enablement is not
viewed pages, all submitted form val- the security team as appli- just putting the tools in
ues and all user keystrokes. cation security architects, the hands of the develop-
A single injection flaw targeted at
the database within an application
could lead to the remote access of all
the data stored within the central data-
rather than security
assessment resources.
In parallel with this
phased introduction to
• ment and quality assur-
ance teams to ensure that
testing can take place.
One of the key challenges
base. It is important to understand security testing, it is also Security is no of securing software with-
however, that injection flaws can be valuable to roll out an in an organization is that
targeted at the file system, LDAP and incremental vulnerability different than those responsible for the
XML data stores. The potential for test plan – especially in creation and testing of the
large scale data theft is significant. phases four and five. It is any other aspect software are not security
Understanding the vulnerabilities difficult to introduce the practitioners and are
and their potential business impact quality assurance and of software therefore not familiar
allows the organization to make sound development teams to with many of the threats
decisions around whether to accept, the full scope of security quality. It is and attacks that are con-
mitigate or transfer the business risk. issues that may occur. To tinually being developed
facilitate a more priori- only treated and communicated
Securing the SDLC tized and practical through the security and
Once an organization recognizes the approach, it is a worth- differently. underground communi-
vulnerabilities that exist, the causes while exercise to control ties.
the number and type of Education is a signifi-
tests that are transferred
to the quality assurance
team, slowly growing
both their responsibility
• cant and important com-
ponent. The quality
assurance team members
who will be responsible
and knowledge over time. for security testing must
Choosing an initial set of security be aware of the security issues and pos-
issues that can be mitigated through a sess the required skill set to run the
similar remediation strategy makes for automated scanning tools in their
a strong first step. An example of this toolkit. And because turnover is a real-
is the decision to automate the two ity in today’s software development
most common vulnerabilities, cross- world, an automated Web-based train-
site scripting and SQL injection. Table ing system can be considered the most
2 lists the key vulnerabilities that can cost-effective means of ensuring this
be integrated over time. ongoing education.
Over the longer term, the security Corroboration is an equally essential
team will continue to be responsible practice. Ensuring the security team
for the full breadth of security issues, has a close working relationship with
but by passing on the task of testing the quality assurance team is a key ele-

OCTOBER 2008
MONSTER MARRIAGE

ment in designing this plan for suc- deployments. It also


cess. It may be as simple as organizing does not consider the
weekly meetings between the two opportunistic nature of
groups during the rollout, and then the attacks happening
maintaining an internal wiki of key today. As the old story
contacts for future reference. goes, you do not need
Consolidation of policies has also to outrun the bear, you
proven valuable for incorporating merely need to outrun
security into the software development the person who is with
lifecycle. This ensures that security you.
testing is included in all test plans and For the purposes of
will become an ongoing business fac- making good business
tor gateway for all new or changed decisions, it is neces-
applications. sary to implement a
Automation is the final piece of glue standardized scoring and prioritiza- the application through the necessary
for introducing security into the tion method. The industry standard cycles of functionality and performance
SDLC. It helps decrease the probabili- for making this calculation is general- testing to ensure that:
ty of successful attacks by malicious ly the Common Vulnerability Scoring a) the application works as planned
individuals. . Not only can automated System (CVSS). It makes calculations b) it will be able to perform in the
technology find and report security based on exploitability, impact, and deployed environment
vulnerabilities, but its capabilities also environmental and temporal factors. Integrating security into the corpo-
extend beyond testing. It can be used Establishing a common baseline of rate culture is nothing more than
to ensure that the development and this type allows an organization to extending quality management to
quality assurance teams participate in include security as part of its mandate.
ongoing educational classes, and that TABLE 2: BE AFRAID This goes beyond running some testing
the process is effectively implemented tools as part of a final gateway.
Phase Tests
so software is not bypassing required Governance must be incorporated from
1 Cross-site scripting
gateways. Automation can serve as a HTTP response splitting the very beginning of the SDLC through
status check to ensure that production SQL injection to deployment and maintenance.
systems are deployed appropriately. Extending quality management is
2 Blind SQL injection employing early and iterative quality
Measuring the Business Impact Insecure direct object reference management, insight and control busi-
Information leakage and improper
The business impact of a single appli- ness processes, and early and continu-
error handling
cation security breach can be monu- ous application and Web site compli-
mental. For example, CardSystems 3 Malicious file execution ance management. It is deploying a
Solutions was once valued at a billion Cross site request forgery comprehensive and integrated platform
dollars. But following a single data that moves beyond simple functional
breach the company sold in 2006 for make sound business decisions about testing to include security, compliance
$47 million. whether to actually mitigate an issue and policy testing that then integrates
The immediate reaction to the dis- and then to know where to begin. the results into change and defect man-
covery of any security vulnerability is The longer organizations wait to agement systems. It means that collabo-
often that it should be fixed. However, test for security issues, the higher the rative and information sharing systems
this is not always the appropriate cost. Whether or not the decision is to not only include resources from devel-
course of action. Realistically speak- mitigate a security vulnerability, the opment and quality assurance, but also
ing, no software is completely secure. length of time that elapses before test- from the security team.
Just as it should be a business decision ing directly impacts the number of Security is no different than any
to deploy an application, it is also a individuals involved in the entire other aspect of software quality. It is
business decision as to whether a vul- process. There will be an increased only treated differently today because
nerability should be fixed. Sometimes number of people determining what many security practitioners have histori-
it makes more business sense to either should be done and, if necessary, an cally insisted it is so.
accept or to transfer the risk to some increased number of people involved Due to the rich nature of today’s
other mitigating factor. in re-testing after the mitigation is Web environment, vulnerabilities are
The more traditional approach to implemented. The bottom line is sim- bound to appear. To minimize the
determining whether a specific risk ply thattesting early and often is criti- impact on businesses and the con-
should be mitigated is to attempt to cal to minimizing business impact. sumers they serve, the security process
measure the cost of compromise and must begin at the start of the develop-
the potential value obtained. The cost High Quality Software Is ment cycle with the development team
of compromise should exceed the Secure Software fully involved. And it must continue
value gained by the attacker. However, How does an organization determine across all phases of development
this approach does not take into that software is of good quality? The through deployment. Increased soft-

organizational costs. ý
account the increasing value of the answer to this simple question is usually ware quality assurance results in lower
data that is stored in many software that the quality assurance team has put

OCTOBER 2008 www.stpmag.com • 17


By Bob Zeidman

omparing software to find similarities is useful and sometimes nec-


C essary in many areas of computer science and industry. Several
years ago I developed a program to
automate comparison of source code
for determining when one program
had been copied from another. Later I
formalized the process and called it
“software source code correlation.”
Over the years, I’ve created a math-
ematical framework for expanding
source code correlation for different
requirements. In addition to its origi-
nal use for detecting copyright
infringement and trade secret theft, it fields of computer science and high-
also has the potential for detecting tech industry.
patent infringement. And I believe
that software source code correlation The Genesis of Source
can also be used for refactoring and Code Correlation
clone detection, reverse engineering, Years ago I read a fascinating article
software testing and various areas of about artificial intelligence by renown-
intellectual property litigation. ed psychology professor Ulric Neisser.
In this article I begin with a brief Later as a student at Cornell University,
history of the concept of source code I had the opportunity to take his class in
correlation, then describe its theoreti- cognitive psychology. We studied the
cal basis and the mathematical frame- controversies surrounding use of the IQ
work for expanding it, and conclude tests as a measurement of intelligence.
with suggestions on its uses in various Professor Neisser believed that the IQ
test measured one form of intelligence
Bob Zeidman is president of Software but not all forms6. I had come to a simi-
Analysis & Forensic Engineering Corp., lar, though more cynical position years
which offers custom software development,
before when, having joined Mensa, I
training and legal consulting services.
came across a group of people who had

18 • Software Test & Performance OCTOBER 2008


scored highly on IQ tests. But I found
that many members exercised judgment
son can sit on. But people recognize a
dollhouse chair even though no one Source Code
that I personally found hard to describe can sit on it. Some chairs have backs,
as intelligent. some have armrests, some don’t have
Professor Neisser believed that IQ
measured only one aspect of intelli-
either. Rosch proposed that we have a
prototypical concept of a chair in our
Correlation Can
gence. In his view, intelligence was best minds that has many different charac-
described using a Roschian model,
defined by Professor Eleanor Rosch of
teristics based on our experiences with
chairs. When we see a new object we
Show How
the University of California at Berkeley. compare the various features of the new
A Roschian model refers to defining
something by a specific prototype
object to our prototypical concept of a
chair. The more features the object has Honest Your
rather than by description7. in common with features we’ve
For example, consider a chair. This observed a chair possessing, the more
seems pretty easy to describe. You might
say a chair has four legs, but stools are a
likely we are to call the object a chair.
Professor Neisser believed that our
People Are,
form of chair that can have three legs. A concept of intelligence is similar. We
beanbag chair has no legs. A chair
might be described as something a per-
have a prototypical concept of an “intel-
ligent person” based on our experi-
Or Are Not!
OCTOBER 2008 www.stpmag.com • 19
COPYCAT CATCHER

TABLE 1: THE FATAL ELEMENTS from the other. Instead I was perform-
ing a fuzzy comparison of various ele-
Software Source Code Elements Description ments and mentally weighing them
Statements Cause actions to occur.They are sequence dependent. until at some point I declared them
Instructions Signify the actions to take place. similar or not similar – copied or not
Control words Control the program flow (e.g. “if”, “case”, “goto”, copied.
“loop”). This is in fact what most expert wit-
nesses in software IP litigation do.
Operators Manipulate data (e.g. +, -. *. /).
Often one expert will testify that “in
Identifiers Reference code or data.
my 25 years of experience I’ve never
Variables Identify data. seen code that matches as much as this
Constants Identify constants. unless it was copied, therefore this is
Functions Identify code. copied.” Whereas the opposing expert
Labels Specify locations in the program. might say “in my 30 years of experi-
Comments For documentation purposes only. Cause no actions ence, I’ve seen code that looks a lot
to occur. like this, and thus I believe this is not
copied.” Not only did this kind of
ences. When we meet a new person, we tual property disputes. I had been con- examination bother me ethically –
judge their intelligence based on how tacted by an attorney who needed me because of its pure subjectivity – but
many features they share with other to examine thousands of lines of com- I’ve always believed that everything is
intelligent people we’ve met. Because of puter source code to look for cross- quantifiable, even Roschian cate-
this, IQ tests measure only one aspect of contamination (code in one program gories.
intelligence and for each of us it may be getting into another). Being paid by I decided that I could quantify the
an important or unimportant aspect. In the hour, this might seem like the amount of similarity between two pro-
addition to logic reasoning ability, we ideal job, but I grew bored pretty grams by comparing specific ele-
may consider aspects of intelligence to quickly as I skimmed over hundreds ments, which I called “software source
comprise observational powers, memo- of lines of code and ran code correlation.” In sta-


ry, diction, artistic skills, musical ability, simple checks on the tistics, correlation is 0
fluency in multiple languages, common files. I decided to auto- for completely unrelated
sense, perseverance, or combinations of mate the process by writ- variables, 1 for perfectly
these. Upon entering the class I dis- ing a small utility pro- identical variables, and -1
agreed with the professor, but by the
end I had been convinced.
gram.
I realized there were
After discussions for completely opposite
variables2. For our pur-
The idea of Roschian categories and certain aspects of the poses, there is no such
prototypes stuck with me, but I never source code that I con-
with computer thing as source code that
thought I’d have a chance to apply it to sidered important. After is completely opposite (at
anything in my fields of study. That was discussions with other
scientists... least I can’t think
until I began working in software intel- computer scientists and of any such thing), so I
lectual property litigation and needed a expert witnesses, I deter- I determined only considered correla-
way to compare different programs to mined that source code tion values ranging from
find similarity or “correlation.” Slowly it could be divided into that source code 0 to 1.
became apparent that Roschian cate- basic elements. They’re I decided initially that
gories could be applied to software. shown in Figure 1 and could be divided there were three ele-
Even more importantly, I found that a described in Table 1. ments that I considered
mathematical framework could be cre- Once source code is into basic when comparing source
ated to measure similarity based on divided into these basic code – statements, com-
Roschian categories. I didn’t need to elements, I then thought elements. ments, and identifiers[8].
completely describe the program in about what I looked for I was mostly interested in


order to compare it with another pro- when comparing source statements that were
gram. And most importantly, this math- code from two different identical and comments
ematical framework could be applied, I programs to determine that were identical, but
believe, to more complex things in whether they were simi- identifiers that were iden-
other areas of computer science and lar. It occurred to me tical or nearly identical
also to areas of study outside the field. that comparing source code was a were also of interest because identifier
For example, I believe this can be used Roschian categorization process in names could easily be subject to a
to produce a better measure of intelli- that there was no single characteristic global search and replace to try to
gence than the IQ. that I was looking for, but rather a hide copying. Yet identifier names
number of characteristics. There was have a certain amount of useful infor-
Source Code Correlation also no particular element or a certain mation, so identifiers with similar
I first found the need to measure the number of matching elements that names could be a sign of copying.
similarity of computer programs in my would make me say the two programs With this in mind I defined three
work as an expert witness for intellec- were similar or that one was copied “dimensions” of correlation, each

20 • Software Test & Performance OCTOBER 2008


COPYCAT CATCHER

dimension being an important aspect FIG. 2: THE FATAL CALCULATIONS


of similarity. While statement and
comment correlation consider only
exact matches, identifier correlation W-Correlation
considers exact and partial matches.

PS Statement correlation. A-Correlation


PC Comment correlation.
PI Identifier correlation.
S-Correlation
It occurred to me that this method
was not complete, and that two pro-
grams could be completely different
with respect to their individual ele-
M-Correlation
ments but still perform a similar func-
tion in a similar way. I would obviously
consider those programs, and their success of CodeMatch, I began to correlations are normalized such that
source code, to be similar. To perform think about whether a weighted sum correlation values are between 0 and 1
similar functions in similar ways would was really the correct way to calculate (see Figure 2).
require a functional correlation that source code correlation. CodeMatch I then created tests to find which of
could at least be approximated by ranks files by their relative correlation these means of measuring source code
sequences of instructions. So I added a values, but sometimes copied files correlation was best. Being best was
fourth dimension, which I call the would be found in as many as ten or determined by the measurement that
instruction sequence correlation. more places from the top of the list. In popped the greatest number of copied
other words, for any given file it might files to the top of each list. In other
PIQ Instruction sequence require looking at 10 or more differ- words, for each given file, the best
correlation. ent files to find a copy or partial copy. measurement caused the copies of the
This makes sense because most file to be highest in the list of relative
The next question was how to com- source code files in a particular lan- correlation scores.
bine these individual measures into an guage will have constructs that are It was very close between S-
overall measure of correlation. My ini- similar for many different reasons and Correlation and M-Correlation, but S-
tial thought was to simply use a weight- will thus have some correlation 10. Correlation won out, at least for pla-
ed sum8. This worked well and was While this was significantly better than giarism detection11. CodeMatch was
implemented in a program called examining hundreds of files, I began eventually integrated into the
CodeMatch that I developed for com- thinking about other ways of measur- CodeSuite tool suite and the correla-
paring program source code to find ing the overall correlation. I came up tion score is currently implemented as
plagiarism9. This source code correla- with four different possibilities that S-Correlation. However, I do believe
tion worked better in tests than other made sense – a weighted sum, an aver- that the other correlation measure-
methods of detecting plagiarized age, a sum of squares, and a maxi- ments may turn out to be useful for
source code8. mum, that I called W-Correlation, A- other applications. Some of those
Despite the efficiency of the corre- Correlation, S-Correlation, and M- areas include:
lation calculation and the commercial Correlation, respectively. All of these • Plagiarism detection
• Refactoring and clone detection
FIG. 1: THE PERIODIC TABLE OF SOURCE CODE • Reverse engineering
• Software testing
• Intellectual property litigation
We have already discussed plagia-
rism detection. Each of the other areas
is discussed in the following sections.

Refactoring and clone detection


Refactoring source code is the process
of making changes to the code without
changing the functionality to make
the code simpler and easier to under-
stand and maintain. In other words,
refactoring is a fancy word for tidying
up. One recognized way to refactor
code is called clone detection and
elimination. Clones are identical or
nearly identical sections of code that
perform the same function. This hap-

OCTOBER 2008 www.stpmag.com • 21


COPYCAT CATCHER

pens, for example, when multiple pro- statement correlation and instruction piled and the resulting source code
grammers on a project need the same sequence correlation would be impor- can be compared to known source
function and each write it independ- tant whereas comment and identifier code. In that case, statement correla-
ently. It can also happen when pro- correlation would be less important. tion and instruction sequence correla-
grammers use third party code tion would be emphasized because
libraries or open source code in differ- Reverse Engineering identifiers and comments would most
ent places within a project. The ten- Reverse engineering is another field likely not exist in the decompiled
dency for duplication is higher in larg- where source code correlation may code.
er projects with many programmers, turn out to be important. The goal of
particularly if communication is poor. reverse engineering computer code is Software Testing
It also happens when code is modified to understand the functionality of the Source code correlation can be used
over a long period of time, especially code by analyzing it. Reverse engineer- to compare a new version of a pro-
when the modifications are imple- ing is typically performed on object gram’s source code to a previous ver-
sion to locate sections that have the
lowest correlation. These would be the
sections where the most significant
changes have been made, and test
engineers can focus on testing these
new sections. On large projects it may
not be possible for the programmers
to know where the most significant
changes occur without measuring cor-
relation.
Another testing area that could
benefit from the technology would be
comparisons of code output against so-
called golden files. With this kind of
testing, golden files are files that con-
tain program output from test cases
that have been manually verified for
correctness. When the software is
revised, the test cases are rerun and
the outputs are compared against the
golden files.
This technique may be the only
realistic way of testing certain kinds of
software, but it has major problems.
The most significant problem is that it
is rare for subsequent versions of soft-
ware to produce identical outputs.
Instead, the outputs vary from version
to version until the resulting output
mented by different programmers. code, where the source code may be file may be significantly different from
Clones make maintenance more “decompiled” into source code. Such the original golden file. Typically, test
difficult. Among the biggest problems decompiled code has few, if any, rele- engineers will look at differences and
with clones is when a modification vant identifier names and no com- sign off on the differences, but for
needs to be made to a particular algo- ments, since these are usually elimi- software of any complexity, this is an
rithm, especially if the modification is nated during compilation. Reverse error-prone process.
a correction to a bug. The modifica- engineering may involve understand- In the case of programs that output
tion needs to be made multiple times, ing source code that has been pur- program code, such as compilers, syn-
once for each clone. If the clones are posely obfuscated or for which the thesizers, and automatic code genera-
unknown, the bugs will continue to documentation no longer exists. tors, source code correlation can be
crop up during normal testing or use There are reverse engineering tools used to compare the output programs
in the field until all of the clones are to aid in this process3, and source code against the golden programs. In this
discovered and corrected. correlation can offer an important case, rather than looking for exact
Clone detection is the process of means. To reverse engineer source matches, correlation scores can be
automatically detecting clones in soft- code, it could be compared to code of used to determine where the output
ware source code1, 5. Source code cor- known functionality to determine sec- programs are most similar and where
relation is a great way of pinpointing tions that are similar. The source code they are most different. It is those
duplicate or near duplicate code could be compared to entire databases places where the code is most different
based on their high correlations. For of code where the code functionality is that a test engineer should examine
clone detection it would seem that known. Object code can be decom- more closely.

22 • Software Test & Performance OCTOBER 2008


COPYCAT CATCHER

Intellectual Property Litigation However, I consider source code


REFERENCES
As I have found out through experi- correlation to be more of a framework
1. Elizabeth Burd, John Bailey, “Evaluating Clone
ence, software often walks out of one for comparing source code rather Detection Tools for Use during Preventative
company and ends up in another. than a specific set of requirements. Maintenance,” Proceedings of the Second IEEE
International Workshop on Source Code Analysis
Sometimes just the concepts behind Just as with other Roschian categories and Manipulation (SCAM’02), 2002.
the software walk out the door. like intelligence, different people may 2. Wilbur B. Davenport and William L. Root, An
Introduction to the Theory of Random Signals and
According to the law, when the actual have different concepts or different Noise, McGraw-Hill Book Company, Inc., 1958.
code is taken from one company to needs to determine the correlation 3. Adrian Kuhn, Stéphane Ducasse, Tudor Gîrba,
“Enriching Reverse Engineering with Semantic
another without permission, that is between different programs. Someone Clustering,” Working Conference On Reverse
copyright infringement. When the determining patent infringement Engineering (WCRE 2005), 2005.
secret concepts used in software are would need an additional dimension 4. Chao Liu, Chen Chen, Jiawei Han, Philip S. Yu,
“GPLAG: Detection of Software Plagiarism by
taken, that’s trade secret theft. Of to the correlation beyond statement, Program Dependence Graph Analysis,” 12th ACM
course just because one company accus- comment, identifier and instruction SIGKDD international conference on Knowledge
discovery and data mining (KDD’06), August
es another of such practices, it is not sequence correlation. That would be 20–23, 2006.
necessarily the case. Tools are needed called functional correlation. There 5. Jean Mayrand, Claude Leblanc, Ettore M. Merlo,
“Experiment on the Automatic Detection of Func-
to determine the facts. This is where are tools in existence that already map tion Clones in a Software System Using Metrics,”
software source code correlation began out the functionality of different pro- icsm, p. 244, 12th International Conference on
and where it has really found its niche. grams4. A comparison of these maps Software Maintenance (ICSM’96), 1996
6. Ulric Neisser, Cognition and Reality, W.H.Freeman
There are a lot of software patents could be included as another dimen- & Co Ltd, 1976.
being granted and a lot of software sion of source code correlation that 7. Eleanor H. Rosch, “Natural categories,” Cognitive
Psychology 4: 328-50, 1973.
patent lawsuits. Software source code would be particularly useful for detect- 8. Robert Zeidman, “Software Source Code
correlation as described here has only ing patent infringement. Correlation,” 5th IEEE/ACIS International Confer-
ence on Computer and Information Science and 1st
a little use in detecting software patent CodeSuite is free for code compar- IEEE/ACIS International Workshop on Component-
infringement, because patents don’t isons that total less than 1 MB, and can Based Software Engineering, Software Architecture
and Reuse (ICIS-COMSAR’06), July 2006.
always involve direct copying. One per- be downloaded at www.SAFE-corp.biz. I 9. Bob Zeidman, “Detecting Source-Code Plagiar-
son may have thought of a great way of leave it up to readers to apply source ism,” Dr. Dobb’s Journal, July 1, 2004.
implementing a function in software code correlation to the other areas I’ve 10. Bob Zeidman, “What, Exactly, Is Software
Plagiarism?” Intellectual Property Today,
and patented it. Another person may discussed as well as areas I haven’t even February, 2007.
have independently discovered that considered. I hope to hear from those 11. Robert Zeidman, “Multidimensional Correlation
of Software Source Code,” The Third Inter-

and furthering this concept. ý


same method but did not patent it, or of you who are interested in expanding national Workshop on Systematic Approaches to
tried to patent it too late. Digital Forensic Engineering, May 22, 2008.

PERFECTION IN SOFTWARE PROTECTION

CodeMeter - All in One and One for All


Q Security
CodeMeter deploys state-of-the-art encryption technologies for the
maximum protection against illegal copies. The encryption keys
y are
ys
stored securely in the CodeMeter hardware.
Q License models
Every license model, even complexe ones, can be built with Software Protection
Sof
WibuKey and CodeMeter.
Document Protection
Docu
D

Order your Free Software Development Kit now!! Access Protection


A
Phone 1-800-6-GO-WIBU | order@wibu.us
Media Protection
M

Q Options in the CmStick


A license entry in the CodeMeter hardware is characterised
by a Firm Code and a Product Code.
Q Special benefits of CodeMeter
The user can employ the CodeMeter hardware for both a license
manager and for storing sensitive data in a very secure way. WIBU-SYSTEMS USA Inc.
110 W Dayton Street,
Q Programming of the hardware Edmonds, WA 98020
There are many attractive options for programming United States
the CodeMeter hardware both within your company and at your www.wibu.us
customers’ location. info@wibu.us

OCTOBER 2008 www.stpmag.com • 23


wibu_st_and_p_0808.indd 1 03.09.2008 8:28:55 Uhr
d Test ing
M ech anize
P er ils Of ip You U p
One: The The m Tr
Par t t to Let
N o
And How
By Elfriede Dustin

ven though many companies believe that automated software defects), and the usability of most-

E testing is useful, few companies actually succeed at it. At least,


what we’ve discerned from user group ing. Software/testing is now also driv-
often used functionality, plus the relia-
bility. That is defined as the probabili-
ty that no failure will occur in the next
postings and a survey conducted by ing the businessBusiness executives “n” time intervals.
IDT in 2007. can have the best business ideas, but if Testing invariably gets the blame:
This first installment in a series of the software and testing teams don’t Deadlines are looming and the testing
three analyzes why so many automated deliver or the testing efforts are cycle in multiple environments can be
software testing efforts fail and offers behind, the competition is only a few numerous and seemingly endless.
solutions for how to avoid or over- clicks away. First to market is the key. Testing often gets blamed for missed
come the pitfalls. It also seeks to clari- Much attention needs to be paid to deadlines; projects that are over-budg-
fy some of the misperceptions sur- both areas. et; uncovered production defects and
rounding automated software testing lacking innovation.
and explains a lightweight process Perceived vs. Actual Quality But often the real culprits are inef-
based on the Automated Software The best quality processes and stan- ficient system engineering processes,
Testing Lifecycle (ATLM) described dards cannot solve the perception such as the black box approach where
in the book “Automated Software issue— the concept of perceived ver- millions of software lines of code are
Testing,” (Addison Wesley, developed including vast
1999) can help solve many amounts of functionality, only
automated software testing
woes.
Research and development
and its resulting technologies
• for it to be handed over to a
test team so they can test and
peel the layers of code,
painstakingly finding one
have been fueling high tech Some of the real culprits of testing defect after another, some-
product innovation for the last times not uncovering a major
20 to 30 years. Yet our ability being late are bad development showstopper until another
to test these technologies has defect is fixed.
not kept pace with our ability practices resulting in buggy code. In other words, some of the
to create them. real culprits of testing being
While innovation does not late are bad development prac-
seem to consider related
required testing technologies,
testing has become more
important than ever.
• tices resulting in buggy code,
requiring long and repetitive
fixing cycles.
Also problematic is the lack
At IDT we spend much time sus actual quality. For example, 10 of unit testing. Statistics show (and my
researching the latest testing tech- defects that occur very frequently and experience can back them up) that the
nologies and have come up with the a impact critical functionality would be more effective the unit testing efforts
technical approach we are currently perceived by most stakeholders as poor the smoother and shorter the system
using and refining. But we also have quality even if the defect density was testing efforts will be.
determined some interesting trends low relative to the entire project. On Inefficient build practices also play
we need to pay attention to, which the other hand, 100 defects that occur a role. Build and release processes
include: infrequently and have almost no should always be automated. If they
• Software development and test- impact on operations would usually be are not, building software can be time-
ing are driving the business perceived by an end user as good quali- consuming and error prone.
• Paying attention to the issue of ty even if the defect density was rela- Unrealistic deadlines are just
perceived vs. actual quality tively high. that–unrealistic. Deadlines often are
• Testing invariably gets the blame Not much research goes into set in stone without much considera-
• Developers don’t test “usage-based testing,” which exploits tion for how long it will actually take to
• Software development and test- the concept of perceived quality, yield- develop or test particular software.
ing are driving the business ing higher perceived quality, and thus Setting unrealistic deadlines is a
An important shift has taken place happier customers. One such example sure way of setting up deliverables for
that needs to be recognized and man- of great perceived quality and usability failure.
aged. While business needs once is amazon.com versus other online
drove software/testing technologies booksellers. In my experience, ama- Developers Don’t Test
Photograph by Scot Spencer

almost exclusively, the trend is shift- zon.com is most user-friendly I acknowledge that this is a generaliza-
The goal here needs to be to tion, but while many developers con-
Elfriede Dustin is currently employed by improve our perceived quality. We can duct unit testing, and proponents of
Innovative Defense Technologies (IDT), a soft- accomplish this by focusing our testing test driven software development do a
ware testing consulting company specializing
on the most often used functionality really good job testing their software
in automated testing.
(which absolutely has to work without modules, there is still a lack of develop-

OCTOBER 2008 www.stpmag.com • 25


AUTOMATION FAILURE

er integration or system testing. Some include: er, or Web system, log files may be writ-
might suggest shifting away from the • Building testability into the appli- ten on several machines, so it is impor-
testing focus and instead focusing on cation tant that the log includes enough
improving development processes. • GUI/Interface testing considera- information to determine the path of
While this is not a bad idea, even if we tions execution between machines.
implemented the best processes and • Adherence to open architecture It is important to place enough
had the most brilliant developers in standards information into the log that it will be
house: Software development is an art – • Adherence to documentation stan- useful for analysis and debugging, but
integration and system testing will dards to include standard ways of not so much information that it will
always be required, since most develop- documenting test cases and using not be helpful due to an overwhelm-
ers are concerned with their compo- the OMG Interface Description ing volume of information, which can
nents working only, without having the Language, for example; make it difficult to isolate important
big picture view of the system. • Following best practices, such as entries. A log “entry” is simply a for-
There are the human factors as to the Library Concept of Code Reuse matted message that contains key
why developers don’t system test: they Building testability into the applica- information that can be used during
don’t have time; they don’t specialize in tion and GUI/interface considerations analysis. A well-formed log entry will
testing and testing techniques; they are are discussed here, while the other best include the following pieces of infor-
busy churning out new code and func- practices listed will be discussed in the mation:
tionality, and it’s not their responsibility follow on article of this series. Class name and method name. This
to test the integration of the system can also simply be a function name if
code. As Josh Bloch, chief Java architect Building Testability Into the function is not a member of any
at Google one said: “Regardless of how tal- The Application class. This is important for determin-
ented and meticulous a developer is, bugs Software developers can support the ing a path of execution through sever-
and security vulnerabilities will be found in automated testing effort by building al components.
any body of code – open source or commer- testability into the application. Host name and process ID. This will
cial,” “Given this inevitability, it’s critical Building testability into the applica- allow log entries to be compared and
that all developers take the tion can be supported via tracked if they happen on different
time and measures to find various ways. One of the machines or in different processes on
and fix these errors.”
Developers
strapped cranking out
new features while trying
are
• most common ways to
increase the testability of
an application is to pro-
vide a logging or tracing
the same machine.
Timestamp of the entry (to the mil-
lisecond, at least). An accurate time-
stamp on all entries will allow the
to meet unreasonable It is important mechanism that provides events to be lined up if they occur in
deadlines. Again, getting information about what parallel or on different machines.
to market first is the key. to place enough components are doing, Messages. One of the most impor-
While the large corpo- including the data they tant pieces of the entry is the message.
rations focus only on information in are operating on, and any It is a description, written by the
R&D, there needs to be a information about appli- developer, of what is currently hap-
focus on R&D&T. the log that it cation state or errors that pening in the application. A message
Additionally, lack of are encountered while can also be an error encountered dur-
software development will be useful the application is run- ing execution, or a result code from
considerations for auto- ning. The test engineers an operation.
mated software testing, for debugging can use this information Gray box testing will greatly benefit
(i.e. building testability to determine where from the logging of persistent entity
into the application) is
another pitfall to avoid.
but not errors are occurring in
the system, or to track the
IDs or keys of major domain objects.
This will allow objects to be tracked
Automated software test- processing flow during through the system during execution
ing efforts can fail when
overwhelming. the execution of a test of a test procedure.
software development procedure. With these items written to the log
doesn’t consider the auto-
mated testing technolo-
gies or framework in
place. Software develop-
• As the application is
executing, all compo-
nents will write log
entries detailing what
file by every method, or function, of
every component in the system, the
following benefits can be realized:
• The execution of a test procedure
ers can contribute to the methods, also known as can be traced through the system
success of automated testing efforts, if functions, they are currently executing and lined up with the data in the
they consider the impacts to automated and the major objects they are dealing database that it is operating on.
testing efforts, when making code or with. The entries are written typically • In the case of a serious failure, the
technology changes. Additionally, if to a disk file or database, properly for- log records will indicate the
developers consider some of the select- matted for analysis or debugging that responsible component.
ed best practices described here, auto- will occur at some point in the future, • In the case of a computational
mated software testing efforts can reap after the execution of one or more test error, the log file will contain all
the benefits. The selected best practices procedures. In a complex, client/serv- of the components that partici-

26 • Software Test & Performance OCTOBER 2008


AUTOMATION FAILURE

A
pated in the execution of the test
procedure, and the IDs or keys of UTOMATION BEST PRACTICES
all entities used.
Along with the entity data from the A year-long IDT software automated testing survey was conducted; it was posted on com-
mercial QA user group sites; sent to tens of thousands of test engineers; and was posted on
database, this should be enough infor-
government tech sites, such as Government Computer News, Defense Systems, and
mation for the test team to pass on to announced during a webinarii we conducted, called “Automated Testing Selected Best
the development personnel to isolate Practices.” We received over 700 responses, world-wide. Here is a breakdown of the
the error in the source code. respondents’ demographics:
Following is an example of a log file
from an application that is retrieving a Over 73% of the respondents were from the US, while the rest was from other countries
customer object from a database: throughout the world, such as India, Pakistan, China, Europe, and others.
Nearly 70% claimed commercial as their organization type, while 10% claimed
Function: main (main.cpp, line 100) Government, and the rest claimed other, such as educational or independent.
Machine: testsrvr (PID=2201) For 40% the size of the organization was less than or equal to 300 employees, 60%
Timestamp: 8/6/2009 20:26:54.721 claimed an organization size of above 300 employees.
Message: connecting to database [dbserver1, cus-
tomer_db] The outcome of the survey showed that the value of automated software testing is gener-
Function: main (main.cpp, line 125)
ally understood, but often automation is not used or it fails. In the survey we asked respon-
Machine: testsrvr (PID=2201)
dents why in their experience automation is not used and the largest percentage respond-
Timestamp: 8/6/2009 20:26:56.153
Message: successfully connected to database ed with that Automated Software Testing does not get implemented due to lack of
[dbserver1, customer_db] resources, i.e. time, budget, skills.

Function: retrieveCustomer (customer.cpp line 20) Closely related to the above questions, we also received feedback as to why automation
Machine: testsrvr (PID=2201) fails. The highest percentage responded that many Automated Software Testing efforts fail
Timestamp: 8/6/2009 20:26:56.568
and tools end up as shelf-ware; and while 72% state that automation is useful and man-
Message: attempting to retrieve customer record
agement agrees, they either had not implemented it at all or had limited success.Their rea-
for customer ID [A1000723]
son for not implementing Automated Software Testing was:
Function: retrieveCustomer (customer.cpp line 25) • 37% lack of time
Machine: testsrvr (PID=2201) • 17% lack of budget
Timestamp: 8/6/2009 20:26:57.12 • 11% tool incompatibility
Message: ERROR: failed to retrieve customer • 20% lack of expertise
record, message [customer record for ID A1000723 • 25 % other (mix of above, etc.)
not found]

This log file excerpt demonstrates Here are various quotes providing reasons for limited automated software testing success
or failure:
a few of the major points of applica-
• “We have begun implementing, but aren’t allowed significant time to do so”
tion logging that can be used for • “Have implemented some efforts but lack of time, budget and resources prohibits us to
effective testing. fully perform this function”
In each entry, the function name is • “The company has previously implemented automated testing successfully, but this was
indicated, along with the filename and years ago and we currently don’t have the time or budget to re-implement”
line number in the code where the • “I’m the only one automating (so have some automation), but spend too much time on
entry was written. The host and New Feature release, need more people”
• “Accuracy of automated processes are the largest issues we have encountered”
process ID are also recorded, as well as
the time that the entry was written. Our survey results match what our experience has shown over the years: many agree that
Each message contains some useful automated software testing is the best way to approach testing in general, but there is often
information about the activity being a lack of budget, time, or experience available to execute successfully.
performed, for example, the database
server is “dbserver1”, the database is Additional reasons why automated software testing fails include:
• Research and Development does not generally focus on testing (manual or automated)
“customer_db” and the customer ID is
• Myths and Misperceptions about Automated Software Testing persist
“A1000723”. • Lack of Automated Software Testing Processes
From this log, it is evident that the • Lack of Software Development Considerations for Automated Software Testing, i.e.
application was not able to successful- building testability into the application
ly retrieve the specified customer • The Forest for the Trees – Not knowing which tool to pick
record. • Lack of Automated Software Testing Standards
In this situation, a tester could
examine the database on dbserver1,
using SQL tools, and query the cus- The tester is now not only report- scripting language and allow for script
tomer_db database for the customer ing a “symptom,” but along with the playback for baseline verification.
record with ID A1000723 to verify its symptom can document the internal When automated software testing tools
presence. application behavior that pinpoints interact with any type of display console
This information adds a substan- the cause of the problem. and Visual/GUI interface testing is
tial amount of defect diagnosis capa- required, the following recommendations
bility to the testing effort, since the GUI/Interface Testing should be considered, because cap-
tester can now pass this information Recommendations ture/playback tools are sensitive to any
along to the development staff as part Capture/playback tools record the test of the following changes when doing
of the defect information. engineer keystrokes in some type of bitmap recording, for example:

OCTOBER 2008 www.stpmag.com • 27


AUTOMATION FAILURE

• Control “font smoothing” or other GUI testing tool scripts are often invented and implemented but can’t
text characteristics should not be based on object properties and other be tested or is difficult to test how do
changed. GUI controls, and therefore it is best if we know its level of quality?
• Don’t change the color depth of the developers understand how the Testing and Development can
the application under test. GUI testing tools functions and how drive a business, so much attention
• Display settings need to stay the the scripts have been implemented, so needs to be paid to speeding up not
same. the impact of any of his changes to the only development efforts, but also
• If possible, keep the testing efforts.
default settings in the OS Test automation is key.


related to visual settings – When developing software,
use standard visual settings. developers need to consider
Developers need to under- testing efforts and build testa-
stand the impact to the automat- bility into the application.
ed GUI testing scripts before When developing software, developers Additionally, developers
making any GUI changes. need to understand the
Sure, GUI changes are
inevitable and many developers
need to consider testing efforts and impact to the automated GUI
testing scripts before making
will scoff at the idea that their any GUI changes. Other ways
development should be limited
build testability into the application. to improve software develop-
by any automated testing tool. ment that can aid automated
But once a GUI has been base-
lined, the developer should
consider the impact some cos-
metic and possibly unnecessary
• testing efforts will be dis-
cussed in the next article of
this series, such as adhering
to open software develop-

standard documentation. ý
changes can have on the auto- ment standards and using
mated testing scripts. scripts can be lessened.
Items for developers to consider To avoid some of the major automa-
are to ideally minimize modification of tion pitfall failures it is important that REFERENCES
object properties once they are base- R&D also considers testing not just 1. Recommended also by Redstone Software, makers
of Eggplant
lined and/or maintaining the control development of the latest and greatest 2. http://video.google.com/videoplay?docid=8774618
flow. technologies. If the best technology is 466715423597&hl=en

Working with SharePoint?


Announcing

SPTechCon
The SharePoint
Technology Conference

January 27-29, 2009 Tom Rizzo,


Director of Microsoft SharePoint, said: Go Behind the
Hyatt Regency “The phenomenal uptake of Microsoft SPTechCon Portal
Office SharePoint Server is transforming
San Francisco Airport blog.sptechcon.com
organizations of all sizes, all industries and
Burlingame, CA all geographies. Business leaders are envisioning new
uses of SharePoint, and IT professionals and developers REGISTER by
are gearing up to implement new applications as well
as integrate SharePoint to other enterprise software. Oct.17 for the
PRODUCED BY This is creating significant demand for training and eXtreme
BZ Media education. We’re delighted BZ Media is introducing this
new SharePoint technical conference.” Early Bird Rate

For more information, go to www.sptechcon.com

28 • Software Test & Performance OCTOBER 2008


This Way To

The Best
SC M 4 You
Software Configuration Management activities put in place to track the evolu-
tion of software items in the develop-
ment of a system.
Systems: What’s Not to Like? David E. Bellagio and Tom J. Milligan,
authors of Software Configuration Man-
agement Strategies and IBM Rational
By Vinny Vallarine ClearCase: A Practical Introduction, (IBM
Press, 2005) encapsulate a good defini-
onfiguration management is 1. Software Configuration Manage- tion of SCM by quoting the IEEE

C a critical part of any system


development effort. CM in the
ment (SCM) – Source code man-
agement and revision control.
2. Hardware Configuration Manage-
“Standard for Software Configuration
Management Plans” with the following:
SCM constitutes good engineering prac-
ment (HCM) – Management of tice for all software projects, whether phased
technology realm can be defined
hardware/device releases. development, rapid prototyping, or ongoing
as the control or management of modifica- 3. Operational Configuration Man- maintenance. It enhances the reliability and
tions made to software, hardware, or docu- agement (OCM) - Management of quality of software by:
mentation throughout the development and the configuration items (hardware, • Providing structure for identifying and
life cycle of a system. It can be broken software and documentation) with- controlling documentation, code, inter-
down into three main sub-categories: in a technology infrastructure. faces, and databases to support all life-
For our purposes, we were con- cycle phases
Vinny Vallarine is a software developer at cerned with the software configuration • Supporting a chosen development
IDT, a software testing consulting company management (SCM) paradigm. SCM /maintenance methodology that fits the
specializing in automated testing.
can be thought of as a process or set of requirements, standards, policies, organi-

OCTOBER 2008 www.stpmag.com • 29


SCM 4 YOU

zation, and management philosophy of the open source paradigm since occurs, an Atomic Commit ensures that
• Producing management and product it would hinder our ability to mod- all the changes are made or none at all.
information concerning the status of ify the source if needed. Non Atomic Commit systems run into
baselines, change control, tests, releases, 3. Seamless Integration with a defect trouble when, for example, during a
audits, etc. tracking tool commit of a bunch of files, a network
4. Eclipse and IDE support. connection is lost. The repository is then
Benefits of SCM 5. A quick learning curve for devel- put into an unknown and unstable con-
At a high level, a good SCM implemen- opers. dition. An Atomic Commit system would
tation should provide you with: 6. A tool that was widely supported in ensure that, since the network connec-
1. A secure repository to store your the community. There is nothing tion was lost before the entire commit
artifacts. worse than “Googling” a question finished none if the commit changes
2. A stable yet responsive workspace. about a problem you’re having make it into the master repository.
3. The ability to take a snapshot of with a particular piece of software • CVS was the only tool that did not
your baseline at incremental proj- and you get only 7 responses, and fully implement the atomic com-
ect milestones and “tag” it with a they’re in another language! mit model. This was a big minus in
known identifier. the CVS column.
4. The ability to revert to any of these The Tools Evaluated Intuitive Tags: Describes whether
“tagged” repository snapshots at We ended up evaluating a number of meaningful, human-readable tags can
anytime for reproducibility and different products to host our compa- be given to a particular revision. For
redeployment of a software build. nies SCM needs. There are plenty of example, “Latest Release 2.3 to DoD
5. Allow for simultaneous updates vendors offering very good SCM solu- Customer for contract num XYZ.123.”
/changes to artifacts in the reposi- tions but many of them can be eliminat- • All tools used Intuitive Tags. Sub-
tory. For example, it should provide ed immediately for not satisfying a par- version’s implementation of a “tag”
a controlled environment for hav- ticular need. Since we were mainly inter- has been criticized as being more of
ing multiple developers modify the ested in the open source community, a “cheap copy” but still appeared to
same file at the same time. There our options were narrowed down quite a satisfy our needs in this area.
are two main approaches that SCM bit. After researching the available prod- Web Based Interface: Some packages
products take towards solving this ucts, we identified the list of features come with a built in web interface. A
issue via its Concurrency Model: that we considered important and exam- web interface allows for the posting of
• Merge – Any number of users ined 4 tools with these features in mind. data to a web site for audits/analysis. A
may write a file at any time. The four chosen packages are: web interface could also allow for con-
The master repository may • CVS – Dick Grune trol over the SVN from any machine
spit out a “conflict error” if a • Subversion – CollabNet with a web browser.
user attempts to check a file • Mercurial – Matt Mackall • Subversion comes with an Apache 2
into the repository that is not • Bazaar – Canonical module built in. It also offers seam-
based off the latest revision. less integration with TRAC.
The user may then merge Features • Mercurial comes packaged with a
his/her local changes manu- Below you’ll find a list of the features web server.
ally or have the system that we considered, a brief explanation • CVS allows for the integration of
attempt the merge. of this feature and a description of how csvweb and ViewVC.
• Lock – One user at a time is each tool addresses this feature. Many of • Bazaar can be used with any simple
allowed to write a file. The the supporting applications referenced web server along with webserve, log-
master repository hands out below are not fully elaborated on as they gerhead or Trac.
locks one at a time. A user fall outside the scope of this writing. Development Status: Whether the appli-
must give up the lock for Ease of Use/Learning Curve: Is the cation is still being aggressively devel-
another use to edit the SCM package intuitive and overall easy oped, or is simply maintained with occa-
same file. to use. After all, the developers will be sional bug fixes incorporated. An appli-
6. The ability to audit and control interacting with it many times per day. Is cation that is being aggressively devel-
changes to our components. there a steep learning curve? oped is usually the most supported. The
More specifically, we needed: • Subversion, CVS and Bazaar seemed to developers of these applications are con-
1. A tool that adheres to the GNU be at the same level of “ease of use;” stantly adding new features and bug fixes.
General Public License (GPL) relatively easy. Many Website reviews Since the world of technology is con-
licensing model and was Open pointed to Mercurial as the more dif- stantly changing, these applications tend
Source ficult of the SCM tools. Subversion to address the latest needs of the devel-
2. Compatibility with Linux and had the largest “head start” here, opment community better than those
C/C++. We didn’t want to have to however, since some of us on the apps that are less actively developed.
install a proprietary compiler to team had prior experience with it. • Bazaar, Mercurial and Subversion are
build/setup the SCM software. Licensing: The license model the all actively developed.
Our developers know C/C++ and application corresponds to. These can • CVS is more or less in a mainte-
Linux well and having a SCM sys- be free or paid licenses. nance phase. New features are no
tem written in a proprietary lan- • All of the evaluated tools were free. longer being added.
guage would diminish the purpose Fully Atomic Commits: When a commit Programming Language: The lan-

30 • Software Test & Performance OCTOBER 2008


SCM 4 YOU

TABLE 1: HOW THEY MEASURE UP ry files as it is with text files.


• Bazaar handles binary files but
SCM Tool
Evaluation Weight Score Value Score Value Score Value Score Value there is no mention of the efficien-
Criteria (1-5) cy in their documentation. They’re
5 assumed to be handled as text files.
Price 5 5 25 25 5 25 5 25
• CVS handles binary files poorly.
Ease of Use 4 4 16 5 20 3 12 4 16 • Mercurial is said to handle binary
/Learning Curve files well.
Licensing 5 5 25 5 25 5 25 5 25 Symbolic Link Support: The ability for
the SCM implementation to allow sym-
Fully Atomic 5 1 5 5 25 5 25 5 25 bolic links to be put under source con-
Commits trol. Opinion seems to be split whether
Intuitive Tags 5 4 20 4 20 5 25 5 25 this feature is a nice convenience or
represents a hole in security.
Web Based 3 4 12 5 15 4 12 5 15 • CVS was the only tool that did not
Interface
support symbolic links.
Development 5 2 10 5 25 5 25 5 25 Repository Model: Typically, there are
Status two main models that SCM tools imple-
ment. The first is the client-server model
Programming 4 5 20 5 20 4 16 4 16
Language in which the server maintains the master
repository and each user keeps a local
Standalone 1 3 3 3 3 3 3 3 3 copy of the repository on his/her devel-
server option
opment machine. Changes made to
Efficient Binary 4 1 4 5 20 4 16 2 8 local copies must be “checked-in” to the
File support master repository for the changes to
propagate to other users. The second is
Symbolic Link 5 1 5 5 25 5 25 5 25
support the distributed model in which users hold
entire “peer” repositories, with version
Repository 5 5 25 5 25 3 15 3 15 history, on their local machine along
Model* with their working copy.
International 2 1 2 5 10 1 2 5 10 • CVS and Subversion implement the
Support client-server model
• Bazaar and Mercurial implement
File Renaming 5 1 5 5 25 5 25 5 25
the distributed model
Merge Tracking 2 1 2 1 2 5 10 5 10 International Support: Multi-language
and multi-operating system support.
Standalone GUI 2 5 10 5 10 5 10 5 10 • Subversion and Bazaar are Inter-
Speed 5 4 20 5 25 3 15 3 15 nationalized.
• Information on the international
IDE Support 5 4 20 5 25 5 25 5 25 support with CVS and Mercurial was
tough to find. We’ll assume no.
End of Line 4 5 20 5 25 5 25 1 4
Conversion File Renaming: The ability to rename
files while maintaining their version
Score 249 370 336 322 history.
*The scores assigned to the repository model do not indicate an absolute advantage of one over another. We scored the client-
• All packages except CVS supported
server model higher than the distributed model since all of our developers had experience with this model. file renames.
Merge Tracking: A system that sup-
guage the tool was written in. This can slower than their full featured counter ports merge tracking remembers the
be important in the open source com- parts. changes that have been merged
munity when you plan on modifying the • They all seem to have their own between each branch and will only
source of the tool you’re using. version of a standalone server. merge the appropriate changes (miss-
• CVS and Subversion are written in C; Efficient Binary File Support: Some ing changes) when merging one branch
Bazaar in Python; and Mecurial in SCM tools treat binary files as simple into another.
Python and C. text files whereas others recognize bina- • Subversion and CVS do not support
Standalone Server Option: Some SCM ry files as such. For the latter group of merge tracking, while Bazaar and
applications come with a standalone SCMs, their merging and tracking algo- Mercurial do.
server process. This server allows for rithms are designed for this and, thus, Standalone GUI: Some SCM packages
quick uptime and installation when a are much more efficient in handling come with standalone Graphical User
full featured Web server (such as binary files formats. Interfaces, relieving the developer the
Apache 2) is not needed or available. • Subversion utilizes a “binary diffing” need to memorize command line argu-
These standalone servers are very scarce algorithm which, theoretically, ments and formats.
on features and are usually considerably makes it just as efficient with bina- • They all offer some sort of GUI front

OCTOBER 2008 www.stpmag.com • 31


SCM 4 YOU

end. These solutions seem to vary in the EOL characters for files so they the web for answers or do we want to
terms of ease of setup and use, how- match the EOL specific method for the spend the cash on a licensed product
ever. The details of these are outside OS in which it is used where the support will be formal?
our scope here, however. • Bazaar was the only package that Do we want a “tried and true” product
Speed: Describes the efficiency of its did not support EOL conversions. with known suitable features or do we put
branching, tagging and commit algo- Again, as we did with our defect our trust in an “up and coming” product
rithms. Also, a big factor affecting the tracking tool, we assigned a weight and with the potential of monumental
tools overall response time is whether a score value to the above features to enhancements over our older product.
it’s written in a compiled (C/C++) or come with the table below: Can we afford to pick a tool that may
interpreted language (Python). not be the best choice because it inte-
• Subversion and CVS seemed to have The Selection grates with another of our tools? Can
an advantage here seeing as As a result of our analysis, Subversion we afford to go back and re-evaluate
they’re written in C, whereas came out on top. Aside from the features our prior choice of tools to accommo-
Bazaar and Mercurial are written in listed above, one very important factor date this one?
Python. This analysis is simply remained. Our developers had experi- Balancing the benefits and liabilities
based on a compiled versus inter- ence with this application in prior jobs. of a tool is important in any choice. In my
preted language. The overall speed This was a big factor in determining “ease experience, there is nothing more valu-
of any SCM tool is dependent of use.” Picking the right tool for any job able that having a team that is aware of
upon its algorithms used. This is difficult and one needs to balance the the current options. A team of people,
aspect of performance wasn’t benefits versus the liability of any tool. In for example that read technical journals,
explored too deeply, however. this instance we had find the balance discuss new technologies with co-workers,
IDE Support: Describes whether the between the following questions and are generally interested in their field
tool has smooth integration with inte- Do we want a feature rich applica- will have a much better shot at picking
grated development environments.such tion even at the expense of ease of use? the right tool at the beginning of a proj-
as NetBeans, Eclipse and Visual Studio. Do we go with something that every- ect, where it’s the cheapest. Needing to
• All the SCM applications offer a vari- one knows and can start using immedi- change to another implementation of a
ety of IDE integration options. All of ately or we do look for something new, tool in the middle of a project can be
which integrate well with Eclipse, and possibly better. Will schedule allow detrimental. This could happen if
which was one of main needs. this “trial” period? needs/requirements change but should

management from the beginning. ý


End of Line Conversion: Implies Do we want a free product where the not happen as a result of ill-informed
whether or not the SCM tool can adapt support is based on arbitrary searching

32 • Software Test & Performance OCTOBER 2008


Development and Testing:
Two Sides of
the Same Coin

Involving Testers Early in a Project is The


Best Way to Mint a Quality Product

By Prakash Sodhani

t has been well documented that devel- In many of the companies I have worked in,

I opment and testing are two separate


domains, each with certain unique characteris-
testing is considered a luxury. If it happens, it
happens. Otherwise, no big deal. There’s a direct
correspondence between these two fields and I
tics. In a traditional software life cycle, code is argue that testing should be given equal impor-
being developed first and turned over to test to tance to development. Of particular focus is auto-
ensure that a quality product is being delivered. mated testing, in which testers write scripts that
With so many contrasts between development
Prakash Sodhani is a quality control specialist at a glob-
and test, it is easy to overlook the similarities that al IT services company based in Texas.
exist between these two fields.

OCTOBER 2008 www.stpmag.com • 33


TWO CENTS ON TESTING

can be as lengthy as the code under test otherwise it’s nothing more than a writing automated code as well.
itself. wild goose chase. Unless a tester has
I have been doing automated testing clear view of functionalities to be auto- 3) Testing
for years, and have been to a good many mated, creating the automation scripts The actual testing phase consists of
companies during that does not serve their two parts: Unit Testing and Integration
time. It’s interesting to intended purpose. It’s Testing. Let’s look at these two sepa-


learn people’s views of important to understand rately.
automation, which range what needs to be auto- 3a) Unit Testing. Unit testing involves
from manual testing with mated and use that infor- testing a single unit of functionality,
tools to simple record and mation to structure out the smallest possible component of
playback. I will explain It is imperative the automated tests. code. The purpose of this testing is to
some of key characteris- make sure the core components work
tics of automation and for people to 2) Code Generation fine in isolation. So, you pick a small
their relationship to Code generation refers to independent piece of code and make
development. Hopefully,
it will open the eyes of
understand that creating reusable automa-
tion scripts. Based on what
sure it works as per specifications. For
example: in development it may refer
people who have been business process needs to to a class, method, function, or a Web
pretending to know every-
automation is be automated, scripts are service call. You develop, test drive and
thing about automation created that can be run the associated unit with different
but in reality know close
not just record replayed again and again. parameters, both positive and nega-
to nothing. As is the case with devel- tive; to make sure it works as expected.
In a nutshell, the devel- and playback. opment, automated While creating test automation
opment process consists of scripts need to follow cod- scripts, you should structure your code
these distinct steps: You write code. ing guidelines and adhere in manageable units. I divide my code
1. Software Require- to standard practices. into small units called “Actions,” which
ment Analysis
2. Code Generation
3. Unit and Inte-
gration Testing
• It is imperative for peo-
ple to understand that
automation is just not
record and playback. You
are synonymous with classes in object
oriented programming. These classes
can be further subdivided into smaller
units, such as functions and methods,
4. Maintenance write code. The difference just like development methods and
Depending on the between writing code for functions. After the test script is ready,
company and project requirements, automated testing and coding for you test it with different parameters
there may be other steps added on or in development is that in testing you write and sets of data to ensure it works as
between, but automated testing has a not to develop something but to break expected.
one-to-one relationship with each and what is developed. All the considera- Let’s take an example to illustrate
every one of these steps. tions involved in development apply to unit testing correspondence. Let’s say

1) Requirement Analysis
Requirement analysis is always the first
step in a software development
process. The process is simple. Get the
requirements from the requirements
team and use them to design and
develop your code. Just as correct code
can’t be written without clear require-
ments, a correct quality approach can-
not be taken without correct require-
ments.
As much as requirement analysis
applies to almost every domain, it is
even more significant when it comes to
automation. I have been on more than
one occasion been in the evening and
told to start writing automation scripts
next day. Sometimes, it has even been
for projects for which I had no prior
information; I was just told that “every-
thing needs to be automated” within
few weeks.
As a tester, it’s your responsibility to
fully understand the expectations of
automation for a particular project,

34 • Software Test & Performance


TWO CENTS ON TESTING

you have an application with a login piece of code working fine in isolation Testing. You should have tested the
page. but failing when used in conjunction Login page with different sets of data.
Development. A developer will write with other units. Now you can include clicking on the
code to create the required compo- Let’s take an example to illustrate “Search” button as a part of your test
nents in the login page and ensure unit testing correspondence. Let’s say script. You can also add checks that
valid inputs are accepted and correct you have an application with a login ensure that clicking on the “Search”
results returned. He might put this page. When clicked on “Search” but- button brings up the home page (or
code in a function named “Login,” just ton in login page, it takes you to home search page, as appropriate). You run
as it might be called in many places. page of the application. this script with multiple sets of data.
Then he will plan to test the
“Login” unit with various sets 4) Final Check And
of data.
Testing. A tester will write
code with different verifica-
tion checks to make sure that
• Maintenance
There is always some mainte-
nance involved in anything you
do. It’s even more significant
the components in login The most successful projects I have when your code is used repeat-
page accepts valid inputs and edly and by many people.
rejects invalid ones. The worked on have involved testers from Development. Maintenance
tester might put this in a required in the development
function so that he can make the beginning and throughout. code is well known. Often in
it reusable across various projects, much of the total soft-
other automation scripts he
might write, and ease the
effort in maintaining the
code. Then, he might run
this script with various sets of
• ware life cycle involves some sort
of maintenance.
include making sure any
Activities

changes in code are document-


ed, the code is being updated
positive and negative data. Development. You should have regularly as the application changes,
already performed unit testing for and unintentional modifications by
3b) Integration Testing. This testing login page. Now, you want to make people who are not authorized to do
involves testing various units together. sure things are as expected when you so are prevented. Various strategies
As a part of unit testing, various small combine the “Login” function with are formulated to make sure code is
standalone components are tested. As “Search” function. You might call maintained as best as possible.
a part of integration testing, these “Search” function from Login func- Testing. It may seem that testing
units are combined to make sure the tion or vice versa and make sure the doesn’t require much maintenance.
components continue to work when home page shows up as per the But actually, the opposite is true.
put together. It is not rare to see a requirement. Keeping automated scripts usable
requires involves a good deal of main-
tenance. If you fail to keep your scripts
up to date with changes in the appli-
cation, your scripts might just become
obsolete. The longer you wait, the
harder they are to bring up to date. It
is also important to make sure that
scripts are modified only by author-
ized people. If someone is not aware
of what the script does, he might unin-
tentionally modify something which
might break the script. Various proce-
dures are put in place to ensure that
maintainability of scripts is a major
consideration in the project.
The best and most successful proj-
ects I have worked on have involved
testers from the beginning and
throughout every phase of develop-
ment. Even better are developers who
do their coding with testing in mind.
If everyone on the project is kept
aware of the needs of everyone else,
both sides of the coin can work togeth-

change. ý
er to bring about the right kind of

www.stpmag.com • 35
Best Practices

.NET Apps Demand


New Ways of Testing
In his role as director task.” The results work but classes or methods in those classes that
of marketing for Visual are inefficient. may not be very amenable to unit test-
Studio Team System in Concatenation of strings ing. “For that you might want to do
Microsoft’s Developer Div- using “string” rather than assembly-level or application-level test-
ision, Norman Guadagno, “StringBuilder” is one sim- ing,” says Hari Hampapuram, director
sees bad code every day, ple example. “Our research of development at tools vendor
sometimes lots of it. Part shows that these coding Parasoft. The reason for this is that
of that may be because the practices are endemic,” says there may be objects that are too com-
.NET framework’s quick Guadagno. With tools such plex to create on a standalone basis; or
rise in popularity has out- as the VSTS Test Edition, it could depend on a framework being
paced the ability of some “we can flag these types of in place before creating the object, he
Joel Shore
developers to keep up. situations – what they are says.
“We want people to write fast, effi- coding versus what the environment can With a near-cornucopia of testing
cient code and we want them to do deliver to them.” tools available from different sources,
unit testing to make sure the frame- Gwyn Fisher, CTO of source code using several simultaneously makes
work is being used appropriately,” he analysis solutions provider Klocwork good sense, especially when one has
says. “But even with code analysis and recommends incorporating the testing strength where another does not.
performance metrics, it has to be process much further upstream, Applications developer Clarity
understood that testing does not because that’s where the problems Consulting used several, including
equate to quality, testing is part of start. It’s an issue of magnitude. “You IBM AppScan, WinRunner,
quality.” can’t possibly test every configuration LoadRunner, MSTest and NDbUnit
With the .NET framework of appli- variation, so testing with tools as far throughout the life cycle and develop-
cations that live in a browser frame upstream as possible is essential.” ment of the customer-facing Web site
and moving away from the concept of Areas that Fisher says merit special at VW Credit, the financial services
client requests to a server, tools have to attention include ensuring safe subsidiary of Volkswagen Group of
evolve to keep up. “The asynchronous threading and checking for memory America. “For testing, we used these
interface completely changes the way leaks or concurrency violations. tools to set and reset the database, so
that load testing is approached,” says Of course, one can go upstream that regardless of state of data we can
Matt Braylee-Berger, a Silk product only so far. But even that model may create our own data-specific tests, or
domain specialist at Borland. be changing. At least one code analysis reset back to known state,” says soft-
Many of the problems that vendor says the first day of coding is ware development consultant Ryan
Guadagno sees in programs written for already too late when it comes to test- Powers.
the .NET framework are more ground- ing. At Ounce Labs, chief scientist Tools and strategies aside, it is
ed in a lack of best practices, rather Ryan Berg believes that while tools knowing the far-flung capabilities and
than a specific technical faux pas. have their proper place, the time to sit nuances in .NET that contributes to
Echoing Braylee-Berger’s observation, down and write test scenarios is before efficient, fast-running code. “.NET. is a
he notes that the practice of linking coding starts. Doing so forces develop- vast platform with lots of complexity,
the user interface to a data layer ers to think about fail cases first, which and it’s possible for even the most
occurs regularly. By imposing con- can influence in code design. experienced developer to occasionally
straints and flagging specific design While specific tools, such as get overwhelmed,” says Guadagno.

run over by that complexity.” ý


errors, “we’ll catch that,” he says. Microsoft’s FxCop analysis tool does a “It’s up to us to make sure they are not
Reticence to explore beyond one’s good job of pointing out places where
own comfort zone also is a contributor code may be breaking out of the .NET
to performance woes even though the framework, misuse of a class, or incor- Joel Shore is a 20-year industry veteran and
underlying code is error-free. “Coders has authored numerous books on personal com-
rectly formed constructs, security
puting. He owns and operates Research Guide,
tend to use the calls that they know checking is not one of its strengths. a technical product reviewing and documenta-
best over and over, even if they are not It’s not possible to test with the tion consultancy in Southboro, Mass.
the optimal solution for a specific same kind of unit testing as there are

36 • Software Test & Performance OCTOBER 2008


ST&Pedia zations may use routers or more com- suspicious patterns of activity and sends
< continued from page 13
plex solutions to protect internal net- an alert when it detects such activity.
the party you are interacting with is works. Among the most widely adopted IDS
legitimate. This prevents phishing and tool is Snort.
certain MITM attacks. HTTPS
This prefix in a URL indicates that the SSL (SECURE SOCKETS LAYER)
CIPHER A technique to apply a cipher
An algorithm used to encode that provides both authentica-


and decode text. The simplest tion and encryption. In other
cipher is a ROT (or rotation) words, you know you are talk-
cipher. In a ROT-1 cipher, ‘a’ ing to the correct party, and
changes to ‘b’, and so on. Such that no one else can overhear
encoded messages, if captured
by a third party, cannot be
Is there a performance hit with RSA the ‘conversation’ over the
world-wide-web. For example,
understood. Julius Caesar used
such a cipher to transmit
versus SSL? What vulnerabilities most websites that accept cred-
it cards use SSL or a scheme
coded messages during the like it.
Gallic wars. Germany’s Enigma
exist after the latest port scan?
Machine of World War II was a RSA
complex, multi-level ROT
cipher (There’s a free online
ROT cipher generator–at www
.unfiction.com/dev/tutorial
• A specific cipher, complex
enough for both business and
military applications. Wikipedia
refers to RSA as “one of the first
/ROT .html). great advances in public key
Web page is encrypted and transmitted cryptography.” Specifically, RSA pro-
FIREWALL using secure sockets. vides both encryption and authentica-
A software (or hardware) system tion – RSA messages cannot be created
designed to limit access from an out- NETWORK INTRUSION without a “private key.” As long as this
DETECTION SYSTEM (IDS)
encryption and digital “signing.” ý
side source. Firewalls are popular for key is protected, RSA can provide both
personal computers, but large organi- Software that monitors a network for

Index to Advertisers

Advertiser URL Page

Automated QA www.testcomplete.com/stp 10

Eclipse World www.eclipseworld.net 6

Empirix www.empirix.com/freedom 4

FutureTest 2009 www.futuretest.net 2, 3

Hewlett-Packard hp.com/go/quality 40

Ranorex www.ranorex.com/stp 34

Reflective Solutions www.stresstester.net/stp 8

SPTech Con www.sptech.com 28

Software Test & Performance www.stpmag.com 37

Software Test & Performance www.stpcon.com 39


Conference

Wibu-Systems USA www.wibu.us 23

OCTOBER 2008 www.stpmag.com • 37


Future
Future Test
Test

Static Analysis,
For example, using static analysis as
an audit that occurs at later stages of the
SDLC only exacerbates its tendency to
drain development resources. Having
an inline process makes the analysis

Security Failure more valuable and more effective. Since


the code is still fresh in developers’
minds when violations are reported,
developers are more likely to learn from
their mistakes and remediate problems
Static analysis for security own organization, team, faster and more easily.
has a hot topic lately, and I and project. Then they Policy management lies at the core
fear we’re starting to think require compliance to those of such an inline process. You should be
of it as a silver bullet. The carefully-selected rules. As a able to easily configure policies for spe-
quest for application secu- result, rule violations are cific projects without compromising the
rity has breathed new life perceived as suggestions for integrity of the corporate objectives,
into static analysis tech- general code improvements easily deploy and update both project-
nologies, which until —not critical coding issues specific and organization-wide policies,
recently were primarily per- that need to be addressed and automate their application for
ceived as either frivolous immediately. rapid scanning and reporting. A care-
beautification tools or bur- The key to providing fully defined and implemented set of
densome big brother mon- Wayne Ariola the necessary context to policies establishes a knowledge base
itoring systems. Surpris- static analysis is to take a that allows developers to increase their
ingly, the underlying technology was not policy-based approach: use static analy- relative security IQs.
substantially modified to accommodate sis to monitor a non-negotiable set of Putting the policy into practice
the issue of security. Rather, the changes expectations around code security, reli- involves workflow management—defin-
were more like a face lift. As a result, ability, performance, and maintainabil- ing, automating, and monitoring securi-
organizations using static analysis still ity. With this approach, a violation of a ty verification and remediation tasks,
encounter the same challenges in mak- particular guideline is not just another which are ingrained into the team’s
ing it sustainable over time. suggestion for people building soft- workflow. These tasks must be optimized
The secret to making static analysis ware in an ivory tower—it’s notification to ensure that the static analysis process
tools productive is to use them in the that the code failed to meet the orga- is both sustainable and scalable. The lack
proper context. The adoption of this nization’s expectations. of automation, repeatability, or consis-
technology should be driven by a policy- Effective policy management allows an tency will degrade any quality initiative
based approach. This means establish- organization to bridge the gap between that the organization intends to deploy.
ing a policy that defines requirements, management expectations and developer
then enforcing that policy consistently. performance. Essentially, if a static analy- Second Time’s a Charm?
Automation helps ensure that the sis rule enforces something that is part of Static analysis has a history of impacting
required practices are sustained, and the policy, fixing a violation of that rule is productivity to the point where devel-
workflow, task management, and met- non-negotiable. If a developer fails to sat- opers start ignoring it and achieve little
rics enable you to measure how well the isfy the defined policy, he is not executing or no code improvement. Now that hav-
policy is being implemented. In the his job as expected by management. ing secure code is non-negotiable, more
context of policy, static analysis is elevat- people than ever are taking advantage
ed from a “nice-to-have” checker to a What’s Needed to Make it Work of the many benefits that static analysis
critical tool for ensuring that code The rising risk and impact of application- can deliver. This is a great opportunity
meets the organization’s expectations. level security attacks has brought static for the industry to reacquaint itself with
analysis and its challenges into new light. the technique. With a concerted effort
How Policy Provides Context Static analysis has great potential for to focus on policy and workflow man-
The most likely culprit for the reputation ensuring that code is written in ways that agement and workflow optimization, we
static analysis has as a nonessential tech- prevent security vulnerabilities. However, can start off on the right foot with static
nology is its lack of context. Products to ensure that static analysis delivers as analysis for security—and then contin-

tion to improve quality as well. ý


provide “out-of-the box” support for promised here, it’s essential to address ue to build on this new stable founda-
hundreds of rules which could be impor- the challenges that have traditionally
tant in many different contexts. stymied its success. This is where consid-
However, most organizations don’t take erations such as policy management, Wayne Ariola is vice president of strategy at
Parasoft, which recently extended dataflow
the time to determine which rules are workflow management, and workflow
capabilities in its flagship code analysis tools.
most important in the context of their optimization come into play.

38 • Software Test & Performance OCTOBER 2008


The biggest
conference ever:
more classes!
more speakers!
more fun!

March 31-April 2, 2009 San Mateo Marriott


www.stpcon.com San Mateo, CA
SPRING
“STP Con has
produced a
conference that is jam
packed with ‘must
know’ topics by the
best presenters in
their field.”

“Great material, great


speakers, all around a
great experience.”

“The courses were


excellent and very
informative. The other
attendees were a vast
source of knowledge
regarding techniques
and experiences.”

“If you don’t learn


something you are not
paying attention!”
A LT E R N AT I V E T H I N K I N G A B O U T Q U A L I T Y M A N A G E M E N T S O F T WA R E :

Make Foresight 20/20.


Alternative thinking is “Pre.” Precaution. Preparation. Prevention.
Predestined to send the competition home quivering.

It’s proactively designing a way to ensure higher quality in your


applications to help you reach your business goals.

It’s understanding and locking down requirements ahead of


time—because “Well, I guess we should’ve” just doesn’t cut it.

It’s quality management software designed to remove the


uncertainties and perils of deployments and upgrades, leaving
you free to come up with the next big thing.

Technology for better business outcomes. hp.com/go/quality

©2008 Hewlett-Packard Development Company, L.P.

You might also like