T802 MSc in Technology Management

Performance metrics for the Open Innovation model

David Scarlatti
X7922828 February 2010

T802 MSc in Technology Management

Performance metrics for the Open Innovation model

David Scarlatti
X7922828 February 2010
ThisdissertationissubmittedinpartialfulfilmentoftherequirementsfortheMScin TechnologyManagement

David Scarlatti - X792282

Abstract
Innovationisseenasakeyelementofanorganisationscompetitiveness.Membersof seniormanagementteamsandmanagersinchargeofResearchandDevelopmenthave struggledforyearstofindthemostappropriatewaytomeasureinnovation.Whilebest practicesformeasuringinnovationarestillunderdevelopment,anewmodelfor innovationhasemerged,knownasOpenInnovation.Thisnewmodelisexpandingquickly andiscausingresearcherstoquestiontraditionallyacceptedprinciples.Forthosewhoare embracingthenewmodel,akeyquestionarises:Canwemeasureinnovationinthesame waythatwehaveuntilnow?OpenInnovationproponentsseemtohavepaidlittle attentiontothistopictodate.Theaimofthisresearchistoproducerecommendations concerningmetricsforevaluatingtheeffectivenessofanOpenInnovationapproachto research.Amajorobjectiveistoexploretheuseofperformancemetricsinresearch pertainingtoOpenInnovation.Thevalidityofexistinginnovationperformancemetricsis examinedusingtheprinciplesofOpenInnovationandbyapplyingdefinedcriteriaof applicability.Iconcludethatwhilemostmetricscurrentlyinplaceremainuseful,theyneed someadaptation.Inmanycases,additionalsupportingmetrics,especiallythoserelatedto externalresourcesandactivities,mustbeadded.ADecalogueoutliningthechanges neededinthecurrentsetofmetricsisproposedasausefultoolformanagersinchargeof evaluatingthisnewapproachtoinnovation.ThecorollaryoftheDecalogueisthatone shouldavoidusingacurrentinnovationperformancemeasurementsystemforthenew OpenInnovationmodelwithoutadaptingtherelevantmetrics.

David Scarlatti - X792282

TableofContents
Figures ........................................................................................................................................... v Tables ........................................................................................................................................... vi Glossary ........................................................................................................................................vii Chapter1 1.1 1.2 1.3 1.4 1.5 Chapter2 2.1 2.2 2.3 Chapter3 3.1 3.2 3.3 3.4 Chapter4 4.1 4.2 4.3 4.4 Chapter5 5.1 5.2 Introduction............................................................................................................. 1 Backgroundontheproblem/issue .............................................................................. 1 Justificationfortheresearch ....................................................................................... 1 Aimandobjectives....................................................................................................... 2 Scopeoftheresearch .................................................................................................. 2 Outlineofthedissertation........................................................................................... 3 Researchdefinition.................................................................................................. 5 Thepracticalproblem/issue ........................................................................................ 5 Existingrelevantknowledge ...................................................................................... 11 Researchquestions.................................................................................................... 20 Methodology ......................................................................................................... 21 Methodsandtechniquesselected............................................................................. 21 Justification ................................................................................................................ 21 Researchprocedures ................................................................................................. 22 Ethicalconsiderations................................................................................................ 22 Analysisandinterpretation ................................................................................... 23 Summaryofdatacollected ........................................................................................ 23 DataAnalysis.............................................................................................................. 29 Interpretationinrelationtotheresearchquestions................................................. 33 Interpretationinrelationtotheaim ......................................................................... 35 Conclusions............................................................................................................ 37 Conclusionsregardingtheresearchquestions.......................................................... 37 Conclusionsregardingtheresearchaim.................................................................... 37

iii

David Scarlatti - X792282

5.3 5.4 Furtherwork .............................................................................................................. 38 Implicationsofthisresearch...................................................................................... 38

References................................................................................................................................... 40

iv

David Scarlatti - X792282

Figures
Figure11Scopeofthisresearch .................................................................................................. 3 Figure21ProblemDefinitionFlow .............................................................................................. 5 Figure22TheClosedInnovationModel(Chesbrough2003) ...................................................... 6 Figure23TheOpenInnovationModel(Chesbrough2003) ........................................................ 7 Figure24ReasonsformeasuringInnovation ............................................................................ 13 Figure25ChallengesmeasuringInnovation.............................................................................. 14 Figure26ApproachesfacilitatingInnovationMeasurement .................................................... 18 Figure41Themeasurementandcontrolprocess(KerssensvanDrongelen,Cook1997).......... 23 Figure42BalanceinOpenInnovation(Lichtenthaler2008) ..................................................... 32

David Scarlatti - X792282

Tables
Table21ContrastingPrinciplesofClosedandOpenInnovation(Chesbrough2003)................. 8 Table23ContrastingPrinciplesbetweenClosedandOpenInnovation(Chesbrough2003)and measurementquestions ............................................................................................................. 10 Table41ThedimensionsofR&Dperformance(Ojanen,Vuola2003)...................................... 25 Table42GroupedInnovationmetrics ....................................................................................... 26 Table43Innovationmetricsforindividuals............................................................................... 29 Table44InnovationmetricsregardingR&Dvalue .................................................................... 30

vi

David Scarlatti - X792282

Glossary
Businessprocess Atitsmostgeneric,anysetofactivitiesperformedbyabusinessthatis initiatedbyanevent;transformsinformation,materialsorbusiness commitments;andproducesanoutput.Valuechainsandlargescale businessprocessesproduceoutputsthatarevaluedbycustomers.Other processesgenerateoutputsthatarevaluedbyotherprocesses. Effectiveness Firm Innovation Thedegreetowhichtheprocesssstatedobjectiveshavebeenmet. AtermequivalenttoorganisationinOpenInnovationliterature. Theimplementationofaneworsignificantlyimprovedproduct(goodor service)orprocess;anewmarketingmethod;oraneworganisational methodusedinbusinesspractices,workplaceorganisationorexternal relations.(OECD2005) IntellectualProperty.Anyproductofsomeone'sintellectthathas commercialvalue,especiallycopyrightedmaterial,patents,and trademarks. IntellectualPropertyRights.A general term for the assignment of property
rights through patents, copyrights and trademarks. These property rights allow the holder to exercise a monopoly on the use of the item for a specified period of time.

IP

IPR

KPI Measure

KeyPerformanceIndicator.Ametricthatisespeciallyimportantfor measuringtheperformanceofaprocess. Inscience,theprocessofobtainingthemagnitudeofaquantity(e.g., lengthormass),relativetoaunitofmeasurement(e.g.,meterorkilogram). Inmanagement,awaytofindoutwhetheragoalisbeingmet. Ameasureforsomething;ameansofderivingaquantitativemeasurement orapproximationforanotherwisequalitativephenomenon. PerformanceManagementSystem.Asystemforevaluationagainstasetof determinedcriteriawithrespecttotheeconomy,efficiencyand effectivenesswithwhichanorganisationexecutesaparticularactivity. Organisationsmaysetregularobjectivesforparticularmetrics,against whichtheorganisationsperformanceisevaluated. Research&Development.Anorganisationsactivityundertakenwiththe intentofdiscoveringordevelopingnewproductsorservices.R&Dincludes pursuingenhancedversionsof(orqualitiesin)existingproductsorservices, oritmayincludediscoveringordevelopingnewormoreefficientprocesses ofproduction.

Metric PMS

R&D

vii

David Scarlatti - X792282

Chapter1 Introduction
1.1 Backgroundontheproblem/issue
Today,mostorganisationsrelyontheinnovationprocesstoremaincompetitiveandtogrowin ahealthymanner(Kumpe,Bolwijn1994).Suchreliancemakesinnovationandits measurementanextremelyimportanttopic.Thesameattentionshouldbepaidtothisareaof businesspractiseasispaidtootherbusinessprocesses,suchasproductionormarketing (KerssensvanDrongelen,Bilderbeek1999,Andrew,Haanaes2008). Ifinnovationisimportantingeneral,itisespeciallyimportantintechnologyfields.Technology isanenablerofinnovation,andinnovationconfersacompetitiveadvantageintechnology industries(Cumming1998). OpenInnovationwasfirstintroducedbyChesbrough(Chesbrough2003a),whoreferredtoa newmodelforinnovationatthefirmlevel.Heobservedthatsomecompaniesstartedto competeandsubsequentlyfoundsuccessinthemarketplaceusingapproachestoinnovation thathadneverbeenusedbefore.TheOpenInnovationmodelcontrastswiththetraditional closedmodelbecauseitremovesanystrongbarrierdelimitingorshieldingtheinternal researchanditsdevelopmentprocess. Thoseinchargeoffunding,evaluatingorexecutinginnovationmustuseameasurement system.Metrics,targetsandresultsarerequiredtomakesounddecisionsabouttheprocess. Measuringinnovationisnotaneasytask;ongoingresearchcontinuesinitsquesttoidentify goodpracticesandmetrics(Pearson,Nixon&KerssensvanDrongelen2000). Furthermore,forthenovelOpenInnovationmodel,muchresearchisstillrequiredforthe purposeofproperlydefiningthemodel(Chesbrough2006b)andtheappropriatewaysof measuringitseffectiveness.

1.2 Justificationfortheresearch
Thisresearchinvestigatestheinnovationmeasurementprocessforanorganisationthat embracestheOpenInnovationmodel.Manycandidateorganisationsforsuchaninvestigation exist,includingcompaniesalongthelinesofEliLilly,IBM,Procter&GambleandDow Chemical. Havingagoodsetofmeasures(i.e.,keyperformanceindicators)canhelporganisationsto maketherightchoicesaboutinvestmentsandorganisationalchanges.Companiesalsorelyon

David Scarlatti - X792282

theirmeasurementsystemtodemonstratethatpositiveresultsarehappeningasa consequenceoftheactionstaken. OpenInnovationisaverynewmodeland,assuch,thereisstillagreatopportunitytolearn andexploreitsmeasurementmetricsanditsroleasadriverfordecisionmaking. Consideringthatasingleerrorinproductdevelopmentinvestmentcouldhaveaneconomic impactseveraltimesgreaterthantheresearcheffortdescribedhere,itisworthinvestingin researchonOpenInnovation. Additionally,IampersonallyinterestedintheOpenInnovationmodelbecauseIbelievethatit couldhaveamajorimpactonthewaybusinesseswilloperateinthecomingyears.

1.3 Aimandobjectives
Aim Theaimofthisprojectistoproducerecommendationsformetricsdesignedtoevaluatethe effectivenessofanOpenInnovationapproachtoresearch. Objectives Theobjectivesforthisprojectareasfollows: 1. Tosurveycurrentinnovationeffectivenessmetricstocreateacatalogueofknown options. 2. ToidentifythebasiccriteriafortheapplicabilityofmetricsbasedbothontheOpen Innovationprinciplesandontheexamplesavailableintheliterature. 3. TocorrelatethedeliverablesofObjective1andObjective2togenerateaproposal abouttheuseofeffectivenessmetricsforusingOpenInnovationinresearch.

1.4 Scopeoftheresearch
Innovationcanoccurinverydifferentways.Theresearchpresentedherefocuseson innovationintheresearchdepartmentorresearchunitofanorganisation.Moreover,the resultspresentedherehaveaclearbiastowardstechnologyorientedorganisations,which makesperfectsensebecausethisresearchfallsundertheTechnologyManagementdomain. However,thisresearchstillmaybeofinterestforthoseinvolvedinmeasuringinnovationin othercontexts,butsuchindividualsmusttranslateandamendconceptsfromtheresearchand

David Scarlatti - X792282

developmentdisciplineintechnologytotheappropriatebusinessroleandmarket.Figure11 showsthescopeofthisresearchintermsoftheintersectionofthethreeconcepts.

INNOVATION SCOPE

TECHNOLOGY

R&D

Figure11Scopeofthisresearch

Fromahistoricalpointofview,OpenInnovationisaveryyoungmodel,sothisresearchisnot limitedtoanyspecificperiodoftime.Geographically,itistruethatmostoftheavailable informationcomesfromEuropeandtheUSA,sothereisanimplicitlimitationtotheseareas. However,thislimitationarisesonlybecausethesearethegeographicareaswhereOpen Innovationisundergoingactivedevelopment.

1.5 Outlineofthedissertation
Thisdissertationisorganisedintofivechapters. Chapteroneservesasanintroductiontotheproblemthatoriginatedtheresearch,andit definestheaimandthreeobjectivesoftheresearch. Chaptertwoexplains,indetail,theproblemthatthisresearchtriestosolve,anditpresents existingknowledgerelevanttotheproblemanditspossiblesolutions.Fromtheinitial problem,threespecificresearchquestionsareoutlined. Chapterthreefocusesonthemethodologyusedforthisresearchandjustifiestheselectionof themethod.

David Scarlatti - X792282

Chapterfourpresentstheinformationcollected,anditpresentstheinterpretationofthatdata astheyrelatetotheresearchquestionsandtheaim. Chapterfiveservesaswrapupandincludestheconclusion,aswellassomeconsiderationsfor futurework.

David Scarlatti - X792282

Chapter2 Researchdefinition
2.1 Thepracticalproblem/issue
Theobjectiveofthisresearchistoproducerecommendationsfortheimplementationofan OpenInnovationperformancemeasurementsystem. TheOpenInnovationmodelrepresentsadrasticchangefromthetraditionalviewofthe businessrulesrelatedtoinventions,researchandproductdevelopment.However,becausea lotofliteratureisalreadyavailableonthesubject,thisresearchwillnotstudytheintricaciesof OpenInnovationperse.Instead,thispaperwillfocusontheproblemofitsmeasurement, whichhasfarlesscoverageintheavailableliterature. Thissectionfollowsonothersresults((McGrath,Romeri1994),(Chiesaetal.2008),(Bremser, Barsky2004),(Schumann,Ransley&Prestwood1995),(KerssensvanDrongelen,Bilderbeek 1999,Davila,Epstein&Shelton2005),(MarkRogers1998);seesection2.2Existingrelevant knowledge)thatdemonstratethefollowing: 1. Measuringinnovationiswidelyacceptedasagoodpractice 2. Measuringinnovationisnotaneasytask Previousresearchhasalsotriedtodemonstratethefollowingprinciple: 3. TheOpenInnovationmodel,becauseofitsnoveltyandcompletelynewprinciples, requiresmoreresearchintohowitshouldbemeasured. Foraclearunderstandingoftheproblem,thissectionflowsinthemannerdescribedinFigure 21.
Why measure innovation?

Challenges measuring innovation

What is Open Innovation?

Current approaches for measuring innovation

What about measuring Open Innovation?

Figure21ProblemDefinitionFlow

David Scarlatti - X792282

FullystudyingtheproblemofmeasuringOpenInnovationisimportanttogainan understandingofthedifferencesbetweenthisnewmodelandthetraditionalone. Traditionally,innovationhasbeenusedbycompaniestocreatebarrierstoentryintheir markets.Researchanddevelopmentdepartmentswouldmaintainoperationalsecrecy. Talentedpersonnelwerehiredbyfirmsandretainedtoavoidthetransferofknowledgeto competitors.Strongintellectualpropertyclausesincontractsprotectedthecompanyfrom partners,customersoremployeesusingthecompanysknowledgefortheirownbenefit. OpenInnovationwasfirstintroducedbyChesbrough(Chesbrough2003a),whoreferredtoa newmodelforinnovationatthecompanylevel.Theconceptreferstoafreshapproach, focusedonremovingthebarriersthatdelimitordefendtheinternalresearchandits developmentprocess.Theprocessisincontrastwiththeold,closedmodel.Theauthor observedhowsomecompaniesstartedtocompeteandsubsequentlyfoundsuccessinthe marketplaceusingthisnewapproach. Theclosedmodelisusuallyrepresentedwithaclearboundaryisolatingtheinternalandthe externalworld,asshowninFigure22.

Figure22TheClosedInnovationModel(Chesbrough2003)

David Scarlatti - X792282

TheOpenInnovationmodelpresentsaporousboundaryfortheorganisation,aboundarythat allowsknowledgetoenterandexitatpracticallyanystageofthetraditionalR&Dcycle,as showninFigure23:

Figure23TheOpenInnovationModel(Chesbrough2003)

Themodelabovedoesnotassumethatbenefitsonlyarisefromonesownknowledge;rather, themodelrecognisesthatonecanalsobenefitfromseveralotherfactors: Theavailableknowledgeofothers Theabilityofotherorganisationstouseabusinesssknowledge

Thisnewmodeloffersprinciplesthatcontrastwiththoseofthewellestablished,traditional closedmodel.Table21summarisesthesecontrastingprinciplesaccordingtoChesbrough (2003a).

David Scarlatti - X792282

ClosedInnovationPrinciples OpenInnovationPrinciples

Thesmartpeopleinourfieldworkforus. Notallofthesmartpeopleworkforus,sowe mustfindandtapintotheknowledgeand expertiseofbrightindividualsoutsideour company. ToprofitfromR&D,wemustdiscover, developandshipitourselves. ExternalR&Dcancreatesignificantvalue; internalR&Disneededtoclaimsomeportionof thatvalue. Ifwediscoveritourselves,wewillgetit tomarketfirst. Ifwearethefirsttocommercialisean innovation,wewillwin. Ifwecreatethemostandbestideasin theIndustry,wewillwin. Weshouldcontrolourintellectual property(IP)sothatourcompetitors don'tprofitfromourideas.
Table21ContrastingPrinciplesofClosedandOpenInnovation(Chesbrough2003)

Wedon'thavetooriginatetheresearchtoprofit fromit. Buildingabetterbusinessmodelisbetterthan gettingtothemarketfirst. Ifwemakethebestuseofinternalandexternal ideas,wewillwin. Weshouldprofitfromothers'useofourIP,and weshouldbuyothers'IPwheneveritadvances ourownbusinessmodel.

WewillrevisitTable21laterinthereporttodeterminehowtheseprinciplesimpactthe measurementofinnovation. Chesbroughpredictedthateventhemoretraditionalsectorsandcompanieswillevolvefrom theold,closedmodeltoanOpenInnovationmodel(Chesbrough2003a).Theconceptcan alreadybefoundtodayintheITindustry(e.g.,IBM,Linux,etc.)(Andrews2003),biotech companies(e.g.,EliLilly,Pfizer,etc.)(Linder,Jarvenpaa&Davenport2003)andinless advancedtechnologyrelatedcompanies(e.g.,Procter&Gamble)(Huston,Sakkab2006, Sarkar,Costa2008).

David Scarlatti - X792282

Thedifferencesbetweenthismodelandtheclosedmodelaresoimportantthat,evenatthe strategiclevel,thatthenewmodelcannotbeexplainedbydominantbusinesstheory;basic conceptssuchasentrylevelbarriersorthecompetitiveadvantageofresourceownershipare questioned(Chesbrough,Appleyard2007).ThenewOpenInnovationmodelsescapefromthe classicPorter'svisionofbusinessstrategyandsubsequentschools.Atthesametime,thenew modelssurvivalmaydependontheseclassictheories.Appleyardcallsforanew"Open Strategy"conceptthatgainstheadvantagesofthe"emerginganomalies"andstillcreatesa sustainablebusiness.Theopenstrategyurgesabalancebetweenopennessandvaluecapture. Appleyardlinkssustainabilitywithvaluecaptureandcommunityengagementfollowingsome existingbusinessmodelsforopensourcesoftware. OpenInnovationrepresentsanimportantshiftintheinnovationworld,anditisasufficiently newconcepttomeritsubstantialattentionfromresearchers. Duringthepastyear,theEuropeanInnovationScoreboard(EIS)wasreviewedtobetterserve theEuropeanCommunityasatruemeasurementofinnovationattheEUlevel.However,the reportindicatesthatToreflectandmeasurenewformsofinnovationinfutureeditionsofthe EIS,wemustdevelopandincorporatenewindicatorsmeasuringOpenInnovation (Hollanders,vanCruysen2008). In2008,NESTA,theNationalEndowmentforScienceTechnologyandtheArts,publisheda reportaspartoftheInnovationIndexprojectandproposedmeasuresoffirmlevelInnovation. ThereporthighlightsinterestfromparticipantsformeasuringOpenInnovationa respondentintheaerospaceanddefencesectorexpressedinterestinafuturemeasureof OpenInnovation.However,theauthorsreportItislikely,though,thattheoutputofOpen Innovationcanbemeasuredusingthesamemetricsasfortraditionalinnovation(Adamset al.2008).Theargumentisvalidaslongasoneacceptsthatmeasuringinnovationcanbe limitedtomeasuringtheoutputsoftheprocess.Wehavefound,however,amultitudeof metricsrelatedtotheinputoftheprocessaswellasthewayinwhichitisperformed. AlthoughtheneedtomeasureOpenInnovationisbroadlyrecognised,thereisalackof researchontheapplicabilityofexistingmethodsandmetricsforthisnewmodel.Because OpenInnovationprinciplesaresodifferentfrompreviousmodels,itisexpectedthatatleast someadaptationoftheexistingmethodsandmetricswouldbeneeded.

David Scarlatti - X792282

Table22introducesanumberofquestionsrelevanttothemeasurementofOpenInnovation:
ClosedInnovation Principles Thesmartpeopleinour fieldworkforus. OpenInnovationPrinciples Measurementrelated Questions Whichmetricsrelatedto humanresourcesshouldwe use? Howdowemeasure knowledgeoutsideofthe firm? Canweusemetricsto motivateexternalpeople? ToprofitfromR&D,we mustdiscover,develop andshipitourselves. ExternalR&Dcancreate HowdowemeasureR&D significantvalue;internalR&D value? isneededtoclaimsome Howdowemeasurethe portionofthatvalue. portionofeachR&D? Howdoyoumeasureaccess toexternalR&D? Ifwediscoverit ourselves,wewillgetit tomarketfirst. Ifwearethefirstto commercialisean innovation,wewillwin. Ifwecreatethemost andbestideasinthe industry,wewillwin. Wedon'thavetooriginate theresearchtoprofitfromit. Whatmeasuresofthe discoveryphasearerelevant inOpenInnovation? Arespeedmeasuresrelevant? Whatsubstitutesforspeed measuresareofinterest? Whatuseswecanmakeof theseideas?Howdowemix internalandexternalideas? Howcanwemeasuretheuse ofothersideas? Weshouldcontrolour intellectualproperty(IP) sothatourcompetitors don'tprofitfromour ideas. Weshouldprofitfromothers' useofourIP,andweshould buyothers'IPwheneverit advancesourownbusiness model. HowdowetradewithIP? Whatdowedoaboutpatent metrics? Howdowemeasure licensing?

Notallofthesmartpeople workforus,sowemustfind andtapintotheknowledge andexpertiseofbright individualsoutsideour company.

Buildingabetterbusiness modelisbetterthangetting tomarketfirst. Ifwemakethebestuseof internalandexternalideas, wewillwin.

Table22ContrastingPrinciplesbetweenClosedandOpenInnovation(Chesbrough2003)andmeasurement questions

ThequestionsshowninTable22pointtoageneralresearchquestion:CanOpenInnovation bemeasuredinthesamewayastraditional,closedInnovation?

10

David Scarlatti - X792282

Theresearchpresentedhereanalysesthisissueand,basedonanycontrastfoundbetweenthe newandoldmodels,elaboratesonrecommendationsfortheimplementationofmetricsfor OpenInnovation.

2.2 Existingrelevantknowledge
Thissectionsummarisesknowledgefromexistingpublishedresearch. Innovationmeasurement Existingliteraturehelpsonetounderstandthemotivationtomeasureinnovation.Thereis agreementamongpastandcurrentliteratureontheneedtomeasuretheresearchand developmentprocessand,takingawiderviewpoint,tomeasureinnovation. Oneneedsmeasurementtoimprovethings.Themeasurementprocessallowsresearchersto setupareferencebase,todeterminewhetherchangeshavehadanimpact(positiveor negative)andtodeterminethesizeoftheimpacts(e.g.,Doyouexecutemoreprojectson timeusingthisprojectplanningtool?).Intheresearchanddevelopmentprocess,therewasa timewhenimprovementsweremademoreeasily,inlargesteps,butonceperformance approacheditslimit,companiesneededmeasurementmetricstocarefullytracknew improvements(McGrath,Romeri1994).TheperformancemeasurementforR&Dwas acceptedlaterthansuchmeasurementswereacceptedforotherprocesses(Chiesaetal. 2008).Nevertheless,therewasaclear,strongcaseforimplementingsuchmeasuresinR&D (Bremser,Barsky2004). Theintroductionofqualitymanagementsystemsandcertifications(suchasISO9001)hasbeen anotherprecursorofmeasurementinresearchanddevelopmentdepartments(Schumann, Ransley&Prestwood1995). Moderntechniquesforpeoplemanagement,suchasvariablecompensationlinkedto objectives,alsocalledformeasurement.Tosuccessfullytightenthecompensationpackagesof researchanddevelopmentstaff,theremustbeagreementonmeasurableobjectives.(Davila, Epstein&Shelton2005).Davila,Epstein&Shelton(2005)defineinnovationandpresenttwo differentstrategiesacompanycanfollowtoinfluencetheinnovationprocess:PlaytoWinand PlayNottoLose.Thebookofferstipsforinnovationstrategydefinitionandforstructuringa companyforinnovation.MorerelevanttothisresearchisthechapterHowtomeasure Innovation,whichintroducessoftmetricsversushardfinancialmetrics.Thesemetricsare linkedtoanincentivesprogrammetosupportinnovation.

11

David Scarlatti - X792282

Supportforthesedifferentacademictrendsexistsamongcurrentseniormanagementinthis field.Amongrespondentstoa2008BostonConsultingGroupsurveyonmeasuringinnovation, 74%agreedwiththeneedforandrelevanceofsuchmeasurements(Andrew,Haanaes2008). TheBostonConsultingGroupreportalsohighlightedtherealityofinnovationmeasurementin companies(2008survey).Thereisagreementontheneedtomeasureinnovation,butsuch measurementsarerarelyperformed.Moreover,thereisageneraliseddissatisfactionwiththe resultsofeffortsmadetomeasureinnovation.The2008reportdescribesthecostand competitiveimplicationsofinnovationmeasurement.Inaddition,someofthemostinnovative companieshavesomeofthemostrigorousmeasurementsystems,butthereportfailsto outlineitsselectionprocessforidentifyingthemostinnovativecompanies. Mostcompaniesarenothappywiththeirmeasurementsystembecausetheysystemusesonly afewmetricsanddoesnothaveatieintorewardandincentivesystems.Innovationoutputis moreaccuratelymeasuredthantheprocessortheinput.Themostcommonlymeasured componentsarepresentedandcanbecomparedwithDamanpourandWischnevsky's(2006) proposal.Similarmeasuresaretimetomarket,effectivenessandefficiency,timetovolume, andlifecycleperformance.Somecomponentsaddedbythisreportareprofitability(similarto theR&Dindexfrom(McGrath,Romeri1994)),ideagenerationandtheselectionandhealthof portfoliomanagement. Themostcommonlyusedmetricsrelatetofunding/revenueandtimeperformance.Thereport showsthegapbetweenthenumberofavailabletheoriesandtheirimplementationatvarious companies.Researchintomeasurementframeworksisrequiredtofacilitatemethodology adoptionbycompanies. Evengovernmentsindifferentpartsoftheworldagreeontheneedtomeasureinnovation: Thecentralityoftheneedtoadvanceinnovationmeasurementcannotbe understated(InnovationMeasurement.TrackingtheStateofInnovationinthe AmericanEconomy,TheAdvisoryCommitteeonMeasuringInnovationinthe21st CenturyEconomy). Ithasbeenlongunderstoodthatthegeneration,exploitationanddiffusionof knowledgearefundamentaltoeconomicgrowth,developmentandthewellbeingof nations.Centraltothisistheneedforbettermeasuresofinnovation(OECD2005).

12

David Scarlatti - X792282

ItispossibletogroupthereasonsformeasuringInnovationintofourmaincategories(see Figure24).

Figure24ReasonsformeasuringInnovation

Theliteratureindicatesageneralagreementregardingthedifficultiesinvolvedinmeasuring innovation(Chiesaetal.2008)(McGrath,Romeri1994).Someauthorshaverecognised peculiaritiesintheR&Dprocess(KerssensvanDrongelen,Cook1997),andthereisagreement thatthiscomplexityrequirestheuseofseveralcombinedmetrics(Schumann,Ransley& Prestwood1995)(MarkRogers1998).Again,theresearchpresentedherefoundthatsenior managementagreedonthis:innovationperformancemeasurementbelongsinthetoo hardbasket(Adamsetal.2008). KerssensvanDrongelenandCook(1997)explainhowthechangingroleofR&D,fromasupport processtoadriverofcompetitiveadvantage,callsfortheproperperformancemeasurements. TheauthorsreviewtheR&Dperformancemeasurementliteratureandpositionitinthe contextofcontroltheory.Furthermore,theyidentifyfourmainproblemsexperiencedwhen measuringR&Dperformance:R&Dcontributionisolation,timelag,correctnormsand acceptance.Theyalsodiscusstwomainissues,thepurposeofmeasurementandcontingency factors.Thepurposeofmeasurementisthenbrokendownintotwogroups:motivatingand

13

David Scarlatti - X792282

diagnosing.Contingencyidentifiesseveralissues:thetypeofR&D,companysize, aggregationlevel,etc.KerssensvanDrongelenandCook(1997)introduceaninteresting categorisationofmetricsfortheirresearchandalignitwiththefourcategoriesfoundinthe BalancedScorecard.Theauthorsfurtherdefinethesystemrequirements,designparameters andprinciplesrequiredforaperformancemeasurementsystem.Interestingly,becausethe typeofR&Disseenasakeyconceptfordesigningthesystem,onecanviewOpenInnovation asatypeofR&D. Themainchallengescanbegroupedintofourcategories(seeFigure25).

Figure25ChallengesmeasuringInnovation

Mostapproachesproposedforinnovationmeasurementfocusonafewsetsofdimensions andmetrics(Brown,Svenson1988,Driva,Pawar&Menon2000,Godener,Soderquist2004, Hauser1998,Pawar,Driva1999,Werner,Souder1997b,Werner,Souder1997a,Pappas, Remer1985).OnlyafewhavetriedtodefineacompleteframeworkorPerformance ManagementSystem(PMS)(Chiesaetal.2008),frequentlybasedontheBalancedScorecard (BSC)(Bremser,Barsky2004). Chiesaetal.(2008)statethat,inadditiontotheR&Dfunction,theuseofaPerformance MeasurementSystem(PMS)isacriticalsuccessfactor.Theyexploresomeofthereasonswhy itcanbeadifficulttask,butalthoughtheydorecognisesomeapproaches,theyseemostof

14

David Scarlatti - X792282

themasbeinglimitedtoasetofdimensionsandmetrics.Theauthorsfurtherreferenceafew approachestoafullPMSsystem,basedontheBalancedScorecardofKaplanandNorton.They complementthisapproachbyproposingaframeworkapplicabletoanykindofR&Daspartof technologyinnovation.Theydefinethestepsnecessarytoapplytheframework,andtheyoffer apracticalexampleforaClinicalResearchOrganisation(CRO).ThefullsetofPMSelements includesasetofdimensionsandmetrics,togetherwithastructureandaprocessforoperating thesystem.Theframeworktakesintoaccountcontextualfactorstodefinetheelements, namely,strategy,entities,typesofactivity,PMSobjectives,andavailableresources. Chiesaetal.outlinetherelationshipsamongeachfactorintheframework,butdonotalways specifyorclarifytheserelationships.Thestepsgivenintheexamplearenotgeneralised enoughtolearnhoweachfactorcaninfluenceeachelementinthePMS.Thestudyisagood example,butitisnotacompleteguide. BremserandBarsky(2004)focusontheframeworkaspectsofthemeasurementsystemsand worktointegratetheStageGateandBSCapproaches.Themaincontributionofthepaperis theevaluationoftheBalancedScorecardinanR&Dprocess.BremserandBarsky(2004)map thefunctionsoftheperformancemeasurementsystemusingthecharacteristicsoftheBSC anddemonstrateitssuitability.Someexamplesofmetricsthatareusefulforthisresearchare reported. Someorganisationshavedevelopedtheirownframeworks.TheEuropeanFoundationfor QualityManagement(EFQM),publishedin2005,isanadaptationoftheExcellenceModelfor innovation(EFQM2004).TheOsloManualproposesaconceptualframeworkforcollecting quantitativedata(OECD2005). Categorisationorclassificationofinnovationhasbeenusedtosimplifytheprocessof measuringit(Damanpour,Wischnevsky2006).Damanpoursetsinnovationasaprerequisite forcompetitiveadvantageandgrowth,andtheauthorshighlighthowpreviousresearchon thedeterminants,processesandconsequencesofinnovationhasproducedinconsistent results.Damanpourstatesthatcurrentapproachestoexplaininconsistenciesbasedon innovationtypes(productvs.process,technicalvs.administrative,radicalvs.incremental)fail, andproposesamodelbasedonorganisationaltypes(InnovationGeneratingOrganisationsvs. InnovationAdoptingOrganisations).However,Damanpourrestrictshismodeltothestudyof oneoftheaforementionedtypes,namelytechnicalinnovation.Damanpourdoesnotmention applicabilitytoadministrativeinnovations.

15

David Scarlatti - X792282

DamanpourexplainsthatInnovationGeneratingOrganisationsdealwithcreativeprocesses andInnovationAdoptingOrganisationsdealwithaproblemsolvingprocess.Thesedifferent approachesarehighlightedastherootcauseofthedifferencesbetweenthetwoapproaches. InnovationAdoptingOrganisationsaresupposedtousewhatInnovationGeneration Organisationsdeliver,whichiscloselyrelatedtotheuseoftechnologybothasaproductand asanapplication,asshowninOpenUniversityscourseT840.Whathappensifaproductfor oneorganisationbecomesanapplicationforanother,followingahierarchyofcomponents? Damanpourappliedthemodeltoexplainhowinnovationisinfluencedbyotherfactorssuchas anorganisationssizeorageandhowradicalitisintermsofinnovation.Interestinglyforthis research,Damanpouralsodescribestheimplicationsofthesefactorsonthemeasurementof innovation.Damanpourfocusedonlyontheoutcomemeasurement,discardinginput measurement(R&Dexpenditure)orintermediateoutcomes(Patens),indirectdisagreement withKatila(2008),whoseespatentsasanoutcomemeasure. Damanpourreviewedaninterestingnumberofmeasures,eachwithreferencetotheratesand speedofinnovation(e.g.,speedofdevelopment,earlinessofadoption,percentadoptersover aperiodoftime,timeliness,speedofimplementation,extentofimplementation,rateof adoption,andinnovationimpact).TheauthorsalsoexplainedtherelationbetweenInnovation GeneratingOrganisationsandInnovationAdoptingOrganisations.Theaforementionedlists, however,mustberevisitedfromthepointofviewofOpenInnovationtoassesswhetherthese factorsarestillrelevant. Damanpourintroducesorganisationsinwhichthereiscoexistencebetweendifferentunits thatareadoptingandgeneratinginnovationfortheInnovationIntegrationUnit(IIU),a managingorganisationfocusedonstrategicintegration.Itseemsfromthesefindingsthatan InnovationIntegrationUnitwouldneeddifferentmetricsforperformancemeasurement. Attheotherextremeofgenericframeworks,thereareveryspecificproposalssuchastheuse ofpatentsinradicaltechnologyinnovation(Katila2008)orthedefinitionoftheR&D effectivenessindexforproductdevelopment(McGrath,Romeri1994). Katilaproposestheuseofpatentsandpatentcitationsformeasuringinnovation.However, sheproposesthisstrategyforaparticularkindofinnovation,namely,radicalinnovation.More specifically,shefocusesontechnologicalradicalinnovationandexcludestheotherthreekinds ofradicalinnovation:industry,organisationanduserradicalinnovations.

16

David Scarlatti - X792282

Katilasometimesreferstomeasuringtheradicalityofinnovation,anotionthatiswellsuited formeasuringinnovationbecauseitisassumedthatmoreradicalinnovationsarealsobetter innovations.However,thismaynotbethecaseforallorganisations.Katilaoccasionallyimplies thatincrementalinnovationcanalsobemeasuredwithpatents,butthisthoughtisnot explicitlymentioned.Somelimitationsarerecognised,andKatilapointsoutthatprobablythe bestuseofinnovationmeasurementsisforcomparisonwithinanindustry(i.e.,benchmarking use).Oneofthelimitationsmentionedisthatmeasurementsthatareonlyapplicabletoa fractionofthetotaloutputcanbegreaterthanexpectedbecausethesizeofthefractionis notmentionedandislikelysmallforsomeindustries.Moreover,technologicalinnovationsare notalwayspatented.Eitherthereareotheroptionsavailablefortheirprotection(trade secrets)orthereisnoprotectionatall(standards).SpecificallyforOpenInnovation,patents areanoption,butarenotalwayspreferred. Conversely,sometechnologicalinnovationsmaybepartofagroupofotherinventions.For example,anewcellphonecanincludehundreds(ifnotthousands)ofpatentsbehindit.The correctionofsomelimitationsispalliatedusingpatentcitation,basedoncitationonlyin subsequentpatents.Thisendogamyisaweaknessofthesystem,butitclearlymeasuresonly theoutputoftheinnovationprocess. Itseemsthatpatentsandpatentcitationscouldbeusedasametricbutthatothermetricswill alsobeneedediftheorganisationwantstobalanceincrementalandradicalinnovation.The needforothermetricsalsoappliesiftheorganisationwantstodevelopotherkindsof innovation(industrial,organisational,user,etc.).Thisistrueformeasuringinputsandthe processitselfandisnotlimitedonlytothemeasurementofoutputs. McGrathfocusesonproductdevelopmentandindicatesthatitismuchmoredifficultto measurethanotherbusinessprocesses.AcompanyshouldcriticisethepercentageofR&D spendingasameasureandshouldadvocateformeasuringtheeffectivenessofR&D(instead ofthelevelofinvestment).McGrathproposesaveryspecificmeasure:theR&DEffectiveness Index.TheR&DEffectivenessIndexfocusesontheeconomicoutputoftheprocessandis basedonthestrongcorrelationbetweengrowthandprofitability,asdemonstratedthrougha benchmarkingstudy.

17

David Scarlatti - X792282

Figure26summarisestheexistingapproaches:

Figure26ApproachesfacilitatingInnovationMeasurement

OpenInnovation ThefatherofOpenInnovation,HenryChesbrough,definedtheconceptinhisseminalworksin 2003(Chesbrough2003a),(Chesbrough2003b).Chesbroughpresentsthe"OpenInnovation (OI)modelinoppositiontothetraditionalclosedapproach.Heshowshowmanywell established,giantfirmshavebeenchallengedbynewcomersthatbypassthesupposed"entry barrier,"thankstothisnewapproachtoinnovation.ChesbroughstatesthattheOpen Innovationmodelwillbeadoptedsteadilyinalmostallindustriesbyallcompanies,andhe definesthreeareastoshowhowtheadoptionofthemodelishappeningineachofthem: funding,generatinginnovationandcommercialisinginnovation. Bythattime,othershadalreadystartedresearchingotherwaysofsourcinginnovation(Linder, Jarvenpaa&Davenport2003),butChesbroughblendedbothsourcingandexploiting.Since then,themodelinitiallyadoptedbytechnologyfirms(e.g.,IBM,Intel,Lucent,Xeroxand others)(Andrews2003)hasbeenadoptedbymoreandmorecompaniesinverydifferent

18

David Scarlatti - X792282

sectors(e.g.,Procter&Gamble,GeneralMotors,EliLillyColgatePalmoliveandothers) (Gwynne2007). Differentwayshavebeenidentifiedtousethenewmodel.Suchareasforimplementation includecorporateventurecapital,internalspinoffprogrammes,externallicensing programmes,"useitorloseit"initiatives,programmestolicensetechnologies,portalsto outplacetechnologiesandotherportalstosolicitinputtechnologies(Andrews2003).Andrews pinpointsthetwosidesoftheOpenInnovationapproach,namelytheaccesstoexternal knowledgeversusthemonetisationofinternalknowledge.OpenInnovationpractitioners identifiedbyAndrewsincludeIBM,Intel,Lucent,Xerox,Procter&Gamble,Merck,Pfizerand Millennium.Andrewsproposesthefollowingmetrics: Identifythesourcesofthemostimportantideasinyourindustryoverthepastfive years. Identifyhowmanycamefrominsideyourcompany,howmanycamefrominsideother currentindustryparticipants,andhowmanycamefromoutsiders. Identifywhatpercentageofyourownpipelineoffutureprojectscomefromoutside versusfromwithin.Howdoesthatratiocomparewiththefirstratio? Usingthisapproach,relationswithcustomers,competitorsandsuppliersmaybemodified.For example,areasformodificationcanincludethefollowing:customerintegration,supplier integration,competitivealliances,customerofcustomerintegration,integrationofsecondtier suppliers,crossindustryinnovation,universityindustrycooperationandglobalisationof innovation(Enkel,Gassmann2007) Anewconcepthasevenbeencreatedtoenablethemodel:knowledgebrokers(Gwynne 2007),(Sousa2008).GwynneintroducestheideaoftheKnowledgeBrokerasacatalystin theOpenInnovationmodel.ExampleshementionsincludeNineSigma,yet2.comor InnoCentive.GwynnepointstooneveryspecificproblemofOpenInnovation,namely,howto integratethetechnologycomingfromoutsidetheorganisation.Heworkedwithearlyadopters oftheOpenInnovationmodelincludingProcter&Gamble,GeneralMotors,IBM,EliLilly, ColgatePalmolive,andPhilips. Soonitbecameapparentthat,inanOpenInnovationmodel,thereisakeyissue:balancing valuecaptureagainstvaluecreation(Andrews2003),(Laursen,Salter2006).Thesetwo dimensionsofvalue(captureandcreation),plusthedualityofinternalandexternalresources,

19

David Scarlatti - X792282

configuresamapthatisveryrelevantwhenthinkingaboutmeasurements.Thisconceptis discussedlaterinthispaper. ItisimportanttonotethatOpenInnovationexhibitsadiffusedefinition,asrecognisedbythe conceptsownoriginator(Chesbrough2006b).ThelatestChesbroughworkclaimsthatthe newmodelrequiressomechangesevenatthestrategylevelforfullexploitation,whichhe callsOpenStrategy(Chesbrough,Appleyard2007). Interestingly,currentkeytrendsintheresearchonOpenInnovationdonotfocuson measurement.Rather,themainfivethemesarebusinessmodels(balancingvaluecreationand valuecapture),inclusionofexternaltechnologies,identification/assessmentofknowledge, StartupsandIntellectualPropertyManagement(Chesbrough2006a).Oneexceptiontothis lackoffocusonmeasurementandmetricsispresentedinChesbrough(2004),wheretheneed forachangeinmetricsisclearlystated,butisfocusedintheissueofmanagingfalse negatives(i.e.,projectsthatwouldhavebeendiscardedunderaclosedmodelbutmaybe pursuedunderanopenmodel).

2.3 Researchquestions
Thegeneralquestionfromtheintroductionsection,CanOpenInnovationbemeasuredinthe samewayastraditionalclosedInnovation?cannowbesplitintomorespecificquestions: 1. DoesOpeninnovationrequirethesamesortofmeasurementapproachesas traditionalinnovation? 2. WhatmetricsareusefulwhentheOpenInnovationmodelisapplied? 3. WhatchangesareneededregardingmetricswhentheOpenInnovationmodelis embraced?

20

David Scarlatti - X792282

Chapter3 Methodology
3.1 Methodsandtechniquesselected
First,acomprehensiveliteraturereviewrespondstothefirstobjectiveoftheresearch: Determinethecurrentpracticesregardingthemeasurementofinnovation.

AcrosssectionalcasestudyhasbeenusedtoexploreOpenInnovation,withafocusonthe measurementpracticesinplaceorprocessesrelatedtothepracticesidentifiedinthefirst literaturereview.AreviewofthecurrentlyavailableexamplesofOpenInnovationhasbeen usedtorespondtothesecondobjectiveoftheresearch: Determine,basedonOpenInnovationprinciplesandexamplesavailableinthe literature,thebasiccriteriafortheapplicabilityofmeasurementpractices. Analysisandsynthesisofthepreviousdeliverableshavegeneratedthedeliverablefor Objective3: GenerateaproposalforOpenInnovationmeasurementpracticesbasedon correlationofthedeliverablesfromObjective1andObjective2.

3.2 Justification
Thisresearchisempiricalandbasedonobservation.Somesourcesforreviewonthegeneral problemofmeasuringinnovationhavebeenidentifiedasanecessarystartingpoint,butthe availablereferencesexaminingthespecificproblemofmeasuringOpenInnovationarevery limitedatthetimeofthiswriting.Other,alternativeapproacheshavealsobeenevaluated,as discussedinthenexttwoparagraphs. Experimentsdonotseemappropriatebecauseitwouldbeverydifficultwithintheresearch timeframetoconductanyreasonableexperiments.Thetypicalperiodforthemeasurementof abusinessprocessisoneyearbecausethisisthetypicalbudgetingcycle,eveninR&D concepts.Itwouldmakesensetomakeobservationsoverlongerperiodsoftimeasinnovation matures(35years).Similarly,itdoesnotseempossibletofindanorganisationthatwouldbe willingtotestanyconclusionsregardingenhancementsinthemeasurementprocess. Surveyswouldhavepresentedunmanageableriskbecausethepopulationisunknownand dispersed.Anattemptwasundertaken,usingunstructuredinterviewsataworkshoponOpen InnovationconductedinMadridinMay2009,butitwasdifficulttocollectmorethaninformal

21

David Scarlatti - X792282

feedback,andthatfeedbackwasdifficulttocompareoranalyse.Thesurveywasonlyuseful forqualitativeinputabouttheinterestintheresearchtopic,notasadatacollection technique. Casestudies,however,seemedtofitwellwiththescopeofthisresearch.Thankstothe Internet,itwaspossibletoaccessinformationregardingrealexamplesofOpenInnovation adoptionatdifferentcompanies.Asstated,OpenInnovationisnewenoughtodiscardany longitudinalcasestudy.Duetothegeneralapplicabilityoftheresearch,theexemplarcase studyalsodoesnotapply.Therefore,acrosssectionalcasestudyhasbeenselectedandtakes intoaccountavarietyoforganisationsastheyoperatetoday.

3.3 Researchprocedures
Thedatacollectedforthisresearchcomefromtwomainsources: PublicdomaincompanyrecordsonOpenInnovationadoption.Theuseofsuchrecords avoidspotentialconfidentialityissues. Literatureavailableregardingthemeasurementofinnovation.

Asmentionedinsection3.2,aninitialsetofunstructuredinterviewswasattempted,with limitedbenefit.Therefore,noadditionalquestionnairesorinterviewswereundertaken. UseoftheISIWebofKnowledgetoolhasallowedmetoidentifyrelevantexistingliterature onboththemeasurementofinnovationandthethemesofOpenInnovation. Otherrelevantsourceshavebeenusedtocollectinformation: TheOpenInnovation.euplatformfoundedbyProf.Dr.WimVanhaverbeke,in partnershipwithHenryChesbrough. TheUKInnovationIndexProject,ledbytheNationalEndowmentforScience, TechnologyandtheArts(NESTA). TheEuropeanIndustrialResearchManagementAssociation(EIRMA).

3.4 Ethicalconsiderations
Themainethicalconsiderationhadtodowiththedisclosureofprivateinformationused duringtheresearch.However,thefinaldatausedarepubliclyavailable,sonodisclosureof privateinformationhastakenplace,tothebestofmyknowledge.Detailsoftheresponsesto theinformalsurveyhavenotbeenincluded,sonoethicalissuesareapparent.

22

David Scarlatti - X792282

Chapter4 Analysisandinterpretation
4.1 Summaryofdatacollected
Currentpractices Thissectionprovidesanoverviewofcurrentpracticesascitedintheliteratureonmeasuring innovation.ThesepracticesarelateranalysedtodeterminetheirsuitabilityfortheOpen Innovationapproach. Fundamentally,everyapproachfollowsthesameprocessformeasurement,whichiswell definedbyKerssensvanDrongelenandCook(1997)andisshowninFigure41.

Figure41Themeasurementandcontrolprocess(KerssensvanDrongelen,Cook1997)

Themaindifferencesaremorerelatedtowhatismeasured(i.e.,themetricsused).Metricsare sometimesreferencedasKeyPerformanceIndicators(KPIs).Multiplefactorsinfluencethe selectionofmetrics,andsomeresearchhasalreadyaddressedtheidentificationofthese factors.ThesefactorsarewhatKerssensvanDrongelenandCook(1997)callcontingency factorsorwhatChiesaetal.(2008)refertoascontextualfactors.Ourresearchacceptsthe

23

David Scarlatti - X792282

typeofresearchasafactorinfluencingmetricsandwillfocusontheparticularcaseofOpen Innovation. Whenacompleteframeworkisused,thesemetricsarealignedwithdimensions(usuallya dimensioncanbemeasuredwithasetofmetrics)andaformalprocesswithnormsandrules aboutwhodoesthemeasurement,how,withwhatfrequency,andsoon.Agoodexampleofa frameworkistheBalancedScorecard(Bremser,Barsky2004). Reviewingthevastarrayofmetricsproposed(seesection2.2Existingrelevantknowledge),we findmetricsthatcanbegroupedintodifferenttypes,accordingtodifferentcharacteristics.For furtheranalysis,itseemsusefultoexaminegroupsofmetrics,insteadofeachoneindividually. Severalgroupswereidentifiedforthisresearch: Absolutemetricsandratiometrics:Absolutemetricsareexpressedinunitsofsomemagnitude (e.g.,Eurosormonths)andcanbecomparedtotargetorreferencevalues.Ratiometricsare expressedinunitsperanotherunitoraspercentages.Ratiounitsarenormallyusedforbetter comparisonoveraperiodoftimeorbetweendifferentorganisations. Input,intermediateoroutcomemetrics:Thisclassificationismorerelevantforourresearch. Metricscanbefoundrelativetotheinputoftheinnovationprocess(e.g.,abudgetforR&Dor expenditure),theoutput(e.g.,thenumberofnewproducts)or,mostrelevantforus,the processitselforintermediateprocesses(e.g.,thenumberofpatents). Accordingtothetypeofmagnitude,wefoundmetricsforspeed,time,financialsor performancerelatedmeasurements(e.g.,theexecutionofprojectsormarketshare). Quantitativeandqualitative(orobjectiveandsubjective):Somemetricsareanexactmeasure ofamagnitude,butothersareestimationsorrankingsbasedonexpertjudgementor subjectivevalidation. Metricsattheorganisationlevel,grouplevelorindividuallevel:Somemetricscanbevalidat allthreelevels,whilstothersonlymakesenseattheaggregatedorindividuallevel. Metricsrelatedtothecreativityphaseortothevaluecapturephase:Thesearerelatedtothe phasesforinnovationfromDavila,Epstein&Shelton(2005). Furthermore,metricscanserveoneorbothmainobjectivesandcanbeusedtomonitor and/ortomotivate(reward).

24

David Scarlatti - X792282

Interestingly,otherauthorshavefoundverysimilargroupingcategoriesforR&Dmeasures (Ojanen,Vuola2003)(Table41).

Table41ThedimensionsofR&Dperformance(Ojanen,Vuola2003)

Basedontheliteraturereview,acollectionofmetricshasbeenselected.Themetricsare groupedconsistentwithlistedcharacteristics(Table42).Thistableserveasareferencefor theanalysisofthedifferentgroupsofmetricsandishelpfultounderstandwhichgroupsare affectedinanOpenInnovationapproachandtounderstandthenatureoftheeffects.

25

David Scarlatti - X792282

METRIC Time used to adopt an innovation Time used to develop a product Time used to commercialize Project related measures (EVM) Impact on market (market share) Profitability Idea generation and selection Time to market R&D effectiveness and efficiency Health of the innovation portfolio Life cycle performance Time to volume Customer satisfaction Total fund invested in growth projects performance, projected versus actual # of projects that meet planned targets development time (Average) revenue realized form offering in the past X years Cannibalization of existing products sales by new offerings % of ideas funded # of projects killed # of new products per dollar spend on R&D TRLs Technology maturation achieved # product at a given stage of the stage-gate approach Time among stages % passing each stage #acceptable rate Safety incidents (rare) Skill coverage of competencies Training hours # patents # patent citations # patents per employee Effectiveness Index (ratio output/input) Performance: present values of an accomplishment R&D contribution to profit/R&D cost Market share gained due to R&D Scores on surveys-customer satisfaction Scores on surveys-employee satisfaction % of customer driven projects Engineering hours on projects/total engineering hours % projects terminated Hours on projects / total R&D hours Current time to market/ reference time to market Rate of reuse of standard designs or proven technology Sum of revised project duration / sum of planned duration # of times rework % budget spent internally and externally on basic and applied research % of projects in co-operation with third party %of project evaluation ideas applied in new projects Talent, in recruitment, training Knowledge Management Tools use Communication Effectiveness Balance of innovation portfolio

Absolute or Ratio

Phase: Input iNtermediate Outcome

Magnitude

quanTitative quaLitative

Levels: Organization Group Individua

Creativity or Value creation

Function: Monitoring Motivating

A A A R R A A A R R R A A A R A A A A R A R A A A R R A R A A A R R A A A A A R R R R R R R A R R R A R A R

N N N N O O N N N O N N O I N N N O O N N O N N N N N N I N N N N O O O O O O N N N N N N N N I N N I N N O

Time Time Time Index %/ ideas Time Index % Index Time Index % projects Time %/ideas projects products Index products Time %/projects products incidents % Time patents citations index index %/ %/ index index %/projects %/hours %/projects %/hours %/Time index %/Time reworks %/ %/projects %/ideas talent %/time index %

T T T T T T T T T T T T L T T T T T T T T T T T T T T T L T T T T T T T T L L T T T T T T T T T T T L T L T

O/G O/G O O/G O O O/G/I O O O O O O/G O O/G O/G O/G O O O O/G O O/G O/G O/G O/G O/G O/G/I O/G/I O/G/I O/G/I O/G/I O/G O O O O O/G/I O/G/I O/G O/G O/G O O O O/G

C C V C V V C V C C C V V C C C C V V C C C C C C C C C C C C C C V V V V V V C C C C V C C C V C C C C C C

M M M M M M R M M M R M R M R M M M M M M M R M M M M R R R R R R M M M M R R M M M M M M M R M M M R M R M

O O O O/I O/G/I O O

Table42GroupedInnovationmetrics

26

David Scarlatti - X792282

BasicCriteriaofApplicability BasedontheOpenInnovationreferencematerial,thissectionidentifiesthecriteriathat shouldbeusedtojudgetheapplicabilityofcurrentmeasurementpracticestotheOpen Innovationmodel. OpenInnovationprinciplesintroducedinprevioussectionswillserveasaguideforthiscriteria identification: Notallofthesmartpeopleworkforus,sowemustfindandtapintotheknowledgeand expertiseofbrightindividualsoutsideourcompany. ThisprinciplereferstothefactthatintheOpenInnovationmodel,theemployeesofthe company(peoplewhoworkforus)arenottheonlyappropriateparticipantsinthe innovationprocess.Peoplefromoutsidethecompanycanandshouldbeinvitedto participateintheinnovationprocess.However,wemustensurethatthepeoplefromoutside whoparticipateininnovationaretherightones(i.e.,smart,bright,knowledgeable). Accordingtothisprinciple,peoplerelatedmetrics,whichusuallyrefertointernalstaff,should beusedoradaptedtoincludeexternalactors. Atthesametime,metricsrelatedtoknowledgefrompeoplemustcovernotonlyinternal knowledgebutalsotheknowledgeaccessedfrompeopleoutsideofthecompany. Becausepeoplerelatedmetricscanbeaimedatmotivatingpeople,themetricsshouldalso aimtomotivateexternalpeople. Metricsregardingbrightpeopleareneededtoensurethattheorganisationisaccessingthe rightpeople.Brightcanbeassociatedwithtalent,sotalentmeasurementisprobably important. ExternalR&Dcancreatesignificantvalue;internalR&Disneededtoclaimsomeportionof thatvalue. ThisprinciplerecognisesthatbothexternalandinternalR&Dmustbeusedintheinnovation process.InternalR&Diseasytolocateandtomanageinthetraditionalway,butexternalR&D isbeyondtheboundariesofthecompany.Therefore,adifferentapproachisneededfor externalR&D.ThefirstthingtoestablishisaccesstotheexternalR&D.Then,onecandevelop waystomeasureit.

27

David Scarlatti - X792282

MetricsaboutthevalueofR&DmustbeapplicabletoexternalR&D. MetricsmustdifferentiatebetweenexternalandinternalR&D,whichisespeciallyrelevantfor ratiosandcomparisonmetrics. MetricsrelatedtoaccesstoexternalR&Dareneededaswell. Wedon'thavetooriginatetheresearchtoprofitfromit. ThisprincipleisbasedonthefactonecanaccessexternalR&Datanystage.R&Dinitiatedby otherscanberelevanttoanorganisationatanypointintime.Onehasaccesstotheexternal R&D,anditcanbeusedtogenerateprofit. MetricsareneededtomeasureR&Dthatisnotgeneratedinternally. Metricsmustdifferentiateinternallyoriginatedresearchedfromothertypesofresearch.Profit shouldalsobemeasuredandclassifiedaccordingtowhetheritoriginatedfrominternalor externalresearch. Buildingabetterbusinessmodelisbetterthangettingtomarketfirst. Thisprincipleinvitesabusinesstofocusnotonlyonbeingthefirstbutalsoonthinkingabout therightbusinessmodeltoprofitfromtheinnovation. Timetomarketmetricsmustbebalancedwithothers.Measuresofthesuccessofthe businessmodelareneeded.Speedmetricsmayrefertoonlyasinglephaseoftheinnovation processinsteadofthewholeprocess. Ifwemakethebestuseofinternalandexternalideas,wewillwin. Ideascanbeusedindifferentways,andonecanusebothinternalandexternalframeworks. Usecanmeanincorporateintoanexistingproductoritcansimplymeanthesaleoftheidea. Payingforathirdpartyideacanmultiplythevalueofyourownproduct. Metricsofusageareneeded.Again,distinctionbetweeninternalandexternalideasisneeded. Metricsforcreationmustbebalancedwithmetricsforusage. Weshouldprofitfromothers'useofourIntellectualProperty,andweshouldbuyothers' IntellectualPropertywheneveritadvancesourownbusinessmodel.

28

David Scarlatti - X792282

Intellectualpropertymustbemanagedinthesamewayasanyotherasset.Onecanselland buyit,anditisevenpossibletorentit.Thisprincipleagainfocusesonprofitingfrom intellectualproperty,regardlessofhowitisused. Metricsregardingintellectualpropertyareneeded.Metricsmustdifferentiateonesown intellectualpropertyversusothercompaniesintellectualproperty.Metricsregardingthe profitrelatedtointellectualpropertyareneeded.

4.2 DataAnalysis
Inthissection,thebasiccriteriaandcurrentpracticesarecorrelatedtofindsupportfor currentmeasurementpracticesforOpenInnovation.Ialsoproposeanyenhancementsfor caseswherethereisinsufficientexistingsupport. Currentpracticesshowthelowuseofpeoplerelatedmetrics.Attheindividuallevel,11of54 metricscanbeapplied(Table43).
Phase of process Input / iNntermediate / Outcome Levels Organization/ Group/ Individual Phase Creativity/ Value creation Function Monitoring/ Motivating

Absolute/ Ratio

Magnitude

quanTitative quaLitative

Idea generation and selection Safety incidents (rare) Skill coverage of competencies Training hours # patents # patent citations Scores on surveys-customer satisfaction Scores on surveys-employee satisfaction # of times rework Talent, in recruitment, training Knowledge Management Tools use

A A R A A A A A A A R

N N I N N N O O N I N

ideas incidents % Time patents citations index index reworks talent %/time

T T L T T T L L T L T

O/G/I O/G/I O/G/I O/G/I O/G/I O/G/I O/G/I O/G/I O/G/I O/I O/G/I

C C C C C C V V C C C

R R R R R R R R R R M

Table43Innovationmetricsforindividuals

Thesemetricsaremostusableforexternalstaff,butsomeofthemneedtobeadaptedwhen appliedtoexternalpeople.Clearexampleismetricsbasedonsurveys;thesurveysneedtobe extendedtoexternalcollaborators.Othermetricscallforspecialattentionduetotheissueof theircollectingandtreatingpersonaldatafrompeoplewhoarenotemployees.Ingeneral, metricsareapplicablebutmaybecomplexifacompanyneedstoaccountforexternalpeople whodonotcollaborate(i.e.,externalpeoplenotwillingtoreportthetrainingtheyhave received).

29

David Scarlatti - X792282

Especiallydifficultisthemeasurementoftalent.Areexternalcollaboratorsreallybright people?Whataretheparametersbywhichthisdeterminationismade?Knowledgerelated measuresmustbeapplied. Theconvenienceofthesemetricsformotivatingexternalcollaboratorsisstilltobeproven becausetherewardmechanismwillbedifferent(i.e.,themeasuredindividualsarenotonthe payroll). ValueoftheR&DisusuallymeasuredinEuros.Theproblemwithusingthesemetricsfor externalR&Disthattheeconomicvalueisusuallyassociatedwithproducts.However, productswillincludebothinternalandexternalR&D.Tousethesemetricsproperly,itis necessarytoaccountforbothinternalandexternalcontributionsatdevelopmenttime,which enablesmanagerstolatersplitthiseconomicvalue.TypicalmetricsfoundregardingR&Dvalue areshowninTable44.
Phase of process Input / iNntermediate / Outcome Levels Organization/ Group/ Individual Phase Creativity/ Value creation Function Monitoring/ Motivating

Absolute/ Ratio

Magnitude

quanTitative quaLitative

Impact on market (market share) Profitability Total fund invested in growth projects revenue realized form offering in the past X years Cannibalization of existing products sales by new offerings Performance: present values of an accomplishment R&D contribution to profit/R&D cost Market share gained due to R&D % budget spent internally and externally on basic and applied research

R A A A A A A A R

O O I O O O O O I

%/ %/ %/ %/

T T T T T T T T T

O O O O O O O O O

V V C V V V V V V

M M M M M M M M M

Table44InnovationmetricsregardingR&Dvalue

Theabilitytodistinguishbetweenexternalandinternalvalueisespeciallyrelevantformetrics liketheBalanceoftheinnovationportfolio.Suchanabilityalsopermitsacompanyto enhancemetricslikePercentofprojectsincooperationwithathirdparty.Balancecanbe pursuedonthenewaxisofinternal/externalvalue,andpercentageofcooperationcanbe measurednotonlyintermsofthenumberofprojectsbutalsointermsofthevaluethey provide. Whatisdifficulttofindinatraditionalmeasurementsystemaremetricsregardingaccessto externalR&D,whichisacleargaptoclosewhenadoptingOpenInnovation.

30

David Scarlatti - X792282

ApartfromsplittingprofitmetricsintointernalandexternalR&D,itisimportantto differentiatemetricsfortheinput(i.e.,originating)phaseoftheprocess.OutputsofOpen Innovationarenotexpectedtobedifferent(newproducts,methods,patents),butitisthe inputandintermediatephasesthatareexecuteddifferently.Usefultoolsforthissplittingare availablesystemsforclassifyingthematuritylevelofatechnology.OnesuchsystemisNASAs TechnologyReadinessLevels(Mankins1995).Metricscanbeusedtocomparethematurity levelatthestartingpointandattheendingpointforbothinternalandexternalR&D. MeasuringprofitintermsofR&Dsold(e.g.,licensingandpatentselling)isalsoimportant.One canmeasureprofitinEuros,soitispossibletoeasilycomparethesaleofR&Dwiththesaleof traditionalproducts. Metricsarealsoneededforinputsthatcomefromoutsidetheorganisation(i.e.,ideas proposedtothecompany).Thenumberofideasgeneratedshouldbecomparedwiththe numberofideasselectedfromoutsidethecompany. Typically,thereisstrongemphasisontimetomarketmetrics.Tobalancethesemetricswith theOpenInnovationfocusonbusinessmodels,otherindependentmetricsareneeded(i.e., marketadoptionofproducts)todeterminewhetheranorganisationisthefirsttoarrive. TimeamongstagesandPercentpassingeachstageareusefultomeasurespeedatother phases,andthesemetricsareperhapsmorerelevantthanthedurationofthewholeprocess. Usagemetricsareneededatdifferentstages(e.g.,percentofideasused,percentofproducts used,percentofpatentsusedandpercentoftradesecretsused),andsuchmetricsshould includeinformationonwhethertheusageisinternalorexternal.Newratiosarepotentially valuableinthisarea.Forexample,metricsmayincludeinternallycreatedideasversus externallyacquiredideas,ortheymayincludeinternalideasusedversusexternalideasused. Theseratioscanhelptodetectsituationswheremoreweightisontheworst,lesseffective approach. MetricsneedtoreflectthetransitionfromClosedInnovationtoOpenInnovation,andthey helptodetectdeviations.Suchdeviationsincludedesorbinginnovation(i.e.,toolittleexternal technologyacquisition)andabsorbinginnovation(i.e.,toolittleexternaltechnology exploitation)(Lichtenthaler2008).LichtenthalerpresentshowtheOpenInnovationstrategies canvaryfromaproperbalancebetweenexternaltechnologyexploitationandexternal technologyacquisition.Heidentifiesgroupsofcompanies,rangingfromClosedInnovatorsto OpenInnovators,withextremedeviationslikeDesorbingInnovatorsandAbsorbing

31

David Scarlatti - X792282

Innovators,seeFigure42.Inhisresearch,Lichtenthalershowshowthehighprofitabilityof theopeninnovatorssuggeststhatstickingtotheclosedinnovationmodelmayleadtoa substantialweakeningofafirmscompetitivepositioninthefuture.Therightbalancebetween exploitationofinternalandexternalR&Disthedesiredcondition.Ratioscanbemeasuredin termsofeconomicvalueand/ormonetaryinvestment.

Figure42BalanceinOpenInnovation(Lichtenthaler2008)

IntellectualPropertyrelatedmetrics(patents,licenses)areusuallyinplace,butwhenadopting theOpenInnovationmodel,itisnecessarytoaddressthebalanceoftheIntellectualProperty produced,theIntellectualPropertyused,theIntellectualPropertysold,andtheIntellectual Propertybought.Again,differentratioscanbedefinedthatareusefulfordetectingdeviations, likesellingalloftheIPacompanyproducesoronlyusingtheIPbought.IntheOpen Innovationmodelanorganisationissupposedtoblenditsownknowledgewiththeacquired knowledgeandtothensellonlyapartoftheknowledgeitgenerates.

32

David Scarlatti - X792282

4.3 Interpretationinrelationtotheresearchquestions
DoesOpeninnovationrequirethesamesortofmeasurementapproachesas traditionalinnovation? Existingmeasurementapproaches,basedonmetricsandaperformancemanagementsystem, arecompletelyvalidfortheOpenInnovationmodel.Thereareonlythreeconsiderationsto keepinmind: Measuringtheoutputoftheinnovationprocessiscompletelyvalidbecausethe objectivesoftheprocessdonotchange.However,newwaysofgeneratingprofitcan requireadditionalmetricsregardingeconomicresultsbecausethereisthepossibility tosellIPorbenefitindifferentwaysfromothercompaniesproducts(includingusing theminonesowninnovations). Measuringtheprocessrequiresanewaxis,namely,thatofexternality.Mostofthe constituentscomingfrominsidethecompanycancomenowfromoutsidethe companyaswell,callingfornewmetrics(ratios)thatcompareinternalandexternal sources. Externalityposeschallengeswhentheobjectivepursuedbyametriccannotbe achieved,asisthecasewithexternalsources.Therefore,newmetricsareneededto complementtheinitialones. WhatmetricsareusefulwhentheOpenInnovationmodelisapplied? Followingtheapproachoftreatingthemetricsaccordingtothegroupsproposedinsection0, theanswertothisquestionincludesthefollowingsixparts: Absolutemetricsandratiometrics:Absolutemetricsremainuseful,butthefocusisonanew breedofratiometricsthatwillcomparetheinternalandtheexternalcontributionsofeach innovationelement.Inadditiontotheresults,eachmetricoftheprocesscanbesplitintothe internal(traditional)sourceandthenewsources. Input,intermediateoroutcomemetrics:Outputmetricsareusefulbecausetheypermitthe measurementofhowthewholeinnovationprocessisworking.Eventually,ifallisreducedto theeconomicsuccessofthecompany,evenusingthesamesinglemetric(revenue)wouldbe useful.However,underarichermeasurementsystem,theinputandintermediatemetricsare veryusefultodeterminehowtheOpenInnovationmodelworks.Inputstotheprocessare nowbroaderbecausethetechnologymaybeatdifferentmaturationlevels,probablybecause

33

David Scarlatti - X792282

thetechnologywasinventedanddevelopedbyotherswhoenteredtheinnovationprocessat laterstages.Intermediatemetrics,whichcanmeasuretheprocessuntiltheoutputofthenew technologytothirdparties,areconsideredthebestoption.Efficiencycannotbemeasured onlyintermsoftimetomarketmetrics.Somephasesmaymoveatgreaterspeedsthan others.Finally,thereisaneedtomeasuretheIP(acquiredorgenerated)atallstages. Quantitativeandqualitative(orobjectiveandsubjective):Thefactthatsomemetricsmustbe translatedtoexternalsources,wheredirectaccesstothedataisdifficult,willdriveanincrease intheuseofqualitativemetrics.Thesemetricsarestillvaluabletomanagetheinnovation processwhenexternalelementsnotunderthedirectcontrolofthecompanymustbe evaluated. Metricsattheorganisationlevel,grouplevelorindividuallevel:Thesecategoriesofmetrics areslightlylessusefulintheOpenInnovationmodelbecausetheconceptoforganisationmust beextendedtoallthirdpartiesinvolved.Also,theindividualsandgroupsofindividualsnow includepersonsfrommultipleorganisations.Individualmetricsvalidforinsidethecompany cannotbepractically(orevenlegally)measured. Metricscanrelatetothecreativityphaseortothevaluecapturephase:OpenInnovation impactsbothphasesoftheprocess.Duringthecreativityphase,onecancounttheideas comingoutoftheorganisation.Atvaluecapture,onecandecidetotransfertoothersan inventionatagivenpriceorroyalty.Therefore,itisinterestingtomaintainmetricsforboth phasesandtodiscriminatebetweenexternalandinternalcontributions. Metricscanserveoneorbothmainobjectives:tomonitorand/ortomotivate(reward): Metricsformonitoringareusefulandmayneedtobecomplementedwithadditionalmetrics thatmeasurespecificresultsorcontributions.Metricsformotivatingfacethechallengeof motivatingexternalpeoplewhoarenotpartoftheorganisation.Thisissuefindsimilaritiesin thepurchasingprocessortothesuppliermanagementfield,namely,howdoesonemotivate suppliers?(Carr,Pearson1999)(Krause,Ellram1997). WhatchangesareneededintermsofmetricswhentheOpenInnovationmodelis adopted? Consistentwithmyinterpretationofpreviousresearchquestions,itisclearthatthechanges implyadaptationoratleastchangestoexternalmetricsalongwiththeadditionofnewones. Newmetricsfornewresults,liketheprofitcomingfromIPsoldorthecostofbuyingothers licenses,areneeded.Itwillbenecessarytosplitsomemetricstoreflectthecontributionof

34

David Scarlatti - X792282

externalsources.Thisway,onecantracethecontributionfromeachpartwhenanalysingthe revenuecomingfromproducts.Finally,currentmetricsthatareusefulinsidetheorganisation willneedcomplementaryparametersthatgeneratethesameimpactoutsidetheorganisation. Developmentofthesemetricswillrequiresconsiderationofthedifferentrelationships involved,andanydevelopmentsshouldalwaysconsiderthelegalityofthemeasurement process. Therestoftheperformancemanagementsystemcanremain.Afewchangesregarding responsibilitiesmayberequired,dependingonwhoisinthebestpositiontomeasureexternal contributions.

4.4 Interpretationinrelationtotheaim
Theaimofthisprojectwastoproducerecommendationsformetricsusedtoevaluatethe effectivenessofanOpenInnovationapproachtoresearch.Allofthefindingspresentedcan becondensedinthisDecaloguethatacompanyembracingOpenInnovationmodelshould followwhenimplementingaperformancemeasurementsystem: 1. Addnewmetricstomeasurenewprofitabilitymodels.Itisnecessarytomeasurethe revenuederivedfromlicensingorfromsellinginnovationtoothers. 2. Splitallrelevantmeasuresintothoseofinternalandexternalorigin.Forexample,a productcanmakeuseof10patents;measurehowmanyareinternalandhowmany areexternal. 3. Introduceratiometricsforcomparinginternalandexternalcontributions.For example,determinewhichnewproductsaredevelopedinhouseversuswhicharethe resultofproductacquisition. 4. Ifnotpresent,introduceamaturitylevelmodeltomeasuretheprocessstepbystep. 5. Measuretheexternalcontributionsatdifferentstagesofmaturity.Forexample, identifythebasicideascomingfromoutside,thebasicinventionssoldtoothersand thefinalsolutionsfromothersthatmaybeincorporatedintoacompanysproducts. 6. Measureefficiencyatdifferentstagesofthemodel.Forexample,measuretheideas discardedversustheideasselected,theprojectsapprovedversustheprojects cancelledandtheproductsinportfolioversustheproductsdiscontinued. 7. Categorisethecontributionstorevenuesasbeingthoseofinternalandexternalorigin.

35

David Scarlatti - X792282

8. Tomotivateexternalparties,usedifferentmetricsthantheonesusedtomotivate internalparties. 9. Checkthelegalityofmaintainingdatabasesregardingexternalpeople. 10. Introduceatalent(i.e.,knowledge)metricforyourexternalparties. ThereisalsoacorollarytothisDecalogue: Avoidusingyourcurrentinnovationperformancemeasurementsystemwithout adaptationforthenewOpenInnovationmodel.

36

David Scarlatti - X792282

Chapter5 Conclusions
5.1 Conclusionsregardingtheresearchquestions
DoesOpeninnovationrequirethesamesortofmeasurementapproachesas traditionalinnovation? Themainconclusionisthatthesamesortofapproachescanbeusedbutitisimportantto adaptthemetricstocapturethewholevalueofthenewapproach.Ifacompanyembracesthe newmodelbutusesexactlythesameperformancemeasurementsystemandmetrics,then thereisahighriskthattherealvalueofthenewmodelwillnotbeproperlyreflected.Sucha scenarioleadstothewrongconclusionsaboutthesuitabilityofthenewmodelforthe company. WhatmetricsareusefulwhentheOpenInnovationmodelisinuse? MostmetricsremainusefulintheOpenInnovationmodel,butsometypesmustbecarefully considered: Ratiosbetweenexternalandinternalcontributions. Metricsregardingnewwaysofprofiting(e.g.,licensing,venturing,spinoffsetc.). Metricsregardingtheefficiencyatintermediatestepsofthetechnologymaturation cycle. Qualitativemetricsforexternalsourcesnotunderthecontrolofthecompany. Newmetricsformotivatingexternalpeople. IntellectualPropertymetrics.

WhatchangesareneededtometricswhentheOpenInnovationmodelisembraced? Itisclearthatthechangesneededarerelatedtothesetofmetricsalreadyinplace.Itwillbe necessarytoaddnewmetricsfornewresults,tosplitsomemetricstoreflectthecontribution ofexternalsourcesandtocomplementsomemetricstoachievethesameoratleastsimilar impactinsideandoutsidetheorganisation.

5.2 Conclusionsregardingtheresearchaim
Theaimofthisprojectistoproducerecommendationsformetricsusedinevaluatingthe effectivenessofanOpenInnovationapproachtoresearch.

37

David Scarlatti - X792282

Thefirstclearrecommendationistousemetricsregardinginnovationinyourorganisationfor thepurposeofproperlymanagingthetechnologythatacompanydevelopsoracquires. ThesecondrecommendationistoadaptthemetricsystemwhenoneembracestheOpen Innovationmodel.Thesechangesaredesignedtobeeffectiveinmeasuringinnovationandin acquiringappropriatedataformakingdecisions. ThethirdrecommendationistoapplytheDecaloguepresentedinsection4.4.

5.3 Furtherwork
Avaluableresearchobjectivewouldinvolvetakingarealperformancemeasurementsystem fromacompanyembracingOpenInnovationandapplyingthechangesproposedherein.Then, onecouldcomparethemetricsresultsfromtheoldperformancemeasurementsystemwith thosefromthenewscheme.Thepurposeforsuchworkwouldbetodeterminewhetherthe newmeasurementsystembettercapturesthevalueoftheOpenInnovationmodel. Otherpossiblepathsforfurtherresearchincludebroadeningthescopeoftheresearch.The initialscopeshowninFigure11canbeexpandedintwodirections: InnovationcanoccuroutsidetheR&Ddepartment.Forexample,theMarketing departmentcouldadoptanopenmodel. Theinnovationmayoccurnotonlyinthetechnology(productorprocess)sector,but alsointhemethodsusedforcompletinganytaskorintheskillsusedinthebusiness.

5.4 Implicationsofthisresearch
Oneinterestingfindingfromthisresearchisadeterminationofhowbeneficialnewparadigms forinnovationmightbeforothertechnologymanagementactivities.Suchparadigmswould necessarilybeaccompaniedbytherightsetofmetricsormeasurementprocessestoprovide upfrontdemonstrationofthebenefitstheycanprovide. Anotherinterestingquestioninvolvesthefactthatopenness,ingeneral,impliesablurringof theboundariesbetweenincompanyandoutofcompany.Ethicalandlegalconsiderationsare neededtotreat,forexample,thepersonaldataofcollaboratorswhoarenotproper employees.Specialattentiontoconfidentialityissuesmaybenecessaryinthisopen environment.

38

David Scarlatti - X792282

Aquestionarisesonceweacceptthatchangesareneededinthemeasurementsystemwhen theOpenInnovationmodelisembraced:Howcanthesechangesbebetterplannedand implementedfromthetechnologymanagementpointofview? ThisresearchandworkbyothersregardingOpenInnovationstillneedstobereviewedinlight ofnewpublicationsonreallifeexperiencesofadoptingthenewmodel.Whilethisresearch wasconducted,thenumberofpublicationsregardingOpenInnovationwasgrowingsteadily. Fromapersonalpointofview,Iamveryhappytohavefollowedthisprocess.Afterworkingin aresearchanddevelopmentcentrefor5years,Inowrecognisethatdailybusinesspressures cancausemetoforgetsomeofthebasicprinciplesoftheresearchmethod.Weshouldnever forgettheseruleswhiledoingresearchanddevelopment.Ihavefoundthatindustrialactivity canbenefitagreatdealfromamoreacademicpointofview.

39

David Scarlatti - X792282

References
Adams,R.,Neely,A.,Yaghi,B.&Bessant,J.2008,ProposalforMeasuresofFirmLevel InnovationPerformancein12SectorsofUKIndustry,NESTA,UK. Andrew,J.P.&Haanaes,K.2008,MeasuringInnovation2008.SquanderedOpportunities,The BostonConsultingGroup. Andrews,P.2003,Openinnovation:Usingresearchfromeverywherefornewproductand servicedevelopment,IBMBCS. Bremser,W.G.&Barsky,N.P.2004,"UtilizingthebalancedscorecardforR&Dperformance measurement",R&DManagement,vol.34,no.3,pp.229238. Brown,M.G.&Svenson,R.A.1988,"MeasuringRAndDProductivity",ResearchTechnology Management,vol.31,no.4,pp.1115. Carr,A.S.&Pearson,J.N.1999,"Strategicallymanagedbuyersupplierrelationshipsand performanceoutcomes",JournalofOperationsManagement,vol.17,no.5,pp.497519. Chesbrough,H.2004,"Managingopeninnovation",ResearchTechnologyManagement,vol. 47,no.1,pp.2326. Chesbrough,H.W.2006a,"Newpuzzlesandnewfindings"inOpeninnovation:Researchinga newparadigm,eds.H.W.Chesbrough,W.Vanhaverbeke&J.West,OxfordUniversity Press,Oxford,pp.1534. Chesbrough,H.W.2006b,"OpenInnovation:Anewparadigmforunderstandingindustrial innovation."inOpeninnovation:Researchinganewparadigm,eds.H.W.Chesbrough,W. Vanhaverbeke&J.West,OxfordUniversityPress,Oxford,pp.112. Chesbrough,H.W.2003a,"Theeraofopeninnovation",MITSLOANMANAGEMENTREVIEW, vol.44,no.3,pp.3541. Chesbrough,H.W.2003b,Openinnovation:Thenewimperativeforcreatingandprofitingfrom technology,HarvardBusinessSchoolPress,Boston,MA. Chesbrough,H.W.&Appleyard,M.M.2007,"Openinnovationandstrategy",CALIFORNIA MANAGEMENTREVIEW,vol.50,no.1,pp.57. Chiesa,V.,Frattini,F.,Lazzarotti,V.&Manzini,R.2008,"Designingaperformance measurementsystemfortheresearchactivities:Areferenceframeworkandanempirical study",J.Eng.Technol.Manag.,vol.25,no.3,pp.213226. Cumming,B.S.1998,"Innovationoverviewandfuturechallenges",EuropeanJournalof InnovationManagement,vol.1,no.1,pp.2129.

40

David Scarlatti - X792282

Damanpour,F.&Wischnevsky,J.D.2006,"Researchoninnovationinorganizations: Distinguishinginnovationgeneratingfrominnovationadoptingorganizations", J.Eng.Technol.Manag.,vol.23,no.4,pp.269291. Davila,T.,Epstein,M.J.&Shelton,R.2005,MakingInnovationWork:HowtoManageIt, MeasureIt,andProfitfromit,WhartonSchoolpublishing. Driva,H.,Pawar,K.S.&Menon,U.2000,"Measuringproductdevelopmentperformancein manufacturingorganisations",InternationalJournalofProductionEconomics,vol.63,no. 2,pp.147159. EFQM2004,EFQMFrameworkforInnovation ,EuropeanFoundationforQualityManagement. Enkel,E.&Gassmann,O.2007,DrivingOpenInnovationintheFrontEnd,EURAM(Paris, France). Godener,A.&Soderquist,K.E.2004,"Useandimpactofperformancemeasurementresultsin R&DandNPD:anexploratorystudy",R&DManagement,vol.34,no.2,pp.191219. Gwynne,P.2007,OpenInnovation'sPromiseandPerils. Hauser,J.R.1998,"Research,development,andengineeringmetrics",ManagementScience, vol.44,no.12,pp.16701689. Hollanders,H.&vanCruysen,A.2008,RethinkingtheEuropeanInnovationScoreboard: ANewMethodologyfor20082010,INNOMETRICS. Huston,L.&Sakkab,N.2006,"Connectanddevelop:InsideProcter&Gamble'snewmodelfor innovation",HARVARDBUSINESSREVIEW,vol.84,no.3,pp.58. Katila,R.2008,"MeasuringInnovationPerformance"inBusinessPerformanceMeasurement: UnifyingTheoryandIntegratingPracticeCambridgeUniversityPress,,pp.304. KerssensvanDrongelen,I.C.&Bilderbeek,J.1999,"R&Dperformancemeasurement:more thanchoosingasetofmetrics",R&DManagement,vol.29,no.1,pp.3546. KerssensvanDrongelen,I.C.&Cook,A.1997,"Designprinciplesforthedevelopmentof measurementsystemsforresearchanddevelopmentprocesses",R&DManagement,vol. 27,no.4,pp.345357. Krause,D.R.&Ellram,L.M.1997,"CriticalelementsofsupplierdevelopmentThebuyingfirm perspective",EuropeanJournalofPurchasing&SupplyManagement,vol.3,no.1,pp.21 31. Kumpe,T.&Bolwijn,P.T.1994,"TowardtheInnovativeFirmChallengeforResearchAnd DevelopmentManagement",ResearchTechnologyManagement,vol.37,no.1,pp.3844. Laursen,K.&Salter,A.2006,"Openforinnovation:Theroleofopennessinexplaining innovationperformanceamongUKmanufacturingfirms",StrategicManagementJournal; 64thAnnualMeetingoftheAcademyofManagemen.JOHNWILEY&SONSLTD,

41

David Scarlatti - X792282

CHICHESTER;THEATRIUM,SOUTHERNGATE,CHICHESTERPO198SQ,WSUSSEX, ENGLAND,pp.131. Lichtenthaler,U.2008,"OpenInnovationinPractice:AnAnalysisofStrategicApproachesto TechnologyTransactions",IEEETRANSACTIONSONENGINEERINGMANAGEMENT,vol.55, no.1,pp.148. Linder,J.C.,Jarvenpaa,S.L.&Davenport,T.H.2003,TowardanInnovationSourcingStrategy. Mankins,J.C.1995,TechnologyReadinessLevels,AdvancedConceptsOffice,OfficeofSpace AccessandTechnology,NASA. MarkRogers1998,TheDefinitionandMeasurementofInnovation,MelbourneInstitute. McGrath,M.E.&Romeri,M.N.1994,"TheR&Deffectivenessindex",TheJournalofProduct InnovationManagement,vol.11,no.3,pp.213. OECD2005,OSLOMANUAL:GUIDELINESFORCOLLECTINGANDINTERPRETINGINNOVATION DATA,OECDandEurostat,Europe. Ojanen,V.&Vuola,O.2003,CategorizingtheMeasuresandEvaluationMethodsof R&DPerformance,TelecomBusinessResearchCenterLappeenranta,Lappeenranta. Pappas,R.A.&Remer,D.S.1985,"MeasuringRAndDProductivity",ResearchManagement, vol.28,no.3,pp.1522. Pawar,K.S.&Driva,H.1999,"Performancemeasurementforproductdesignanddevelopment inamanufacturingenvironment",InternationalJournalofProductionEconomics,vol.60 1,pp.6168. Pearson,A.W.,Nixon,W.A.&KerssensvanDrongelen,I.C.2000,"R&Dasabusinesswhatare theimplicationsforperformancemeasurement?",R&DManagement,vol.30,no.4,pp. 355366. Sarkar,S.&Costa,A.I.A.2008,"Dynamicsofopeninnovationinthefoodindustry",TRENDSIN FOODSCIENCE\&TECHNOLOGY,vol.19,no.11,Sp.Iss.SI,pp.574580. Schumann,P.A.,Ransley,D.L.&Prestwood,D.C.L.1995,"MEASURINGRANDD PERFORMANCE",ResearchTechnologyManagement,vol.38,no.3,pp.4554. Sousa,M.2008,Openinnovationmodelsandtheroleofknowledgebrokers. Werner,B.M.&Souder,W.E.1997a,"MeasuringR&DperformanceStateoftheart", ResearchTechnologyManagement,vol.40,no.2,pp.3442. Werner,B.M.&Souder,W.E.1997b,"MeasuringR&DperformanceUSandGerman practices",ResearchTechnologyManagement,vol.40,no.3,pp.2832.

42