You are on page 1of 39

A Bayesian Model for Sales Forecasting at Sun Microsystems

Response to comments of the editor and referees begins after page 33.

Abstract An accurate short-term forecast of product sales is vital for the smooth operation of modern supply chains, especially where the manufacture of complex products is outsourced internationally. As a vendor of enterprise computing products whose business model has long emphasized extensive outsourcing, Sun Microsystems Inc. has a keen interest in the accuracy of its product sales forecasts. Historically, the company has relied on a judgment-based forecasting process, involving its direct sales force, marketing management, channel partners, and so on. Management recognized, however, the need to address the many heuristic and organizational distortions to which judgment-based forecasting procedures are prey. Simply replacing the judgmental forecasts by statistical methods with no judgmental input was unrealistic; short product life cycles and volatile demand demonstrably confounded purely statistical approaches. This article documents a forecasting system developed in Suns research laboratory and currently deployed by the company that uses Bayesian methods to combine both judgmental and statistical information. We discuss the development and architecture of the system, including steps that were taken to ease its incorporation into the companys existing forecasting and planning processes. We also present an evaluation of the systems forecasting performance, as well as possible directions for future development. Key words: Sales forecast, Bayesian statistics, prior elicitation, Markov chain Monte Carlo

Introduction
Sun Microsystems
Founded in 1982, and currently headquartered in Santa Clara, California, Sun Microsystems Inc. is a premier supplier of enterprise computing products, employing some 33,500 people worldwide. In the scal year ending in June 2008, its revenues amounted to approximately $14 billion. Suns sales offerings range from microprocessors to IT services,

A Bayesian Model for Sales Forecasting at Sun Microsystems

but the bulk of its income is derived from sales of computer servers and storage systems, which range in price from below $1,000 to more than $10 million each. 1 Throughout Suns history, the companys business model has been distinguished by a very high degree of outsourcing, with a heavy reliance on contract manufacturers and resellers. Baldwin and Clark (1997) relate that on the whole, this strategy has proven very advantageous to Sun, and indeed the past few years have witnessed the adoption of similar approaches by the companys competitors (Davis 2007, Dean 2007). It does, however, present signicant challenges for supply chain managementchallenges that are compounded by the character of computer products themselves, which have lifecycles measured in months or even weeks, and parts that can depreciate catastrophically in inventory. Accordingly, Sun has devoted considerable effort to the engineering of its supply chain, as exemplied by its recently-implemented One Touch program, which involves extensive use of drop shipment and cross-docking (Whiting 2006). Nonetheless, as articles such as (de Kok, Janssen, van Doremalen, van Wachem, Clerkx, and Peters 2005) recount, the effectiveness of even the most agile and well-coordinated supply chain rests on accurate forecasts of demandin fact in a survey documented by Wisner and Stanley (1994), managers of lean supply chains like Suns placed a greater reliance on demand forecasts than did those operating more traditional supply chains.

Forecasting at Sun
Suns supply chain forecasting process, in common with many of those in industry, revolves around predictions for product sales over a quarterly time horizonsgenerally the current and next two scal quarters, in Suns case. These numbers serve as input to planning and procurement processes. The quarterly sales of a small selection of Suns server and storage products are displayed in gure 1. 2 Before the rollout of the program described in this paper, Suns sales forecasting rested heavily on the production of so-called judgmental forecasts (Lawrence, OConnor, and Edmundson 2000), assembled by polling the direct sales force and channel representatives and combining the results in committee with input from marketing, sales and senior executives. Sun is hardly alone in such a practice; judgment has long formed the basis of commercial sales forecasting. It gures prominently, for example, in an instructional article written in the 1950s by Boulden (1958), and surveys published in recent decades
1 2

All information current as of February 2009c.f. (SMCI). In the interests of condentiality, sales data appearing in the gure are mildly disguised. A label such as 2Q3 on the horizontal axis of a graph in the gure denotes the third nancial quarter of the second year of the products life.

100

150

200

250

300

500

1000

1500

2000

2500

3000

10 0 0

20

30

40

50

60

50

0 Q3 Q4 2Q1 Q2 Q3 Q4 3Q1 Q2 Q3 Q4 4Q1 Q2 Q3 Q4 5Q1 Q2 Q3 Q4 6Q1 Q2 Q3 Q4 2Q1 Q2 Q3 Q4 3Q1 Q2 Q3 Q4 4Q1 Q2 Q3 Q4 5Q1 Q2 Q3 Q4 6Q1 Q3 Q4 2Q1 Q2 Q3 Q4 3Q1 Q2 Q3 Q4 4Q1 Q2 Q3 Q4 5Q1 Q2 Q3 Q4 6Q1

A Bayesian Model for Sales Forecasting at Sun Microsystems

Figure 1. Sample product demands (units)

3
10000 12000 2000 0 4000 6000 8000 2000 4000 6000 8000 0 3Q1 Q2 Q3 Q4 4Q1 Q2 Q3 Q4 5Q1 Q2 Q3 Q4 6Q1 Q4 2Q1 Q2 Q3 Q4 3Q1 Q2 Q3 Q4 4Q1 Q2 Q3 Q4 5Q1 Q2 Q3 Q4 6Q1

1000

1500

2000

2500

500

0 1Q1 Q3 2Q1 Q3 3Q1 Q3 4Q1 Q3 5Q1 Q3 6Q1

A Bayesian Model for Sales Forecasting at Sun Microsystems

including (Dalrymple 1987, Wisner and Stanley 1994, Sanders and Manrodt 1994, Klassen and Flores 2001, Sanders and Manrodt 2003, Fildes, Goodwin, Lawrence, and Nikolopoulos 2009)attest to the continuing prominence of judgment in the forecasting processes of the vast majority of companies. According to Sanders and Manrodt (2003), for example, of 240 US corporations surveyed, almost 90% based their sales forecasts exclusively on judgment, and of those who used quantitative techniques, some 60% routinely made judgment-based adjustments to the forecasts produced. Certainly, some of the reasons for the continued widespread use of judgment in industry have little to do with improved forecast accuracy: Sanders and Manrodt (1994) cite a lack of acquaintanceship with mathematical techniques and a desire to conate forecasting with organizational goal-setting. But Lawrence, Goodwin, OConnor, and nkal (2006) note that of late, the research community has come to recognize that judgment has an important and technically legitimate role to play in forecasting. This is especially true as Sanders and Ritzman (2001) observein highly dynamic environments such as Suns, where structural changes may undermine a statistical forecasting model. Moon, Mentzer, and Smith (2003) insist that both judgment-based and statistical elements are indispensable to the sales forecasting process in particulara point illustrated vividly in the episode related by Worthen (2003), when Nike Corporation was compelled to take massive writeoffs against inventory, thanks to an exclusive reliance on statistical forecasts. Our experience at Sun also led us to conclude that judgmental input was vital, in that exclusively statistical methods that dispensed with judgmental inputs were found to be largely ineffective at forecasting company product sales. There is, however, a wealth of literature documenting the biases and errors associated with judgment-based forecastingc.f. (McGlothlin 1956, Tversky and Kahneman 1974, Wright and Ayton 1986) or (Bolger and Harvey 1998), for example. Mentzer and Bienstock (1998) and Tyebjee (1987) point out that in addition to these problems, judgmental sales forecasts may be distorted by other factors, such as organizational pressures. That commercial sales forecasts are indeed subject to distortion is conrmed by recent survey data from Fildes et al. (2009). Mindful of this, supply chain managers in Suns Worldwide Operations business unit looked for statistical techniques that could be used to enhancebut not supplantthe companys judgmental forecasts, and engaged the research unit of Sun (Sun Labs) to this end. The ensuing joint project culminated in the Sun Labs Forecasting System, the suite of software described in this paper.

A Bayesian Model for Sales Forecasting at Sun Microsystems

Related Work
Many researchers and practitioners have sought to combine judgmental and statistical forecasting techniques. Broadly speaking, such efforts may be categorized under the headings: (1) adjustment, (2) combination, (3) correction, (4) judgmental bootstrapping and (5) Bayesian methods. Work in each of these categories is reviewed briey below.

Adjustment
Articles such as (Sanders and Manrodt 1994, Fildes and Goodwin 2007, Franses and Legerstee 2009) and (Fildes et al. 2009) evidence the widespread manual adjustment of statistical forecasts ex post factofrequently, by eyeball analysis of forecast graphs, as described in (Bunn and Wright 1991) and (Webby and OConnor 1996). Without doubt, therefore, this approach has intuitive appeal. However, Armstrong and Collopy (1998) point out that if adjustments are carried out in an unstructured or undisciplined fashion, this approach risks simply reintroducing the distortions of judgmental forecasting. In fact the accounts of Fildes and Goodwin, Franses and Legerstee, and Fildes et al. (ibid.) suggest that more often than not in practice, adjustments reduce the accuracy of a statistical forecast. Bunn and Wright (1991) also aver that without sufcient ancillary detail, it can be difcult to see how an apparently arbitrary adjustment was justied, leading to possible contention within the organization employing the forecast.

Combination
An alternative approachdiscussed by Blattberg and Hoch (1990), Webby and OConnor (1996), Lawrence et al. (2006) and Franses (2008)is to originate statistical and judgmental forecasts independently and use a mechanical procedure to combine them. This builds upon work on forecast combination which began with (Bates and Granger 1969), and which is surveyed by Clemen (1989) and Timmermann (2006). Combination is generally effected by taking a weighted sum of the values of the constituent forecasts in each period. A number of methods have been proposed to estimate the combination weights: Granger and Ramanathan (1984) suggest least squares regression, for example, whereas Bates and Granger (1969) compute weights based on the forecast errors of the constituent forecasts, and Diebold and Pauly (1990) use Bayesian shrinkage regression. In practice, a simple unweighted average of the component forecastsas expounded by Blattberg and Hoch (1990) and Armstrong (2001b) amongst othershas been found to perform consistently well.

A Bayesian Model for Sales Forecasting at Sun Microsystems

Timmermann (2006) lists a number of factors that recommend forecast combination: It synthesizes the information sets used to produce the component forecasts, it dilutes bias in individual forecasts and it increases robustness with respect to model misspecication and structural breaks. Set against this, Timmermann also notes that estimated combination weights can be very unstable in practicewhich helps explain the remarkably good relative performance of the simple average. In principle, avers Diebold (1989), where the information sets underlying the component forecasts are available, 3 it is always preferable to construct a single encompassing forecast model, rather than simply to combine the forecasts themselves.

Correction
Rather than combining a statistical and a judgmental forecast, some authorsincluding Theil (1971), Ahlburg (1984), Moriarty (1985), Elgers, May, and Murray (1995) and Goodwin (1996, 2000)have explored statistical methods for correcting judgmental forecasts in the light of observed outcomes. Generally, such methods are based on Theils (ibid.) optimal linear correction, which involves regressing observed outcomes on forecasts, using the estimated regression coefcients to produce a revised prediction from new forecasts; Goodwin (1997) accommodates time-varying coefcients using a weighted regression. A technique related to statistical correction originates with Lindley (1983), and is applied explicitly to time series forecasting by West and Harrison (1997, sec. 16.3.2). Lindleys methodology is an example the so-called supra-Bayesian approach to the reconciliation of expert opinion developed by Pankoff and Roberts (1968) and Morris (1974, 1983). Here, the value of a judgmental forecast is construed as a linear function of the actual value, and Bayesian updating is used to produce a revised forecast.

Judgmental bootstrapping
Researchersparticularly in psychologyhave long sought to capture judgmental reasoning in a tractable mathematical form. Efforts centered on linear models date back at least to (Hughes 1917); important contributions were made by Meehl (1957), Hammond, Hursch, and Todd (1964), Hursch, Hammond, and Hursch (1964) and Tucker (1964). Surveying this work, Dawes (1971) coined the term bootstrapping to describe the process by which an experts judgment is modeled by a linear expression involving the environmental factors (usually referred to as cues) that enter into the experts consideration. Recent au3

Such a set might comprise a judgmental bootstrap model of the forecaster in the case of a judgmentbased forecastsee later.

A Bayesian Model for Sales Forecasting at Sun Microsystems

thors exploring the application of such a process to judgment-based forecasting, including as Armstrong (2001a), OConnor, Remus, and Lim (2005) and Batchelor and Kwan (2007) use the qualied term judgmental bootstrapping to avoid confusion with the (quite distinct) statistical bootstrap technique developed in the late 1970s by Efron (1979). 4 Armstrong (ibid.) also applies the term to models that go beyond simple linear combination of cues. Evidence for the efcacy of judgmental bootstrapping in forecasting is mixed: Ashton, Ashton, and Davis (1994) nd a bootstrap model out-performed by a statistical forecasting model, and stebro and Elhedhli (2006) and Batchelor and Kwan (2007) cannot conclude that a bootstrap model forecasts more accurately than the experts it represents. Lawrence and OConnor (1996) and Fildes et al. (2009) assert that bootstrapping is less effective in the context of time series extrapolation, where cue information tends to be autocorrelated; the error bootstrapping technique developed in response by Fildes (1991) seeks to model the errors in a judgmental forecast, much like the forecast correction approach described above.

Bayesian methods
The Bayesian paradigm of statistical inference (see e.g. (Gelman, Carlin, Stern, and Rubin 2003) for an overview), with its incorporation of subjective information in the form of prior distributions, seems a natural means of combining judgmental and statistical elements in forecasting. Indeed, a substantial number of Bayesian models have been devised for product demand, both at the inventory level (Silver 1965, Hill 1997, Dolgui and Pashkevich 2008a,b), and in the aggregate (Lenk and Rao 1990, Montgomery 1997, Moe and Fader 2002, Neelamegham and Chintagunta 1999, van Heerde, Mela, and Manchanda 2004, Neelamegham and Chintagunta 2004, Lee, Lee, and Kim 2008)amongst many others. Sales applications also feature prominently in the seminal work of Pole, West, and Harrison (1994) and West and Harrison (1997) on Bayesian forecasting. Bunn and Wright (1991) note that despite the apparent attractions of Bayesian modeling, and a research literature that dates back several decades, there is a dearth of Bayesian models with judgmental priors used routinely in forecast applications. Bunn and Wright suggest that the chief impediment is the expense (in time and effort) of repeatedly eliciting subjective priors of sufcient qualitya point reinforced by researchers such as Wright and Ayton (1986), who highlight the difculties involved in obtaining reliable judgmental priors. In fact many of the models cited in the previous paragraph circumvent the need
4

Alternatives terms, such as the statistical approach, actuarial modeling (both of which appear in the clinical literature) or the (mildly grandiloquent) paramorphic representation of Hoffman (1960) would doubtless invite even greater confusion.

A Bayesian Model for Sales Forecasting at Sun Microsystems

for informative priors by relying on a hierarchical (Gelman and Hill 2006) structure to pool information from analogous historical situations in order to produce forecasts.

System Overview
As the title of the paper suggests, we began the Sun Labs Forecasting System with a Bayesian model of product sales. Mindful of the need to incorporate judgmental information, we built a model (described in detail in the next section) intended to formalize the judgmental framework used by the companys forecasters. The model included parameters specifying the level of sales achieved by a product in its mature phase, the time required to achieve maturity after launch, and so on. Our intention was to elicit priors for these parameters from the forecasters, using Bayesian updating to reconcile these priors with actual sales and to extrapolate future sales. 5 This system relied on the frequent elicitation of priors from the forecasting staffideally, each time they came into possession of new information. We tried to minimize the work required of the forecasters to produce the priors, constructing the model so that its parameters were easy to interpret, and implementing a number of graphical tools to assist with estimation. Despite this, we fell afoul of a Catch-22 (Heller 1961) that afrmed the conjecture of Bunn and Wright highlighted above: Our colleagues were unwilling to invest signicant time and effort until the efcacy of the Forecasting System had been proven, but without good priors, the Systems performance was bound to be subpar. Faced with this problem, we attempted rst to deploy a hierarchical Bayesian model that drew priors from historical records, 6 combining the statistical forecasts produced by the model with the judgmental forecasts prepared by company forecasters. The heterogeneity of Suns product line and rapid changes in its product markets made the performance of the hierarchical model disappointing, however, and combinations of the models forecasts with judgmental ones performed inconsistently. The solution eventually adopted for use in the Forecasting System resembles the converse of the judgmental bootstrapping process described in the previous section. For consider that the companys forecasters routinely produce judgmental sales forecasts, and that the model used in the Forecasting System embodies the way in which those forecasts are assembled from beliefs about sales at maturity, time to reach maturity and so on. Therefore tting the model to the judgment-based sales forecasts produces estimates of the forecasters that underlie themcues in the parlance of judgmental bootstrapping, or parameter
5 6

An early version of the Forecasting System based on this idea is described in (Yelland 2004). Again, see Yelland (ibid.) for details.

A Bayesian Model for Sales Forecasting at Sun Microsystems

Processing step

Data

Prior deduction

Sales force forecasts

Initial parameter estimates

Forecasting model

Parameter correction

Historical records

Parameter priors

Bayesian updating

Actual sales

Revised forecast

Figure 2. System schematic

priors, to a Bayesian. In the Forecasting System, these prior estimates are then corrected using a device similar to the linear correction used by Theil and his successors: Using historical records of forecasts and actual sales, parameter estimates derived from actual sales are regressed on estimates for the same parameters derived from forecasts of those sales. The coefcients of the regression are used to produce adjusted prior estimates from the new judgmental forecast. Finally, the adjusted prior estimates are revised by Bayesian updating in the light of actual sales to produce the Systems forecast. A functional schematic of the Sun Labs Forecasting System is given in gure 2. Details of the forecasting model and a full description of prior estimation and correction are provided in the next section.

A Bayesian Model for Sales Forecasting at Sun Microsystems

System Details
Model
Dynamic Linear Model From its inception, the Sun Labs Forecasting System centered on the use of dynamic linear models (DLMs), as described by West and Harrison (1997) (hereinafter abbreviated W&H). Bayesian formulations of the structural time series models also discussed from a classical statistical viewpoint by Harvey (1989), 7 DLMs have been employed for sales forecasting by a number of authors, including W&H themselves, van Heerde et al. (2004) and Neelamegham and Chintagunta (2004). For reference, the general formulation of a DLM representing a scalar time series (W&H, p. 102) is presented below: Observation equation: Evolution equation: Initial information: yt = Ft t + t , t = Gt t 0
1

t t

N(0, Vt ), N(0, Wt ),

(1a) (1b) (1c)

+ t ,

N(m0 , C0 ).

Here, observation equation (1a) takes the form of a regression of the time series value, yt (which in the case of the Forecasting System is quarterly product sales), on (the transpose of) a design vector, Ft , of known covariates. The coefcients in this regression, t , may vary over time; they comprise the state vector of the model. Changes in the state vector itself are described by the evolution equation (1b): In each time period, the state is subject to a linear transformation and randomly perturbed by the addition of the noise component t . 8 The specication is completed by the provision of a multivariate normal prior for the state vector in the period preceding that of the rst observation. The DLM framework is extraordinarily exible: Harvey, West and Harrison, Durbin and Koopman (ibid.) demonstrate the encoding of trend, seasonal and dynamic regression componentssingly or in combinationas DLMs, as well as providing DLM versions of the familiar ARIMA models (Box, Jenkins, and Reinsel 1994). As Harvey points out, structural time series models are generally more intuitively accessible than alternatives such as the ARIMA modelsan important consideration in view of our original intention to parallel the reasoning of Company forecasters. Unfortunately, we struggled to represent the important characteristics of Suns product demands directly in DLM form; in particular, we found DLMs ill-suited to the distinctive life cycle curve illustrated in gure 1, and the
7

The more recent account of Durbin and Koopman (2001) also concentrates on a classical treatment of structural models, with some attention is given to the Bayesian perspective. 8 Noise terms and are assumed to be uncorrelated. t t

10

A Bayesian Model for Sales Forecasting at Sun Microsystems

multiplicative seasonality evinced by many products (the DLM formulation provides only for additive seasonality). Obvious devices, such as log transformations, produced forecasts that were far from satisfactory, and elaborations of the basic DLM such as mixture or linearized models (W&H, chp. 12, 13, resp.) quickly became unwieldy. 9 The model currently used in the Sun Labs Forecasting System is actually derived from the most basic form of DLM, called the rst-order polynomial model by West and Harrison (p. 32), and the random walk plus noise model by Harvey (ibid. p. 19). 10 The model is dened as follows: yt = S(t)L(t)t + t ,
1

t
t

N(0, V ), N(0, W ),

(2a) (2b) (2c)

t = t 0

t,

N(m0 , C0 ).

Here the state vector consists of a single scalar quantity, t usually referred to as the level of the processwhich evolves according to the random walk set out in equation (2b). In a further simplication, the variances of the noise terms in equations (2a) and (2b) are constant over time. The only departure from the DLM framework is in the (scalar) quantity corresponding to the design vector Ft in equation (1a); instead of the known (if possibly time-varying) values assumed in the usual DLM, the design vector in this model is determined by the product of two stochastic quantities, S(t) and L(t), which represent respectively the effects of seasonality and life cycle stage in the period t. These are described in further detail below. Seasonality Seasonality is described straightforwardly using dummy variables (see Ghysels, Osborn, and Rodrigues (2006), for example). Parameters Q1t , Q2t and Q3t give the size of the multiplicative changes in the respective quarters of the nancial year, relative to sales in the fourth quarter of the year: 1 if t is the jth quarter of a year,

S(t) = 1 + Q1t 1 + Q2t 2 + Q3t 3 ,


9

where Q jt =

See (Yelland and Lee 2003) for a description of some of our early investigations. Incidentally, West and Harrison (loc. cit.) indicate that the rst-order polynomial model is widely used for forecasting in inventory and operations management applications like that of the Forecasting System.
10

0 otherwise.

11

A Bayesian Model for Sales Forecasting at Sun Microsystems

Life cycle phase Ramp up Mature Ramp down

Span 0t<
1 1 1

Behavior Geometric growth from a value when t = 0 to 1 as t 1 .


1

t< +
2

Constant value 1. Geometric decay from 1 when t = t 1 + 2 + 3.


1

<t

to as

Table 1. Interpretation of the life cycle coefcient

Life cycle Mirroring the conceptual framework generally used by Company forecasters, the model divides a products life cycle into three phases: (1) a ramp up phase, in which product sales increase consistently period over period, (2) a mature phase, in which product sales are roughly constant and (3) a ramp down phase, in which product sales dwindle again. In the model, lengths of these phases are delimited by the three positive random variables 1 , 2 and 3 , respectively. The value of the life cycle coefcient varies between 0 and 1, the latter corresponding to sales in maturity. The ramp up and down phases are characterized by geometric growth and decay, dened by the positive constants and . 11 For the sake of identication, is set (arbitrarily) to 0.05, so that sales dwindle to 5% of mature sales during the ramp down. In symbols, the life cycle coefcient is dened as follows: ( 1 (t
1 2 )/ 3 1

t)/

L(t) =

if

if 0 t <
1 1

1, 1

if

t<
2

2,

< t.

Figure 3 illustrates the general shape of the life cycle curve in the model, and Table 1 summarizes its interpretation. Formal modeling of product life cycles has a long history, dating back (at least) to the work of Rogers (2003, rst published in 1962) and Bass (1969); see (Mahajan, Muller, and
It is generally the case that and are less than 1. Values of and in excess of 1 are permissible, however, though in such instances the value of L(t) may exceed 1 in some periods, and the epithets ramp up and ramp down are probably inappropriate.
11

12

A Bayesian Model for Sales Forecasting at Sun Microsystems

L(t) 0 0 1 t
Figure 3. Life cycle coefcient over time

Ramp up

Maturity

Ramp down

1 + 2

1 + 2 + 3

Wind 2000a) for a compendium of work in the area. The vast majority of models of this sort are structural in nature; product demand emerges endogenously as a result of consumer responses to the experience of other consumers and to the marketing actions of the producing rm. By contrast, the life cycle representation used in the Sun Labs Forecasting System is a purely empirical account of what Mahajan, Muller, and Wind (2000b) refer to as strategy-driven diffusionthe life cycles of the products modeled by the Forecasting System do not result from consumer actions, but from strategic and operational decisions taken by Sun, its suppliers and its distributors. In particular, the ramp up phase in the model reects product quality assurance processes and supply chain inertia, and the ramp down phase is initiated not by market saturation, as in the Bass model and its successors, but by a decision on the part of the Company to withdraw the product in the face of technological development. We should note that of late, mainstream work in the area of life cycle modeling has employed piecewise representations like that used in the Forecasting Systemsee (Niu 2006, Huang and Tzeng 2008) and (Sood, James, and Tellis 2009).

13

A Bayesian Model for Sales Forecasting at Sun Microsystems

Prior Estimation
Regarding prior estimation, assume that a judgment-based forecast has been produced at the end of quarter t a products life cycle for sales of the product in quarters t + 1, . . . , t + h. Let {, , 1 , 2 , 3 , 1 , 2 , 3 , 0 } 12 denote a model parameter for which a prior is required. The prior estimation procedure is as follows: (1) Prepend actual sales (y1 , . . . , yt ) to the forecast, producing a combined series y of length t + h. (2) Using a non-linear least-squares regression, t to the series y the model (listing explicitly all the parameters that enter into S(t) and L(t) ): Let + be the estimated value of in the above regression. The model in equation (3) is a restriction of the full model in (2), without the level evolution specied in equation (2b). In practice, we found the restriction necessary to ensure reliable tting of judgment-based forecastsespecially early in a products life, where few actual demand points are available to supplement the forecast demands. The quality of the resulting priors was not substantially compromised. (3) Collect together a set y1 , . . . , y N of combined forecasts and actuals for a set of recentlyobsolete products similar to the current one, 13 where the forecasts were made in the corresponding quarter t of the products lives. Since these products are obsolete, their entire sales histories y1 , . . . , yN are also available. (4) Fit the model in equation (3) (using non-linear least squares again) to the series collec tions y1 , . . . , y N and y1 , . . . , yN . For i 1, . . . , N, let i be the estimate of the parameter of interest that results from tting to yi , with i the estimate from the corresponding series yi . Thus i represents the parameter value that characterizes the actual trajec tory of the ith products sales, whereas i is the value suggested by the forecast in the tth quarter of its life cycle. (5) Fit a linear regression i = a + bi + i to the estimates from step 4, 14 yielding coef cient estimates a and b and an estimate of the residual standard deviation .
The prior for 0 actually constitutes the values m0 and C0 in (2c). Suns products are generally categorized by market, such as enterprise storage, low-end server, etc., and this categorization is used to identify like products. 14 For parameters restricted to the positive half-line, the linear regression is t on a log scale to ensure coherent predicted parameter values. The resulting estimates are actually used to derive truncated (not log-) normal distributions on the original scale (by matching rst and second moments), as we have found that truncated normal priors make for far better forecast performance than log-normal ones.
13 12

yt = S(t; 1 , 2 , 3 )L(t; ,

1 , 2 , 3 ) 0

+ t .

(3)

14

A Bayesian Model for Sales Forecasting at Sun Microsystems

(6) The prior for derived from the value + estimated for the new forecast in step 2 is given by standard results for classical normal regression (e.g. Gelman and Hill 2006, p. 48)i.e., a normal distribution with mean a + bi and standard deviation: 1 2 1 (+ mean i )2 1+ + . N iN 1 (i mean i )2 = (7) The set of priors required for the model is completed by priors for the observation and evolution noise variances in equation (2a) and equation (2b), respectively. Following a rule of thumb we have found to work well in practice, these are set to relatively diffuse scaled inverse 2 distributions (see Gelman et al. 2003, p. 480) Inv2 (4, 0.2 m0 ) and Inv2 (4, 0.1 m0 ), resp., where m0 is the prior mean for 0 .

Bayesian Updating
The Bayesian updating step in gure 2 (which revises the forecast priors in light of observed sales to date) is carried out using a Gibbs sampling routine, schematic descriptions of which now abound in the literaturesee (Gilks, Richardson, and Spiegelhalter 1996, chp. 1), for example. The individual steps of this particular sampler are described in the Appendix. Many of the steps rely on standard results concerning conjugate updating in Bayesian analysis, which may be found in reference texts such as (Gelman et al. 2003) or (Bernardo and Smith 1994). Where such closed-form updates are not available, we resort to Metropolis-Hastings sampling (also discussed by Gilks et al.); proposals are generated using Geweke and Tanizakis (2003)s Taylored chain procedure, details of which are also provided in the Appendix. The Gibbs sampler is run for 10,000 iterations, with the rst 3,000 samples discarded; convergence is veried using Gewekes (1992) single chain diagnostic (in additional to occasional sample-path plots). In fact execution of the sampler for 10,000 iterations is quite conservativewe are almost invariably able to establish convergence by about 4,000 iterations.

Forecast Performance
To demonstrate the efcacy of the Sun Labs Forecasting System, we compare its forecast performance with that of alternative forecasting methods, using the demand histories of a sample of Suns products.

15

A Bayesian Model for Sales Forecasting at Sun Microsystems

Setup
The test is conducted using a collection of 32 products, representing a cross-section of Suns recent product lines. Of these products, 27 are randomly selected for calibration, and holdout forecasting is performed on the remaining 5. 15 Point forecasts at horizons of 1, 2 and 3 quarters are prepared for each quarter of the holdout products demand histories, yielding some 77 forecast values in total for each method at each horizon. Four forecast methods are compared in the test: Sys This forecasting method relies on posterior predictive distributions calculated by the Forecasting System in the Bayesian updating step of gure 2. 16 With calibration data (i.e. demand histories and judgmental forecasts) available at time t collectively denoted Dt , the posterior predictive distribution for the forecast value yt+h at horizon h is given by p(yt+h | Dt ), which is itself calculated by marginalizing over the posterior density for the model parameters (here represented by ): p ( y t + h | Dt ) =

The integral expression in equation (5) is readily approximated using the Gibbs sampler specied in the Appendix, as described by Albert (2008), for example. For the point forecast yt+h|t (the forecast for yt+h made in period t), we use the mean of the posterior predictive distribution of yt+h . DLM This method is a straightforward implementation of a univariate dynamic linear model, as set out in (W&H, chp. 58). Following West and Harrisons prescription, the model comprises a second-order polynomial- (local level with trend) and four-period seasonal component. As Gardner and McKenzie (1989) suggest, to improve forecasts at over longer horizons, the trend in the polynomial component is damped, so that it decays over time; the design and evolution matrices necessary to accomplish this are adapted from analogous structures described for the closely-related single source of error (SSOE) structural time series models in (Hyndman, Koehler, Ord, and Snyder 2008, p. 48).

p ( y t + h | ) p ( | Dt ) d p ( y t + h | ) p ( Dt | ) p ( ) d .

(4) (5)

15

The relatively large number of products in the calibration collection is necessary to ensure reliable operation of forecast methods Sys, DLM and CJudg, which require a representative set of products from each of the companys product groups in order to correct priors and forecasts. 16 This is the standard approach to forecasting with Bayesian modelssee (Neelamegham and Chintagunta 1999, 2004), for instance.

16

A Bayesian Model for Sales Forecasting at Sun Microsystems

Horizon 1 Sys DLM Judg CJudg 0.53 (1) 0.73 (3) 0.73 (4) 0.66 (2) 2 0.84 (1) 1.13 (4) 1.10 (3) 1.00 (2) 3 0.80 (1) 0.97 (4) 0.96 (3) 0.94 (2)

Table 2. Mean absolute scaled error by forecast horizon

Formulae for updating and forecasting with this DLM are standard: We use the unknown, constant variance results summarized on (W&H, p. 111), which incorporate the estimation of a time-invariant observation noise variance in equation (1a). Multiple discount factors are used to specify the evolution noise component in equation (1b)c.f. op. cit. pp. 196198 for details. Discount and damping factors are derived using a grid search based on forecast accuracy for a sample of the calibration set, and initial conditions are set from the same corrected priors used for the Sys method, using a procedure similar to that described by Hyndman et al. (ibid., sec. 2.6.1). All components in the state are uncorrelated in the prior, so that the matrix C0 in (1c) is diagonal. Judg This method simply reproduces the companys judgmental forecastthe forecasts from which the priors for methods Sys and DLM are derived.

CJudg For this method, the companys judgmental forecast is corrected using Theils (1971) optimal linear correction. Recall from the above that this involves regressing actual sales on predicted sales for products in the calibration set, using the estimated coefcients to compute revised prediction from forecasts in the holdout setsee Theil (1971, p. 34), or more recently, Goodwin (2000), for a detailed discussion. Separate regressions are calculated for each forecast horizon and each of the product categories identied for prior correction.

Test Results
Table 2 summarizes the performance of the candidate methods in the forecasting test. We have used the mean absolute scaled error (MASE) of Hyndman and Koehler (2006) as a performance metric; the Appendix denes the MASE, and sets out the considerations that lead to its adoption. MASEs are given for each method at each forecast horizona smaller

17

A Bayesian Model for Sales Forecasting at Sun Microsystems

entry indicates better performance. The rank (smallest to largest) of each method at each horizon is recorded in parentheses after the corresponding entry. As the table shows, the Forecast System consistently exhibits superior overall performance, with the purely statistical DLM method generally turning in the worst performance, and the corrected judgmental forecast also consistently outperforming its uncorrected counterpart. The MASE is dened such that a value less than 1.00 indicates a performance better (on average) than that of the nave random-walk forecast, which simply uses the most recent observation to produce a forecast. Thus according to the table, only the Sys method consistently improves on the benchmark. For a more detailed perspective, Figure 4 is a box-and-whisker plot (Tukey 1977) of the distribution of absolute scaled errors of the forecast methods at each forecast horizon. Though the distributions of the ASEs are clearly skewed, the plot broadly conrms the superiority of the Sys method established by the MASEs.

Implementation
A depiction of the current implementation of the Forecast System is given in gure 5. Remarkably, we found it possible to implement the System almost entirely using open source and/or freely-available software. The core of the System is written in the statistical programming language R (Venables and Smith 2002). It resides on a dedicated server and carries out the following operations:

Calculating priors from judgmental forecasts, Updating the priors given actual product sales, and Forming sales predictions from the updated priors.
This portion of the System is run once a week to incorporate the latest sales gures. Input to the R code comes from the companys data warehouses by way of Java interface modules, and output is stored in a dedicated MySQL database (Williams and Lane 2004) on the same server. Access to the forecast data is Web-based, relying on the Adobe Flash/JavaScript/PHP/Apache combination common in modern Web applications (Wiedermann 2006, Schafer 2005). Figure 6 provides a (mildly disguised) snapshot of the forecast display for a typical product: On the left of the page, the set of forecast products is arrayed by category, while at the top of the page is a selection of the base week in which the forecast was produced, allowing the user to review historical records as desired. The animated chart (provided by amCharts: http://www.amcharts.com/) provides an interactive pictorial representation

18

Horizon: 1

Horizon: 2

Horizon: 3

2 2

A Bayesian Model for Sales Forecasting at Sun Microsystems

19
1 1 0 Sys DLM Judg CJudg Sys DLM Judg CJudg 0 Sys DLM Judg CJudg

Figure 4. Distribution of ASEs by model and forecast horizon

A Bayesian Model for Sales Forecasting at Sun Microsystems

!"#$%"&'$()#*

!"#$%&'(%&)( +"'",-"(./0$1. 2$#,3"41, 50(.&"1')#*, 261'.7 /)01()2' 3-.)"%(#( /)01()2' 3-.)"%(#( 50(.&"1', 7..')#* ,-.)"%(#( 2$99:6,&/")# 7"#"*.7.#'

8.4,)#'.(%"&.

*%&)(+ ,-.)"%(#(

*%&)(+ ,-.)"%(#(

4&%5(

Figure 5. How the Forecast System supports Suns supply chain management process

of forecasts and actual sales, which are also presented in tabular form at the bottom of the page. The system also provides forecasts in the form of PDF reports, produced on the A server using L TEX (Lamport 1994).

Concluding Remarks
The Sun Labs Forecasting System has been in use at Sun for over a year at the time of writing. The System provides an effective combination of judgmental and statistical forecasting information that consistently improves upon the forecast accuracy of both its constituents. The system operates almost entirely unattended, extracting the requisite information from the Companys data warehouses and delivering forecasts in a regular and expeditious manner through a simple, intuitive user interface. Furthermore, with Bayesian priors derived automatically from the output of the existing sales forecasting process, the Forecasting System imposes no additional procedural burdens on its users. Taken to-

20

A Bayesian Model for Sales Forecasting at Sun Microsystems

Figure 6. Snapshot of Forecast System user interface

gether, these factors no doubt account for the warm reception the System has received from the Companys supply chain managers. Development of the Forecasting System continues. Most immediately, we are investigating means by which forecasters can inform the System of upcoming events such as very large orders or promotions that are likely signicantly to perturb the sales of one or more products. The technical mechanisms by which such interventions may be achieved are fairly straightforward to implementsee (West and Harrison 1997, chp. 11) for a discussion. More delicate are the organizational issues involved, for articles such as Sanders and Ritzman (1991), Fildes et al. (2009) and Sanders (2009) testify that although management intervention generally increases forecast accuracy when managers are possessed of spe-

21

A Bayesian Model for Sales Forecasting at Sun Microsystems

cic and accurate information about special events, forecasts are usually degraded if managers are allowed to intervene willy-nilly. Our current intentions are to provide some form of decision support systemalong the lines of that described by Lee, Goodwin, Fildes, Nikolopoulos, and Lawrence (2007)to help forecasters make only considered interventions. The Forecasting System makes no attempt to provide normative informationsuch as inventory levels or purchase quantities (Zipkin 2000)based on the forecasts it produces. This is quite deliberate, since the complexity of Suns supply chain and the sourcing arrangements upon which it rests makes the derivation of policy recommendations from demand forecasts formidably complex; inventory-, depreciation- and expediting costs, costs of lost sales, contractual supply arrangements, sales of complementary products and many other factors vary from product to product, between different congurations of the same product, and across quarters. The Forecasting System system therefore passes off responsibility for operating decisions to the users of its forecasts. A somewhat vexatious upshot of this is that it is very difcult to place a rm dollar amount on the benets that accrue from superior accuracy of the Systems forecasts. Certainly, the suggestions of Mentzer (1999) and Kahn (2003) provide means of reaching plausible rough estimates, but a truly denitive gure would require a thorough-going appraisal of the companys forecasting and supply arrangements along the lines proposed by Moon et al. (2003). Policy prescriptions are easier to make at a less aggregate level of demand: Purchasing and inventory management recommendations are much more tractably computed from a demand forecast for a single manufacturing part. In regard of this, we have begun to apply many of the techniques used in the Forecasting System (such as Bayesian updating and a semi-parametric life cycle representation) to the forecasting of part demands. At this level of disaggregation, priors derived from judgmental forecasts of product demand are frequently unreliable, because a multitude product congurations and frequent substitution of manufacturing parts due to changes in technology or sourcing arrangements makes the correspondence between product and part demands too unstable. However, we have found hierarchical priors (Gelman and Hill 2006)which pool information across sets of partsto be effective replacements. On a more pragmatic note, we should note that in developing a parts forecasting solution, our dealings with Suns operating units have been greatly enhanced by the credibility garnered by the Sun Labs Forecasting System.

22

A Bayesian Model for Sales Forecasting at Sun Microsystems

Appendix
Gibbs Sampler
The following details the Gibbs sampler used for Bayesian updating in the Forecasting System. Each step in the sampler is introduced by the full conditional distribution from which a sample is to be drawn. Variables of which the sampled quantity is conditionally independent are omitted from the conditioning set. 0 , . . . , T yt , , , V, W, m0 , C0 Conditionally on the other parameters of the model, 0 , . . . , T constitutes a sample path of the state vector of a rst-order polynomial dynamic linear model. Procedures for sampling the state of a DLM (generally under the moniker forward ltering/backwards sampling algorithms) are described by Frhwirth-Schnatter (1994), Carter and Kohn (1994), West and Harrison (1997) and Durbin and Koopman (2001), amongst others. W 0 , . . . , T Again, a standard step in DLM samplingc.f. (W&H, p. 568), for example.
1 , 2 , 3 , yt , t , , V

The full conditional is proportional to the expression:

N(yt |L(t)S(t)t ,V )
t =1

2 N[0, ) (| , ).

This is sampled using a Taylored chain proposal in a Metropolis-Hastings stepthe calculation of the Taylored chain proposal is described in detail in the following section. yt , t , , V Draw from the posterior distribution of the coefcients 1 , 2 , 3 in the linear regression: yt

given independent priorssee e.g. (Gelman et al. 2003, chp. 8).

L(t)t = [L(t)t Q1t ] 1 + [L(t)t Q2t ] 2 + [L(t)t Q3t ] 3 + t ,

N(0, V ),

23

A Bayesian Model for Sales Forecasting at Sun Microsystems

V yt , , , t
T The conditional likelihood of V is t=1 N(yt |S(t)L(t)t ,V ); with prior V the posterior is given by conjugacysee e.g. (Gelman et al. 2003, chp. 2).

Inv2 (, ),

The Geweke- anizaki (2003) 17 Taylored Chain T


With x the current state of the sampler, to produce a proposal x + for a target kernel p(z), let q(z) = log p(z), with q (z) and q (z) the rst and second derivatives thereof. Proceed by cases: Case 1: q ( x ) < , where is a suitable small constant, such as 0.1 . 18

Rewrite the Taylor expansion of q(z) around x: q(z) q( x ) + q ( x )(z 1 x ) + q ( x )(z x )2 2 2 1 q ( x ) ( q ( x )) z x 2 q ( x )

= q( x )
Since q ( x )

Case 2: q ( x )

< 0, the component () of the latter expression constitutes the exponential part of a normal distribution, which implies that the target kernel in the vicinity of x may be approximated by a normal distribution with mean x q ( x )/q ( x ) and standard deviation 1/ q ( x ); sample x + accordingly.
and q ( x ) < 0
Approximate q(z) by a line passing through x and x1 , the largest mode of q(z) smaller than x: q ( x1 ) q ( x ) q ( z ) q ( x1 ) + ( z x1 ) x1 x

Sampling techniques similar to the Taylored chain are also discussed by Qi and Minka (2002). By ensuring that | q ( x ) | > 0, using rather than 0 reduces the occurrence of proposed values that depart too markedly from the current state.
18

17

24

A Bayesian Model for Sales Forecasting at Sun Microsystems

In this case, the component () indicates an exponential distribution, and the proposal is: 19 x + = x1 + w, where w Exp( 1 ), q ( x1 ) q ( x ) , 1= x1 x 1/
1

x 1 = x1

Case 3: q ( x )

and q ( x ) > 0

Approximate q(z) by a line passing through x and x2 , the smallest mode of q(z) larger than x. The proposal is developed in a manner parallel to that in Case 2:

x + = x2

w,

where w

x 2 = x2

Exp( 2 ), q ( x2 ) q ( x ) = , 2 x2 x 1/
2

Case 4: q ( x )

and q ( x ) = 0

In this instance, x + is sampled from a uniform distribution over a range [ x1 , x2 ], such that x1 < x < x2 . End points x1 and x2 are set to suitable modes of q(), if they can be found, and to user-supplied values otherwise.

Measures of Forecast Accuracy


As operationalizations of the quadratic loss criterion that pervades statistical inquiry, the mean square error and its related metric, the root mean square errorwe refer to both metrics jointly as the (R)MSEhave long been a staple of academic research in forecastingsee (Granger and Newbold 1973), for example. Unfortunately, though the (root) mean square error is analytically attractive, researchers such as Armstrong and Collopy (1992) point to a number of practical problems with its use: (1) Performance with respect to the (R)MSE may be signicantly affected by outliers. Such problems may be ameliorated by eliminating outliers, though this might draw the objectivity of the procedure into question. (2) More seriously, the (R)MSE is inherently scale dependent, in that its magnitude depends not only on forecast accuracy, but also on the level of the underlying series; ceteris paribus, a forecast 10% in excess of an actual value of 1,000,000 will result in a substantially greater (R)MSE than one 10% above an actual of 1,000. This largely invalidates summary
The origin of the proposal, x1 , is offset from the mode x1 in order to guarantee irreducibility of the resulting Markov chain; see (Geweke and Tanizaki 2003) for details.
19

25

A Bayesian Model for Sales Forecasting at Sun Microsystems

measures based on the (R)MSE of performance across a heterogeneous collection of series. 20 Since the series in this test described in the paper vary in maximum between 200 and 4000, the (R)MSE is unsuited to this application. The mean absolute percentage error (MAPE) favored by practitioners, which expresses error as a fraction of the associated actual value, avoids the scale-dependency of the (R)MSE. The MAPE has disadvantages of its own, however: (1) In contradistinction to the (R)MSE, summary MAPE measures may be skewed by small actualsindeed, the MAPE is innite for an actual value of 0. Some researchers, such as Coleman and Swanson (2007) have suggested taking logarithms as a way of mitigating the problem, though this makes the resulting metric more difcult to interpret. (2) The MAPE exhibits a rather counter-intuitive asymmetry; a forecast of 5 units on an actual of 10 produces an absolute percentage error of 50%, whereas a forecast of 10 units on an actual of 5 gives an APE of 100%. Attempts to amend the MAPE in order to overcome this problem (Makridakis 1993) have met with limited success (Koehler 2001). In light of these problems, some authorsincluding Fildes (1992) and Armstrong and Collopy (1992)have proposed the use of metrics based on relative absolute errors (RAEs). These are absolute forecast errors divided by the corresponding error from a benchmark method (normally the nave or random walk method, which simply repeats the last observation). Hyndman and Koehler (2006) point out, however, that RAE-based metrics suffer from some of the same problems from which the (R)MSE and MAPE suffer: They are sensitive to outliers, and may be skewed by small benchmark forecast errors (again, the RAE is innite if the forecast error is 0); remedies involving outlier elimination and log-transformation are subject to the same criticisms, too. Hyndman and Koehler (ibid.) propose the mean absolute scaled error (MASE) as a robust, scale-independent metric that largely avoids the problems set out above. Formally, for series y1 , . . . , y T , and denoting by yt+h|t the forecast for yt+h made in period t, the absolute scaled error ASEth is dened: ASEth =
T h T h t =1 1

| yt+h|t

| y t + h

yt+h |

y t |

Then the mean absolute scaled error, MASEh at horizon h for the entire series is simply the T mean T 1 t=1 ASEth , and a summary metric for a collection of series may be calculated by taking the mean of the ASEs across all the series. In recommending the MASE, Hyndman
20

Chateld (1988), for instance, points out that Zellners (1986) analysis was unduly affected by the outsized contribution to summary MSE metrics of only 5 out of 1,001 test series.

26

A Bayesian Model for Sales Forecasting at Sun Microsystems

and Koehler note that normalization with respect to the benchmark forecast error confers scale-independence, while use of the in-sample aggregate in the denominator of scaled errors makes the MASE metric more stable than those based on relative errors.

References
D. A. Ahlburg. Forecast evaluation and improvement using Theils decomposition. Journal of Forecasting, 3:345351, 1984. J. Albert. Bayesian Computation with R. Springer, 2008. J. Armstrong and F. Collopy. Error measures for generalizing about forecasting methods empirical comparisons. International Journal of Forecasting, 8:6980, 1992. J. M. Armstrong. Judgmental bootstrapping: Inferring experts rules for forecasting. In J. M. Armstrong, editor, Principles of Forecasting: A Handbook for Researchers and Practitioners, pages 171192. Kluwer, 2001a. J. M. Armstrong. Combining forecasts. In J. M. Armstrong, editor, Principles of Forecasting: A Handbook for Researchers and Practitioners, pages 417439. Kluwer, 2001b. J. M. Armstrong and F. Collopy. Integration of statistical methods and judgment for time series forecasting: Principles from empirical research. In G. Wright and P. Goodwin, editors, Forecasting with Judgment, chapter 10, pages 269293. John Wiley & Sons, New York, 1998. A. H. Ashton, R. H. Ashton, and M. N. Davis. White-collar robotics: Levering managerial decision making. California Management Review, 37:83109, 1994. Carliss Y. Baldwin and Kim B. Clark. Sun wars: Competition within a modular cluster, 1985-1990. In David B. Yofe, editor, Competing in the Age of Digital Convergence, pages 123158. Harvard Business School Press, 1997. F. Bass. A new product growth model for consumer durables. Management Science, 15: 215227, 1969. R. Batchelor and T. Y. Kwan. Judgemental bootstrapping of technical traders in the bond market. International Journal of Forecasting, 23:427445, 2007. J. M. Bates and C. W. J. Granger. The combination of forecasts. Operational Research Quarterly, 20:451468, 1969. J. M. Bernardo and A. F. M. Smith. Bayesian Theory. Wiley, 1994. R. C. Blattberg and S. J. Hoch. Database models and managerial intuition: 50% model + 50% manager. Management Science, 36(8):887899, August 1990. F. Bolger and N. Harvey. Heuristics and biases in judgmental forecasting. In G. Wright and P. Goodwin, editors, Forecasting with Judgment, chapter 5, pages 113137. John Wiley & Sons, New York, 1998. J. Boulden. Fitting the sales forecast to your rm. Business Horizons, 1:6572, 1958.

27

A Bayesian Model for Sales Forecasting at Sun Microsystems

G. E. P. Box, G. M. Jenkins, and G. C. Reinsel. Time Series Analysis. Prentice-Hall, 3rd edition, 1994. D. Bunn and G. Wright. Interaction of judgemental and statistical forecasting methods: Issues & analysis. Management Science, 37(5):501518, May 1991. C. Carter and R. Kohn. On Gibbs sampling for state space models. Biometrika, 81:541543, 1994. C. Chateld. Apples, oranges and mean square error. Journal of Forecasting, 4:515518, 1988. R. T. Clemen. Combining forecasts: A review and annotated bibliography. International Journal of Forecasting, 5:559583, 1989. C. D. Coleman and D. A. Swanson. On MAPE-R as a measure of cross-sectional esimation and forecast accuracy. Journal of Economic and Social Measurement, 32:219233, 2007. D. Dalrymple. Sales forecasting practices: Results from a United States survey. International Journal of Forecasting, 3:379391, 1987. Jessica Davis. Dell takes wraps off channel partner program, 2007. eWeek.com, December 5, http://www.channelinsider.com/article/Dell+Takes+Wraps+O+Channel+Partner+ Program/220950\_1.aspx. R. M. Dawes. A case study of graduate admissions: Application of three principles of human decision making. American Psychologist, 26:180188, 1971. Ton de Kok, Fred Janssen, Jan van Doremalen, Erirk van Wachem, Methieu Clerkx, and Winfried Peters. Philips electronics synchronizes its supply chain to end the bullwhip effect. Interfaces, 35(1):3748, JanuaryFebruary 2005. Jason Dean. The forbidden city of Terry Gou, 2007. The Wall Street Journal, August 11, http://online.wsj.com/public/article/SB118677584137994489.html. F. X. Diebold. Forecast combination and encompassing: Reconciling two divergent literatures. International Journal of Forecasting, 5:589592, 1989. F. X. Diebold and P. Pauly. The use of prior information in forecast combination. International Journal of Forecasting, 6:503508, 1990. A. Dolgui and M. Pashkevich. On the performance of binomial and beta-binomial models of demand forecasting for multiple slow-moving inventory items. Computers and Operations Research, 35:893905, 2008a. A. Dolgui and M. Pashkevich. Demand forecasting for multiple slow-moving items with short requests history and unequal demand variance. International Journal of Production Economics, 112:885894, 2008b. J. Durbin and S. J. Koopman. Time Series Analysis by State Space Methods. Oxford University Press, Oxford, 2001. B. Efron. Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7:126, 1979.

28

A Bayesian Model for Sales Forecasting at Sun Microsystems

P. T. Elgers, H. L. May, and D. Murray. Note on adjustments ot analysts earning forecasts based upon systematic cross-sectional components of prior-period errors. Management Science, 41:13921396, 1995. R. Fildes. Efcient use of information in the formation of subjective industry forecasts. Journal of Forecasting, 10:597617, 1991. R. Fildes. The evaluation of extrapolative forecasting methods. International Journal of Forecasting, 8:8198, 1992. R. Fildes and P. Goodwin. Good and bad judgement: Lessons from four companies. Foresight: The International Journal of Applied Forecasting, 8:510, 2007. R. Fildes, P. Goodwin, M. Lawrence, and K. Nikolopoulos. Effective forecasting and judgmental adjustments: An empirical evaluation and strategies for improvement in supplychain planning. International Journal of Forecasting, 25:323, 2009. P. Franses. Merging models and experts. International Journal of Forecasting, 24:3133, 2008. P. H. Franses and R. Legerstee. Properties of expert adjustments on model-based SKUlevel forecasts. International Journal of Forecasting, 25:3547, 2009. S. Frhwirth-Schnatter. Data augmentation and dynamic linear models. Journal of Time Series Analysis, 15:183102, 1994. E. S. Gardner and E. McKenzie. Seasonal exponential smoothing with damped trends. Management Science, 35(3):372376, March 1989. A. Gelman and J. Hill. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press, 2006. A. Gelman, J. Carlin, H. Stern, and D. Rubin. Bayesian Data Analysis. Chapman & Hall/CRC Press, Boca Raton, 2nd edition, 2003. J. Geweke. Evaluating the accuracy of sampling-based approaches to calculating posterior moments. In J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, editors, Bayesian Statistics, volume 4, pages 169193. Clarendon Press, 1992. J. Geweke and H. Tanizaki. Note on the sampling distribution for the Metropolis-Hastings algorithm. Communications in Statistics Theory and Methods, 32(4):775789, 2003. E. Ghysels, D. R. Osborn, and P. M. M. Rodrigues. Forecasting seasonal time series. In G. Elliott, C. Granger, and A. Timmermann, editors, Handbook of Economic Forecasting, volume 1, pages 660706. North-Holland, 2006. W. R. Gilks, S. Richardson, and D. J. Spiegelhalter. Markov Chain Monte Carlo in Practice. Chapman & Hall, London, 1996. P. Goodwin. Statistical correction of judgmental point forecasts and decisions. Omega: International Journal of Management Science, 24:551559, 1996. P. Goodwin. Adjusting judgemental extrapolations using Theils method and discounted weighted regression. Journal of Forecasting, 16:3746, 1997. P. Goodwin. Correct or combine? Mechanically integrating judgmental forecasts with statistical methods. International Journal of Forecasting, 16:261275, 2000.

29

A Bayesian Model for Sales Forecasting at Sun Microsystems

C. W. J. Granger and P. Newbold. Some comments on the evaluation of economic forecasts. Applied Economics, 5:3547, 1973. C. W. J. Granger and R. Ramanathan. Improved methods of combining forecasts. Journal of Forecasting, 3:197204, 1984. K. R. Hammond, C. J. Hursch, and F. J. Todd. Analyzing the components of clinical inference. Psychological Review, 71:255262, 1964. A. C. Harvey. Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge Univ. Press, Cambridge, 1989. Joseph Heller. Catch-22. Simon & Schuster, New York, 1961. R. Hill. Applying Bayesian methodology with a uniform prior to the single period inventory model. European Journal of Operational Research, 98:555562, 1997. P. J. Hoffman. The paramorphic representation of clinical judgment. Psychological Bulletin, 57:116131, 1960. C. Huang and G. Tzeng. Multiple generation product life cycle predictions using a novel two-stage fuzzy piecewise regression analysis method. Tehnological Forecasting and Social Change, 75:1231, 2008. H. D. Hughes. An interesting seed corn experiment. The Iowa Agriculturist, 17:424425, 428, 1917. C. J. Hursch, K. R. Hammond, and J. L. Hursch. Some methodological considerations in mutliple-probability studies. Psychological Review, 71:4260, 1964. R. J. Hyndman and A. B. Koehler. Another look at measures of forecast accuracy. International Journal of Forecasting, 22:679688, 2006. R. J. Hyndman, A. B. Koehler, J. K. Ord, and R. D. Snyder. Forecasting with Exponential Smoothing: The State Space Approach. Springer, 2008. K. Kahn. How to measure the impact of a forecast error on an enterprise? Journal of Business Forecasting, pages 2125, 2003. Spring. R. Klassen and B. Flores. Forecasting practices of Canadian rms: Survey results and comparisons. International Journal of Production Economics, 70:163174, 2001. A. B. Koehler. The asymmetry of the sAPE and other comments on the M3-competition. International Journal of Forecasting, 17:570574, 2001. A L. Lamport. LTEX: A Document Preparation System. Addison-Wesley Professional, 2nd edition, 1994. M. Lawrence and M. OConnor. Judgement or models: The importance of task differences. Omega: International Journal of Management Science, 24:245254, 1996. M. Lawrence, P. Goodwin, M. OConnor, and D. nkal. Judgmental forecasting: A review of progress over the last 25 years. International Journal of Forecasting, 22:493518, 2006. M. J. Lawrence, M. OConnor, and R. H. Edmundson. A eld study of sales forecasting accuracy and processes. European Journal of Operational Research, 122(1):151160, April 2000.

30

A Bayesian Model for Sales Forecasting at Sun Microsystems

C.-Y. Lee, J.-D. Lee, and Y. Kim. Demand forecasting for new technology with a short history in a competitive market: The case of the home networking market in South Korea. Technological Forecasting and Social Change, 75:91106, 2008. W. Y. Lee, P. Goodwin, R. Fildes, K. Nikolopoulos, and M. Lawrence. Providing support for the use of analogies in demand forecasting tasks. International Journal of Forecasting, 23:377390, 2007. P. Lenk and A. Rao. New models from old: Forecasting product adoption by hierarchical Bayes procedures. Marketing Science, 9:4257, 1990. D. V. Lindley. Reconciliation of probability distributions. Operations Research, 31:866880, 1983. V. Mahajan, E. Muller, and Y. Wind, editors. New-Product Diffusion Models. Kluwer, 2000a. V. Mahajan, E. Muller, and Y. Wind. New-product diffusion models: From theory to practice. In New-Product Diffusion Models Mahajan et al. (2000a), chapter 1. S. Makridakis. Accuracy measures: Theoretical and practical concerns. International Journal of Forecasting, 9:527529, 1993. W. H. McGlothlin. Stability of choices among uncertain alternatives. American Journal of Psychology, 69:604615, 1956. P. E. Meehl. When shall we use our heads instead of the formula? Jounal of Counseling Psychology, 4:268273, 1957. J. T. Mentzer. The impact of forecasting improvement on return on shareholder value. Journal of Business Forecasting, pages 812, 1999. Fall. J. T. Mentzer and C. C. Bienstock. Sales Forecasting Management: Understanding the Techniques, Systems and Management of the Sales Forecasting Process. Sage Publications, Thousand Oaks, CA, 1998. W. W. Moe and P. S. Fader. Using advance purchase orders to forecast new product sales. Marketing Science, 21(3):347364, Summer 2002. A. Montgomery. Creating micro-marketing pricing strategies using supermarket scanner data. Marketing Science, 16:315337, 1997. M. A. Moon, J. T. Mentzer, and C. D. Smith. Conducting a sales forecasting audit. International Journal of Forecasting, 19(1):525, 2003. JanuaryMarch. M. Moriarty. Design features of forecasting systems involving management judgments. Journal of Marketing Research, 22:353364, 1985. P. A. Morris. Decision analysis expert use. Management Science, 20:12331241, 1974. P. A. Morris. An axiomatic approach to expert resolution. Management Science, 29:2432, 1983. R. Neelamegham and P. Chintagunta. A Bayesian model to forecast new product performance in domestic and international markets. Marketing Science, 18(2):115136, 1999. R. Neelamegham and P. Chintagunta. Modeling and forecasting the sales of technology products. Quantitative Marketing and Economics, 2:195232, 2004.

31

A Bayesian Model for Sales Forecasting at Sun Microsystems

S.-C. Niu. A piecewise-diffusion model of new-product demands. Operations Research, 54: 678695, 2006. M. OConnor, W. Remus, and K. Lim. Improving judgmental forecasts with judgmental bootstrapping and task feedback support. Journal of Behavioral Decision Making, 18:247 260, 2005. L. D. Pankoff and H. V. Roberts. Bayesian synthesis of clinical and statistical prediction. Psychological Bulletin, 80:762773, 1968. A. Pole, M. West, and J. Harrison. Applied Bayesian Forecasting and Time Series Analysis. Chapman Hall, 1994. Y. Qi and T. P. Minka. Hessian-based Markov chain Monte Carlo algorithms. In First Cape Cod Workshop on Monte Carlo Methods, 2002. T. stebro and S. Elhedhli. The effectiveness of simple decision heuristics: Forecasting commercial success for early-stage ventures. Management Science, 52:396409, 2006. E. M. Rogers. Diffusion of Innovations. Free Press, 5th edition, 2003. N. R. Sanders. Comments on Effective forecasting and judgmental adjustments: An empirical evaluation and strategies for improvement in supply-chain planning. International Journal of Forecasting, 25:2426, 2009. N. R. Sanders and K. B. Manrodt. Forecasting practices in US corporations: Survey results. Interfaces, 24:92100, 1994. N. R. Sanders and K. B. Manrodt. Forecasting software in practice: Use, satisfaction, and performance. Interfaces, 33:9093, 2003. N. R. Sanders and L. P. Ritzman. On knowing when to switch from quantitative to judgmental forecasts. International Journal of Operations and Production Management, 11:2837, 1991. N. R. Sanders and L. P. Ritzman. Judgmental adjustment of statistical forecasts. In J. M. Armstrong, editor, Principles of Forecasting: A Handbook for Researchers and Practitioners, pages 405416. Kluwer, 2001. Steven M. Schafer. Web Standards Programmers Reference: HTML, CSS, JavaScript, Perl, Python, and PHP. Wrox, Chicago, IL, 2005. E. Silver. Bayesian determination of the reorder point of a slow moving item. Operations Research, 13:989997, 1965. SMCI. Sun microsystems company info, 2009. http://www.sun.com/aboutsun/company/ index.jsp. A. Sood, G. James, and G. Tellis. Functional regression: A new model for predicting market penetration of new products. Marketing Sciences, 28:3651, 2009. H. Theil. Applied Economic Forecasting. North-Holland, 1971. A. Timmermann. Forecast combinations. In G. Elliott, C. Granger, and A. Timmermann, editors, Handbook of Economic Forecasting, chapter 4, pages 135194. North-Holland, 2006.

32

A Bayesian Model for Sales Forecasting at Sun Microsystems

L. R. Tucker. A suggested alternative formulation of the developments by Hursch, Hammond and Hursch and by Hammond, Hursch and Todd. Psychological Review, 71:528 530, 1964. J.W. Tukey. Exploratory data analysis. Addison-Wesley, 1977. A. Tversky and D. Kahneman. Judgment under uncertainty: Heuristics and biases. Science, 185:11241131, 1974. T. T. Tyebjee. Behavioral biases in new product forecasting. International Journal of Forecasting, 3:393404, 1987. H. J. van Heerde, C. F. Mela, and P. Manchanda. The dynamic effect of innovation on market structure. Journal of Marketing Research, 41:166183, 2004. W. N. Venables and D. M. Smith. An Introduction to R. Network Theory, 2002. R. Webby and M. OConnor. Judgmental and statistical time series forecasting: A review of the literature. International Journal of Forecasting, 12:91118, 1996. M. West and P. J. Harrison. Bayesian Forecasting and Dynamic Models. Springer-Verlag, New York, 2nd edition, 1997. Rick Whiting. Sun takes build-to-order to the next level, 2006. InformationWeek, September 12, http://www.informationweek.com/story/showArticle.jhtml?articleID=192700331. J. Wiedermann. Web Design Flash Sites. Tachen, 2006. H. E. Williams and D. Lane. Web Database Applications with PHP and MySQL. OReilly Media, 2nd edition, 2004. J. Wisner and L. Stanley. Forecasting practices of JIT and non-JIT purchasers. European Journal of Purchasing and Supply Management, 1:219225, 1994. B. Worthen. Future results not guaranteed. CIO, 16:14, 2003. URL http://www.cio.com. au/article/168757/future_results_guaranteed. G. Wright and P. Ayton. The psychology of forecasting. Futures, 18:420439, 1986. P. M. Yelland. A model of the product lifecycle for sales forecasting. Technical Report 127, Sun Microsystems Laboratories, 2004. URL http://research.sun.com/techrep/2004/smli_ tr-2004-127.pdf. P. M. Yelland and E. Lee. Forecasting product sales with dynamic linear mixture models. Technical Report TR-2003-122, Sun Microsystems Laboratories, March 2003. A. Zellner. A tale of forecasting 1001 series: The Bayesian knight strikes again. International Journal of Forecasting, 2:491494, 1986. Paul H. Zipkin. Foundations of Inventory Management. McGraw-Hill/Irwin, New York, 2000.

33

A Bayesian Model for Sales Forecasting at Sun Microsystems


Response to editors report

Excerpts of the editors comments are reproduced below in boxes, each followed by the corresponding response.

The other referee. . . asks specically why the authors did not initially consider the Bayesian models of West and Harrison (a very good point). We should apologize: West and Harrisons Dynamic Linear Models have formed the basis of the Forecasting System since its earliest incarnationsee (Yelland and Lee 2003) and (Yelland 2004) for detailsand continue to do so (albeit in a somewhat altered form) today. We hope that the revision (c.f. the discussion of the model in particular) makes the centrality of DLMs much clearer.

This manuscript is of potential interest to Interfaces readers but is in need of substantial revisions (including an extensive update of the references) to improve its scholarly tone. In accordance with the Editors request, the paper has been radically revisedreferences included. We hope that the tone of the revision is now appropriate.

A Bayesian Model for Sales Forecasting at Sun Microsystems


Response to editors report

Excerpts of the editors comments are reproduced below in boxes, each followed by the corresponding response.

(Neelamegham and Chintagunta 2004) is a reference I would like to see cited. Apologies: References to the authors earlier works appeared in a precursor to this submission (Yelland 2004), but the reference in question was inadvertently omitted from the paper. This has been corrected in the revision.

The rst paragraph of background is not clear. I dont know what is the starting point and the current point. The section concerning related work has been completely rewritten in the revision, and I hope that the result is more perspicuous.

The second paragraph of background uses a dangerous wordbootstrappingwhen the references are not adequate. This is the case. We hope that the referee with agrees that in the revision, the section describing bootstrapping (now termed judgmental bootstrapping, in keeping with recent literature, to lessen confusion with statistical bootstrapping) is amply provided with references.

A Bayesian Model for Sales Forecasting at Sun Microsystems

The third paragraph of background must be improved including adequate references to forecast combination. The paragraphexpanded into a subsection in the revisionhas been rewritten, and now contains numerous references.

References to relative errors in page 16 must be updated with recent papers. A sectionwhich includes extensive referencesdevoted entirely to the denition of and motivation for the error metric used in the paper has been added to the Appendix in the revision.

Conclusions must be modied. They are of limited usefulness for the readers. The Conclusions have been rewritten, emulating those of other papers in Interfaces. We hope that the results meet with the referees requirements.

The estimation paragraph in the Appendix is very ambiguous. Authors should give information in a precise way. For example, using a step-by-step approach. In the revision, the Appendix provides a thorough specication of each step of the Gibbs sampler used to update the forecasting model, including the algorithm used to make Metropolis-Hastings proposals in the sampler.

Suns Web page provides 33,350 people worldwide. As of February 2009, the Web page (SMCI) records 33,556 employees worldwide; this is rendered as some 33,500 in the introduction, and a note concerning the vintage of the information has been added.

Page 9 (Model). In the second paragraph you should eliminate to this paper. This section has been rewritten entirely.

ii

A Bayesian Model for Sales Forecasting at Sun Microsystems

Pages are wrong in Moon et al. (2003). Corrected.

In page 25 you should cite Gilks et al. (1996). Corrected (BibTEX problem).

iii

A Bayesian Model for Sales Forecasting at Sun Microsystems


Response to comment of referee #2

Excerpts of the editors comments are reproduced below in boxes, each followed by the corresponding response.

As I rst read through the narrative, I found myself wondering why the author had not initially considered the Bayesian models of West and Harrison. These state-space type dynamic models are designed to do just what the author was challenged to accomplish in this situation, but with a seemingly more rational and formal structure having time varying parameters. They allow for constant updating and feedback from the forecaster together with a user-directed red ag feature to alert for structural changes. They are also simple, depending upon ad hoc normality assumptions with parameters that must be provided via elicitation (although the variance is treated using a discounting methodology). After completing my reading, it appears the author is aware of this model (having cited the relevant work), but has relegated it to the realm of future work. That is a shame. Such models are no more complex (a quality the author argues is essential in the business environment in which they are working), and would be subject to the same amount of expert input for parameter determination. At a minimum, some discussion of these models should be incorporated. . . We should apologize: West and Harrisons Dynamic Linear Models have formed the basis of the Forecasting System since its earliest incarnationsee (Yelland and Lee 2003) and (Yelland 2004) for detailsand continue to do so (albeit in a somewhat altered form) today. We hope that the revision (c.f. the discussion of the model in particular) makes the centrality of DLMs much clearer.

A Bayesian Model for Sales Forecasting at Sun Microsystems

[The Bayesian models of West and Harrison should be] included in their comparison section. I have no desire or expectation for an enlargement of the paper to include details of the seasonal ARIMA model since these are generally well-known. In addition, they are not as exible as the Bayesian models, failing to take expert opinion into account. In fact, I would be happy to see that aspect of the comparison replaced by the dynamic Bayesian model if conservation of space is an issue. After all, the former model is not Bayesian, the latter is, and this paper is about Bayesian models. I believe it is a given that Bayesian models will outperform non-Bayesian ones. In the revision, a DLM has been substituted for the SARIMA model in the comparison section.

I was hoping for some detailed discussion of exactly how the experts provided their input. I am sure readers would nd this as interesting as the details concerning the modeling aspect. Too often, discussions of Bayesian approaches gloss over this pivotal consideration, yet we (or at least I) spend much time in my teaching of Bayesian methods on how to elicit expert judgment. The revised version of the paper incorporates a much expanded discussion (in the section Prior Estimation) of the way in which priors for the system are derived.

ii

You might also like