A Comparison of Neural Network and Fuzzy Clustering Techniques in Segmenting Magnetic Resonance Images of the Brain Lawrence O. Hall, Member, EEE, Martin S. Abstract Magnetic resonance (MIR) brain section images are segmented and then synthetically colored to give visual represen- {ation of the original data with thre approaches: the tral and proximate fury comenne unsupervised custering algorithms ia supervised computational neural network, a dynamic ‘muliayered perception trxined with the cascade correlation {earning agorthm, Initial clinical results are presented on both itormalvolunters and selected pallens with brain tumors sur- founded by edema. Supervised snd unsupervised segmentation Techniques provide broadly similar results. Unsupervised fuzzy lgorihins were visually observed to show beter segmentation ‘rhea ‘compared with raw image data for volunteer sti, However, for 8 more complex segmentation problem with tu- Imoriedema or cersrospinal fuld boundary, where the tissues fave similar MR relaxation behavior, Inconsistency io rating lamong experts was observed, with fzzy-comeans approaches being slighty preferred over feedforward cascade correlation ‘suls afer several trations inthe selection of traning repons {We compare various facets of bath approaches, sch as supervised ‘ersus unsupervised learning, time complexity, and wit for The diagnostic process, Each approach seems to provide useful information about different aspects of the problem Indes Terme— Cluster analysis, fay comeans algorithms, fcr Sets, neural networks, Image segmentation, magnetic resonance ‘imaging 1. sropucrion [eee sie ss machine vision platforms that attempt o interpret scenes with various kinds ‘of sensor and contextual information, almost always include {and depend upon) one or more pattern recognition algorithms for low and intermediate-evel image data processing. The area addressed by the present paper is image segmentation. ‘Treatments of the many conventional approaches fo this prob- lem include the texts by Duda and Hart [1], Gonzalez and Wintz [2], and Serra [3]. Image segmentation techniques can Manus ett ane 1, 191; reve October 2,191. This work set saan) te Nal cece Foust ney Gan TRI-S259 lou by'NASA” (OTL: Keely ‘Space, Cem), ACS Foss Division ‘Slonen Med Sytner), nd Sen rst Inept (Gio LOT and A.M, Renate withthe Deparient of Computer Science an Eines, Unery of South rT FL 3360 TL Chates iP. Veliizen, ant M Sige ate with the Depriest of aio, Clie of Medica, Usher South Pes, Tape FL "Te: Beaks wih the Divison of Computer Scene, Universi of Wes TEE Log Name 6201468. ‘Amine M. Bensaid, Laurence P. Clarke, Robert P. Velthuizen, ilbiger, and James C. Bezdek, Fellow, IEEE be categorized in @ number of ways. Haralick and Shapiro [4] sive a comprehensive survey to techniques that predate the approaches discussed here ‘Mote specifically, segmentation techniques can be generally escribed in terms of their axiomatic basis (deterministic, ‘tatstical, or fuzzy), the principle on which image subareas are idemified (region-based, edge-based, or pixel classification), implementation architecture (serial or parallel), and type of data (labeled or unlabeled). This paper concerns segmenta- tion algorithms based on two very different schemes. One, the fuzzy c-means cliscring approach, is a fuzzy, serial, unsupervised pixel classification technique based on iterative approximation (0 local minima of global objective functions, Readers may consult Bezdek [5] or Pal and Mujumdar (6] for introductions (oa variety of models ofthis kin. The other ‘approach we take to magnetic resonance (MR) image segmen- lation is a deterministic, hybrid, supervised pixel classification algorithm based on a modification ofthe standard feedforward bck-propagation computational neural network called cascade correlation [7], [8]. ln what follows, we find it convenient to use HCM, PCM, AFCM, and EFCC as acronyms for the rd (ot conventional), fuzzy, and approximate fuzzy c-means laorithms and the feedforward cascade correlation algorithm, respectively. Previous Attempts at Image Segmentation ‘The use of pattcn recognition methods to segment MR image data sels has been described in the literature {9}, {10} Although characteristics of different issue types overlap, Imultispecral analysis may still be sufficiently successful, 4 {10} satisfactory preliminary results with Bayesian max- mum likelihood were reported, and in [9] the use of an attificial neural net showed great promise, with improved performance compared with the maximum likelihood method The performance of these wo methods was compared in terms. of stability for variations in the data set (interslice and intersubject classifications) and in terms of the subjective interpretation by experienced radiologists {9} In [11] Dempster-Shafer evidential reasoning is used in conjunction with TL and T2 weighted MR images to inter pret segmented MR images and improve upon segmentation ‘done vin localized feature analysis and statistics. Chen et al tlso used previously classified images in the process, and 101s 92279280800 © 1992 IEEE‘other modalities such as X-ray computed tomography (X- ray CT). These additions render the method discussed in [11] ly more complicated than those discussed below. n [12], 8 3D extension of the Marr-Hildreth operator is used for edge detection in MR images ofthe brain. In adtion, morphological filters are used 10 refine the detected edges, tissue labeling is done interactively, and a surface shading method (the “gray level gradient” method) is used for a 3D display of the results. In [13], some low-level features based on the proton- density and T2-weighted parameters of MR images ae used to separate voxels of diferent structures. Some segmentation heuristics based on these low-level structures are developed, and they ate coded into set of rules that guide the segmen- In (14), X-ray CT, MRI, and positzon emission tomography images ate segmented using a neural network algorithm, The mage Segmentation problem is formulated as a constraint sal- jsfacton problem, and a CSNN (constraint satisfaction neural network) is used (o perform the segmentation (Kohonen’ssell- organizing feature map is used for the assignment of intial conditions). ‘Section II preserbes our notation and provides an overview of the FCM/AFCM models and algorithms. Section Il de- scribes the FFCC model. Section IV contains a description of the MR dal including its collection and physical meaning. ‘The main results ate_given in Section V as comparative computational examples using various MR data seis specif ‘chosen to illustrate the similarities and differences as as the feasibility and utility of the wo approaches Section VI detils our conclusions and lists some ideas for further research I, THe CMEANS PARTITIONING MoDELs “The structure of partion spaces underlying clustering gorithms is generally well known, so this section is brief, following [S]. Let (¢) be an integer, 1 < ¢ 1st 20 w= (SE, DalDa OD) Lsiceleken orit Gat) a> 12, £0 ue = 0 forall € (F~ fe) and arbitrarily. SO ton = 1 (2) od, 1, Da = min({Da: 1S 8; Given unlabeled data set X = {24,2 aaj Fic Le cml ems ‘ce; positive definite weight matrix A (0 induce an inner product norm on °. Choose «, small positive constant Guess Uy € Myon (0, iitilize vp (ei P205---¥e0) € RF. To avoid be ing at a Fixed point, the rows of Uo (or vectors 0) must be distinc :: For a = 1 © aMAX: <: Calculate vj with Uy and (5), 1< i <6 Sho: Update Uy 10 Uy, ith (Wg) and (4a if m > 1) o¢ (4b if m Joo: AT max( lias ~ taaaenal) $6 then slop and put Wr) =U se Next a ‘This procedure converges «linearly from any initialization to a locel minimum or saddle point (local maxima are impos- sible) of Jn [19]. Several algorithms have been proposed to test whether (U*, 0°) i a local minimum or saddle point [20] ‘This, however, has been less important in applications than the related questions of whether FCM terminates at the same point when begun from various initalizaions and whether termination points yield “good” solutions. We shall point out {in Section V that “good” initialization is crucial to successful segmentation of MR data with FCM/AFCM. ‘When ¢, p, andlor n becomes large, FCM is a time- ‘consuming process. The data of interest to us have values for these three parameters in the ranges 2 Sc < 7, p and n= 256 x 256 = 65536, Thus, each iteration of FCM for a single cross section involves finding, say, five or six clusters in a set of 65536 vectors in 38. Moreover, typical MR studies consist of 20-128 slices, each having the dimensions given. Each iteration of FCM on one slice might take, eg, 30 of CPU time on a SUN 4, so methods for acceleration of FCM are of great practical interest. ne method for accomplishing this isthe AFCM, or approximate fuzzy c- means algorithm, This epproach is based on replacing some of | the calculations (the exponentiations) in (4) and (5) by lookup table approximations (21). To contol the size of lookup tables and the speed of computation, AFCM is defined only for data whose features are discrete: image intensities =, are usually such that 0< tj < Gr, where G, = some number of gray levels. The data in our case have Gz, = 256 possible values for each feature. Details of the AFCM algorithm are well documented in (21), and will be omitted in the interests ‘of brevity, Suffice it 10 say that AECM provides (roughly) one ‘order of magnitude speedup in the FCM procedure outlined above; on the other hand, the optimization problem that AFCM solves is NOT the one specified by (5), because round off in the AFCM approach may cause the effective value of (m) in 'AFCM to vary slightly at each iteration. Consequently, FCM and AFCM may terminate at different pairs (U*,o"). We will have further comments about this in Section V. Next, we turn to the feediorward cascade corelation multilayered perception. IL, CASCADE-CORRELATION Cascade correlation is a supervised computational neural network algorithm that has a dynamic architecture. It was developed as a result of efforts to solve the speed problem that the standard feedfoward back-propagation (FFBP) algorithm safes from, and 10 allow incremental learning. In FFBP, the network has a fixed topology, and the learning algorithm involves two passes each time an input vector is presented. In the forward pass, network outputs are computed. In the backward pass, changes fo the network weight vectors are ‘made based on minimizing the diference between the network ‘outputs and the desired outputs (the error) by back-propagation through the same connections (ie, in the reverse direction) [22]. The gradient ofthe eror with respect to each weight is computed, and every weigh! is updated in accordance with the ‘method of steepest descent [23]. Fig. 1 isa schematic diagram of the familiar FFBP archtec- ture. Although the Kolmogorov theorem asserts the existence ‘of a feedforward network with one hidden layer having (2p ++ 1) nodes that can exactly represent any continuous (cassifie) function, there is no way {0 find this “optimal” three-layer network (7 Consequenlly, practicality often dictates more than one hidden layer. In this scheme, all the weight vectors tae attached to nodes in the “hidden layers.” as in Fig. 1, may be updated after each presentation of a labeled input vector ‘The architecture itself is statie—only the weight vectors can change (without intervention). Two factors are believed to contribute to backprop’s slow= ness; the step-size problem and the moving target problem [8]. As shown in Fig. 1, the input to FFBP (and FPCC) is a set of labeled feature vectors. Let X = fx3,22y--"nu}i Tet U = {ayeasss-stta,} be the set of hard labels, w, of i,t © No = (eqsensss*se-). where ng is the number ining (sign) vectors, ¢ is the number of classes, and (0,042, 1,0,--»,0) bas its 1 im the ith place. The Iackprop (BP) network error function is defined as Fap(w) = |BP(x —w)| o fonction f= 1,2,-++,h, where BP(a,) = network output ‘of network weights w. Of course, BP(z;) € N. ‘The step-size problem is due to backprop’s taking ifin- itesimal steps down the error curve 9 minimize Epp over tw, The moving target problem occurs because weight updates ‘within hidden layers compete for eror reduction of Epp; and ‘weight updates across different hidden layers do not aceount for changes in Fp which updates just made in adjacent layers will cause. Tn other words, at each node the goal of this update is to solve the problem defined by the error signal back-propagated to that node, However, this problem is constantly changing because while a hidden node is working ‘on minimizing the eror, basing its computations on a given state of the network, other nodes ate updating their weights with conficting objectives. ‘To achieve a faster move to a minimum of Epp, cascade correlation uses the second-order method used inthe quickprop algorithm [23]. This technique is based on two assumptions: the firs is thatthe error as a function of each component of cach weight vector wy: can be approximated by a concave: up parabola; the second assumption is that the change in the derivative of the error with respect 10 wsay of a given connection is not affecied by the simultaneously changing ‘weights (wae) of the other connections at node (st). While these two assumptions cannot be verified for a particular network, the empirical evidence (good performance) suggests that they are probably satisfied. The idea behind quickprop is to define a parabola using, for every weight, independently, the previous and curzent error slopes and the weight change between the pois at which these slopes were measured. ARAL rat A COMPARISON OF NEURAL NETWORK AND FUZZY CLUSTERING TECHMQUES os % P rater Wiener Fig. 1. Convent FFBP clase acbiecu, rect move tothe minimum point of the parabola can then be performed using the formula oe) My) where s(t) and s{¢ ~ 1) are the current and previous values ‘of BE'/Bw, respectively. In (7) we have omited the subscripts (ty) for clarity, The resulting value ofthe weight is only an approximation to the optimum value, but this method seems very effective when applied iteratively Cascade correlation remedies the moving-taget problem by inroducing the cascade architecture and a learning algorithm associated with it [8]. The cascade architecture initially con- sists of input and output layers as dictated by the problem t0 be solved, Every input is connected to every output, and all the connections have adjustable weights. Moreover, there is a bias input, permanently set 19 +1. The cascade architecture is shown in Fig. 2. The output nodes may just produce @ linear sum of their weighted inputs, or they can use some nonlinear activation function. AS the network is trained, new hidden nodes are added tothe network, one by one as needed, and as dictated by the learning algorithm. Cascade corclation Starts with no hidden nodes, and using quickprop it trains the direct connections between the input and output nodes over the entice traning set until no significant error reduction is achieved. Then the error is computed by running the network ‘over the whole training set I the eror is less than ¢ (Set by the user), the algorithm terminates; otherwise, a new hidden node is added in an attempt to eliminate or reduce the residual feror. The inputs ofthe new node come from every input node and all other hidden nodes in the network; each hidden node adds a new one-node “layer” to the network. Initially, the ‘outputs of the new node are not yet connected to the active ietwork. A numberof passes are run over the traning pater, adjusting the weights ofthe input connections tothe new nodes. after each pass, This adjustment is done in the direction that ‘maximizes S, the sum overall output nodes ofthe magnitude ofthe correlation between the candidate node's output value=, fs shown in Fig. 2, andthe residval output error, Ep, observed cach output node o. Strictly speaking, $ is « convariance; if Aw(t) = Aw(t =), o 1 “ages » ig. 2._The cascade schilesre rom alate rough te aon of wo ees, The weit the rn odes te cen when the hin mer ‘is the number of ouput nodes and ny the numberof training patterns, is defined as =Y|Sea-9&.-Bh @ where Big = 20 — thoy #1 the output of the candidate nove on taining vector 2, the dsized output vale a ‘ouput o, and 2 and Ey ae the values of = and B, averaged 8% Then 6S;/bu, the panial derivative of S with respect 10 each ofthe candidate node's incoming Weights ws, can be computed LY volEe - Fe) fila, o 58/6; ‘where a is the sign ofthe correlation between the candidate's Value and output o, ff is the derivative for pattem kof the tandidate node's activation function with respect to the sum of its inputs, and I, isthe inpot the candidate node receives from node 4 for pattern: Finally, quickprop is used for a fas-converging ascent to maximize S, When the magnitude of the correlation stops improving, the new candidate node i installed in the active network; its outputs are now connected to the output nodes. ‘Then its input weights are frozen, and its output weights are trained to minimize the network's output error. This is doneinthe same way the direct connections were trined, using uickprop. I the networks's performance isnot satisfactory land no significa error rdicton is achieved, mew hidden ‘ode is ntoued; the cycle is started. This cycle is repeated tnt the enor E(w) becomes accepably small (or wnt ‘we give up). Im addition, cascade curlation can use & poo fof candidate nodes instead of just one node. These nodes Se inalized with different random weighs; the candidate ‘vith the best corcation is installed. Some implementation Improvements can also be found i [8 IV, Tae NATURE OF MRI DATA Magnetic resonance imaging (MRI) systems measure the spatial distribution of several soft tissve related parameters sch as T1 relaxation (spin late), T2 relaxation (ansverse), ‘and proton density. By discrete variations of the radio fr ‘quency (RF) timing parameters, a set of images of varying soft tissue contrast can be obtained. The use of time-varying magnetic eld gradients provides spatial information based on the frequency or phase of the precessing protons using both multislice 2DFT) or volume (3DFT) imaging methods [24], [25] Hence, a multispectral image dat sti produced (similar to NASA Landsat data [26)) as shown in Fig. 3, Feature space representation of each tissue cluster is also shown in the same figure (Les, white or gray matter, cerebrospinal fuid (CSF), fat, and muscle) [26]-[28}. Bone and air, although similarly classified, do not yield any MR signal. Tn our work, male volunteers (25-85 years) and patient tumor studies were performed on a high field MRI system (15 tesla) using a resonator quaduature detector head RF coil. Transverse images of 5 mm thickness were obtained using a standard spin echo (SE) technique for TI weighted images (pulse repetition time TR = 600 ms, echo time TE 20 ms) and proton density (p) and T2 weighied images (TR 3000 ms, TE = 20 and 80 ms, respectively), using the 2DFT tmulislice technique (27]~(29}- Volunteers were imaged for the same anatomical location, Pixc-intnsity-based classification methods were employed in this work [27] as opposed to methods based on the caleu- lation of magnetic resonance relaxation parameters. The later methods equire tailored RF pulse sequences (24) (25]-Image- intensity-based methods can be applied to any imaging proto- col and are not restricted to the number of images acquired; ie, its possible to accommodate images with features other than MR relaxation parameters, suchas perfusion and diffusion imaging, metabolic imaging, and the addition of images from ‘other diagnostic modalities (29]. The transverse images were acquired, centrally located in the resonator RF head coll, and hence did nol require uniformity corrections for RF coil ‘geometry or dielectric loading characteristics as developed at ‘this institute (30, Similarly, the subjects studied didnot move significantly during the imaging procedure; hence corrections were not requited for related registration problems LV. EXPERIMENTS AND RESULTS X = fates tensa) CR is the data set from which our computations begin. Bach xe = (rita). with sas € 0,255), has the following physical meaning feature 1 = TI = value of the TH-weighted pixels; feature 2 = T2 = value of the T2-weighted pixels feature 3 = p= value of the spin-densty-weighted pixels Fig. illstrates the flow of compoting forthe two schemes. Because they are unsupervised, FCM and AFCM are applied directly to X inthe figure. The rest of eustering X this way is a pair (U,);U © Mg i a fuzzy e-panition on the spatial locations in the image. In order to use U for segmentation, ‘we ean convert {10 a hard e-arttion in a number of ways The usual method isto assign each pixel to the tissue class of. ‘maximum membership Alternatively, we can devise a scheme for displaying the fuzzy membership grades obtained in U. Both methods are wiz in our numerical examples. “The input to AFCM is 8 65536 3 integer-value matix, where cach zow vector consists ofthe three features associated with a pixel (T1, 72, and spin density p parameter values). The same matrv in oatng-pont format s used as input to FCM. In addition, the number of clusters (0) is set to the number of tissue types believed (by the expert) 10 make up the image being segmented. FCM was run for diferent values of parameter m ranging ftom 1.5 10 2.0; the best results, based ‘on quality of segmentation (as evaluated by a radiologist) ané (CPU time, were obtained with m = 2. A value € = 0001 was used for FCM and AFCM, and the Euclidian inner prod- ‘ct norm was used to compute the distance between vectors in Ga, On the other hand, FFCC is « supervised method. This requires us to divide into ining (design) and test sets, say x| 6 [Xal = may [Xe = ry 1 = ny a8 shown in Fig. 4. Because 2, © Ry, the FFCC network shown in Fig. 2 has three inpot nodes for these data, The three features of each input vector to FFCC are scaled values ofthe pixel parameters; scaling is achieved by dividing every pixel value parameter by 255, the maximum value that i¢ can take on. The training set consists of ny (pine, class) pairs, where each pixel is a three-dimensional vector as described above, and the (crisp) clas is the tissue type that the piel fll into. The taining set is chosen by an expert who interactively selec regions of interest fom the three raw data images andor from a plot ofthe feature space. ‘The operator assigns each region to one of the fssue types that are believed to make up the image being processed. The traning set, in our experiments, consists on the average of| ‘ny = 250 of the m= 65 536 paters for one image. FFCC is run using forthe integrator Function the usual Euclidean dot product, y: = f(z) = (ws -2), and forthe activation function = Fil) = (1/(1-+e-)) at node i. Furthermore, a pool of eight candidate nodes is trained before a new hidden node is added tothe network. The hidden node inserted at each step isthe one with the best performance inthe pool “The experiments performed with FFCC, FCM, and AFCM. were done using 12 subjecs. OF these, eight were normal and four were sbnorml. Furthermore, our experiments with FECC also include segmentation using interstice and intesubject traning data to test stability of segmentation using a set of labeled pixels from one image across different slices andsubjects. In the case of intersice training, the traning set is Selected from an MR image that i taken at a diferent position Of the brain thatthe image being segmented, but from the same brain. On the other hand, when intersubject tating is performed, the taining dats are chosen from an image of the brain of a eitferent subject, For display purposes each output class is a “The colors are chosen from a palete to provide a -8 9 unique calor good! visualization of the segmented regions. AFCM and FCM generate different segmentations, depending upon the defuzziieation method used t© make U exsp. The oust is, for each pixel, a veetor of fszzy membership values for the pixel in cach cass, A thresholding method can be used, so that each pixel is uniquely assigned to the class which has the highest membership Value. An alternative method) sill w the highest membership class, isto assign a color shaded by weighting with the actual value. That is, the membership grade will determine the shade of color assigned fo each pixel. We refer (0 this latter metho as weighted defuzzfication More specifically, let me = (1nstuasss*-tbe)” be the column of U & Myon associated with pixel (vector) 2p Unweighted defuzitication (heesholding) simply replaces ui, by the hasis vector e, € NY. for which was > ass 1S) Se 7 (les are resolved arbitrarily). Then pixel 2 is assigned the color representing the tissue class (ef. Table I). Weighted Gefuzzification, on the other hand, uses the act that ay is not only the largest we, but also its actual value. Thus, the higher the actual value the lighter the shade of color chosen AA value of | obtains the lighest coloration and a value of 0.6 (for example) will be a darker shade of a given color. A, Iniialzation Strategies Initialization is important all three algorithms. For FFCC wwe found that random initialization of the network weightSven Coton ron MR Ts OAs yalow etoile) franee as mater Nowe i pare tne pit ‘sea vectors eaused no problems in tems of speed of convergence ‘oF limit point approached. On the other hand, both fuzzy gots exhibited sensitivity tothe inital guess with regard to oth speed and sabity. This is hardly suprising, since the system of equations at () involves, for example atc = 6, (on) + (ep) = (6 65536) + (6 x3) = 393234 unknowns. Thus Jm_probably possesses many more local minima and ponsibly sade points than Ep, which involves fra typical Confguction with two hidden nodes. (36) + (26) 4 (+4) = 31 weighs, making Eyp 4 much less complex error surface than the ne defined bY J "We found that FCM seemed most lable when intalized with 1100 on oo11o8 0 Uy a0) o0000. 1140 is ac x 65596 matix. AFCM worked best when ined with vp obtained alter running one iteration of FCM, ‘We found that by running FCM for one iteration, and then using the resulting cluster center values to initialize AFCM, the results of FCM and AFCM. were always identical In Addition, in these experiments, identical segmentations were also obtained when the final output (U' or v) resulting from the classification of a different image is used to initialize the U matrix in FCM or the cluster centers in AFCM. ‘There isthe question of whether FCM and AFCM will con- verge to anything meaningful at all based on the initialization and in general, In this domain, we have found that they will ‘converge to meaningful casters for normals with five cluster ‘centers and for abnormals with between seven and ten cluster ‘enters, ‘The CPU times used by the different algorithms to gen crate segmented image were surprisingly dependent on the inititizations. CPU times presented below aze all obtained using a Sun6/470. When a “random” initialization is used 10 clasify an image of 2 normal braia using five clusters, FCM takes about 40 min and AFCM takes about 18 min, When AFCM js initialized using the cluster centers obtained after ‘one iteration of FCM, a segmented image is produced in about 12 min. Moreover, i the data from a segmented image of a different slice of the brain of a different subject are used 10 initialize AFCM, the algorithm performs the classification in about 8 min. For images of brains with tumors, atleast six clusters are needed, To segment such an image, FCM runs for about 1 hr, $0 min. AFCM, when randomly initialized, takes about 30 min, and it runs for about 15 min when initialized after one iteration of FCM. ‘The performance of cascade correlation, measured by its CPU time and the size of ils network, also varies from one run (© the next since its weights are randomly initialized. For a normal subject, using five classes, FFCC performs the classification in an average time of 32 (including both taining land testing). For an abnormal case with six classes, FPCC takes an average of 7 min to do both the taining and the testing (which serves as the classification). We also note that the number of hidden nodes introduced by FFCC differs with ferent initalzations; furthermore, this number is greater for cases with pathology. ‘The residual sum of square errors in FFCC was always less than 0.001 B, Evaluation of Segmented Images Qualitative comparison of the results of FCC, FCM, and |AFCM is done by visual evaluation and comparison with raw MR images. The images obtained by MR contrast material (gadolinium) for the brain tumor cases were assumed to be ‘round truth, within the known constraints ofthe spatial and temporal changes in MR contrast and related twmor staging factors, In the following, we examine the performance of the algorithms ona typical normal volunier and the most difficult (Jo segment) tumor patient in our limited study. The color scheme in Figs. 5 and 6 is listed in Table L Fig. 5 presens the segmentations of an MR image of a volunteer's brain, using ¢ = 5 clustrs. The segmentations by FCM and AFCM were obtained using arbitrary ini izations, and both yield “correct” segmentations. Evaluating these segmentations, FCM’s results showed an improvement 10 FFFOC and the physician prefers the image generated wsing the weighted defuzification because this image provides the most ‘detail. On the other hand, FECC’s segmentation does not seem to convey enough details, parly mixing gray matter (colored in orange) with CSF (yellow) in some regions, and mixing fat with ar on the boundaries. AFCM's segmentation intialy did not show CSF at all; given five clusters, it comes up with air White matter, gray matter, and two different fat tissues. The latter result may be parly due 10 peripheral inbomogencity ‘caused by the RF coil, However, if AFCM is run withthe same parameters, using six clusters rather than five, it generates the same segmentation as FCM with the addition of the new fat tissue cluster (ths result i not shown inthe figures). Fig. 6 shows the results of segmenting an MR image of 4 patent study with suspected reoccurtent tumor that had both chemo and radiation therapy, using © = 6 clusters ‘The patient had malignant glioma removed and it was suspected that it might be reoccurring in the cerebral edema inthe lower le-hand section of the image. In this ease no eoceurrent tumor was found; however with radiation therapy tceatment resulted in a spatially variant change in MR tissue relaxation parameters, i., representing a very dificult or extreme case ofa segmentation problem. In this study, FFCC’s segmentation, after several trials with diferent training data, identified an additional sixth cluster (putple color code), 2s edema surrounding the area of suspected tumor, while AFCM and FCM classified this abnormal region (edema) as belongingCSF; both uids have similar MR. relaxation ehsvios For this particular ease FFCC was thought 10 provide better segmentation and cortelation with the raw MRI data. Tis was Apparently de to the different color provided for CSF and edema by supervised FOC which was told that another class txised. Similar small differences in segmentation coloring have been observed in one of the other cases with patients having @ tumor with surrounding edema Using AFCM's weighted defuzztiation on output yielded better separation of the edema and CSF. We also note that in the ease of this patent study, FCM generates exactly the same segmentation as AFCM. . General Comparisons Our general study suggests that FCM and AFCM are considered superior for volunteer cases, FECC is considered better by one radiologist, who examined the segmentation ff each of the 12 images in this study, in the tumorous atca of some abnormal and not away’ from the tumor where FCMIAFCM provide better detail. On the other hand, as we shall see in the following, FCM/AFCM provide preferred segmentations for cerain abnormals, ‘When the results of FCM/AFCM were interpreted by rep resenting every cluster by different shades of the same color based upon their membership values, the radiologist prefered these results for normal cases, This method of defuzification resulted in images that were believed to convey more accurate ‘etal, For volunteer subjects, these results were qualified as very good; furthermore, they offered more details in the tumorous area for abnormal cases. ‘At the current time the images produced by FCMIAFCM are not completely unsupervised, since the colors foreach cluster fre chosen by a radiology technician, Without being able to ‘utomatially label the clesters of tissues, itis highly Tikey that ‘ulomation of tissue coloring would cause tissue types to be Colored differently across images. Some tess have been done With differen colors for the same tissue type on images and the color changes do not appear to have a significant effect on the physician's inegpretation, Hence itis unclear what effect futomation wouké have on the interpretation proces. For the few intersice ad intersubject classifications done, which only apply tothe FECC method, CPU times and quality fof segmentation (as evaluated by a radiologist) are comparable to the results obiained when the aining data come from the same subject and the same slice asthe image being segmented. Tn order to provide & more quantitative summary of the results, a blind study with three expert radiologists was done. They were given a survey form and asked to respond tothe frst three questions shown below on a 0 (very bad) 10 (excellent) scales, with the last two questions on a percentage basis. The issues addressed were the fllowing: 1) the differentiation between CSF, white matter, and gray 2) the differentiation between normal tissues and pathol- ogy3) the differentiation between tumor and surrounding edema: 4) the percentage of false positives (tumor where it should not be) 5) the percentage of false negatives (normal tissue, where tumor or edema should exist) Four images were considered in the survey, the two shown here and an abnormal study both precontrast and postoontrast Ina posteonirast study, the patient has been injected with a (MR contrast (gadolinium) designed to enkance the boundary between tumor and the surrounding regions. For the normal volunteer, FCM with crisp output was rated as vety good by the three physicians in difereniating CSF, white matter, and gray matter. FCM with fuzzy output was {ated as good, while FOC was rated inthe middle ofthe sale. FCM/AFCM were rated slightly higher than FFCC on the abnormal patient shown in Fig. 6, but the average rating was in the mile of the scale. In the precontrast and postcontrest study FCM/AFCM were rated 3s good 10 very good and FFCC was slightly below the mide ofthe range. (On differentiating normal tissues from pathology, FCM/ AFCM were tated good to very good with both exisp and fuzzy output on all three abnormal images. FFCC was rated in the middle of the scale. Differentiation between tumor and edema was only ex- amined forthe procontrast and postcontast study, since the patient in Fig, 6 did not have observable tumor. Each of the Techniques was) poor in differentiating precontast and fair (FFCC) to very good with fuzzy output (FCMIAFCM) for the posteontras study. "There were no fase positive regions found according tothe survey, but from 20% for FCM/AFCM to 35% for FECC, on average was felt tobe false negative. For the patent in Fig. 6, a false negative is considered to be missing edema, Vi. SUMMARY AND FUTURE DIRECTIONS “The ultimate goal ofthis research i to develop unsupervised segmentation methods for for 3-D MRI data as required t0 ‘enhance tumoriedema boundary detection (27}~(29}, for 3-D surgery simulation [31], [32] and for 3-D radiation treatment planning [33], i. to generate something akin to probability ‘maps of normal/abnormal tissue boundaries as opposed 10 simple gray seale methods employed using X-ray CT data {31}, We have previously shown that supervised segmentation methods are cttically dependent on the choice of traning regions and are often unstable for interslice or intrsubject (traning/lassiication) [28]. These limitations have stimulated tur efforts towards unsupervised methods such as FCM and ‘AFCM [27] The preliminary results suggest that FCM and ‘AFCM show broadly similar segmentations compared with supervised methods. In te following, we examine the perfor- mance ofthe algorithms on the criteria of accuracy, amount ‘of needed interaction, and time complexity (costs). |AFCMIFCM are more accurate, according 10 our expert plysician, in segmenting images of volunters than FEC. ‘The weighted defuzziication of AFCMIFCM yields the best segmentation yet obtained on a volunteer, in the radiologis’s evaluation, It i possible that selecting “better” training seis ‘Would produce a better segmented image from FFCC. I is not clear how to select these sets in a reproducible manner. "According to our primary exper, data from three patient ceases are generally better sepmented between edema and normal tissues by FFCC. While FECC does not do as well way from the tumorous area, it i usually the region of Jnerest. The fat that tumor and edema would have different colors from surrounding tissue classes was a major factor im his decision. The weighted defuzziication of AFCM/ECM provides improved detail over the unweighted method in the Tumorous region, However, other studies [37] (de la Paz) suggest that better tumor definition using AFCM may ensve if 10 < ¢ < 1G; we shall investigate this in the future. "The panel of physicians (which included the primary exper) preferted FCM/AFCM over FCC for all the cases shown, ‘which included the patient in Fig. 6 and one study not before considered by the primary exper Tn terms of time, FPCC is much faster on volunters than FCMIAFCM, and at least twice as fast on patient studies ‘We note, however, thatthe faster speed of FFCC comes at the cost of human intervention to acquire the trsining daa Moreover, if one #8 not interested in realtime display of the segmented images the times of both techniques may be wholly aceeptable. That i, if the images are to be segmented for review off-line, CPU times may be acceptable forthe faster unsupervised learning methods. Good initialization, which ‘appear generally available, make the worst case for clustering fon the order of 10 min using a SUN 4/470. Considering the ratio of unknowns FCM:FFCC=393234:37 = 39000:1, the (CPU times ate certainly understandable 'As noted above, the FFCC algorithm requires that someone well versed in picking out areas of MR images choose pixels for training. This may require intervention for each patent, and this does not seem desirable. If i is the case the FFCC may be trained upon # generic set of classes (regions) taken from Volunteer and “typical” tumor patien(), retraining for each patient would be unnecessary. This would make the method essentially unsupervised inthe long run, While inital interlice fad intersubject tess are promising, many more subjects must bye segmented before we ean make any valid inference about this. For example, we would have to know the number of classes for the segmentation {0 work, with one training set for teach group of classes (5,6,7, et) ‘On the other hand, the only thing that is necded in AFCM/ECM isthe number of classes (c) forthe segmentation. Currently, this requies. someone 10 Took at the image and eclare the number of regions to expect. The unsupervised {echnigue can be completely automated if the mumber of clusters needed can be chosen in an automated fashion and the coloring of the segmented regions is automated. Ths i an ‘old problem for clustering algorithms (18). For this application there are several possibilities that may be explored and will be discussed below. Both FFCC and AFCM/FCM provided results that the radiologist found acceptable. Which i bette? IFone considers patient tumor eases 10 be the most important, and outing, the tumor 10 be the crucial issue in evaluating them, theFFOC algorithm has yielded better tumorledema definition in some cases in this pilot study, while FCM/AFCM has done a better job for at Teast one ease in which choosing & food traning Set was very difficult because of tissue overlap. ‘The weighted defuzzification of AFCMUFCM partitions gives improved tumor/edema definition and is close to FFCC, with the major exception of not necessarily putting the fumor/édema into a separate class. The separate class would be shown in a iferent colo, which makes it much easier forthe radiologist to idemify the tumor. ‘The technology of MRI is stil rapidly advancing, with ad tional improvements in imaging time for both 2DFT and 3DFT imaging methods using fast spin echo (FSE) methods or echo planar imaging that allow an order of magnitude reduction in data acquisition time, These methods should reduce the problem of involuntary motion and associated artifacts. The frowth in image data presented tothe radiologist, in tun, will also increase by an order of magnitude, thus requiring highly interactive unsupervised segmentation methods to intelligently fase the image data for ease and speed of interpretation. unre Directions FFCC needs to be evaluated with intersice and intersubject, ata on a wider range of test subjects. Trials with che number fof classes known will be used for traning; testing on normals and abnormals will then be used to evaluate the possibility ‘of not needing a taining set for cach individual patient. Improvements in the method of choosing the training set may provide better segmented normals and better segmentations ‘way from the tumorous areas in abnormal. 'ARCM/FCM might use the same training information as FCC does. That is, Xy can be used to fix columns in U with crisp labels, This may speed up convergence, since fewer pixels ae processed t each step and we already have a partial clustering, Further, it may help the segmentation performance with the patient tumor cases when another class (over the ‘ones used inthis study) is add. That is, using seven classes instead of six with the six clases having fixed taining data ‘may provide beter tumor det, as well as better detail away from the tumor. “Methods for choosing the number of clusters dynamically (uring clustering) would make AFCM/FCM completely unsu- Pervised and relatively subject independent. A heuristic based Upon the strength of memberships across clusters might be used a6 a criterion for adding a new cluster. Starting with, say, ve clusters (88 for a normal subject) and adding clusters fs needed may provide this feature Finally, in order to provide better definition around the tumor, the tumorous area might be identified. This smaller area could then be processed in an attempt fo provide beter ‘etal (Substructure). In summary, each approach discussed in this paper has its advantages and Hiniations; furore reports will hopefully resolve the issues just raised, REFERENCES. 1D] R. Dads ad an, Paton Clasieaon and Scone Anais Yate Wey, 1973 New (1 8 Gon an Win itl nase Presi A aod [3] J'Scem Imogen on Nethontcl Morphology, New Yok Acatemie Pres 582 [a] Robt Hae snd 1-0. Stag, Survey: age segmentation ech apes CVGI, wo 29, 1-182 198. (5) BBeedek, Pury Rocogaion wih Fy Objective Farcton Alo films New ok: Pea, 198 {o) Sik"Pa and. Miata Fn’ Mathemacal Approach Patern ‘ecogntion New York: Wey, 1586 71 S8Pan adap Ptern Renton ond Newel Neworks the, MA" Aloe Wey, 1993. (o1 Stralnun wd Leis “The cede sotto ening nchiee ture” Carete Melon, Compu Soence TR, 190. [pL LP Gai er sy "Compr of supe pate ecogson {echigues snd unsipetvied metodo MI penton, presented ‘SMa tmapng Vt, Newport Bench, CA, Fe. 192 LW Vans ¥t Spee ans DL Ruan, “Map sone Inoging mali petal ie cuscaion” IPS vo 3, Ang. 191. tony SRT viectan. €Chen eM! fmage unestging 35 ten bused on Benya Sher esoing” SPIE Biomedical Image ‘Proce Sane, CA. 1991 rma KW, Hoe "ede, aM. Rimes, “3D segmentation ‘SEM ge ofthe head fer 3D apa” TEEF Tron: Met. mei $e 9, oJ 1980 lis) SP. hig, Suamskel spmentin of 3D magne romance bain insgesh tet sytem” IEEE Tron. Med. Inaping V9, 8 Sep 180, (14) Wn Eta €. Che, sd Y. Feng, “Neal ntwors for met Image sepnenion: Nonwesters Unive, Bee Engierting ta Compatr Sees. Fe bubond Ainn grt Csr Data. Eelewond Ci (16) Gr ulvand DLA Hal “Chater etnigue for surmarcng mal ‘ne dts Bolo, Si yl 1p 1582158, 158 [17] J'Bexik, "Soe mw sind closing sgt” in evelomens in Namercl Ely, P Lepnde aad Leone, Bs Bes, ‘enmnny Spring Nra, 1087p. 225-287 (us) E"obrowsh at Beek “Cringe the 1 and 1 sms IEEE Dna. Sst Ma Cibers, 1291 Berd and Hhawsy “Roce conegence ret fr the fanny comeana charg grins” J. Clason, vl S, 8.2, pp s?-2i7 188. 20) Fin Wet, and R Hath, “Opin es fo fed points of he fay ermine agri” Pater Recognition vol 21, ho. 8 t21) Rie Gham 2 ve, ad 1 Bende, “sient implemen of he fziy mers hating lgnihs” IEEE Tras Pat. Ava Mach Ie, vt 8962, 1988 (22) £"thinm tnd tine, “Comectoni aritecares for ari! Iicligence, IEEE Computer, 1987 (29) $'Flan “Tascam ration on ck popgaton: An emp! Shula Prac 1084 Conmctanat Model Somer Sct YR oom, Poser Rs Arwesige nd ML Pie, “A Fevew of mmol se hyogen NAGE veo es ad elaton Irecnonins fom 1-100 Mile: Depndcce ae tue ype, NMR Treen temper, apc, eon snd gt.” Medal Pics ol Th no 4 p25 4, 19, (25) TL thm UF Koran, 6.€ Loy, and. Stoop, “Caravan srl ta ss ig seven clue? MRI parameters ni 3 Sotacal aly rym, loge Mo, sl 1, pp. 22-34 19. 1 ae i! ts me ler) AM Bena P'S" EP. Cte na ALP. Velen, MRL ‘Samentaton tung servile and sored meio” in TEEE ed & Bil, Poe 1h Amma! Meeting (Oras, FL), Oc Steno. 3.1981 To Shear, W.C. Nel, a4 TP. Cate, “Appleton of til near teers or tse laine fm mises Image soem ages of he hea TEE Sp Comper Based Melia ome Tid Anal Coro. (Cape il, NC Dae 3-6, 00) bs) us, 4 bs 10. UP caste, “Compson of Byes maximim ietood (MLM) tnd atl ol new fo Sapevised tse casifeaton end 3D ‘gmefaton by MRL” preset at Sei Symp, Aree [AStaion of Pye n Stdine (AAPM), 3d Amal Meine Sen Fans, CA July 199: Med. Pi, vol 8. po 3:9, 673 194 Dn! loner, (oP Clarke, RP. Vln, MCT” Sibige, “MK Image nonualoeiy cameo techies” THEE Fg. Med Bll rr poProc, 13h Aa Mecing (Os, FL) Oc 31-Now 3, 19 Robert, Velen secre be MS, ee tou SF zie Sw Rematy, DAL Long B.S Caso, and CR Beate ney of Une Nendo 8 BuiceS3 eppicnten in newodlogy.” Mespmedion v0 8 Fotar vison sae He hn woke i mesial (oy Be OR maps nna eyo ge pecsing ce 1990, iy in MR image it and PET ingen Rely a7, pp. 783-78, 1989 HE'S cent poting be PED. pe (9) Rt inet Copa ps Wo 3-48 Unive of Soh lon, Tame (04) 2 and R Leth, “Tie csc in MR igs wing Fat segmcetton” Proc IEEE Ine Conf Medal map. oe. 1580 tas) B'A Grad and JW. Can, “Sepmenaton of magne eso tune ages ing 2 cstrng” Pro. 0 Int. Cot formation ‘reesing i Medic! Imaging, 198, tao) 2 ea hy A uh bere proach to image seen Toa Minas in Pac SPIISPSE's Symp. Elect mage Se and Tenor V. SPIE-14S0, Feb, 198 (a7) dc Pa, Aden fo (AAPA, Sd Anal Meng Sen Fransio, Cain 19 Martin S.Sbge, MLD. 6 Profesor and Chi Meter tc Deparment of Rog. Uae of ‘Sout ia ps is esearch ners fae fesomce aogogapy ad bod ow, ed the ‘penton compas for 3D Suing eas “Lawrence 0. Hal S'85-M'M) sce he PAD. gpa of Te toepote scence fm Ponda Site Univer Fen associate Poesr i the Compe Se (Soe aed Engosing Deparment, Uniesiy of ‘San Moe Tange, nh rests zy set {feo puter ecogilon, machine eri, a pool ca eigence ames €. Bezdek (M'90-SM'90-F)rccve HE'BS. epee fom the Univeriy of Neva (Reno) a 1900 and the PD. dee fom Coe nn Amine M, Bena cine he degee BS te as hel cei pslan at SUOO ne, ‘Memon stems from te Unies of Soa Maret Unerey, Ua Ste Unies andthe Fa, Hescareny » Pk teen compet UR a Sout Cain an nds postions ngiceing thee. Hi esearch acti & i pt Bong Acsonpc (1982) nl he Boclg Fe (28 fcopnion,nea newts, fry Tog od tone igh Tecingy Cet (1957 1 198) De ‘ppt fee meal aging Beale cue bode» Hora Eminent Scot {Cal wih be Divison of Capt See the ‘nlvrsy of West Pr Hs rere meres cde pt enpion gore eral eons, mage procsing a machine vison, medic ‘Sopwtng, od expe ses a ee ise tendng ite ofthe Iteration Jounal of Appr me Resening om ste et ofthe Tas TO OF SYST Maw Crntaners, The IEEE Teatro on Nrinat NETO, tenn Journal of Man ache Sates, he Ieraaionl Journal Lanreace PCa is Asie Profesor ofRa> of App Imelgone teInteratinal ural of General Systems, sth are se icc Unity of South eration Jounal of Faz Ses and Ses Fore amp, He oto Asoce Press of Fay and Computer ‘Scene the Cobege of Engen thee ad conhnoma fhe Cero Engncring and Medi Image Ass (CEMIA), Hin acch mere fale soot Sgn ad peiseaon in msl Imapng sl the apis e Em ot ings eomencent ed pater recopaion {echlges far improvd ener ago an ‘Sener sed aoe nl sging modal