Professional Documents
Culture Documents
in Colonoscopy
Nima Tajbakhsh1(B) , Changching Chi1 , Haripriya Sharma1 , Qing Wu2 ,
Suryakanth R. Gurudu3 , and Jianming Liang1
1
Dis-
Introduction
Optical colonoscopy is the preferred method for colon cancer screening and prevention, during which a tiny camera is inserted and guided through the colon.
The goal of a colonoscopy is to detect and removal colorectal polyps, which are
precursors to colorectal cancer. Colonoscopy is an eective screening method
and has led to a signicant decline of 30 % in the incidence of colorectal cancer [1]. However, colonoscopy is an operator dependent procedure whose ecacy
largely depends on human factors such as attentiveness, diligence, and navigational skills. A Canadian study [2] reports a 6 % cancer miss-rate after a negative
c Springer International Publishing Switzerland 2014
H. Yoshida et al. (Eds.): ABDI 2014, LNCS 8676, pp. 151158, 2014.
DOI: 10.1007/978-3-319-13692-9 14
152
N. Tajbakhsh et al.
colonoscopy and attributes this to the polyps that are missed due to insucient
quality of procedures. Other clinical trials have also reported signicant levels
of polyp miss-rates [35].
In response to the increasing concerns for missed polyps and cancers, American Society for Gastrointestinal Endoscopy (ASGE) established a number of
guidelines for a quality colonoscopy in 2002. Among the suggested guidelines is
maintaining minimum withdrawal time of 6 min, which is designed to discourage
colonoscopists from a hasty colon examination. However, such a constraint on
raw withdrawal time may not guarantee a quality and thorough inspection. This
is because a colonoscopist may spend a large amount of time in one segment
of the colon (for removing a polyp or getting a biopsy) but performs a quick
examination in the other parts. Therefore, more eective ways are needed to
monitor the quality of colonoscopy and encourage diligence during procedures.
Fig. 1. Comparison between informative and non-informative frames. (a) An informative frame where the information content is well-spread all over the image. (bd)
Examples of non-informative frames: (b) an out-of-focus image with bubbles inside,
(c) an image captured during wall contact with light reection artifacts, (d) a motion
blurred image captured close to wall contact. Two distinguishing characteristics of noninformative frames are blurry edges and appearance of salient features in only a few
local regions of the images (e.g., bubbles in (b) and reection artifacts in (c)).
We propose a system that can monitor the quality of colonoscopy and warn
colonoscopists against hasty or poor colon examinations. We base our method on
the observation that a poor colon examination shot consists of a large number of
non-informative frames including blurry, out-of-focus images, or those captured
during wall contact (see Fig. 1). Our method assigns each colonoscopy image a
quality or informativeness score. By monitoring the informativeness scores during
a procedure, we can detect the onset of a hasty or low quality colon examination
upon identifying a number of consecutive non-informative images. In addition to
the warning mechanism, the suggested method can report a segmental quality
score during a procedure as the average of informativeness scores recorded in a
segment of the colon. The overall quality score of a procedure can be computed
as the average of the segmental scores, which can be used along with overall
withdrawal time for quality monitoring purposes.
We trained and evaluated the suggested system using our collection of 5500
colonoscopy images. Quantitative evaluations demonstrate that our method provides a signicantly higher area under the ROC curve, yielding higher sensitivity
153
and specicity to non-informative frames than the existing image quality assessment methods for colonoscopy videos. Qualitative comparisons also show that quality scores computed by our system are more inline with that of human perception.
The rest of this paper is organized as follows: Sect. 2 reviews the existing
works for image quality evaluation in colonoscopy. We describe the suggested
method in Sect. 3, and present the experimental results in Sect. 4. Section 5
explains the related research aimed at reducing polyp miss-rate, discusses their
limitations, and underlines the importance of image informativeness assessment
as an eective tool for improving the quality of colonoscopy. Finally, this paper
is concluded in Sect. 6.
Related Works
There are several methods for automatic image quality assessment in colonoscopy.
Filip et al. [6] proposed a simple method based on image variance and contrast to
detect blurry and out-of-focus images. Oh et al. [7] suggested a more sophisticated
method based on gray level co-occurrence matrix (GLCM) in the Fourier domain.
The main idea was to identify frequency patterns associated with non-informative
images. Arnold et al. [8] computed the l2 -norm of 2D discrete wavelet transform
(DWT) coupled with a Bayesian classication approach for image quality classication. Park et al. [9] trained a hidden Markov model using entropy-related features. Our experiments, however, reveal that these methods fail to achieve high
sensitivity and specicity for our diverse set of informative and non-informative
images. This motivates our research to develop a more eective image quality
assessment for colonoscopy.
Proposed Method
We base our methodology for detecting non-informative frames on 2 key observations (see Fig. 1): (1) non-informative frames most often show an unrecognizable scene with few details and blurry edges and thus their information can be
locally compressed in a few Discrete Cosine Transform (DCT) coecients; however, informative images include much more details and their information content
cannot be summarized by a small subset of DCT coecients; (2) information
content is spread all over the image in the case of informative frames, whereas
in non-informative frames, depending on image artifacts and degradation factors, details may appear in only a few regions. We use the former observation to
design our global features and the latter to design our local image features.
Our method begins with dividing an input image to non-overlapping image
patches. We then apply 2D DCT transform to each patch and reconstruct the
patch using the dominant DCT coecients. The reconstructed patches are then
put together to form the reconstructed image. We create a dierence map by
taking the absolute dierence between the original input image and the reconstructed image. The dierence maps are computed in multiple scales, their
154
N. Tajbakhsh et al.
histograms are constructed, and then concatenated to produce a global feature vector. We also compute a local feature vector to measure how information
is spread all over the input image. To do so, we divide each dierence map
into a 3 3 grid and then in each cell, we compute the energy as the sum of
squared intensities. The local feature vector is formed by concatenating all the
local energy values computed in multiple scales. We experimentally found out
that feature fusion is the best way to combine local and global information, outperforming other alternatives such as score-level and decision-level fusion. Once
fused feature vectors are formed, we train a classier to assign a probabilistic
score to each input image with 0 and 1 indicating images with minimal and
maximal information content, respectively. We refer to this score as quality
score or informativeness score throughout the paper. We use random forest
for classication. Random forest has been successfully applied to a variety of
computer vision and medical image analysis applications, and has been shown
to outperform other widely-used classiers such as AdaBoost and support vector machines [10]. The two main ingredients of random forest are bagging of a
large number of fully grown decision trees and random feature selection at each
node while training the trees, which together achieve high generalization error
Fig. 2. Overview of the suggested method for image informativeness assessment. Our
method is based on global and local image features that are extracted by histogram
pooling over the entire image reconstruction error and region-based energy pooling, i.e.
l2 -norm of reconstruction error in each of the 9 sub-regions.
155
and high quality probabilistic outputs. Figure 2 illustrates how the suggested
method works. We should note that our method is fundamentally dierent from
JPEG image compression. Unlike JPEG, our method is not meant to compress
images, rather, it is designed to generate global and local feature vectors from
the image reconstruction error.
Fig. 3. (a) Comparison between the proposed method, DWT [8], and GLCM [7]. Our
method excels in all operating points with a large margin. (b) Image informativeness
assessment for a short colonoscopy video. The higher the score, the more informative the
frame. Segments of the signal with average low quality scores correspond to hasty or low
quality colon examination, in which case our method warns colonoscopists, encouraging
a more diligent examination. (ce) Three colonoscopy frames and their corresponding
quality scores (our method in blue, GLCM in green and DWT in red). A seen, the
scores assigned by our method are in more agreement with human perception (Color
gure online).
Results
156
N. Tajbakhsh et al.
random forest classier consisting of 100 fully grown decision trees. Randomness
was induced by randomly selecting a subset of features at the tree nodes.
We use receiver operating characteristic (ROC) curves to compare the performance of the suggested method with that of Oh et al.s [7] and Arnold
et al.s [8]. Figure 3(a) shows the resulting ROC curves. As seen, our system
outperforms the other two methods in all the operating points, achieving higher
sensitivity and specicity to non-informative images. To test the statistical signicance of the dierence between the ROC curves, we employ the method of
DeLong et al. [11]. Our statistical analysis shows that the proposed method
achieves area under curve (AUC) of 0.948 (95 % CI, 0.935 to 0.959), which signicantly (p < 0.0001) outperforms Oh et al.s [7] with AUC of 0.880 (95 % CI,
0.862 to 0.897) and Arnold et al.s [8] with AUC of 0.867 (95 % CI, 0.848 to
0.885). Statistical analyses are performed using MedCalc for Windows, version
13.3 (MedCalc Software, Ostend, Belgium).
Figure 3(b) demonstrates image informativeness assessment for a segment of
a colonoscopy video. Segments of the signal, highlighted with the ellipses, correspond to low quality colon examination because of their poor average quality
score. For qualitative comparison, we have selected 3 frames from this video and
compared the corresponding informativeness scores assigned by our suggested
system and the other methods. Figure 3(c) shows a very informative colonoscopy
frame, which is assigned the high score of 97 % by our method, 62 % by GLCM [7],
89 % by DWT [8]. Figure 3(d) shows a rather blurry colonoscopy frame and the
corresponding scores given by the three methods. As seen, the score assigned by
our method (49 %) describes the image quality more accurately. Finally, Fig. 3(e)
depicts a non-informative colonoscopy frame captured during wall contact, to
which our method assigns a poor score of 1 % where as DWT gives a relatively
high score of 53 %. In summary, the scores generated by our system are more
inline with that of human perception.
On a desktop computer with a 2.4 GHz quad core Intel, our MATLAB implementation runs at 10 frame/s, which outspeeds [7] performing at 6.5 frame/s.
Compared with [8] that operates at 65 frame/s, our method is signicantly
slower; however, for real-time clinical applications, our method requires a speedup of only 2.5, which is achievable using C++ Multithreaded Programming.
Discussion
The concerning issue of polyp miss-rates has been approached from dierent
angles. One can categorize such approaches in three major groups:
Computer aided diagnosis (CAD) for automatic polyp detection [1214]. These
systems are useful for detecting the polyps that appear in the videos clearly
but get overlooked due to operators eye fatigue or inattentiveness. The major
drawbacks with such CAD systems are computational complexity and high
false positive rates.
157
References
1. Siegel, R., DeSantis, C., Jemal, A.: Colorectal cancer statistics. CA: Cancer J. Clin.
64, 104117 (2014)
2. Bressler, B., Paszat, L.F., Chen, Z., Rothwell, D.M., Vinden, C., Rabeneck, L.:
Rates of new or missed colorectal cancers after colonoscopy and their risk factors:
A population-based analysis. Gastroenterology 132, 96102 (2007)
158
N. Tajbakhsh et al.
3. Van Rijn, J.C., Reitsma, J.B., Stoker, J., Bossuyt, P.M., van Deventer, S.J., Dekker,
E.: Polyp miss rate determined by tandem colonoscopy: a systematic review. Am.
J. Gastroenterol. 101, 343350 (2006)
4. Heresbach, D., Barrioz, T., Lapalus, M.G., Coumaros, D., et al.: Miss rate for
colorectal neoplastic polyps: a prospective multicenter study of back-to-back video
colonoscopies. Endoscopy 40, 284290 (2008)
5. Gelder, R.E.V., Nio, C., Florie, J., Bartelsman, J.F., et al.: Computed tomographic
colonography compared with colonoscopy in patients at increased risk for colorectal
cancer. Gastroenterology 127, 4148 (2004)
6. Filip, D., Gao, X., Angulo-Rodrguez, L., Mintchev, M.P., et al.: Colometer: a realtime quality feedback system for screening colonoscopy. World J. Gastroenterol.
WJG 18, 4270 (2012)
7. Oh, J., Hwang, S., Lee, J., Tavanapong, W., Wong, J., de Groen, P.C.: Informative
frame classication for endoscopy video. Med. Image Anal. 11, 110127 (2007)
8. Arnold, M., Ghosh, A., Lacey, G., Patchett, S., Mulcahy, H.: Indistinct frame
detection in colonoscopy videos. In: 2009 13th International Machine Vision and
Image Processing Conference, pp. 4752 (2009)
9. Park, S.Y., Sargent, D., Spoord, I., Vosburgh, K.: Colonoscopy video quality
assessment using hidden markov random elds. In: SPIE Medical Imaging, pp.
79632P79632P. International Society for Optics and Photonics (2011)
10. Criminisi, A., Shotton, J.: Decision Forests for Computer Vision and Medical Image
Analysis. Springer, New York (2013)
11. DeLong, E.R., DeLong, D.M., Clarke-Pearson, D.L.: Comparing the areas under
two or more correlated receiver operating characteristic curves: a nonparametric
approach. Biometrics 44(3), 837845 (1988)
12. Tajbakhsh, N., Gurudu, S.R., Liang, J.: A classication-enhanced vote accumulation scheme for detecting colonic polyps. In: Yoshida, H., Wareld, S., Vannier, M.W. (eds.) Abdominal Imaging 2013. LNCS, vol. 8198, pp. 5362. Springer,
Heidelberg (2013)
13. Tajbakhsh, N., Chi, C., Gurudu, S.R., Liang, J.: Automatic polyp detection from
learned boundaries. In: 2014 IEEE 10th International Symposium on Biomedical
Imaging (ISBI) (2014)
14. Bernal, J., Sanchez, J., Vilarino, F.: Towards automatic polyp detection with a
polyp appearance model. Pattern Recogn. 45, 31663182 (2012)
15. De Groen, P.C., Tavanapong, W., Oh, J., Wong, J.: Computer-aided quality control for colonoscopy: Automatic documentation of cecal intubation. Gastrointest.
Endosc. 65, AB354AB354 (2007)
16. Oh, J., Hwang, S., Cao, Y., Tavanapong, W., et al.: Measuring objective quality
of colonoscopy. IEEE Trans. Biomed. Eng. 56, 21902196 (2009)
17. Hong, D., Tavanapong, W., Wong, J., Oh, J., de Groen, P.C.: 3D reconstruction
of virtual colon structures from colonoscopy images. Comput. Med. Imag. Graph.
38, 2233 (2014)
18. Vicari, J.: Performing a quality colonoscopy: Just slow down!. Gastrointest. Endosc.
71, 787788 (2010)