11
Bowel Polyp Detection in Capsule Endoscopy Images with Color and Shape Features
Baopu Li and Max Q.-H. Meng
The Chinese University of Hong Kong Hong Kong, China
CONTENTS
Abstract
Capsule endoscopy (CE) has been widely applied in hospitals because it can be used to directly view the whole small intestine in the human body. However, a major drawback of this technology is the tedious review process of about 50,000 images produced in each examination To relieve physicians and provide support for their decision making, computerized detection of disease is highly desired In this chapter, we put forward a novel scheme for bowel polyp detection for CE images that integrates color and shape information, which are important visual clues for physicians An illumination-invariant color feature built upon a chromaticity histogram is suggested. Combining it with Zernike moments that are scale, translation, and rotation invariant, we exploit the integrated information as color and shape features to discriminate polyp CE images from normal ones By using a multilayer percetron neural network and support vector machine as classifiers, we perform experimental results on our collected CE data, illustrating encouraging performance of detection for polyp CE images.
11.1 Introduction
The gastrointestinal (GI) tract is a 30-foot-long structure composed of a pharynx, esophagus, stomach, small intestine, large intestine, and rectum Diseases of the GI tract, such as stomach and intestinal cancer, pose a great threat to human's health. According to a publication in Hong Kong (Hospital Authority of Hong Kong 2006), GI tract-related cancers of the colon, stomach, and rectum are ranked as the third, fourth, and fifth causes of cancer deaths, respectively, accounting for 18% of the total cancer deaths in Hong Kong in 2003 Gastroscopy and colonoscopy are the most commonly used approaches for diagnosis of GI diseases Although they can provide good images that reveal the status of the GI tract, neither can reach the small intestine. Moreover, these techniques require sedating the patient, are uncomfortable, and may include a risk of perforation Furthermore, these operations require that doctors be very skillful, concentrated, and experienced to navigate the endoscope because the GI tract is very flexible. In 2000, a new kind of GI endoscopy, that is, capsule endoscopy (CE), was developed This new endoscopy technology, developed by Given Imaging Corporation in Israel, has revolutionized the diagnosis methodology for the digestive tract because this small device can directly view the entire small intestine without pain, sedation, or air insufflation, which is a significant breakthrough.
As demonstrated in Figures 11.1 and 11.2, a CE, measuring 26 mm x 11 mm, is a pill-shaped device that consists of a short-focal-length complemen-tary metal oxide semiconductor (CMOS) camera, light source, battery, and radio transmitter A patient must fast for about 12 hours before ingesting the CE After ingestion, this small device starts to work with the peristalsis of mucosa of the GI tract and takes images while moving forward along the digestive tract Images recorded by the miniature camera are sent out wire-lessly to a special recorder attached to the waist Such a process continues for about 8 hours until CE battery life ends Finally, all of the image data in the recorder are downloaded into a personal computer or a computer workstation, and physicians can view the images and analyze potential sources of different diseases in the GI tract with the help of specific software. The diagnosis process is time consuming due to the large amount of video, so the diagnosis is not a real-time process, paving a potential way for off-line postprocessing and computer-aided diagnosis. CE was approved by the U. S. Food and Drug Administration (FDA) in 2001, and it has been reported that this novel technology has provided great value in evaluating GI bleeding, Crohn's disease, ulcers, and other diseases of the digestive tract (Adeler and Gostout 2003) Moreover, over 800,000 patients have been diagnosed using CE technology (Given Imaging Ltd. 2009).
One major issue associated with CE is that it takes a long time for physicians to evaluate the large number of images produced There are about 50,000 images per examination for one patient, and it costs an experienced clinician about 2 hours on average to review and analyze all of the video data (Adeler and Gostout 2003) In addition, abnormalities in the GI tract may be present in only one or two frames of the video, so they might be missed by physicians due to oversight Moreover, some abnormalities cannot be detected by the naked eye due to their size, texture, and distribution Furthermore, different clinicians may have different findings when viewing the same images Such problems motivate researchers to design reliable and uniform assistive approaches to relieve physicians However, such a goal is very challenging because true features associated with diseases are not exactly known or well defined. Moreover, different diseases have different symptoms in the digestive tract, and some diseases show great variations in color and shape
FIGURE 11.1
Capsule endoscopy.
FIGURE 11.2
M2A capsule endoscopy: (1) optical dome, (2) lens holder, (3) lens, (4) illuminating sources, (5) CMOS imager, (6) battery, (7) transmitter, and (8) antenna.
Due to CE's wide application, many efforts have been made to develop computer-aided detection of CE images to decrease the burden on doctors Bashar et al. (2008) proposed a method using color and texture to choose informative frames from the CE video. A novel scheme for choosing MPEG-7 visual descriptors as a feature extractor to recognize several diseases such as ulcers and bleeding in the GI tract was proposed in Coimbra and Cunha (2006). Based on this work, they proceeded to develop two approaches to segment the GI tract into four major topographic areas (Cunha et al 2008), and the first software that utilizes these approaches aiming for CE examination was also introduced in this paper A scheme using color distribution to discriminate stomach, intestine, and colon tissue in CE images was proposed by Berens, Mackiewicz, and Bell (2005). In Zabulis, Argyros, and Tsakiris (2008), the authors employed two cues, that is, lumen and illumination highlight, for navigation of active CE Recently, Bejakovic et al (2009) proposed making use of color, texture, and edge features to analyze Crohn's disease lesions from CE images Szczypinski et al (2009) suggested a novel model of deformable rings to interpret CE video, which allows a quick review of the whole video. We have investigated bleeding, ulcer, and tumor region detection for CE images in our previous works (Li and Meng 2009a, 2009b, 2009c, 2009d), which mainly concentrate on features that are suitable to describe bleeding, ulcers, and tumors in CE images
Because polyps are common in the GI tract, we focus on polyp CE image recognition in this chapter To achieve this goal, we advance a new scheme that exploits color and shape features The proposed new feature integrates a chromaticity histogram with a Zernike moment shape descriptor to differentiate between a normal CE image and a polyp image Experimental results from our present data validate this new scheme's ability to achieve performance for polyp CE image detection when using a multilayer perceptron neural network as the classifier.
The remainder of this chapter is organized as follows The color and shape features for polyp detection are discussed in detail in the following section In Section 11.3, experimental results are presented and discussed Conclusions are drawn at the end of this chapter
11.2 Color and Shape Feature Analysis
A polyp is an abnormal growth of tissue protruding from the mucous membrane. Polyps are commonly found in the intestine, stomach, and so on Intestinal polyps grow out of the lining of the small and large bowels, and they come in a variety of shapes—round, droplets, and so on. They also exhibit different colors. Figure 11.3 shows a few normal CE images, and Figure 11.4 shows some CE images with different polyps that vary in color, size, and shape (refer to color originals in Li and Meng 2009e) From these illustrations, it can be noticed that polyps show some differences in color compared to normal CE images Moreover, polyps demonstrate some specific shape features. These interesting properties motivated us to investigate the combination of color and shape features to discriminate normal CE images from polyp CE images We proposed a novel color feature based on a histogram of the chromaticity channel in hue, saturation, intensity (HSI) color space and then integrated it with a traditional but powerful shape feature, that is, Zernike moments, for CE images
FIGURE 11.3
Representative normal intestinal CE images.
FIGURE 11.4
Representative intestinal CE images with polyps.
11.2.1 Color Feature
CE images usually suffer from illumination variation due to the specific imaging circumstances such as camera motion and the rather limited range of illumination in the digestive tract Moreover, different images from different patients in the database may be obtained under different imaging conditions with a great deal of variation in lighting and so on Therefore, it may be beneficial to consider illumination variations and geometric deformation effects on the colors of CE images because colors are different when objects are viewed under different angles and different lighting conditions.
There are many techniques to solve the problem of identifying the presence of an object under varying imaging conditions (Gevers and Stokman 2004; Gevers, Voortman, and Aldershoft 2005). One kind of algorithm estimates transformations and compensates for such effects An alternative method is to obtain invariant features, namely, deriving features that are robust to different transformations. Benefits of the latter method is that it avoids expensive parameter estimations such as camera and light source calibration and so on In this chapter, we adopt the latter strategy due to this advantage. Specifically, we take into account color invariance because CE images are color images
Gevers, Weijer, and Stokman (2005) demonstrated that the following equations hold for hue and saturation in HSI color space:
(11.1)
(11.2)
where for
is the constructed variable that depends only on sensors and surface, fc (X) is the channel sensor response function, and r(X) is the surface reflectance function. They further verified that hue (H) and saturation (S) in HSI color space are invariant to viewing orientation, illumination direction, and illumination intensity The HSI color space is devised to be used intuitively in manipulating color and to approximate the manner in which humans perceive and interpret color In the human vision system, perception of an image is decomposed into luminance and chroma components, and HSI color space separates an image into intensity and chromaticity just as in human vision perception Three properties of color, that is, hue, saturation, and intensity, are defined in order to differentiate the color components HSI color space will facilitate our investigation of features for polyp image detection because we can study color features in chromaticity channels while investigating shape features in intensity channels. Moreover, the invariant property of HSI color space compared to other color models is attractive for employment as the basis for the color feature analysis
Distribution of colors in an image provides useful cues for object recognition Physicians also use color as a primary clue to conduct diagnosis for CE images (Li and Meng 2007). To represent color features, we resort to a two-dimensional (2D) histogram of HS channels in HSI color space because a histogram is robust to image scale changes, translation and rotation about the viewing axis, and partial occlusion (Swain and Ballard 1991). Because HS only represents the chromaticity information for a color image, we call this 2D histogram a chromaticity histogram. However, direct usage of a chromaticity histogram may be computationally intensive if full use of the histogram is made; for example, a quantization scheme of 180 bins in H and 50 bins in S To overcome this shortcoming, we apply a discrete cosine transform (DCT) to compress the chromaticity histogram Because the lower frequencies can represent most of the energy of an image, we truncate the higher frequency coefficients to reduce the number of features to characterize color features. Through experiments, we found that using only twenty-eight coefficients, which lie in the upper left corner of the DCT coefficients matrix, worked well for our purpose, as shown in Figure 11.5. Using these twenty-eight coefficients of the chromaticity histogram, we obtained color features for CE images that were invariant to illumination change Furthermore, the color features obtained were also robust to scale, translation, and rotations
The quantization scheme may affect the performance of such a color representation. However, because the emphasis of this chapter is not to find an optimal quantization scheme for detecting polyp CE images, this remains as one point for our future work At present, we experimentally use the quantization scheme of forty bins in H and twenty bins in S, producing satisfactory detection results
11.2.2 Shape Feature
As illustrated in Figure 11.4, the polyps in CE images also show different characteristics of shape Shape is another primary low-level image feature exploited by clinicians There are two main types of shape representation methods; namely, contour-based methods and region-based methods (Zhang and Lu 2003). Because it is very challenging to obtain an accurate and clear contour of polyps in CE images due to the complex background, in this chap-ter we turn to the region-based shape descriptor.
FIGURE 11.5
Coefficients chosen from DCT.
Taking into account the specific imaging circumstances required for CE images mentioned previously, we need a shape feature that is invariant to rotation, scale, and translation. Fortunately, Zernike moments satisfy these properties. A basis function for the Zernike moment is defined by (Teague 1980):
(11.3)
where
(11.4)
where p is the radius from (x,y) to the shape centroid, 9 is the angle between p and the x-axis. n,m are integers that satisfy the condition n-|m| = even, | m |< n. A Zernike moment can be then defined as:
(11.5)
where * represents complex conjugate. Anm is a complex number and the magnitude of Anm is rotation invariant. The unit disk can be centered on the center mass of an image, which enables both scale and translation invariance of the moments (Ye and Peng 2002). Zernike moment invariance can be constructed in an arbitrary order. In our implementation, we obtained different orders of Zernike moments, that is, tenth-order and fifth-order, directly on the I channel in HSI color space, thus obtaining the shape feature for each CE image
11.3 Experimental Results
Gastroenterologists selected a data set composed of 300 representative polyp (150) and normal (150) CE images from two patients' video data. The original images were manually labeled to provide the ground truth A CE image containing any polyp region is labeled as a positive sample; otherwise, it is labeled as a negative sample. In order to prevent overfitting of the classifica-tion results, we exploited threefold cross-validation for all of our classification experiments To demonstrate the performance of the proposed features, we compare the proposed scheme with color wavelet covariance (CWC) features used in Karkanis et al. (2003) to detect tumors in traditional endoscopic images CWC features are new techniques to represent color features that are built upon the covariance of second-order textural measures in the wavelet domain of color channels of images
We also exploited a multilayer perceptron (MLP) neural network and support vector machines (SVM) to find a better classifier for our work. A three-layer MLP with two nonlinear outputs was employed in the experiments The number of input nodes of the MLP depends on the number of input features, and the number of epochs for training the MLP was set to 5,000 Because the number of hidden nodes has a strong impact on the final classification results, several variations on the number of hidden layer neurons, ranging from five to fifty, with five-node increments, were carried out. For SVM implementation, we referred to the work of Chang and Lin (2001) The radial basis function was found to be the kernel function that yielded the best classification performance in our experiments. The highest classification accuracy of an SVM from seven different sets of the key parameters in an SVM, that is, the penalty parameter and the kernel parameter, was chosen as the classification performance.
Classification of CE images using MLP or SVM is measured by accuracy, specificity, and sensitivity, which are widely employed to evaluate the per-formance of classification. Some definitions are as follows:
Accuracy (11.6)
Specificity redictions (11.7)
Sensitivity (11.8)
We performed experiments using different orders of Zernike moments, that is, fifth- and tenth-order, and the number of these moments corresponds to twelve and thirty-six, respectively, so the number of whole features is forty and sixty-four, respectively The average recognition results of the proposed features using MLP and SVM with fifth-order and tenth-order Zernike moments are shown in Tables 11.1 and 11.2, respectively. The classification results of CWC features are demonstrated in Table 11.3.
From these three tables, it can be noticed that the proposed color and shape feature shows much better performance for polyp recognition from CE images compared to the CWC method when choosing MLP or SVM as the classifier. This is expected because the proposed color and shape features hybrid color invariance with shape invariance Moreover, the proposed feature with fifth-order Zernike moments and a chromaticity histogram shows an encouraging recognition accuracy of 94.20% when using MLP as the classifier, together with a promising specificity (93.33%) and sensitivity (95 07%) An unexpected result is that the performance of an SVM is inferior to that of an MLP for the proposed color and shape features Such a result may be due to the fact that the parameters used in SVM experiments are not optimized.
TABLE 11.1
Classification Results of the Proposed Algorithm with Fifth-Order Zernike Moment and the Chromaticity Histogram (%)
MLP |
SVM |
|
Accuracy |
94.20 |
85.33 |
Specificity |
93.33 |
80.67 |
Sensitivity |
95.07 |
90.00 |
TABLE 11.2
Classification Results of the Proposed Algorithm with Tenth-Order Zernike Moment and the Chromaticity Histogram (%)
MLP |
SVM |
|
|---|---|---|
Accuracy |
93.47 |
89.00 |
Specificity |
93.53 |
81.34 |
Sensitivity |
93.42 |
96.67 |
TABLE 11.3
Classification Results of CWC Method (%)
MLP |
SVM |
|
|---|---|---|
Accuracy |
58.67 |
70.50 |
Specificity |
63.67 |
63.67 |
Sensitivity |
53.67 |
76.33 |
11.4 Conclusions
In this chapter we have presented a new scheme that uses color and shape features to detect intestinal polyps from CE images The novel features combine the advantages of a chromaticity histogram and Zernike moments in HSI color space, leading to color invariance and shape invariance Thus, the proposed feature shows greater discriminative ability for polyp detection in CE images compared to a CWC method Experiments with our present CE images show that this method is promising for detection of polyp images Future work will be directed to collecting more patients' data in order to test the robustness of the proposed scheme Moreover, a suitable quantization approach for HS histograms is worth further investigation to achieve better performance
Acknowledgment
This work was supported by SHIAE project #8115021 of the Shun Hing Institute of Advanced Engineering of The Chinese University of Hong Kong, awarded to Max Meng
References
Adeler, D.G., and Gostout, C.J. 2003. Wireless capsule endoscopy. Hospital Physician, 39(5): 14-22.
Bashar, M.K., Mori, K., Suenaga, Y., Kitasaka, T., and Mekada, Y. 2008. Detecting informative frames from wireless capsule endoscopic video using color and texture features. Paper read at the 11th Medical Image Computing and Computer-Assisted Intervention, New York, September 6-10, 2002.
Bejakovic, S., Kumar, R., Dassopoulos, T., Mullin, G., and Hager, G. 2009. Analysis of Crohn's disease lesions in capsule endoscopy images. Paper read at the IEEE International Conference on Robotics and Automation, Kobe, Japan, May 12-17, 2009
Berens, J., Mackiewicz, M., and Bell, D. 2005. Stomach, intestine and colon tissue discriminators for wireless capsule endoscopy images. Proceedings of SPIE on Medical Imaging, 5747: 283-290.
Chang, C. -C. and Lin, C. -J. 2001. LIBSVM: A library for support vector machines. Available at http://www. csie. ntu. edu. tw/cjlin/libsvm. Accessed August 18, 2010.
Coimbra, M. T. and Cunha, J. P. S. 2006. MPEG-7 visual descriptors—Contributions for automated feature extraction in capsule endoscopy. IEEE Transactions on Circuits and Systems for Video Technology, 16(5): 628-637.
Cunha, J S, Coimbra, M., Campos, P., and Soares, J M 2008 Automated topographic segmentation and transit time estimation in endoscopic capsule exams IEEE Transactions on Medical Imaging, 27(1): 19-27
Gevers, T and Stokman, H 2004 Robust histogram construction from color invariants for object recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(1): 113-118
Gevers, T., Voortman, S., and Aldershoft, F 2005 Color feature detection and classification by learning. Proceedings of the IEEE International Conference on Image Processing, 2: 714-717.
Gevers, T., Weijer, J.V. D., and Stokman, H. 2006. Color Image Processing: Emerging Applications . Boca Raton, FL: CRC Press.
Given Imaging Ltd. 2009. Corporate overview. Available at http://www. givenimag-ing com Accessed July 12, 2010
Hospital Authority of Hong Kong 2006 Highlights on cancer statistics 2003 Hong Kong Cancer Registry, 1-29.
Karkanis, S A, Iakovidis, D K, Maroulis, D E, and Korras, D A 2003 Computer-aided tumor detection in endoscopic video using color wavelet features IEEE Transactions on Information Technology in Biomedicine, 7(3): 141-152.
Li, B. and Meng, M. Q. -H. 2007. Analysis of wireless capsule endoscopy images using chromaticity moments Paper read at the IEEE International Conference on Robotics and Biomimetics, Sanya,China, December 15-18, 2007.
Li, B. and Meng, M. Q. -H. 2009a. Computer aided detection of bleeding regions in capsule endoscopy images. IEEE Transactions on Biomedical Engineering, 56(4): 1032-1039.
Li, B and Meng, M Q -H 2009b Computer-based detection of bleeding and ulcer in wireless capsule endoscopy images by chromaticity moments Computers in Biology and Medicine, 39(2): 141-147
Li, B and Meng, M Q -H 2009c Small bowel tumor detection for wireless capsule endoscopy images using textural features and support vector machine Paper read at the IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO. October 11-15, 2009.
Li, B and Meng, M Q -H 2009d Texture analysis for ulcer detection in capsule endos-copy images. Image and Vision Computing, 27(9): 1336-1342.
Swain, M. S. and Ballard, D. H. 1991. Color indexing. International Journal of Computer Vision, 7(1): 11-32
Szczypinski, P M, Sriram, R D, Sriram, P V J, and Reddy, D N 2009 A model of deformable rings for interpretation of wireless capsule endoscopic videos Medical Image Analysis, 13(4): 312-324
Teague, M R 1980 Image analysis via the general theory of moments Journal of the Optical Society of America, 70(8): 920-930.
Ye, B and Peng, J 2002 Invariance analysis of improved Zernike moments Journal of Optics A: Pure and Applied Optics, 4: 606-614.
Zabulis, X., Argyros, A. A., and Tsakiris, P. D. 2008. Lumen detection for capsule endoscopy Paper read at the IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice, France, September 22-26, 2008.
Zhang, D and Lu, G 2003 Evaluation of MPEG-7 shape descriptors against other shape descriptors Multimedia Systems, 9: 15-30.