You are here: Research

Image Segmentation for 2D/2D Series Sonograms

 

My master thesis work mainly focused on the image segmenation problem to delineate the boundary of breast lesion in 2D and 2D series ultrasound images. The difficulty of delineating the breast lesion boundary might consist in that there is almost no general shape/texture prior of breast lesion exploitable in ultrasound images. Without any available prior to guide/constrain the segmentation task, we proposed data-driven approaches, which all base on watershed transformation to attack this difficult problem. In our approahes, the concrete information of edges and regions provided by watershed transformation can substantially promote the efficacy and efficeincy.

2D Segmentation:

To outline the contours of breast lesions in 2D ultrasound images, the segmentation job is realized in two phases. The first phase is to partition image/Region of Interest (ROI) into several prominent components. The prominent components here is defined as homogenious regions with perceivable boundaries. The second phase conducts the edge grouping task upon the result of first phase. The feature utilized for edge grouping is the bilater consistence along the boundary of the object of interest. At the end, our algorithm proposes five boundary candidates for user selection. The following figures demonstrate the every significant steps of our 2D segmentation method. The 2D segmentation method had been tested on 300 or more breast ultrasound images by comparing four sets of manual delineations. More results of our 2D segmentation algorithm can be found in my thesis defense slides and the appendix of my thesis.

Benign case:


 Malignant case:
 

Note: The edge segment emphasized ellipsoid in Prominent Component is the initial edge for the second phase.

2D Series Segmentation:

As for 2D series segmentation, the derived boundary in each slice should satisfy the contextual coherence. To attend this goal, we formulated the boundary derivation in every slice as a watershed-based two region compeition scheme within the Maximum A Posteriori (MAP) framework. The posterior probability to be maximized is further decomposed into three components, i.e., region appearance probability model, contour model, and prior model. The region appearance probability model describes the texture property for the object- and background-regions. The contour model characterizes the edge properties, i.e., the contextual coherence and the boundary salience, of the contour in-between the two competing regions, while the prior model regularizes the labeling for each catchment basin. The opmization is sough with the Expectation-Maximization (EM) fashion. The 2D series segmentation algorithm had been tested on 10 series by also comparing four sets of manual delineations. The follwing figures compare our results with manual outlines and the result of level set (Chan and Vese). The first column shows the original clips and the second, third, and fourth columns demonstrate the manual delineations, levet set results, and our results, respectively. More details of 2D series segmentation algorithm can be found in our 2007 UMB paper.

   

Related works:

  1. J. Z. Cheng, C. M. Chen, Y. H. Chou, C. S. K. Chen, C. M. Tiu, K. W. Chen. “Cell-based Two-region Competition Algorithm with A MAP Framework for Boundary Delineation of A Series of 2D Ultrasound Images,” Ultrasound in Medicine and Biology, 2007. To Appear.
  2. C. M. Chen, Y. H. Chou, C. S. K. Chen, J. Z. Cheng, Y. F. Ou, F. C. Yeh, K. W. Chen. "Cell Competition Algorithm: A New Segmentation Algorithm for Multiple Objects with Irregular Boundaries in Ultrasound Images," Ultrasound in Medicine and Biology, vol. 31, no. 12, pp. 1647-1664, 2005.
  3. J. Z. Cheng. Cell-Based Image Segmentation for 2D and 2D series ultrasound images," Master thesis, Insitute of Biomedical Engineering, National Taiwan University, 2007.

Future Work:

In the future, we are looking for a seamless model to fuse the region and edge clues provided by watershed trasnformation. Aslo, we would like to apply graph-cut method upon the watershed result to see if there will be more convincing performance than pixel-/voxel-based graph-cut approach.


Motion Compensation for Multimodal Image Registration

Registering images  from different modalities is valuable in the applications like Computer Aided Surgery. It can help to fuse the complementary information from images of different modalities and update the anatomical information during the surgical operation. However, for the operation on soft tissue, like liver, the rigid registration might not be practical for that the object of interest might deform dramactically. Also, the performance of registration result might also be disturbed by the motion of the subject, e.g., respiratory motion, heart beating, etc., during the acquisition. In the first step, we would like to detect the respiratory cycles from abdominal 2D+t ultrasound image series to compensate the image registration. 

Reference:

  1. W. Wein, A. Khamene, D. A. Clevert, O. Kutter, N. Navab. "Simulation and Fully Automatic Multimodal Registration of Medical Ultrasound," Medical Image Computing and Computer-Assisted Intervention (MICCAI 2007), Brisbane, Australia, October 2007.
  2. W. Wein, B. Röper, N. Navab. "Integrating diagnostic B-mode ultrasonography into CT-based radiation treatment planning," IEEE Transactions on Medical Imaging, 26(6): pp. 866–879, 2007.
  3. M. Nakamoto, et al. "Recovery of respiratory motion and deformation of the liver using laparoscopic freehand 3D ultrasound system," Medical Image Analysis, 11(5): pp. 429-442, 2007.
  4. J. M. Blackall, G. P. Penney, A. P. King, D. J. Hawkes. "Alignment of sparse freehand 3-D ultrasound with preoperative images of the liver using models of respiratory motion and deformation," IEEE TMI, 24(11), pp. 1405-1416, 2005.
  5. G. P. Penney, J. M. Blackall, M. S. Hamady, T. Sabharwal, A. Adam, D. J. Hawkes. "Registration of freehand 3D ultrasound and magnetic resonance liver images," Medical Image Analysis, 8: pp. 81-91, 2004.

Special Interest Reference

  1. S. C. Zhu, A. Yullie. "Region competition: unifying snakes, region growing, and Bayes/MDL for multiband image egmentation," IEEE TPAMI, 18(9): pp. 884-900, 1998.
  2. X. Han, C. Xu, J. L. Prince. "A topology preserving level set method for geometric deformable models," IEEE TPAMI, 25(6): pp. 755-768, 2003.
  3. Z. Tu, X. Chen, A. Yullie, S. C. Zhu. "Image Parsing: Unifying Segmentation, Detection, and Recognition." IJCV, 63(2): pp. 113-140, 2005.
  4. L. Fei-Fei, R. Fergus, P. Perona. "One-Shot learning of object categories," IEEE TPAMI, 28(4): pp. 594-611, 2006.
  5. L. Vincent, P. Soille. "Watershed in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations," IEEE TPAMI, 13(6): pp. 583-598, 1991.
  6. M. Isard, A. Blake. "CONDENSATION—Conditional Density Propagation for Visual Tracking," IJCV, 29(1): pp. 5-28, 1998.
  7. G. Jacob, J. A. Noble, C. Behrenbruch, A. D. Kelion, A. P.  Banning. "A shape-space-based approach to tracking myocardial borders andquantifying regional left-ventricular function applied inechocardiography," IEEE TMI, 21(3): pp. 226-238, 2002.
  8. A. Chakraborty, J. S. Duncan. "Game-theoretic integration for image segmentation," IEEE TPAMI, 21(1): pp. 12-30, 1999.
  9. X. S. Zhou, A. Gupta, D. Comaniciu. "An information fusion framework for robust shape tracking," IEEE TPAMI, 27(1): pp. 115-129, 2005.