Current Research Interests :

  • Image Processing, Computer Vision and Pattern Recognition

  • Image-Based Measurements and Applications


 

Ongoing Projects :

 

 

Head Pose Estimation and Face Tracking

Hysteroscopy Video Summarization and Analysis

Intelligent Transportation Systems: Vehicle Tracking and Analysis of Vehicle Behavior

Object Oriented Video Coding and Representation

Watershed Drusen Detection In Eye Fundus Images

Identification of Retinal Structures and Lesions in Color Eye Fundus Images

Color Image Segmentation

Medical Image Denoising and Enhancement

Pigmented Skin Lesions Classification using Standard Color Camera Images

Visual Information Retrieval

Medical Visual Information Exchange on the WEB

 


Head Pose Estimation and Face Tracking


PROJECT DESCRIPTION:

In this project we investigate new methods to compute the head pose in monocular images by comparing the positions of specific facial features with the positions of these facial features in multiple instances of a prior 3D face model. Given an image containing a face, we locate facial features such as nose, eyes, and mouth. Then these 2D feature locations are used as references in the comparison with the corresponding feature locations in multiple instances of our 3D face model, projected on the 2D image space. To estimate the depth of these feature points, we use the 3D spatial constraints imposed by our face model (e.g. eyes are at a certain depth with respect to the nose, and so on). The head pose is estimated by minimizing the comparison error between the face feature locations in the image and in a given instance of the face model. Our preliminary experimental results are encouraging, and suggest that our approach potentially can provide accurate results.

Some preliminary results are illustrated above: (a),(d) and (g): Original face images; (b), (e) and (h): Best matching mask instances; and (c),(f) and (i): Best matching mask instances overlayed on the faces.

We intend to evolve this work and track faces in different poses in video sequences. At this stage of our work we are investigating face segmentation methods based on the detection of skin regions in monocular images. Below, some preliminary results of a segmented head and torso are shown: (left) original video frame ; (center) detected skin regions ; (right) segmented head.

 

Hysteroscopy Video Summarization and Analysis


PROJECT DESCRIPTION:

Hysteroscopy is a surgical procedure in which a gynecologist uses a small lighted telescopic instrument called a hysteroscope to diagnose and treat many uterine disorders. A hysteroscopic examination has different phases, and usually only the relevant findings are used for the diagnosis and prognosis in fertilization studies and in the Gynecology practice. Consequently, most visual information in such videos is not relevant, and only a reduced number of frames are used. Currently, a summary of a hysteroscopic video is obtained manually during the diagnosis process, by selecting only the relevant frames of the different phases of a hysteroscopic procedure, and later the findings in these frames are reported and described in the patient records. Therefore, in this project we intend to develop a simple and accurate method to extract concise representations of hysteroscopic videos contents that could be used in the clinical practice.

Application: Given a library with thousands of diagnostic hysteroscopy videos, which are only indexed according to a patient ID and the exam date. Usually, users browse through this library to obtain answers to queries and retrieve images of submucosal myomas or recover images whose diagnosis is endometrial polyp. This work will allow to identify clinically relevant information in diagnostic hysteroscopy videos, since only portions of the recorded videos are relevant from the diagnosis/prognosis point of view. Our goal is to capture the specialist intention by tracking image points through the image sequence. We demonstrate that the resulting representation is a helpful way to organize the hysteroscopy video content, allowing specialists to perform fast browsing without introducing spurious information in the video summary. The preliminary experimental results indicate that our method produces compact video summaries (data-rate reduction around 97.5%) without discarding clinically relevant information.

 
PRELIMINARY RESULTS:

Uninteresting frame

Interesting frames provided by our method

Video segment trees computed for a particular hysteroscopy video. X-axis represents the frames in the video sequence. Red line segments represent the video segments manually selected by specialists. Notice the relevant video segments appear associated with taller trees.

 

Intelligent Transportation Systems: Vehicle Tracking and Analysis of Vehicle Behavior


 

PROJECT DESCRIPTION :  

 

Intelligent transportation systems (ITS) are systems that use advanced computer vision techniques to gather traffic monitoring information. These information can be used in surveillance, traffic control and identification of unusual vehicle behavior, among others applications. Vehicles tracking is a fundamental task in ITS, because it allows to identify the vehicle location at any time. However, the vehicle tracking process can be interrupted under different types of occlusions, like bridges, traffic signs and other vehicles on the road. We are developing a new adaptive particle filter approach to handle vehicle occlusions, and we also are working on the analysis of vehicle behavior using the obtained vehicle tracking information with promising results.

 

PRELIMINARY RESULTS:

 

Tracking results for a video sequence containing vehicle occlusions. On the left, there are vehicle tracking results under several occlusions (frames 03 (a), 35 (b) and 55 (c) of the video sequence).  Our Adaptive Particle Filter approach is able to resume the vehicle tracking process after the the vehicle becomes disoccluded. The solid/dashed line shows the vehicle tracking results obtained by our adaptive particle filter.

 

Analysis of vehicle behavior. On the top a truck changes lanes dangerously (blue ribbon). Below it, a truck is tracked even under the occlusion by a bridge (green ribbon). These figures show the vehicles path and a “stroboscopic effect” illustrating the vehicle motion and behavior.

 


Object Oriented Video Coding and Representation


PROJECT DESCRIPTION:

Object video representation provides a convenient approach for several compression and transmission tasks. However, the coding efficiency is directly related to the effectiveness of the object segmentation. In this work, we develop a video coding approach using video segmentation based on motion coherence. Long-range motion patterns are identified by computing correspondence of sparse points (i.e. particles) and the segmentation of particle trajectories using an ensemble clustering approach. Then, a dense video frame representation (i.e. a pixel-wise representation) is obtained, leading to object tunnels in the spatio-temporal domain (see figure below). Instead of motion boundaries, the segmentation is guided by the consistent motion behavior of sample points of the frames. This strategy allows to extract longer tunnels in the spatio-temporal domain. The proposed approach generates a simple scene representation, adequate for object video coding, and also delivers a more redundant and temporally persistent partition of the scene than direct video segmentation methods and motion prediction strategies.

PRELIMINARY RESULTS:

Particles

Particle trajectories segmentation

Pixel-wise segmentation and object tunnels 1 and 2 (green and blue bands, respectively)


Watershed Drusen Detection In Eye Fundus Images


PROJECT DESCRIPTION:

Age-related macular degeneration (ARMD) can evolve rapidly, and cause severe losses to the central vision of patients. An early sign of ARMD is the formation of drusen in the retina (i.e. white-yellow spots located in the center of the macula).  Therefore, the early identification of drusen in the ARMD process, and their delimitation and quantification over time, is very important for its medical treatment. This project is focused on the development of robust methods for drusen detection and segmentation, delimiting precisely individual drusen, even when the drusen spots are spatially close. We intend to improve drusen segmentation and overcome common drusen segmentation difficulties reported in the literature.


PRELIMINARY RESULTS:

Original Image

Literature

Our result

 
 

Identification of Retinal Structures and Lesions in Color Eye Fundus Images : Diabetic Macular Edema


PROJECT DESCRIPTION:

In this project, we are developing automatic methods to solve several problems related to the detection and classification of the Diabetic Macular Edema (DME) using color eye fundus images. We have developed methods to detect the optic disk rim, the fovea center, and the exudate lesions. Our experimental results are encouraging, and indicate that our approach potentially can achieve a better performance than other known methods available in the literature. Some results are illustrated in Figures 1(a)  and 1(b).


PRELIMINARY RESULTS:

 

Figure 1: A typical eye fundus image. (a) Detected optic disk rim superimposed on the original color eye fundus image. (b) Exudate lesions and polar fundus coordinates (which are centered on the fovea center) superimposed on a color eye fundus images.

 


Color Image Segmentation



PROJECT DESCRIPTION:

Image segmentation is, by definition, the problem of decomposing images into regions that are semantically uniforma (e.g. correspond to the visible objects in the scene), and is often the first important step of an image understanding system. The goal of this project is to develop image segmentation techniques that suitably separate the relevant classes of objects in a scene, based on color and spatial information.

Some preliminary results obtained with our feature space tesselation approach and posterior data clustering in joint spatio-color domain are illustrated below.


PRELIMINARY RESULTS:

Original Images:

Segmented by our method:

Segmented by Mean Shift::

PSNR: 23.80

Clusters (colors): 220

Regions: 2480

 

PSNR: 23.22

Clusters (colors): 5088

Regions: 5196

 

PSNR: 23.36

Clusters (colors): 33

Regions: 936

 

PSNR: 18.91

Clusters (colors): 49

Regions: 63

 

We are investigating the effect of shading attenuation in low level color vision, specially in face detection problems. Some preliminary results obtained with our shading attenuation method and and posterior face segmentation are illustrated below.


PRELIMINARY RESULTS:

Face segmentation examples. In the first and second columns  the original images nd their respective segmentation results are shown. In the third and fourth columns, the images after using our shading attenuation method, and their respective segmentation results are shown.

 


Medical Image Denoising and Enhancement


 

PROJECT DESCRIPTION: 

 

This project is focused on medical image noise suppression and enhancement. Different image modalities are studied and suitable denoising and enhancement methods are investigated. Two examples are illustrated below.

 

 

IMAGE DENOISING AND ENHANCEMENT IN MAMMOGRAPHY:

 

In this work, wavelet based methods are investigated. At each resolution, coefficients associated with noise are modelled by Gaussian random variables; coefficients associated with edges are modelled by Generalized Gaussian random variables, and a shrinkage function is assembled based on posterior probabilities. Given a resolution of analysis, the image denoising process is adaptive (i.e. do not require troublesome parameter adjustments), and the selection of a gain factor provides the desired detail enhancement. The enhancement function must be designed to avoid introducing artifacts in the enhancement process, which is essential in mammographic image analysis. We intend to develop techniques that are easy to use (by mammographists), and can help detecting microcalcifications and other suspicious structures in situations where their detection would be difficult otherwise.

PRELIMINARY RESULTS:

Original mammographic image

Denoised and enhanced version showing a

microcalcification cluster (middle right)

 

DENOISING AND ENHANCEMENT OF PROSTATE ULTRASOUND IMAGES:

 

In this study, we investigate the applicability of a Monte Carlo approach to despeckling transrectal ultrasound (TRUS) images of the prostate. The particularities and noise statistics of TRUS images of the prostate are incorporated into a likelihood-weighted Monte Carlo estimation scheme. Our preliminary in-silico and in-vivo experimental results are promising, which was confirmed by a clinical evaluation of the in-vivo test cases, and indicate that our method potentially can perform better than other state-of-the-art methods recently proposed.

 

This project has been developed in collaboration with Dr. Alexander Wong (U. of Toronto, Canada).

PRELIMINARY RESULTS:

Despeckled log-envelope images produced by state-of-the-art tested methods, and by the proposed

despeckling method. The image plate shows transverse transrectal ultrasound images of the prostate

with left posterior adenocarcinomas confirmed by biopsy.


Pigmented Skin Lesions Classification using Standard Color Camera Images


 

PROJECT DESCRIPTION: 

 

Pigmented skin lesions have two forms, benign (called nevi) and malignant (called melanoma). Since melanoma is one of the most dangerous cancers nowadays, there is a strong interest in digital image analysis systems to help, and maybe improve, the physician diagnosis and pre-screening. However, most of these efforts are based on dermoscopy, a noninvasive tool that improves the recognition of submacroscopic morphologic structures and vascular patterns, facilitating the use of a computer system in melanoma diagnosis, but with significant investments. Oour goal in this project is to develop a system to segment and classify skin lesions just using standard camera images (a simple color photograph of the skin lesion). Our preliminary results indicate a pre-screening accuracy of 98%, without any false negatives.

PRELIMINARY RESULTS:

    

Examples of pigmented skin lesion segmentation. In the
first and second columns, the original images and their respective
segmentation results by state-of-the-art methods. The third and fourth columns show the
resulting images after the application of our proposed shading attenuation
method, and the respective segmentation results.


Visual Information Retrieval


 

PROJECT DESCRIPTION :  

 

We are studying content-based indexing and retrieval of compressed color images, mostly for image retrieval in the World Wide Web and in medical image repositories. We are interested in multi-resolution techniques that allow user interaction, and are based on probabilistic texture models.

 

 

PRELIMINARY RESULTS:


Medical Visual Information Exchange on the WEB


PROJECT DESCRIPTION : 

The web has become such an extensive health information repository in the world that it is increasingly difficult to search for relevant medical information. Most medical information available on the web is not peer reviewed, and is retrieved imprecisely by current web search mechanisms (i.e. based on keywords). This project aims at developing a metadata model that allows to describe medical visual information (i.e. medical images) of different modalities, including their properties, components, relationships, and authorship. The model uses the web architecture and supports the international classification of diseases and related health problems (i.e. ICD-10). An RDF schema derived from this metadata model is integrated to each medical image, and specifies the semantics of each property in the image. Thus, relevant information can be extracted directly from the images, and data integrity is better preserved in the web.  

 

PRELIMINARY RESULTS:

Prototype built (MedISeek) to validate our metadata model,

and mechanism for medical visual information exchange on the web.

Authorized system users have been able to describe, store and

retrieve medical images and their associated diagnostic information.