Brief Introduction

With recent advances in communications, computers, and storage capacities, multimedia streams (i.e., videos, photos, music) are becoming increasingly important information sources. To deal with such enormous and diverse information, there arise challenging theoretical problems and strong industrial needs. We will preliminarily address such issues in this course. Students in the course will gain practical experiences through intensive hand-on homeworks. The topics include the following:

Machine learning techniques such as graphical models, discriminative models, clustering approaches, etc.
Multimedia (video/photo/music) feature representations
Construction of high-level indices
Content analysis and object recognition
Multimedia indexing and retrieval
Multimedia data mining
Information exploitation in social media
Summarization, personalization, and visualization of large-scale multimedia databases
Standards and applications in video, photo, and medical image databases
Benchmarks and evaluation metrics

Prerequisites: Background in image processing (or signal processing related courses), probability, and linear algebra. Experience with machine learning or statistical pattern recognition will be useful but not required.

Lecturer: Winston Hsu (office: R512, CSIE Building)

PAST Course Projects

Purpose: posting student projects in the course and recruiting project members

[Fall 2010]

雷禹恆、曾開瑜、廖航緯 - People Search beyond Text in Consumer Videos
柯政祥、張孟修、楊立德 - IMAGETUBE
鄭鉅翰、林俊廷、鄭耿維 - Tag Recommendation for image
顧廷緯、趙祈鈞、許永健 - http://2010mmai.blogspot.com/
涂宗瑋、陳冠瑋、楊智鈞 - Baseball Automatic Pitch by Pitch Making System
劉昱承 - Bridge the Multimedia Contents and the Real Objects by Object Recognition in Mobile Device
陳品周 - Event detection and highligh summarized for baseball video
王建元、張博詞、柏傑 - https://sites.google.com/site/vishintmmai/
黃彥翔、張嫚家、林士涵 - http://mmaiproposal.blogspot.com/2010/11/mmai-final-project-proposal.html
蘇小杭、高介其 - Photo Quality Assessment
盧子珣、傅承堯、李明璋 - Beauty Dictionary: Using face similarity to look up your love
吳家祥、蔡承志、洪晟峰 - Automatic interesting shots extraction by face recognition
蔡典哲、陳雍協、林霓苗 - PHOTOTYPE
林耿生、陳白駿 - Automatic Background Music Generation System
朱冠宇、李哲君、施孟甫 - WHAT FOOD IS THIS?
鄧雅文、廖文慶 - extract highlight from a given basketball game video
蘇昱銓 - Medical Image Retrieval
黃重翰、顏吟霞、林佳怡 - 求婚大作戰
楊皓中 - 談話性節目 - 人物說話片段切割與整
陳奕魁 - Naming every face in social network
羅際巧、蔡明宏、陳群元 - Secure Face Identification
郭宗倫 - the tag generator

[Fall 2009]

吳家翔、王顥學 - Paper folding assistant system
林弘偉、周瑋慈 - Search for Human Actions
陳昌毅、黃柏淳、唐玥蓮 - virtual instructor
蔡雅淳、何元臣 - Choreographer assistant system
高紹航、戴瑋彥、李佳蓉 - Image Re-Ranking Based on Image Quality
顏孝杰、張峻華、王紹丞 - http://mmai09.blogspot.com/
蘇彥禎 - Multidimensional indexing for volume rendering
唐立宇、劉怡君 - Travel Spot Recommender
薛琇文 - Audio fingerprinting and retrieval
蔡佳良 - Video Photo Montage for Music Emotions
趙偉崙 - Baseball game event detection on MLB videos
蔡旻光、黃彥達、陳柏村 - "Peeping! - Character Recognition in Movie Clips"
江欣倩 - Motion Recorder
陳則諭、曾鵬叡 - Person Filter for Variety Show
蔡淳宇、郭子豪 - Photo Slideshow
鄒奇軒 - Query by sketch
許裕明、戴佑家- Snap2read: Retrieving and segmenting your magazine within one snapshot
李政德、柯俊宇- Speaker Recognition
黃大源、林衛成、周奕宏 - 基於內容的個人相簿管理
吳冠龍 - Context Affinity Propagation

[Fall 2008]

朱慶瑜、洪韻蕙、陳怡伶 - NTU Walker
陳怡君、祁恆昱、龔建瑞 - Automatic Classification for Social Albums
呂鈺達、陳晉暉、呂哲安 - Visual Art - Peeping Montage
王政堯 - CBIR System For Clothes in Yahoo Auction
陳祺文、林芳而、黃彥傑 - Music Changes Your BORING film
謝良奇 - Annotation by Search
吳瑞欣、劉維倫 - Love Home
陳怡伶、陳思佑、帥宏翰 - Google @ home
邱孟傑、李世強 - Funny Face
李文瑜、王聖媛、曾柏儂 - http://smilelibby.blogspot.com/
郭盈希、江建興 - Query expansion by applying hierarchical k-means and LSH
陳志帆 - Object detector by using shape information
白育姍、林志宏 - http://g.csie.org/~meconin/mmai/doku.php?id=start
劉峻瑋、鄭玨玉民- dress.com
鄭安容、黃琦雅 - http://lenaqueen.blogspot.com/
林晏慶、黃鴻昇、李長曄 - http://orz4mmai.pixnet.net/blog
曾賢舜、柯佳伶 - Tour Recorder
程瀚平、胡傳甡、邱柏叡 - http://dspgg.blogspot.com/
葉容瑜、林敬棋 - Cast Indexing for Talk Show Videos by Face, Cloth, and Background
黃柏瑜、賴韻芊 - http://lovelyzoo.blogspot.com/
林彥良、邱子軒、莊舒婷 - http://mmai08.blogspot.com/
陳俊瑋、何星翰、陳俊勳 - http://jstfunny.blogspot.com/

[Fall 2007]

Wen-Kwang Tsao, Ying-Ho Liu, "Video Matching for video duplication protection"
Jen-Wei Huang, Peter Chen, Chi-Yao Tseng, "Progressive Image eXtraction And Recommendation"
Photo Kao, "Analysis of photo composition - lines and shapes"
Chun-Chao Yen, Cheng-Te Li, Chun-Min Chang, "Painting Style"
Tse-Wei Chen, "Image segmentation and Region-based Image Retrieval"
Tien-Jung Chuang, roastfish, "Semantic Photo Synthesis"
郭姿玲, "Detecting and Tracking Moving Objects in Seguences of Color Images"
徐兆良, Intersection of Image-Music Searching System (交錯式圖片-音樂搜尋系統)
Kai-Wei Chang, Cho-Jui Hsieh, "Hotter or Notte- Hot Ranker"
Tyng-Ger Li, Chui-Chiu Kung, "P2P Recommendation System."
Carson Liao, "video advertising."
Chia-Ping Chen, Yen-Yu Lin, "Multi-modal Feature Fusion for Retrieval."
吳映嫻, 鄭逸廷, "music semantic query."
張銘修, 陳鴻銘, 張炳傑, "Sheep Dog."
林厚達, 張瀠文, "face recognition in personal albums."
Chia-yu Wan, "Clustering and Organization for personal digital album."
李勁葦, 袁國訓, 陳鵬仁, "Dressing matcher."

[Spring 2007]

Pei-yu (Peggy) Chi, "Cooking Video Indexing,"
"..I'd like to make use of video analysis and indexing techniques to improve [cooking] activity recognition..."
Terry Wu and Ming-Fang (Alpha) Weng, "Emotion-based Media Player Show,"
"..A novel method is exhibited here to show a fancy media player show, integrating auditory and visual perception..."
Yi-Hsuan (Eric) Yang and 馬智釗, "What are you looking at!,"
"..knowledge about where a person is looking provides important clues which enable a better understanding of the scene..."
Ting-ting Hu, "Environment Adaptive Picture Frame,"
"..做出ㄧ個虛擬畫框，由一個display + 1~2 個camera組成，當它進入某個環境時，會對攝影到的環境畫面做分析，可能包括顏色、線條、空間複雜度、object recognition等等，透過這些分析，去找一或多幅適合這個環境的畫，當使用者掛上牆壁時，適合的畫就顯示在display上..."
Hsin-Chien Huang, "2007 IPPR Contest,"
"..Face Detection in Video..."
Su-Chen Lin, 陳泓稜, 林與絜, 宋品頡, "Personalized Beauty Detection System,"
"..build a personalized model for beauty detection...by analyzing photos of persons who are recognized as beauties by a user"
Ya-Fan Su, Yu-Ching Lin, "Who's Singing?,"
"..we're interested in is the analysis of music video (or opera) structure..."
Chun F. Hsu, "People Detection based Embedded Cameras,"
"..Detect strangers at the front door or the entrance lane in the office..."
Janet Huang, Emily Chen, Evelyn Hung, "Photo Social Network Visualization,"
"We use the photo's metadata to build a personal social network, and use a novel way to visualize the information of the photos."
謝其璋, 謝秉諺, 謝逵繹, "Singer Retrieval,"
Chun-Kang Chen, Victor Hsieh, "Visual Layers for Semantic Concept Detection,"
"...we want to separate every key-frame into foreground and background to decrease the noise [for concept detection]."
周辰威, 葉仰庭, 王奕棠, "Billiards Video Analysis,"
"Since there are lots of research for Sports Analysis, we think that billiards analysis might be a interesting topic."