Fall 2016 Teaching

August 10th, 2016 Irfan Essa Posted in Computational Photography, Computer Vision No Comments »

My teaching activities for Fall 2016 areBB1162B4-4F87-480C-A850-00C54FAA0E21

AddThis Social Bookmark Button

Spring 2016 Teaching

January 10th, 2016 Irfan Essa Posted in Computational Photography, Computational Photography and Video, Computer Vision, Computer Vision No Comments »

My teaching activities for Spring 2016 areBB1162B4-4F87-480C-A850-00C54FAA0E21

AddThis Social Bookmark Button

Fall 2015 Teaching: Computer Vision and Computational Photography for Online MSCS.

August 15th, 2015 Irfan Essa Posted in Aaron Bobick, Computational Photography, Computer Vision No Comments »

In fall 2015 fall term, I am teaching two classes. Both for the Georgia Tech’s Online MSCS program.Cursor_and_CS6475___Computational_Photography___Georgia_Tech

AddThis Social Bookmark Button

Computational Photography (CS 6475) for Georgia Tech’s Online MSCS Program (via Udacity)

January 5th, 2015 Irfan Essa Posted in Computational Photography, Computational Photography and Video No Comments »

Today, the Inaugural Offering of the Computational Photography (CS 6475) was launched for the Georgia Tech’s Online MSCS Program using the Udacity platform.

Course Description

CS 6475* (3-0-3): Computational Photography – (Instructor: Irfan Essa) – This class explores how computation impacts the entire workflow of photography, which is traditionally aimed at capturing light from a (3D) scene to form an (2D) image. A detailed study of the perceptual, technical and computational aspects of forming pictures, and more precisely the capture and depiction of reality on a (mostly 2D) medium of images is undertaken over the entire term. The scientific, perceptual, and artistic principles behind image-making will be emphasized, especially as impacted and changed by computation. Topics include the relationship between pictorial techniques and the human visual system; intrinsic limitations of 2D representations and their possible compensations; and technical issues involving capturing light to form images. Technical aspects of image capture and rendering, and exploration of how such a medium can be used to its maximum potential, will be examined. New forms of cameras and imaging paradigms will be introduced. Students will undertake a hand-on approach over the entire term using computation techniques, merged with digital imaging processes to produce photographic artifacts.

DO NOTE that there are programming assignments in this class, and working knowledge of Linear Algebra, Calculus, Probability, and Programming in C++/Python/Matlab/Java will be required. OpenCV OR Matlab are used in this class as appropriate. More information on this class at Computational Photography Class Website.

Video Preview

AddThis Social Bookmark Button

At ICVSS (International Computer Vision Summer School) 2013, in Calabria, ITALY (July 2013)

July 11th, 2013 Irfan Essa Posted in Computational Photography, Computational Photography and Video, Daniel Castro, Matthias Grundmann, Presentations, S. Hussain Raza, Vivek Kwatra No Comments »

Teaching at the ICVSS 2013, in Calabria, Italy, July 2013 (Programme)

Computational Video: Post-processing Methods for Stabilization, Retargeting and Segmentation

Irfan Essa
(This work in collaboration with
Matthias Grundmann, Daniel Castro, Vivek Kwatra, Mei Han, S. Hussian Raza).

Abstract

We address a variety of challenges for analysis and enhancement of Computational Video. We present novel post-processing methods to bridge the difference between professional and casually shot videos mostly seen on online sites. Our research presents solutions to three well-defined problems: (1) Video stabilization and rolling shutter removal in casually-shot, uncalibrated videos; (2) Content-aware video retargeting; and (3) spatio-temporal video segmentation to enable efficient video annotation. We showcase several real-world applications building on these techniques.

We start by proposing a novel algorithm for video stabilization that generates stabilized videos by employing L1-optimal camera paths to remove undesirable motions. We compute camera paths that are optimally partitioned into con- stant, linear and parabolic segments mimicking the camera motions employed by professional cinematographers. To achieve this, we propose a linear program- ming framework to minimize the first, second, and third derivatives of the result- ing camera path. Our method allows for video stabilization beyond conventional filtering, that only suppresses high frequency jitter. An additional challenge in videos shot from mobile phones are rolling shutter distortions. Modern CMOS cameras capture the frame one scanline at a time, which results in non-rigid image distortions such as shear and wobble. We propose a solution based on a novel mixture model of homographies parametrized by scanline blocks to correct these rolling shutter distortions. Our method does not rely on a-priori knowl- edge of the readout time nor requires prior camera calibration. Our novel video stabilization and calibration free rolling shutter removal have been deployed on YouTube where they have successfully stabilized millions of videos. We also discuss several extensions to the stabilization algorithm and present technical details behind the widely used YouTube Video Stabilizer.

We address the challenge of changing the aspect ratio of videos, by proposing algorithms that retarget videos to fit the form factor of a given device without stretching or letter-boxing. Our approaches use all of the screens pixels, while striving to deliver as much video-content of the original as possible. First, we introduce a new algorithm that uses discontinuous seam-carving in both space and time for resizing videos. Our algorithm relies on a novel appearance-based temporal coherence formulation that allows for frame-by-frame processing and results in temporally discontinuous seams, as opposed to geometrically smooth and continuous seams. Second, we present a technique, that builds on the above mentioned video stabilization approach. We effectively automate classical pan and scan techniques by smoothly guiding a virtual crop window via saliency constraints.

Finally, we introduce an efficient and scalable technique for spatio-temporal segmentation of long video sequences using a hierarchical graph-based algorithm. We begin by over-segmenting a volumetric video graph into space-time regions grouped by appearance. We then construct a region graph over the ob- tained segmentation and iteratively repeat this process over multiple levels to create a tree of spatio-temporal segmentations. This hierarchical approach gen- erates high quality segmentations, and allows subsequent applications to choose from varying levels of granularity. We demonstrate the use of spatio-temporal segmentation as users interact with the video, enabling efficient annotation of objects within the video.

Part of this talks will will expose attendees to use the Video Stabilizer on YouTube and the video segmentation system at videosegmentation.com. Please find appropriate videos to test the systems.

Part of the work described above was done at Google, where Matthias Grundmann, Vivek Kwatra and Mei Han are, and Professor Essa is working as a Consultant. Part of the work were efforts of research by Matthias Grundmann, Daniel Castro and S. Hussain Raza, as part of their research efforts as students at GA Tech.

AddThis Social Bookmark Button