Thesis: Antonio Haro’s PhD (2003): “Example based processing for image and video synthesis”

November 6th, 2003 Irfan Essa Posted in Antonio Haro, Computational Photography and Video, PhD, Research, Thesis No Comments »

Antonio Haro (2003) “Example based processing for image and video synthesis” PhD Thesis, Georgia Institute of Technology, College of Computing, Atlanta, GA, [URI] [PDF] (Advisor: Irfan Essa)


The example based processing problem can be expressed as: “Given an example of an image or video before and after processing, apply a similar processing to a new image or video”. Our thesis is that there are some problems where a single general algorithm can be used to create varieties of outputs, solely by presenting examples of what is desired to the algorithm. This is valuable if the algorithm to produce the output is non-obvious, e.g. an algorithm to emulate an example painting’s style. We limit our investigations to example based processing of images, video, and 3D models as these data types are easy to acquire and experiment with.

We represent this problem first as a texture synthesis influenced sampling problem, where the idea is to form feature vectors representative of the data and then sample them coherently to synthesize a plausible output for the new image or video. Grounding the problem in this manner is useful as both problems involve learning the structure of training data under some assumptions to sample it properly. We then reduce the problem to a labeling problem to perform example based processing in a more generalized and principled manner than earlier techniques. This allows us to perform a different estimation of what the output should be by approximating the optimal (and possibly not known) solution through a different approach.

AddThis Social Bookmark Button

Paper:ICPR (2002) “Learning video processing by example”

August 11th, 2002 Irfan Essa Posted in Antonio Haro, Collaborators, Computational Photography and Video, Machine Learning, PAMI/ICCV/CVPR/ECCV No Comments »

Haro, A. Essa, I. (2002), “Learning video processing by example” In Proceedings of 16th International Conference on Pattern Recognition, 2002, 11-15 Aug. 2002 Volume: 1, page(s): 487 – 491 vol.1, Number of Pages: 4 vol.(xxix 834 xxxv 1116 xxxiii 1068 xxv 418), ISSN: 1051-4651, ISBN: 0-7695-1695-X, [Digital Object Identifier: 10.1109/ICPR.2002.1044771][IEEEXplore#]


We present an algorithm that approximates the output of an arbitrary video processing algorithm based on a pair of input and output exemplars. Our algorithm relies on learning the mapping between the input and output exemplars to model the processing that has taken place. We approximate the processing by observing that pixel neighborhoods similar in appearance and motion to those in the exemplar input should result in neighborhoods similar to the exemplar output. Since there are not many pixel neighborhoods in the exemplars, we use techniques from texture synthesis to generalize the output of neighborhoods not observed in the exemplars. The same algorithm is used to learn such processing as motion blur color correction, and painting.

AddThis Social Bookmark Button

Paper: EGSR (2001) “Real-time Photo-Realistic Physically Based Rendering of Fine Scale Human Skin Structure”

October 1st, 2001 Irfan Essa Posted in Antonio Haro, Computational Photography and Video, Papers, Research No Comments »

Real-time Photo-Realistic Physically Based Rendering of Fine Scale Human Skin Structure

AddThis Social Bookmark Button