Papers: ACM SIGGRAPH (2003) “Graphcut textures”

July 25th, 2003 Irfan Essa Posted in Aaron Bobick, ACM SIGGRAPH, Arno Schödl, Computational Photography and Video, Greg Turk, Papers, Vivek Kwatra No Comments »

Vivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk, Aaron Bobick (2003), “Graphcut textures: image and video synthesis using graph cuts” In ACM Transactions on Graphics (TOG), Volume 22 , Issue 3, Proceedings of ACM SIGGRAPH 2003, Pages: 277 – 286, July 2003, ISSN:0730-0301. (DOI|Paper| SIGGRAPH Video (160 MB, 50 MB) | Video Results 87 MB | Project Site)


In this paper we introduce a new algorithm for image and video texture synthesis. In our approach, patch regions from a sample image or video are transformed and copied to the output and then stitched together along optimal seams to generate a new (and typically larger) output. In contrast to other techniques, the size of the GC-TOCpatch is not chosen a-priori, but instead a graph cut technique is used to determine the optimal patch region for any given offset between the input and output texture. Unlike dynamic programming, our graph cut technique for seam optimization is applicable in any dimension. We specifically explore it in 2D and 3D to perform video texture synthesis in addition to regular image synthesis. We present approximative offset search techniques that work well in conjunction with the presented patch size optimization. We show results for synthesizing regular, random, and natural images and videos. We also demonstrate how this method can be used to interactively merge different images to generate new scenes.

AddThis Social Bookmark Button

Paper: ACM SCA (2002): “Controlled animation of video sprites”

August 1st, 2002 Irfan Essa Posted in Arno Schödl, Computational Photography and Video, Outdated, Papers, Research No Comments »

Controlled animation of video sprites

  • Arno Schödl and Irfan Essa (2002), “Controlled animation of video sprites” Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation. ACM Press, Pages: 121 – 127 July 2002, San Antonio TX, ISBN:1-58113-573-4.  (DOI|PDF|WebSite)


We introduce a new optimization algorithm for video sprites to animate hamstersrealistic-looking characters. Video sprites are animations created by rearranging recorded video frames of a moving object. Our new technique to find good frame arrangements is based on repeated partial replacements of the sequence. It allows the user to specify animations using a flexible cost function. We also show a fast technique to compute video sprite transitions and a simple algorithm to correct for perspective effects of the input footage. We use our techniques to create character animations of animals, which are difficult both to train in the real world and to animate as 3D models.

AddThis Social Bookmark Button

Paper in CVPR 2001: “Depth Layers from Occlusions”

December 10th, 2001 Irfan Essa Posted in Arno Schödl, Computational Photography and Video, PAMI/ICCV/CVPR/ECCV, Papers No Comments »

Depth Layers from Occlusions

  • A. Schödl and I. Essa (2001), “Depth layers from occlusions,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Kauai, HAWAI, US, 2001, pp. 639-644. [PDF] [WEBSITE] [DOI] [BLOG] [BIBTEX]
    @InProceedings{    2001-Schodl-DLFO,
      address  = {Kauai, HAWAI, US},
      author  = {A. Sch{\"o}dl and I. Essa},
      blog    = {},
      booktitle  = {Proceedings of IEEE Conference on Computer Vision
          and Pattern Recognition (CVPR)},
      doi    = {},
      organization  = {IEEE Computer Society},
      pages    = {639 - 644},
      pdf    = {},
      publisher  = {IEEE Computer Society Press},
      title    = {Depth layers from occlusions},
      url    = {},
      volume  = {1},
      year    = {2001}


We present a method to extract relative depth information from an uncalibrated monocular video sequence. Our method detects occlusions caused by an object moving in a static scene to infer relative depth relationships between scene parts. Our approach does not rely on any strong assumptions about the object or the scene to aid in this segmentation into layers. In general, the problem of building relative depth relationships from occlusion events is underconstrained, even in the absence of observation noise. A minimum description length algorithm is used to reliably calculate layer opacities and their depth relationships in the absence of hard constraints. Our approach extends previously published approaches that are restricted to work with a certain type of moving object or require strong image edges to allow for an a-priori segmentation of the scene. We also discuss ideas on how to extend our algorithm to make use of a richer set of observations.

AddThis Social Bookmark Button

Paper: ACM SIGGRAPH (2000) “Video textures”

August 1st, 2000 Irfan Essa Posted in ACM SIGGRAPH, Arno Schödl, Computational Photography and Video, David Salesin, Papers, Research, Rick Szeliski No Comments »


  • A. Schödl, R. Szeliski, D. H. Salesin, and I. Essa (2000), “Video textures,” in ACM SIGGRAPH Proceedings of Annual Conference on Computer graphics and interactive techniques, New York, NY, USA, 2000, pp. 489-498. [BIBTEX]
    @InProceedings{    2000-Schodl-VT,
      address  = {New York, NY, USA},
      author  = {A. Sch{\"o}dl and R. Szeliski and D. H. Salesin and
          I. Essa},
      booktitle  = {ACM SIGGRAPH Proceedings of Annual Conference on
          Computer graphics and interactive techniques},
      pages    = {489--498},
      publisher  = {ACM Press/Addison-Wesley Publishing Co.},
      title    = {Video textures},
      year    = {2000}



Still Image of a VideoTexture of 4 Fish, Plants, and Bubbles in a Fish Tank

This paper introduces a new type of medium, called a video texture, which has qualities somewhere between those of a photograph and a video. A video texture provides a continuous infinitely varying stream of images. While the individual frames of a video texture may be repeated from time to time, the video sequence as a whole is never repeated exactly. Video textures can be used in place of digital photos to infuse a static image with dynamic qualities and explicit actions. We present techniques for analyzing a video clip to extract its structure, and for synthesizing a new, similar looking video of arbitrary length. We combine video textures with view morphing techniques to obtain 3D video textures. We also introduce video-based animation, in which the synthesis of video textures can be guided by a user through high-level interactive controls. Applications of video textures and their extensions include the display of dynamic scenes on web pages, the creation of dynamic backdrops for special effects and games, and the interactive control of video-based animation.

AddThis Social Bookmark Button