1932

Abstract

From mainstream media outlets to social media and everything in between, doctored photographs are appearing with growing frequency and sophistication. The resulting lack of trust is impacting law enforcement, national security, the media, e-commerce, and more. While some types of manipulations can be detected with a careful visual examination, our visual system seems unable to reliably detect other types of manipulations. The field of image forensics has emerged to help return some trust in photography. I describe the perceptual limits of detecting manipulated images, as well as representative examples of computational techniques for authenticating images.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-vision-091718-014827
2019-09-15
2024-03-28
Loading full text...

Full text loading...

/deliver/fulltext/vision/5/1/annurev-vision-091718-014827.html?itemId=/content/journals/10.1146/annurev-vision-091718-014827&mimeType=html&fmt=ahah

Literature Cited

  1. Adelson EH 2000. Lightness perception and lightness illusions. The New Cognitive Neurosciences ed. MS Gazzaniga 339–51 Cambridge, MA: MIT Press 2nd ed.
    [Google Scholar]
  2. Ashcroft v. Free Speech Coalition, 535 US 234 (2002)
  3. Bertamini M, Parks TE 2005. On what people know about images on mirrors. Cognition 98:85–104
    [Google Scholar]
  4. Bertamini M, Spooner A, Hecht H 2003. Predicting and perceiving reflections in mirrors. J. Exp. Psychol. Hum. Percept. Perform. 39:982–1002
    [Google Scholar]
  5. Bianchi T, Piva A 2011. Analysis of non-aligned double JPEG artifacts for the localization of image forgeries. Proceedings of the IEEE Workshop on Information Forensics and Security1–6 Piscataway, NJ: IEEE
    [Google Scholar]
  6. Bianchi T, Piva A 2012. Image forgery localization via block-grained analysis of JPEG artifacts. IEEE Trans. Inform. Forensics Secur. 7:31003–17
    [Google Scholar]
  7. Bravo M, Farid H 2001. Texture perception on folded surfaces. Perception 30:7819–32
    [Google Scholar]
  8. Cao H, Kot AC 2009. Accurate detection of demosaicing regularity for digital image forensics. IEEE Trans. Inform. Forensics Secur. 4:4899–910
    [Google Scholar]
  9. Carvalho T, Farid H, Kee E 2015. Exposing photo manipulation from user-guided 3-D lighting analysis. Proceedings of the SPIE Symposium on Electronic Imaging, art. 940902. Bellingham, WA: SPIE
    [Google Scholar]
  10. Casati R 2008. The copycat solution to the shadow correspondence problem. Perception 37:4495–503
    [Google Scholar]
  11. Cavanagh P 2005. The artist as neuroscientist. Nature 434:7031301–7
    [Google Scholar]
  12. Cavanagh P, Wang D, Chao J 2008. Reflections in art. Spatial Vis. 21:3–5261–70
    [Google Scholar]
  13. Chan C, Ginosar S, Zhou T, Efros AA 2018. Everybody dance now. arXiv:1808.07371 [cs.GR]
    [Google Scholar]
  14. Chen W, Shi YQ, Xuan G 2007. Identifying computer graphics using HSV color model. Proceedings of the IEEE International Conference on Multimedia and Expo1123–26 Piscataway, NJ: IEEE
    [Google Scholar]
  15. Conotter V, Bodnari E, Giulia B, Farid H 2014. Physiologically-based detection of computer generated faces in video. Proceedings of the IEEE International Conference on Image Processing248–52 Piscataway, NJ: IEEE
    [Google Scholar]
  16. Dang-Nguyen D-T, Boato G, De Natale FGB 2012a. Discrimination between computer generated and natural human faces based on asymmetry information. Proceedings of the 20th IEEE European Conference on Signal Processing1234–38 Piscataway, NJ: IEEE
    [Google Scholar]
  17. Dang-Nguyen D-T, Boato G, De Natale FGB 2012b. Identify computer generated characters by analysing facial expressions variation. Proceedings of the IEEE Workshop on Information Forensics and Security252–57 Piscataway, NJ: IEEE
    [Google Scholar]
  18. Dehnie S, Sencar H, Memon S 2006. Digital image forensics for identifying computer generated and digital camera images. Proceedings of the IEEE International Conference on Image Processing2313–16 Piscataway, NJ: IEEE
    [Google Scholar]
  19. Dempster A, Laird N, Rubin D 1977. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. 99:11–38
    [Google Scholar]
  20. Dirik AE, Bayram S, Sencar HT, Memon N 2007. New features to identify computer generated images. Proceedings of the IEEE International Conference on Image Processing4–433 Piscataway, NJ: IEEE
    [Google Scholar]
  21. Fan S, Ng T-T, Herberg JS, Koenig BL, Xin S 2012. Real or fake? Human judgments about photographs and computer-generated images of faces. SIGGRAPH Asia 2012 Technical Briefs New York: ACM
    [Google Scholar]
  22. Farid H 2009. Exposing digital forgeries from JPEG ghosts. IEEE Trans. Inform. Forensics Secur. 4:1154–60
    [Google Scholar]
  23. Farid H 2016. Photo Forensics Cambridge, MA: MIT Press
  24. Farid H 2019. Photo Authentication Cambridge, MA: MIT Press
  25. Farid H, Bravo MJ 2007. Photorealistic rendering: How realistic is it. J. Vis. 7(9)766
    [Google Scholar]
  26. Farid H, Bravo MJ 2010. Image forensic analyses that elude the human visual system. Proceedings of the Conference Media Forensics and Security II, Vol. 7541, art. 754106. Bellingham, WA: SPIE
    [Google Scholar]
  27. Farid H, Bravo MJ 2012. Perceptual discrimination of computer generated and photographic faces. Digital Investig. 8:226–35
    [Google Scholar]
  28. Ferrara P, Bianchi T, Rosa AD, Piva A 2012. Image forgery localization via fine-grained analysis of CFA artifacts. IEEE Trans. Inform. Forensics Secur. 7:51566–77
    [Google Scholar]
  29. Gallagher AC, Chen T 2008. Image authentication by detecting traces of demosaicing. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops1–8 Piscataway, NJ: IEEE
    [Google Scholar]
  30. Gloe T 2012. Forensic analysis of ordered data structures on the example of JPEG files. Proceedings of the IEEE Workshop on Information Forensics and Security139–44 Piscataway, NJ: IEEE
    [Google Scholar]
  31. Green D, Sweats J 1966. Signal Detection Theory and Psychophysics Newport Beach, CA: Peninsula Publ.
  32. Holmes O, Banks MS, Farid H 2016. Assessing and improving the identification of computer-generated portraits. ACM Trans. Appl. Percept. 13:27
    [Google Scholar]
  33. Huang F, Huang J, Shi YQ 2010. Detecting double JPEG compression with the same quantization matrix. IEEE Trans. Inform. Forensics Secur. 5:4848–56
    [Google Scholar]
  34. Ittelson W, Mowafy L, Magid D 1991. The perception of mirror-reflected objects. Perception 20:567–84
    [Google Scholar]
  35. Jacobson J, Werner S 2004. Why cast shadows are expendable: insensitivity of human observers and the inherent ambiguity of cast shadows in pictorial art. Perception 33:111369–83
    [Google Scholar]
  36. Johnson MK, Farid H 2005. Exposing digital forgeries by detecting inconsistencies in lighting. Proceedings of the ACM Multimedia and Security Workshop1–10 New York: ACM
    [Google Scholar]
  37. Johnson MK, Farid H 2007. Exposing digital forgeries in complex lighting environments. IEEE Trans. Inform. Forensics Secur. 3:2450–61
    [Google Scholar]
  38. Kee E, Farid H 2010a. Digital image authentication from thumbnails. Proceedings of the SPIE Symposium on Electronic Imaging, art. 75410E. Bellingham, WA: SPIE
    [Google Scholar]
  39. Kee E, Farid H 2010b. Exposing digital forgeries from 3-D lighting environments. Proceedings of the IEEE Workshop on Information Forensics and Security1–6 Piscataway, NJ: IEEE
    [Google Scholar]
  40. Kee E, Johnson M, Farid H 2011. Digital image authentication from JPEG headers. IEEE Trans. Inform. Forensics Secur. 7:31066–75
    [Google Scholar]
  41. Kee E, O’Brien J, Farid H 2014. Exposing photo manipulation from shading and shadows. ACM Trans. Graph. 33:5165
    [Google Scholar]
  42. Khanna N, Chiu GTC, Allebach JP, Delp EJ 2008. Forensic techniques for classifying scanner, computer generated and digital camera images. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing pp. 1653–56 Piscataway, NJ: IEEE
    [Google Scholar]
  43. Kim H, Garrido P, Tewari A, Xu W, Thies J et al. 2018. Deep video portraits. ACM Trans. Graph. 37(4):163
    [Google Scholar]
  44. Kirchner M 2010. Efficient estimation of CFA pattern configuration in digital camera images. Proceedings of the SPIE Conference on Media Forensics and Security , art. 754111. Bellingham, WA:: SPIE
    [Google Scholar]
  45. Kornblum JD 2008. Using JPEG quantization tables to identify imagery processed by software. Digital Investig. 5:1S21–25
    [Google Scholar]
  46. Kubovy M 1988. The Psychology of Perspective and Renaissance Art Cambridge, UK: Cambridge Univ. Press
  47. Lalonde JF, Efros AA 2007. Using color compatibility for assessing image realism. Proceedings of the 11th IEEE International Conference on Computer Vision1–8 Piscataway, NJ: IEEE
    [Google Scholar]
  48. Lalonde JF, Efros AA, Narasimhan SG 2012. Estimating the natural illumination conditions from a single outdoor image. Int. J. Comput. Vis. 98:2123–45
    [Google Scholar]
  49. Lukas J, Fridrich J 2003. Estimation of primary quantization matrix in double compressed JPEG images. Proceedings of the Digital Forensic Research Workshop5–8 Trumansburg, NY: DFRWS
    [Google Scholar]
  50. Lyu S, Farid H 2005. How realistic is photorealistic. IEEE Trans. Signal Proc. 53(2):845–50
    [Google Scholar]
  51. Mader B, Banks MS, Farid H 2017. Identifying computer-generated portraits: the importance of training and incentives. Perception 46:91062–76
    [Google Scholar]
  52. Mahdian B, Saic S, Nedbal R 2010. JPEG quantization tables forensics: a statistical approach. Proceedings of the International Workshop on Computational Forensics150–59 Buffalo, NY: IAPR
    [Google Scholar]
  53. Milani S, Tagliasacchi M, Tubaro S 2012. Discriminating multiple JPEG compression using first digit features. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing2253–56 Piscataway, NJ: IEEE
    [Google Scholar]
  54. Nagano K, Seo J, Xing J, Wei L, Li Z et al.tal> 2018. paGAN: real-time avatars using dynamic textures. SIGGRAPH Asia 2018 Technical Papers, art. 258. New York: ACM
    [Google Scholar]
  55. New York v. Ferber, 458 US 747 (1982)
  56. Ng T-T, Chang S-F, Hsu J, Xie L, Tsui M-P 2005. Physics-motivated features for distinguishing photographic images and computer graphics. Proceedings of the ACM International Conference on Multimedia239–48 New York: ACM
    [Google Scholar]
  57. Nightingale SJ, Wade KA, Watson DG 2017. Can people identify original and manipulated photos of real-world scenes. Cogn. Res. Princ. Implic. 230
    [Google Scholar]
  58. Nillius P, Eklundh J-O 2001. Automatic estimation of the projected light source direction. Proceedings of the 13th IEEE Conference on Computer Vision and Pattern Recognition239–48 Piscataway, NJ: IEEE
    [Google Scholar]
  59. O’Brien J, Farid H 2012. Exposing photo manipulation with inconsistent reflections. ACM Trans. Graph. 31:14
    [Google Scholar]
  60. Ostrovsky Y, Cavanagh P, Sinha P 2005a. Perceiving illumination inconsistencies in scenes. Perception 34:1301–14
    [Google Scholar]
  61. Ostrovsky Y, Patrick C, Sinha P 2005b. Perceiving illumination inconsistencies in scenes. Perception 34:1301–14
    [Google Scholar]
  62. Popescu A, Farid H 2004. Statistical tools for digital forensics. Proceedings of the International Workshop on Information Hiding128–47 New York: ACM
    [Google Scholar]
  63. Popescu A, Farid H 2010. Exposing digital forgeries in color filter array interpolated images. IEEE Trans. Signal Proc. 53:103948–59
    [Google Scholar]
  64. Potter M 1976. Short-term conceptual memory for pictures. J. Exp. Psychol. Hum. Learn. Memory 2:509–22
    [Google Scholar]
  65. Raiturkar P, Farid H, Jain E 2018. Identifying computer-generated portraits: an eye tracking studyTech. Rep., Univ. Fla., Gainesville
  66. Roose K, Mozur P 2018. Zuckerberg was called out over Myanmar violence. Here's his apology. . The New York Times, Apr. 9. https://www.nytimes.com/2018/04/09/business/facebook-myanmar-zuckerberg.html
    [Google Scholar]
  67. Shane S, Mazzetti M 2018. Inside a 3-year Russian campaign to influence U.S. voters. The New York Times, Feb. 16. https://www.nytimes.com/2018/02/16/us/politics/russia-mueller-election.html
    [Google Scholar]
  68. Sinha P, Balas B, Ostrovsky Y, Russell R 2006. Face recognition by humans: 19 results all computer vision researchers should know about. Proc. IEEE 94:111948–62
    [Google Scholar]
  69. Suwajanakorn S, Seitz SM, Kemelmacher-Shlizerman I 2017. Synthesizing Obama: learning lip sync from audio. ACM Trans. Graph. 36:495
    [Google Scholar]
  70. Swaminathan A, Wu M, Liu KR 2007. Nonintrusive component forensics of visual sensors using output images. IEEE Trans. Inform. Forensics Secur. 2:191–106
    [Google Scholar]
  71. Taub A, Fisher M 2018. Where countries are tinderboxes and Facebook is a match. The New York Times https://www.nytimes.com/2018/04/21/world/asia/facebook-sri-lanka-riots.html
    [Google Scholar]
  72. Thies J, Zollhöfer M, Stamminger M, Theobalt C, Nießner M 2018. Headon: real-time reenactment of human portrait videos. ACM Trans. Graph. 37(4):164
    [Google Scholar]
  73. Vishwanath D, Girshick A, Banks M 2005. Why pictures look right when viewed from the wrong place. Nat. Neurosci. 10:81401–10
    [Google Scholar]
  74. Wandell B 1995. Foundations of Vision Sunderland, MA: Sinauer Assoc.
  75. Wang Y, Moulin P 2006. On discrimination between photorealistic and photographic images. Proceedings IEEE International Conference on Acoustics, Speech and Signal Processing, Vol. 2. Piscataway, NJ: IEEE
    [Google Scholar]
  76. Westheimer G, McKee S 1975. Visual acuity in the presence of retinal-image motion. J. Opt. Soc. Am. 65:847–50
    [Google Scholar]
  77. Zach F, Riess C, Angelopoulou E 2012. Automated image forgery detection through classification of JPEG ghosts. Pattern Recognition: Joint 34th DAGM and 36th OAGM Symposium185–94 Buffalo, NY: IAPR
    [Google Scholar]
/content/journals/10.1146/annurev-vision-091718-014827
Loading
/content/journals/10.1146/annurev-vision-091718-014827
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error