Introduction to Computer Vision This is a simple introduction to computer vision along with important and significant applications of computer vision in real-life.

1. O TO COMPUTER VISION Dr. M. Aksam
Iftikhar

2. WHAT IS COMPUTER VISION?
Nice
sunset!
“Making computers see and understand”

3. MAIN OBJECTIVES: THEORY +
ALGORITHMS
Development of the theoretical and algorithmic
basis by which useful information about the 3D
world can be automatically extracted and analyzed
from a single or multiple 2D images of the world.

4. COMPUTER VISION, ALSO
KNOWN AS ...
Computational Vision
 Includes modeling of biological vision
Image Understanding
 Automated scene analysis (e.g., satellite images, robot
navigation)
Machine Vision
 Industrial, factory-floor systems for inspection,
measurements, part placement, etc.

5. CONNECTIONS TO OTHER
DISCIPLINES
Computer Vision
Image Processing
Pattern Recognition
&
Machine Learning
Artificial Intelligence
Robotics
Psychology
&
Neuroscience
Computer Graphics

6. IMAGE PROCESSING
Image Enhancement

7. COMPUTER GRAPHICS

8. Computer Graphics
Image
Output:
Geometric Models
Synthetic
Camera
Projection, shading, lighting models

9. Image (or
video)
Sensing
device
Interpreting
device
Interpretation
s
garden,
spring,
bridge,
water,
trees,
flower,
green, etc.
What is (computer) vision?

10. THE GOAL OF COMPUTER
VISION
• To bridge the gap between pixels and
“meaning”
What we see What a computer sees
Source:
S.
Narasimhan

11. Image (or
video)
Sensing
device
Interpreting
device
Interpretation
s
garden,
spring,
bridge,
water,
trees,
flower,
green, etc.
What is (computer) vision?

12. WHY IS COMPUTER VISION
CHALLENGING?
Viewpoint variations
Illumination changes
Scale changes
Deformation
Occlusions
Background clutter
Motion
Intra/Inter-class variations
…

13. Viewpoint variations

14. Illumination changes

15. Scale changes

16. Deformations

17. Occlusions

18. Background clutter

19. Motion blurring

20. OBJECT INTRA-CLASS
VARIATION

22. THREE PROCESSING LEVELS
(1) Low Level
(2) Mid Level
(3) High Level

23. LOW LEVEL VISION

24. LOW LEVEL VISION -
EXAMPLES
Corner and blob detection
Edge detection

25. LOW LEVEL VISION -
EXAMPLES
• Region segmentation

26. MID LEVEL VISION

27. MID LEVEL VISION -
EXAMPLES
• 3D Reconstruction

28. MID LEVEL VISION -
EXAMPLES
• Structure (i.e., 3D) from motion
3D teacup model reconstructed from a 240-frame video sequence
Optical flow

29. HIGH LEVEL VISION

30. slide credit: Fei-Fei, Fergus & Torralba
Scene Interpretation

31. Object categorization
sky
building
flag
wall
banner
bus
cars
bus
face
street lamp
slide credit: Fei-Fei, Fergus & Torralba

32. Qualitative geometric information
slanted
rigid moving
object
horizontal
vertical
slide credit: Fei-Fei, Fergus & Torralba
rigid moving
object
non-rigid moving
object

33. Scene and context categorization
• outdoor
• city
• traffic
• …
slide credit: Fei-Fei, Fergus & Torralba

35. COMPUTER VISION
APPLICATIONS
Industrial inspection/quality control
Surveillance and security
Face recognition
Gesture recognition
Space applications
Medical image analysis
Autonomous vehicles
Virtual reality and much more …...

36. INDUSTRIAL COMPUTER VISION (MACHINE
VISION)
Industrial computer
vision systems work
really well.
Make strong
assumptions about
lighting conditions
Make strong
assumptions about the
position of objects
Make strong
assumptions about the
type of objects
COGNEX

37. OPTICAL CHARACTER RECOGNITION
(OCR)
Digit recognition, AT&T labs
http://yann.lecun.com/exdb/lenet/
• Technology to convert scanned docs to text
License plate readers
http://en.wikipedia.org/wiki/Automatic_number_plate_recognition
Automatic check processing

38. BIOMETRICS

39. LOGIN WITHOUT A PASSWORD…
Fingerprint scanners on
many new laptops,
other devices
Face recognition systems now
beginning to appear more widely
http://www.sensiblevision.com/

40. HAND-BASED BIOMETRICS

41. How the Afghan Girl was Identified by Her Iris Patterns
IRIS BIOMETRICS

42. OBJECT RECOGNITION

43. APPLICATIONS…
AUTOMATED IMAGE
CAPTIONING
Karpathy, Andrej, and Li Fei-
Fei. "Deep visual-semantic
alignments for generating
image descriptions." arXiv
preprint
arXiv:1412.2306 (2014).

44. MOBILE VISUAL SEARCH:
Google Lens – Search what you see

45. FACE DETECTION

46. FACE DETECTION
Many new digital cameras now detect faces
 Canon, Sony, Fuji, …

47. FACIAL EXPRESSION
RECOGNITION
http://www.youtube.com/watch?v=M1WgnisIyPQ&feature=related

48. SMILE DETECTION?
Sony Cyber-shot® T70 Digital Still Camera

49. HAND GESTURE RECOGNITION
Smart Human-Computer User Interfaces
Sign Language Recognition

50. Video Analytics…
Human Behavior Analysis…
Categories of human behavior analysis

51. VISUAL SURVEILLANCE AND
HUMAN ACTIVITY RECOGNITION
Surveillance and security

52. Reverse Engineering of Images…
Image Depth Estimation & Surface Reconstruction
Self Driving cars / Robot
navigation
3D Printing
3D games/ movies
Medical images for Robotic
surgery

53. VISION-BASED INTERACTION
AND GAMES
Kinect

54. TRAFFIC MONITORING
http://www.honeywellvideo.com/

55. SMART CARS:
 Vision systems currently in high-end Tesla, BMW, GM, Volvo models.
Mobileye

56. AUTOMATIC PANORAMA
STITCHING

57. AUTOMATIC PANORAMA STITCHING
(CONT’D)

58. AUTOMATIC 3D RECONSTRUCTION FROM
INTERNET PHOTO COLLECTIONS
“Statue of Liberty”
3D model
Flickr photos
“Half Dome, Yosemite” “Colosseum, Rome”

59. INPAINTING & COLORIZATION

60. ROBOTICS
http://www.robocup.org/
Semantic Robot Vision Challenge
http://www.semantic-robot-vision-challenge.org/
http://www.youtube.com/watch?v=GItjILILB50

61. VISION IN SPACE
• Vision systems used for several tasks
– Panorama stitching
– 3D terrain modeling
– Obstacle detection, position tracking
NASA'S Mars Exploration Rover Spirit

62. MOVIE SPECIAL EFFECTS
Movie special effects
• Insert synthetic objects in real image sequences;
• Change artificially the position or the orientation of a camera;

63. MEDICAL IMAGING
Skin/Breast Cancer Detection
3D imaging
MRI, CT
Enable surgeons to visualize internal
structures through an automated overlay of
3D reconstructions of internal anatomy on
top of live video views of a patient.
Image guided surgery
Grimson et al., MIT

64. Applications…
Agriculture & Live Stock
Remote sensing of crops and vegetables
To explore and develop new ways to improve
sustainable food production using satellite
technologies.

65. Applications…
GIS…
Earth viewers (3D modeling)

66. Applications…
Content Based Image Retrieval

67. Applications…
Forensic Science…
Original (left and middle) and Forged using Splicing (right)

68. Forensic Science…
How forged images effect our society?
Counterfeiting (currency, identification, licenses, etc.)
Evidence tampering
Antique faking (for online shopping)
Political propaganda
Yellow journalism
Scientific research (forging results or observations)
DEEP FAKE
A never ending
research…

69. Forensic Science…
Machine Learning Created Fake Images
o Use of neural networks to create fake
images.
o People on the right aren’t real; they’re the
product of machine learning.
o Generative Adversarial Network (GAN)
architecture is used to create these images
D. Kim, H.-U. Jang, S.-M. Mun, S. Choi, and H.-K. Lee, "Median filtered image restoration and anti-forensics using adversarial
networks", IEEE Signal Processing Letters, vol. 25, pp. 278-282, 2018.
Powerful editing tools & GAN
images are big challenges
https://thispersondoesnotexist.

70. Forensic Science…
Examples of real world forgery
Obama is leading (Original) Hosni Mubarak is leading
R. D. Berenger, and M. Taha, “Technology disruption theory and Middle East media,” In Proceedings of Association for Education
in Journalism and Mass Communication, USA, Chicago, 2012

71. Forensic Science…
Examples of real world forgery
Iranian missile test (Original) Removed the failed missile
M. Nizza, and P. J. Lyons, "In an Iranian Image, a Missile Too Many," The Lede, The New York Times News Blog, July, 10, 2008.

72. OTHER SCIENTIFIC
APPLICATIONS
Astronomy
Weather
Aerial/Satellite

73. WHAT SKILLS DO YOU NEED
TO SUCCEED IN THIS FIELD?
Strong programming skills)
Good knowledge of Data Structures and
Algorithms
Good skills in analyzing algorithm performance
(i.e., time and memory requirements).
Good background in mathematics, especially
in:
 Linear Algebra
 Probabilities and Statistics
 Numerical Analysis