Color Imaging on the Internet

Color Imaging on
the Internet
Robert Buckley
Xerox Innovation Group

Giordano Beretta
Hewlett-Packard Laboratories

http://www.inventoland.net/imaging/cii/
www
Visual Communications and Image Processing 2003

Course objectives 1

• List and describe the current and emerging methods for
Internet image exchange

• Develop a systematic understanding of the principles of
color encoding, image compression, ﬁle formatting,
protocols, and Internet imaging applications

• Understand the differences between the various methods
for each imaging function

• Develop an intuition for specifying well-balanced scalable
architectures for Internet imaging

R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet

Rationale 2
Why a course on color imaging specifically for the Internet?

• A picture is worth ten thousand words…
A screenful of text requires 24×80 = 1,920 bytes
•
A VGA size image requires 640×480×3 = 921,600 bytes
•

• …but requires almost 500 times as much bandwidth…
• data compression is essential for images on the Internet
• which compression is best for my image?

• …and the server and client are unknown a priori
• which color representation is suitable to both?
• which ﬁle format can be understood by both?
• how can they negotiate the above?
• how can we provide for unknown viewing conditions?

To be successful, systems must be designed in a top-down approach


Course roadmap 3

Application
Protocol
Format
• Systematic bottom-up
Compression
presentation and Color image
comparison of methods

• Intended for top-down
system design


1 Basics 4
Evolution of Internet imaging

• Internet developed over 30 years, now mature and in
incremental engineering mode

• Although the Internet has been used for scientiﬁc
visualization from the beginning, it has become a visual
medium only since the advent of the free Mosaic browser
in 1993

• Outline of this module:
• the Internet
• protocols
• media types
• intelligent image processing


1.1 The Internet 5
Born Arpanet, Fall 1969

• Originally a high-speed packet-switching network
connecting research super-computers
• packet switching allows building a reliable system that is based on an
infrastructure assumed at all times to be unreliable
• each packet is individually addressed and each node just forwards
packets not addressed to itself
• the routing of packets is irrelevant
• based on TCP/IP

• Today the Internet is the communications medium for
• individuals
• businesses
• communities of practice (extended knowledge networks)


1.1.1 The communication process 6
Claude Shannon, 1941

1. Information source: person or thing generating original
message
2. Transmitter: intrument that transforms the message into a
signal suitable for transmission
3. Communication channel: medium that conducts the signal
4. Receiver: instrument that takes the signal and tries to
reconstruct the message
5. Destination: person or thing the message is intended for


1.1.2 Multi-layer models for networking 7

OSI 7-layer model
DoD 4-layer model

Application
Process
FTP, SMTP, HTTP
Presentation

Host-to-Host Session
TCP, UDP
Transport
Internet
Network
IP, IPv6

Data Link
Network Access
Ethernet, FDDI Physical

Used in newer designs
Used in the original
development of
the Internet


1.1.3 IETF standards development 8
4000
3559
3500

3000

2500
RFC No.

2000

1500

1000

500

0
1970 1975 1980 1985 1990 1995 2000


1.1.4 The WWW 9
Born World Wide Web, March 1989

• Using hypertext links to connect chunks of information on
the Internet

• The WWW is a set of three speciﬁcations
• URL, Uniform Resource Locator, to locate information
• HTML, Hypertext Markup Language, to write simple documents
• HTTP, Hypertext Transfer Protocol, to transfer HTML ﬁles

• The WWW became popular when
• the Internet became commercialized
• fast data connections became pervasive
• graphical browsers made navigation easy and appealing
• early adopters understood the value of the new communication medium
and invented disruptive technologies


1.2 Protocols 10
• A protocol is a set of conventions or rules governing
communications

• Protocols allow networks to interconnect and ensure
compatibility between devices of different manufacturers

• Examples:
• FTP — file transfer protocol
• HTTP — hypertext transfer protocol
• IIP — Internet imaging protocol
• IPP — Internet printing protocol
• SMTP — Simple Mail Transfer Protocol

• Protocols become standards when signed off by an ofﬁcial
body like IETF, W3C, ITU-T, ISO, or IEEE
• de facto and de jure standards


1.2.1 FTP — File Transfer Protocol 11
Transfer ﬁles from one machine to another

• Based on TCP/IP
• TCP (transmission control protocol) converts messages into streams of
packets at the source, then reassembles them back into messages at the
destination
• IP (Internet protocol) handles addressing, seeing that packets are routed
across multiple nodes and even across multiple networks with multiple
standards

• Requires explicit directory navigation both at the source
and the destination

• Allows anonymous login

• Can perform end-of-line conversion in ASCII ﬁles


1.2.2 HTTP — HyperText Transfer Protocol 12
Transfer compound documents with links

• Application-level protocol for distributed, collaborative,
hypermedia information systems

• Requires a reliable transport such as TCP/IP

• Request the components of a document identiﬁed by
hypertext links

• Provides support for HTML forms

• Typing and negotiation of data representation allows
systems to be built independently of the data being
transferred
• example: color images


1.2.3 Protocols for wireless applications 13
The PC is no longer at the center of the world

• WASP — wireless application service providers

• WAP — wireless application protocol

• WML — wireless mark-up language for WAP
• existing commercial applications for automatic translation from HTML to
WML by AvantGo and Phone.com

• iMode — uses HTML, but the screen is still small
• created by NTT DoCoMo
• does not require translation from HTML


1.2.4 Protocol evolution for services 14
distributed objects web services
remote messages remote procedures

DLL CLR .NET

DDE OLE 1.0 COM DCOM

MSRPC SOAP
COM/
LPC
CORBA
NCS
UDP/TCP DCE
RPC
CORBA
J2EE
ONC
ONC+
RPC
IPC JAVA/RMI
SHL RRBC

1960 1970 1980 1990 2000


1.3 Internet media types 15
• Identify type and encoding of transmitted data
• type/subtype
• used by Multipurpose Internet Mail Extensions (MIME) and others
• used to be called MIME types
• standard types registered with Internet Assigned Numbers Authority
(IANA)

• Standard types, sample subtypes
text plain, html
multipart mixed, related
message rfc822, http
application pdf, vnd.ms-powerpoint, ipp
image tiff, jpeg, png, gif, vnd.fpx
audio basic, 32kadpcm
video mpeg, quicktime


1.4 New trends in image processing 16
User’s expectations

• Many users access the Internet in the ofﬁce on fast
workstations connected over fast links to the Internet

• At home users often have fast graphics controllers for
playing realistic computer games

• Increasingly, private homes are equipped with fast
connections over DSL, cable modem, 802.11g, FTTH, …

• The latest video game machines are very powerful graphic
workstations

These user experiences set very high expectations for color
imaging on the Internet

1.4.1 Polarization of devices 17
The nomadic workforce

• The new generation grew up on video games & WWW

• At work, they expect concise answers immediately on
multiple media

• The new working world is mobile and wireless
• a comprehensive fast ﬁber optics network provides a global backbone
• the “last mile” is wireless
• computers are wearable

• An appropriate viewing device has not yet been invented
• but it will not be printed paper
• the viewing conditions will be unpredictable
• likely, a plethora of viewing devices will be in use


1.4.1.1 Global Crossing’s peak network 18


1.4.2 How fast is the Internet? 19
It is both fast and slow

• There is a lot of global fiber
• example: Global Crossing planned to circumscribe all continents

• Backbones will have ample bandwidth
• oversupply: a large amount of fiber is dark
• competition is fierce
• movies on demand and telepresence will consume this bandwidth

• Most users will access the data wireless
• color imaging over the Internet must be efficient

• Today’s game machines have much more processing
power than desktop machines
• trade-off data for computation on the client
Trend: separation of data from control

1.4.3 Leveraging on vision theory 20
• To conserve power, wireless devices will have low effective
transmission bandwidth and small display areas

• Concomitantly the new users are impatient

• Progressive encoding based on region of interest will be
crucial
• JPEG 2000 and MPEG-21 provide the frameworks
• algorithms are required

• Automatic cropping based on region of interest is a
necessary capability for major commercial sites

• Leverage on vision theory for Internet imaging
• Intelligent image processing technologies
• Lawrence Stark & Claudio Privitera, UC Berkeley


1.5 Anatomy of a Web page 21

computer graphics

plain text

full color image


1.5.1 Web page elements 22


2 Color representations 23

Application
Protocol
Format
Compression
Color image


2.1 Requirements 24

• Color must be encoded in standards that
• support communication over the Internet
• the total size of a page should be such it can be transferred quickly
• hence the color space must compress well
• are suitable for heterogeneous environments
• there is no a priori knowledge of the user platform
• the Internet is more like a bazaar than a cathedral
• can easily be implemented efﬁciently and robustly
• Internet imaging applications are not implemented by color scientists
• images must be displayed reliably (no unexpected rendering)
• there is no a priori knowledge of the user’s machine power


2.2 Viewing condition issues 25
see also §2.6.2, slide 40

• There is no control over the user’s viewing conditions
• users often work in poor viewing conditions
• viewing conditions can change during a session
• there is a plethora of viewing devices
• an applications implementation may not be aware of the difference, e.g.
between colorimetric RGB and device RGB

• Issues too complex to expect users controlling their
viewing conditions

• Color integrity is more important than color ﬁdelity
• Ralph Evans: consistency principle


2.3 Color representations 26
Color model operators

• XYZ
• basis for all colorimetry
• deﬁned by CIE for 1931 2˚ and 1964 10˚ Standard Observers
• most applications refer to 2˚ Observer

• RGB
• scanners and digital cameras — linear, non-CIE
• monitors and displays — non-linear, CIE-based

• Luma-chroma
• luminance (lightness) and 2 opponent color signals
• color television — luminance-chrominance YIQ, YUV, YCbCr
• uniform color spaces — CIELAB, CIELUV
• color fax uses CIELAB


2.4 Luma-chroma spaces 27
fR ( R )
L
C1 = A ⋅ f ( G )
G
fB ( B )
C2

YIQ YUV YC1C2

NTSC EBU SMPTE CCIR sRGB
XYZ RGB RGB RGB 709

Photo
CIELAB YES YCC


2.4.1 RGB separations 28

R

G B


2.4.2 CIELAB separations 29

L*

a* b*


2.4.3 Chroma subsampling 30

L*

a*
b*


2.5 Some popular schemes 31
to represent color on the Internet

• sRGB is a colorimetric standard based on common CRTs
• gamma function is built-in for efﬁcient display
• does not require computations in most cases
• viewing conditions are part of the standard, but are not realistic for
casual users on the Internet
• extended sRGB color spaces are under development

• CIELAB and YUV are opponent color spaces that compress
well in the case of pictorial images

• YCbCr is an opponent color space that was used
extensively in developing the JPEG standard
• Y is the same as in YUV
• U and V are scaled and zero-shifted so that Cb and Cr are in [0, 1]; then
they are scaled by 255 to be represented by a byte


2.5.1 RGB speciﬁcation 32
Transformation from sRGB to 1931 CIE XYZ values
R’sRGB = R8bit / 255.0
• Primaries
G’sRGB = G8bit / 255.0
• red: (xR, yR) B’sRGB = B8bit / 255.0
• green: (xG, yG) If R’sRGB, G’sRGB, B’sRGB ≤ 0.04045
• blue: (xB, yB) RsRGB = R’sRGB / 12.92
GsRGB = G’sRGB / 12.92
• White point: (xN, yN) BsRGB = B’sRGB / 12.92
else R’sRGB, G’sRGB, B’sRGB > 0.04045
• Non-linearity (gamma) RsRGB = [(R’sRGB + 0.055) / 1.055]2.4

GsRGB = [(G’sRGB + 0.055) / 1.055]2.4
• Example: sRGB
BsRGB = [(B’sRGB + 0.055) / 1.055]2.4
• IEC 61966-2-1
and
• (xR, yR) = (0.64, 0.33)
0.4124 0.3576 0.1806 R sRGB
• (xG, yG) = (0.30, 0.60) X
Y = 0.2126 0.7152 0.0722 G sRGB
• (xB, yB) = (0.15, 0.06)
Z 0.0193 0.1192 0.9505 B sRGB
• (xN, yN) = (0.3127, 0.3290)
• same as ITU-R BT.709-2


2.5.1.1 sRGB viewing conditions 33
Reference display conditions
Display parameter Reference condition
80 cd/m2
luminance level
white point D65
gamma 2.2

Reference viewing conditions
Viewing parameter Reference condition
screen background 20% of reference display area
surround 20% of ref. ambient illuminance level
proximal ﬁeld 20% of ref. display luminance level
ambient illuminance level 64 Lux
ambient white point D50
veiling glare 1%


2.5.2 CIELAB 34
• CIE standard for color difference evaluation
• uniform color space
• illuminant Xn, Yn, Zn
• L* range: [0, 100]
L* = 116 ⋅ 3 Y ⁄ Y n – 16
a* = 500 ⋅ { 3 X ⁄ X n – 3 Y ⁄ Y n }
b* = 200 ⋅ { 3 Y ⁄ Y n – 3 Z ⁄ Z n }

• Xn, Yn, Zn: reference white
• D50: 96.422, 100, 82.521; D65: 95.047, 100, 108.883

• von Kries type adaptation

• Color fax, ICC Proﬁle Connection Space


2.5.2.1 8-bit CIELAB encodings 35
• CIE encoding (TIFF)
• scale L* = [0, 100] to [0, 255]
• limit a* and b* to [-128, 127]

• ICC encoding
• scale L* = [0, 100] to [0, 255]
• add offset 128 and limit a* and b* to [0, 255]
• white point: D50

• ITU-T encoding
• scale L* = [0, 100] to [0, 255]
• apply scale/offset so a* = [-85, 85] maps to [0, 255]
• apply scale/offset so b* = [-75, 120] maps to [0, 255]
• white point: D50


2.5.3 The YUV color space 36
Used in the PAL television system

Y 0.299 0.587 0.114 R
U = – 0.148 – 0.289 0.437 ⋅ G
V 0.615 – 0.515 – 0.100 B
or
Y = 0.299 ( R – G ) + G + 0.114 ( B – G )
U = 0.493 ( B – Y )
V = 0.877 ( R – Y )

Reference: Bhaskaran & Konstantinides


2.5.4 The YCbCr color space 37
Popular for JPEG

• From ITU-R BT.601-2 for color television

Y 0.299 0.587 0.114 R
Cb = – 0.169 – 0.331 0.500 ⋅ G
Cr 0.500 – 0.419 – 0.081 B
• 8-bit encoding in digital ﬁles

Y 0.299 0.587 0.114 R 0
Cb = – 0.169 – 0.331 0.500 ⋅ G + 128
Cr 0.500 – 0.419 – 0.081 B 128


2.6 More color representations 38
• CMYK
• color print separations — device specific, non-linear
• example: SWOP printing process specification

• Palette
• color map or lookup table
• color represented by an index into a table of N colors
• see §3.3.1, slide 70

• ICC profiles
• profile is a transform between a given color space and a Profile
Connection Space (PCS)
• defines color explicitly in terms of its transform to PCS
• PCS is XYZ or CIELAB
• ICC has defined standard formats for profiles


2.6.1 ICC profile concepts 39
• Profile classes
• input devices (scanners, digital cameras)
• display devices (monitors, LCD projectors)
• output devices (printers, film recorders)
• DeviceLink (dedicated device to device)
• ColorSpace
• Abstract (PCS-to-PCS, effects, e.g., contrast adjustment)
NamedColor (Pantone®, Trumatch®)
•

• Rendering intents
• colorimetric: absolute, relative; perceptual; saturation

• Models
• shaper/matrix (shaper is a 1-D LUT)
• shaper/multi-D LUT


2.6.2 ICC profile based color reproduction 40
• One cannot assume that a casual Web User works in a
controlled environment

• sRGB is considered a safe bet for “average” situations

• Tools are available to control color rendering on the
Internet server side

• It is imperative that the entire workflow is characterized
and ICC profiles be always embedded in images, instead
of assumed

• For an example on how to set up an ICC based
environment see
//www.hpl.hp.com/techreports/1999/HPL-1999-110.pdf


2.7 Color interchange models 41
S R D
color color
Type I T1 T2
source destination

destination parameters

S R D
color color
Type II T1 T2
source destination

S R D
color color
Type III T1 T2
source destination

source parameters

color values: S = source, R = reference, D = destination
Ti = color conversion


2.7.1 Color interchange model types 42
• Type I
• interchange uses device color values
• source prepares color data for known destination
• example: traditional graphic arts CMYK workﬂow

• Type II
• interchange uses device-neutral, reference color space
• examples: color TV broadcasting, color facsimile

• Type III
• source transmits source values + source characteristics
• similar to type II, but with delayed conversion
• examples: PDF CIE-based color spaces, ICC workﬂow


2.8 Server side color management 43
• On the client side, a set of filters is used to create visually
an ICC profile with an applet running in the browser

• High-end systems are based on spectroradiometry &
compensate for brightness level differences among
monitors

1. On the server side, a servelet pushes each image through
a color management system before it is sent to the client
• E-Color True Internet Color, Imation Verifi
2. …or servelet sends applet that does correction at browser
• Gretag-Macbeth WebSync
3. …or the HTML page is tagged with a trigger
• WayTech Coloreal, Praxisoft RealNetColor


2.8.1 True Internet Color architecture 44

send uncorrected images

3
2
Merchant’s E-Color’s
image URL
send page points to
Server Server
with E-Color’s
image URL server
4
1 send color-
request corrected
page image

Browser


2.8.2 Veriﬁ Accurate Web Color architecture 45

2
Merchant’s Imation’s
send raw
uncorrected
Image Server Profile Server
3
image
request
6
1 profile
send color-
request
corrected
page 4
image send cookie
5 with
send profile profile
from
cookie

Browser


2.8.3 WebSync architecture 46
4
replace
5 image
send page tags
and
color-matching WebSync
applet
Software

1
request 2
page forward
3 request
send
Browser page

Merchant’s
Server


2.8.4 Coloreal architecture 47
• Polynomial for display monitor’s gamma curve is stored in
monitor’s EDID chip

• On merchant’s Web server all images are encoded in sRGB

• Web server adds a Coloreal tag to each HTML file

• When monitor is first connected, installer reads gamma
curve from EDID chip to create an ICC profile for the
monitor

• When an HTML page contains the Coloreal
tag, Windows ICM is invoked to use the IC
profile to compute device RGB counts


2.8.5 RealNetColor architecture 48

• On Web server all images are encoded in sRGB or are
tagged with an ICC profile

• Web server adds a RealNetColor tag to each HTML file

• Each use of the RealNetColor tag triggers a payment from
the Web retailer to Praxisoft

• When an HTML page contains the
RealNetColor tag, a plug-in converts the
color using the ICC profile or assuming sRGB
values


2.9 Display trends 49
LCDs are not CRTs

• CRT displays are being replaced
by LCD displays

• LCDs are brighter, smaller, and
use less power

• However, the colorimetry can
be quite different
• with careful calibration,
characterization & color management,
an LCD can be made to perform close
to a CRT in terms of linearity, gamma,
and white point
• the color gamut can be very different
• today’s LCDs can outperform CRTs (monitors above are from 1995, 1998)


2.9.1 Display gamuts, the whole story 50
• Gamut renderings in chromaticity diagrams are
misleading, because of colorfulness and appearance mode

21” studio CRT 23” LCD

reproduced with permission, © Apple


2.9.2 Future display technologies — OLED 51
Organic light-emitting-diode displays

• LCD displays use absorption ﬁlters and polarizers, limiting
the gamut in the blues and the brightness

• OLED displays are emissive and are brighter
• no ﬁlters nor polarizers

• Current limited lifetime of blue OLEDs limits the gamut in
the blues even more than for LCDs

• Wafer size still limited
• today’s applications: car stereo, portable DVD players
• largest prototype display shown: 13”
• manufacturing process more expensive than LCD


2.9.3 Future display technologies — MEMS 52
• MEMS (Micro-ElectroMechanical System) technology
makes is possible to build displays based on interference

glass substrate V
air thin film stack
metallic membrane reproduced with permission, © Iridigm

• Voltage between thin ﬁlm stack and metallic membrane
controls their gap and therefore the pixel’s color

• Luminance by ﬂickering or dithering

• Typical resolutions: 400–1000 dpi

• Bistable, only draws power during switching


2.10 Appearance mode 53
CRT at 80 cd/m2 is darker than surroundings
•
• perceived as object in ﬁeld of view
• viewing conditions must be controlled
• color ﬁdelity is important

LCD at 300 cd/m2 is brighter than indoor surroundings
•
• similar to illuminator viewing condition
• visual system adapts to white point, memory colors

OLED achieves 30,000 cd/m2 in military applications
•
expect 1,000 cd/m2 in consumer applications
•
• MEMS interference displays can be brighter than any surroundings

• Consistency principle (Evans)
• reproduction of relation among colors more important than absolute
colorimetry


2.11 Rendering state 54

• Stock photo agency images are rendered to a normalized
intent

• Typical consumer images are the raw output of digital
cameras or scanners

• Many CBIR algorithms rely on color histograms

• Need to specify when images are unrendered

• RIMM/ROMM RGB

• Need algorithms to perform automatic rendering
operation


2.11.1 Digital color image ﬂow 55

sensor unrendered rendered device
space space space space

colorimetric colorimetric output device
input device estimate of estimate of a specific RGB
specific RGB original scene reproduction or CMYK

device and/or device and/or
image specific
image specific image specific
transformation
transformation transformation


2.11.2 Rendering in Photoshop 56

connection space display
(XYZ) (RGB counts)

input connection space working color space
(Adobe RGB, or …) rendered image
(RGB counts) (CIELAB or XYZ)

connection space printer
(CIELAB or XYZ) (CMYK counts)


3 Data compression 57

Application
Protocol
Format
Compression
Color image


3.0.1 Approach 58

• Waveform coding of color images
• with “waveform” we put the emphasis on the signal, as opposed to its
meaning

• “Avoiding the transmission of information which the eye
cannot use” A.V. Bedford, 1950

• Reducing statistical or visual redundancy
• source vs. sink coding
• lossless vs. lossy (visually lossless) coding
• lossless: decompressed image identical to original
• lossy: decompressed image tolerably different


3.0.2 General compression system 59

• 3 stages: transform — quantize — code
• quantize — lossy
• code — lossless

transform quantize code
original compressed
T Q C

spatial ﬁlter scalar quantizer Huffman coding
color transform vector quantizer arithmetic coding
spatial transform color palette Lempel-Ziv


3.1 Coding methods 60

• Achieve compression by exploiting statistical redundancy
in the symbol set
• average number of bits cannot be less than the entropy H
H = – ∑ pi log (pi), where ∑ pi = 1 (pi is the probability of symbol i)
•
• entropy sets bound on performance

• Not all symbols are equally likely
• use short codewords for more probable symbols
• use long codewords for less probable ones


3.1.1 Encoding methods (cont.) 61
• Statistics known
• Huffman coding
• method of constructing the optimum preﬁx code
• Arithmetic coding
• represents a symbol string as a binary fraction
• typically 5–10% better than Huffman coding, but more complex

• Statistics not known
• Lempel-Ziv (dictionary methods) in 3 ﬂavors: LZ77, LZ78, LZW
• represent a string in terms of previous occurrences using:
• a pointer to the previous occurrence and its length (LZ77)
• a dictionary of previous occurrences (LZ78, LZW)

• Flate
• LZ77 followed by Huffman coding
• in some contents authoring tools, Flate encoding is labelled as ZIP


3.1.2 Huffman coding 62
Developed 1952 by D.A. Huffman

• Produces the optimum preﬁx code
• ﬁxed-length symbols to variable-length codewords

1. Order the symbols according to their probabilities
• frequency of occurrence of each symbol known a priori
• in practice, a training set of data is used
2. Merge the two symbols with the smallest probabilities
3. Repeat step 2 until one merged symbol is left
• step 2 can be viewed as construction of a binary tree, since at each
recursion we merge two symbols
• at end of recursion, all symbols will be leaf nodes of this tree
• the codeword for each symbol is obtained by traversing the binary tree
from root to the leaf node corresponding to that symbol


3.1.2.1 Huffman coding example 63
• In most Internet imaging applications the size of the
alphabet composing these symbols is restricted to at most
64,000 symbols

• Average number of bits cannot be less than the entropy
H = – ∑ p i log ( p i )
pi 1 1.00
1
V1 000 0.5
1
01
V2 001 0.2
1 0
0.50 0011
V3 010 0.1 1
0.18
0 0010
V4 011 0.08 0
0.30
1
0001
V5 100 0.06
0
0.12
1
00001
V6 101 0.03
0
0.06
1
000001
V7 110 0.02 0
0.03
0 000000
V8 111 0.01

2.19
Entropy = 2.16


3.1.3 Arithmetic coding 64
• Achieves higher compression than Huffman by combining
several symbols into a single unit
• a message is encoded as a whole new symbol instead of as separate
symbols
• geometric interpretation: symbols correspond to subintervals in [0, 1)

• Separates coding from modeling
• this allows for the dynamic adaptation of the probability model without
affecting the design of the coder

• Many image compression standards allow to substitute
Huffman with arithmetic coding
• Huffman coding is often the baseline requirement
• arithmetic coding can be used in critical applications

• Covered by patents from IBM, Mitsubishi, and AT&T


3.1.4 LZ coding method 65
Jacob Ziv and Abraham Lempel, 1977 and 1978

• A sliding window is moved across the data stream

• LZ77:
• a string is represented in terms of a pointer to the previous occurrence
and its length

• LZ78:
• a string is represented in terms of a pointer into a dictionary of previous
occurrences
• a dictionary is built that maps variable length bit strings from the data
stream into ﬁxed length codes
• the decoder parses the code sequence, recursively builds the same
dictionary, and reconstructs the data stream


3.1.5 LZW coding method 66
Lossless compression of graphics

• Improvement of LZ proposed by Terry Welch in 1984

• Dictionary is initialized with the character set

• Bytes from the input stream are read and used to
progressively form larger and larger sequences until a
sequence is formed that is not in the dictionary

• The last known sequence’s encoding is output and the
new sequence is added to the dictionary

• Typical compression ratio: 2:1

• Implementing LZW may require licensing USP 4,558,302
• see http://www.unisys.com/about__unisys/lzw/


3.1.6 Flate and deﬂate 67
Proposed in 1996 by L. Peter Deutsch

• L77 cascaded with Huffman
• window size up to 32K bytes
• Huffman coding of pointers and lengths

• Performance
• substantially better compression than LZW
• considerably slower encoding speed than LZW
• same decoding speed

• Usage
• PNG format
• gzip, StuffIt, and ZIP archives
• PDF 1.2 and later to compress text, graphics, and indexed image data

• Speciﬁcation Ver. 1.3, IETF RFC 1951

3.2 Binary image compression 68
• Group 3 1-d (MH) and 2-d (MR)
• ITU-T Rec. T.4

• Group 4 (MMR)
• ITU-T Rec. T.6

• JBIG — progressive bi-level image compression
• ISO 11544 / ITU-T Rec. T.82
• ITU-T Rec. T.85 — application proﬁle for fax
• ITU-T Rec. T.43 — bit-plane coding for color fax images using JBIG

• JBIG2 — lossy/lossless coding for bi-level images
• ISO 14492 / ITU-T Rec. T.88
• text halftone, and generic modes
• add color tags to symbols in text mode


3.3 Palette color 69
Counting colors

• 24-bit pixels can represent 16 million colors

• Humans can distinguish 10 million colors

• A 2×3K image contains
6 million pixels

• A 512×512 image contains
250 thousand pixels

A “typical” 5122 image has
•
26 thousand colors

• One byte can represent 256 colors


3.3.1 Color palettes (mapped color) 70
• Represent original colors by indices into a map with
reduced set of colors (paint by numbers)
• choose N colors (palette)
• image dependent (adaptive) or image independent (ﬁxed)
• e.g., median cut
• quantize (map) original to palette colors
• use look-up table to map index to palette color
• may use dither in palettized image

quantize
original index
Q


3.4 Transform coding 71

• Represent pixels p(x, y) as linear basis functions ci(x, y)
p ( x, y ) = k ∑ C i c i ( x, y )

• Coordinate transformation / spectral decomposition
• decorrelating original pixels
• compacting signal energy
• matching quantizer to human visual system

• Quantize and code transform coefﬁcients Ci
• emphasis on T step of T-Q-C compression model

original compressed
T Q C


3.4.1 Transform coding (cont.) 72

• Discrete Karhunen-Loeve Transform (KLT) is optimal
• uncorrelated coefﬁcients, best energy packing
• image dependent, no fast implementation

• Discrete Cosine Transform (DCT)
• image independent, fast transform exists
• performance approaches KLT
7 7
( 2x + 1 )kπ ( 2y + 1 )lπ
1
∑∑
Y ( k, l ) = -- C ( k )C ( l ) S ( x, y ) cos --------------------------- cos -------------------------
-
-
16 16
4
x = 0y = 0

• Baseline JPEG standard uses block DCT
• Joint Photographic Experts Group


3.5 JPEG compression method 73
ISO/IEC 10918–1, ITU-T Rec. T.81

• Lossy compression of images

• Pixels are correlated across space
• the compaction efﬁciency of the Discrete Cosine Transform (DCT) is close
to the optimal transform (KLT)
• DCT is an orthogonal and separable transform

• Transformed data is quantized

• Compression is achieved with cascaded entropy coder

• Typical compression ratios (depends on resolution)
• 10:1 in RGB
• 25:1 in opponent color spaces


3.5.1 JPEG sequential modes of operation 74

• Sequential DCT
• image blocks are coded in scan-like sequence
• Huffman coding (baseline)
• arithmetic coding

• Sequential lossless
• DPCM predictive


3.5.1.1 JPEG non-sequential modes of operation 75

• Progressive DCT
• image blocks are processed sequentially, but coding is completed in
multiple scans
• spectral selection: successively more coefficients are coded in zig-zag
• successive approximation: DCT coefficients are divided by power of 2
before encoding and slices from MSB to LSB are coded
• requires buffering

• Hierarchical coding
• each image component is encoded as a sequence of frames
• first frame is a low-resolution version of image
• subsequent frames are differential frames between source
components and reference reconstructed components
• useful for multi-resolution applications


3.5.2 Color in JPEG 76
Very ﬂexible

• No color space speciﬁcation

• Baseline JPEG: 4 or less color components
• Colorimetric color representation is possible

• Full JPEG: 256 or less color components
• Discrete spectral color representation is possible

• Compression can be improved with chroma subsampling

Conclusions:

• JPEG can be used for full color communication

• Find way to solve artifact problem in JPEG

3.5.2.1 Examples of problems 77
The same image: original, GIF, JPEG

en
radi ts
Hot colors
on g
Cool colors o gradients
n
Soft colors gradients
on

• GIF can cause color quantization problems due to
palettization before LZW compression

• Just changing the q-factor introduces ringing and
blockiness artifacts

3.5.3 JPEG sequential (baseline) pipeline 78

compressed
original stream
raster to entropy
block DCT quantization coding
translation

Q
Huffman
tables

quantization
tables

critical knob
for image quality


3.5.4 The DCT and its kernels 79
7 7
( 2x + 1 )kπ ( 2y + 1 )lπ
1
∑∑
Y ( k, l ) = -- C ( k )C ( l ) S ( x, y ) cos --------------------------- cos -------------------------
-
-
16 16
4
x = 0y = 0

m ⎛ n + --⎞ π
1
-
⎝ 2⎠
[ C 8 ] mn = k m cos ---------------------------
-
8

The 64 kernels of the
discrete cosine transform:


3.5.5 Classical approach: the q-factor 80
The same image compressed with the same parameters except
for an increasing q-factor


3.5.6 Perceptually lossy compression 81
• Internet images often include text

• Readability of text is preserved when small features are
preserved

• Optimize quantization tables to preserve typeface parts

itag ear
bar

stem

serif terminal

stress

USP 5,883,979


3.5.7 More than just compression 82
Image processing in the compressed domain

• Optical shortcomings can be compensated
• cost reduction

• Geometric transformations

• Preferred rendering

USP 5,850,484


3.5.8 Strategies 83
to optimize the JPEG method

• A discrete quantization table (DQT) can be used for all
images of the same class
• text
• business graphics
• maps
• drawings
• gradients in various directions
• etc.

• But: image is created only once, downloaded many times
• it can be more efﬁcient to compute custom tables for each image:
adaptive algorithm

• Main goal is system balancing!


3.6 JPEG 2000 — overview 84
ISO/IEC 15444, ITU-T Rec. T.800

• Wavelet-based follow-on to JPEG
• same committee, different contributors

• Single compression architecture
• continuous-tone and binary compression
• lossy, lossless, and lossy-to-lossless coding
• progressive rendering
• by quality or by resolution via order of codestream packets

• Offer better compression (~25%) with more features

• More parts coming with extensions, proﬁles, ﬁle formats
& conformance

• Part 1 (core decoder) approved as January 2001

3.6.1 JPEG 2000 — applications 85

• Internet and WWW images
• low bandwidth, multiple resolutions, random access
• replacement for FlashPix with multiple, tiled JPEG images?

• Mobile applications
• error resilience, rate control, progressive decompression
• low bit rate

• Digital photography

• Facsimile and multi-function products

• Compound images


3.6.2 JPEG 2000 — features 86
• Superior low-bit rate performance

• Random access within compressed image

• Multiple resolutions with multi-level wavelet transform

• Can specify bit rate

• Error resilience
• re-synchronization of decoder

• Regions of Interest (ROI)
• some parts of the image compressed with higher ﬁdelity

• 1–256 color (spectral) components


3.6.3 JPEG 2000 — operation 87

LH
LL
2-D DWT
LH
LL LL LH
2-D DWT HL HH
Original
HH
HL HH
HL

original compressed
T Q C

Component transform Scalar by Modeling followed
• reversible for lossless mode by binary adaptive
sub-band
• RGB-to-YCbCr for lossy mode arithmetic coder
• none
skip for lossless mode
Pixel transform
• 2-D discrete wavelet transform
• separable, by tiles, multilevel


3.6.4 JPEG 2000 — wavelet transform 88

2-D DWT LH
LL
LL LH LL LH
2-D DWT HL HH
Original
HH
HL HH
HL

• 2-level wavelet transform
• with JPEG 2000 9×7 ﬁlter


3.6.5 Image compressed with JPEG 89

0.125 bpp


3.6.5.1 Image compressed with JPEG 2000, no ROI 90

0.125 bpp


3.6.5.2 JPEG 2000 codestream is packetized 91

• First few packets are such that you can decompress and
obtain an image with more quality in the ROI (face) than
in the periphery (surround)

• As more packets arrive, you obtain the data to produce
better quality in the surround, so that the entire image is
rendered at the same quality

• User can truncate the process anywhere in between


3.6.5.3 Image compressed with JPEG 2000 @ 0.125 bpp 92
ROI coding (face)

equivalent to 0.125 bpp

ROI coding


3.6.5.6 Image compressed with JPEG 2000 @ 1 bpp 95
ROI coding

equivalent to 1 bpp

ROI coding

equivalent to 2 bpp

ROI coding

equivalent to 4 bpp

3.7 Mixed Raster Content — background 98
T.4
T.6
black-and-white black-and-white
MH
MMR text and line text and line
diagrams diagrams

T.85 in1 in1
out out
in2 in2
JBIG
black-and-white
text, halftones,
PSTN
stipples, line art,
and so on

Multiple, independent
compression methods—
T.43
T.42 each optimized for one
kind of image content
JBIG
JPEG
CIELAB
CIELAB


3.7.1 Mixed Raster Content — solution 99
T.44
black-and-white
text & digrams
Mixed
as before,
colored Raster
text Content
too

interchange
black-and-white
text and line
diagrams
black-and-white
text, halftones,
stipples, line art,
color text and
in1 and so graphics
on
out
in2

MRC is a method for using
multiple compression methods
in raster documents that contain
multiple kinds of content


3.7.2 Fax implementation tree 100
T.4
black-and-white
text and line
diagrams MH

in1
out
in2

T.42
black-and-white
black-and-white
text, halftones,
text and line
stipples, line art,
diagrams
JPEG
and so on
CIELAB

in1
out
in2

T.85
T.6
JBIG
MMR
T.44 black-and-white
T.43 text & digrams
as before,
Mixed colored
Raster text
JBIG
Content too
CIELAB


3.7.3 Mixed Raster Content — overview 101
• MRC = Mixed Raster Content
• multi-layer model for representing compound images
• described in ITU-T Recommendation T.44
• originally proposed in joint Xerox/HP contribution
• efﬁcient processing, interchange and archiving of raster-oriented pages
with a mixture of multilevel and bilevel images

• Technical approach
• segmentation of an image into multiple layers (planes), by image content
• use spatial resolution, color representation and compression method
matched to the content of each layer

• Compound image architecture
• framework for using compression methods

• Performance
• can achieve compression ratios of several 100 to 1 on typical documents


3.7.4 Mixed Raster Content — model 102
Image

3-layer model
black-and-white
text & digrams
• Foreground
colored text

• multilevel, e.g., text color
bla
ck
• JBIG @ 12 bpp, 100 dpi
red

• Mask
bla
• bilevel, e.g., text shape
tex ck-a
n
t
co & dig d-wh
• MMR @ 1 bpp, 400 dpi
lor i
ed rams te
tex
t
• Background
• multilevel, e.g., contone im.
• JPEG @ 24 bpp, 200 dpi
Image = M • FG + M’ • BG


3.7.4.1 MRC model — decomposition by stripe 103
1 strip/page, 3 layers vs. multiple strips/page, 1-3 layers/strip
If we do not make Recommendations Stripe 1: M
together, we will surely not make them at all

Stripe 2: M, B
Did you
ever get a
sinking
Time to pull
feeling?
Stripe 3: M, B, F
together?

Stripe 4: B
Stripe 5: M, B, F
Too late!
Too late!

Stripe 6: B

But it is better to have proposed a Recommendation
Stripe 7: M, F
and failed, than to never have proposed at all.
But better still to propose and see success in both
document and marketplace Based on Fig. 3 & 8 in T.44


104
3.7.5 MRC — test
Create the same-sized ﬁles using JPEG and using MRC


3.7.5.1 MRC test — decomposition by stripe 105

Stripe 1: Mask image with FG = red
Stripe 2: Mask image only

Stripe 3: Mask image with graphic in FG

Stripe 4: Mask image only

Stripe 5: Mask (white) with image in BG

Unless otherwise noted, FG is defaulted
to black and BG is defaulted to white


3.7.6 MRC — performance 106

Original
@ 200 dpi

JPEG
@200 dpi
CR = 95:1

MRC
M — MMR @ 400 dpi
FG — JPEG @ 200 dpi
BG — JPEG @ 200 dpi
CR = 382:1 @ 400 dpi


3.7.7 MRC speciﬁcations 107

• Standards
• ITU-T Rec. T.44
• TIFF-FX Proﬁle M
• JPM (JPEG2000 standard, Part 6)

• Proprietary
• ScanSoft PagisPro
• LizardTech Document Express (DjVu)
• Luratech LuraDocument


3.8 Which compression method should I108
use for my images?
• Users do not like plug-ins — avoid them if possible

• Static pictorial image: JPEG

• Image with few colors: LZ (no dithering)
• GIF (§4.3, slide 114) is being replaced by PNG; currently there are still
differences in what is native in browsers

• Vector graphics: do not rasterize
• SVG ﬁle format (see §4.10, slide 135)
• plug-in still new and huge; not available for all platforms

• High concept vector graphics: scripting
• Flash ﬁle format
• requires plug-in; not available for all platforms


3.8.1 Slide 21 revisited in bytes 109
uncompressed compressed

238,960 8,838

1,427

393,448 56,331

619K bytes total 65K bytes total


4 File formats 110

Application
Protocol
Format
Compression
Color image


4.1 Color images 111

• Image: a rectangular array of pixels
• a pixel is an array of samples
• image document: an array of page images

• Two things a ﬁle format should do
• provide sufﬁcient information to decode an image or rendering or
processing
• height, width, samples per pixel, bits per sample, resolution,
color space, compression method, associated images
• image structure, bye ordering
• provide useful information about the image
• metadata, e.g., image description, OCR data


4.1.1 File formats 112
How is the data and its metadata stored?

• Specify the structure of a ﬁle
• ﬁle consists of
• metadata (e.g., color space, white point, little endian, big endian)
• compressed data

• Text, structure, and meta oriented: HTML, XML

• Image oriented: GIF, PNG, JFIF, FlashPix, TIFF-FX

• Compound document oriented: TIFF-FX, PDF


4.2 What is metadata? 113

• Metadata is machine understandable information about
ﬁle resources

• The architecture is of metadata represented as a set of
independent assertions
• assertions about resources are attributes of the resource
• this architecture facilitates programming

• The set of valid attribute names for a context are deﬁned
by convention in a vocabulary

• Metadata increases the value of information

• See §4.9.1, slide 129


4.3 GIF — Graphics Interchange Format 114
• Developed by CompuServe, Inc. in 1987

• Protocol for the on-line transmission and interchange of
raster graphic data.

• Colors speciﬁed in uncalibrated device dependent RGB

• Color is palettized & restricted to power of 2 in [0, 7]

• A GIF data stream can contain several raster-based
graphics — this can be used for animations
• a optional global color map and a local optional map per image

• The raster data is a string compressed with LZW
• sliding window is moved across data stream and dictionary is built
• code size is limited to 12 bits per code
• there are special codes for resetting tables and end-of-stream

4.4 PNG — Portable Network Graphics 115
Patent-free replacement for GIF

• Developed within W3C as license-free alternative to GIF

• Supports palettized color, grayscale, and RGB color
• extension chunk for sRGB and ICC proﬁles
• allows for gamma correction for better cross-platform performance
• not supported by all browsers

• Optional 8-bit alpha channel can be used for transparency
• not supported by all browsers

• Only supports single images (no animation)
• proposed multi-image version is MNG (Multi-image Network Graphics)

• Compression method is ﬂate


4.5 JPEG ﬁle formats 116
The JPEG standard does not specify a file format; several
different formats have been proposed
• ANPA/IPTC — newspaper industry
• ITU-T — ITU-T Rec. T.4 Annex E for color fax
• ETSI — photo videotext, video telephony
• EXIF and Exif Print — digital cameras
• TIFF/EP — digital cameras, ISO/DIS 12234–2
• IOCA — IBM Image Object Content Architecture
• NITFS — intelligence community, DoD
• TIFF — Tag Image File Format rev. 6.0 and later
• PDF — Portable Document Format
• JFIF — JPEG File Interchange Format
• SPIFF — ISO 10918 Part 3


4.5.1 JFIF — JPEG File Interchange Format 117
• Developed by C-Cube Microsystems as a simple file format
to exchange JPEG bitstreams
• just adds APP0 marker segment with application specific information to a
JPEG datastream, as defined in ISO 10918
• baseline or progressive JPEG

• Simpler than TIFF, but for JPEG only
• single codestream, with thumbnail in APP0 marker segment
• quantization and Huffman tables in codestream

• Allows for additional attributes over those of JPEG

• The color space is YCbCr
• no provisions for gamma correction
• an offset is applied to turn CbCr into non-negative numbers (see §2.5.4,
slide 37)


4.5.2 Exif and Exif Print 118
Exchangeable Image File

• Exif being revised from V2.1 to V2.2, called Exif Print

• New features:
• enhanced metadata
• scene modes (portrait, landscape, etc)
• more manditory camera data
• user / image preference data (sharpness, chroma, …)
• formalized use of sYCC for larger than sRGB color gamut
• ExifPrint sYCC is display referenced for a display with sRGB properties
but with no gamut limitations
• ExifPrint sYCC can include ICC proﬁle

• Exif Print was adoped April 2002

• Often used with Digital Print Order Form (DPOF)


Color Imaging on the Internet

Recommended

Recommended

More Related Content

Similar to Color Imaging on the Internet

Similar to Color Imaging on the Internet (20)

More from Giordano Beretta

More from Giordano Beretta (19)

Recently uploaded

Recently uploaded (20)

Color Imaging on the Internet