Getting touchy - an introduction to touch and pointer events (1 day workshop) / JavaScript Days 2016 / Berlin / 10 October 2016

gettingtouchy
ANINTRODUCTIONTOTOUCHANDPOINTEREVENTS
Patrick H. Lauke / JavaScript Days 2016 / 10 October 2016 / Berlin

about me...
•   senior accessibility consultant at The Paciello Group
•   previously developer relations at Opera
•   co-editor Touch Events Level 2
•   WG chair and co-editor Pointer Events Level 2

github.com/patrickhlauke/getting-touchy-presentation
"evergreen" expanded version of this presentation
(and branches for speciﬁc conferences)

Touch/pointer events test results

my JavaScript sucks...
(but will hopefully convey the right concepts)

“how can I make my website
work on touch devices?”

you don't need touch events
browsers emulate regular
mouse events

patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html

compatibility mouse events
(mouseenter) > mouseover > mousemove* > mousedown >
(focus) > mouseup > click
* only a single “sacriﬁcial” mousemove event ﬁred

on ﬁrst tap
(mouseenter) > mouseover > mousemove >
mousedown > (focus) > mouseup > click
subsequent taps
mousemove > mousedown > mouseup > click
tapping away
(mouseout) > (blur)
focus / blur only on focusable elements; subtle differences between browsers
Mobile/tablet touchscreen activation/tap event order

emulation works,
but is limiting/problematic

1. delayed event dispatch
2. hover/mouseover interfaces
3. mousemove doesn't track

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

patrickhlauke.github.io/touch/css-dropdown/mouse-only.html

patrickhlauke.github.io/touch/particle/2

touch events
introduced by Apple, adopted
in Chrome/Firefox/Opera
(and belatedly IE/Edge – more on that later)
www.w3.org/TR/touch-events

caniuse.com: Can I use touch events?

touchstart
touchmove
touchend
touchcancel

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
Bug 128534 - 'mouseenter' mouse compat event not ﬁred...

events fired on tap
touchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover > mousemove > mousedown >
(mouse events only fired for single-finger tap)

on ﬁrst tap
subsequent taps
mousemove > mousedown > mouseup > click
tapping away
mouseout > (mouseleave) > (blur)

•   too many touchmove events prevent mouse compatibility events
after touchend (not considered a "clean" tap)
•   too many touchmove events on activatable elements can lead to
touchcancel (in old Chrome/Browser versions)
•   not all browsers consistently send touchmove
•   differences in focus / blur and some mouse compatibility events
(e.g. mouseenter / mouseleave )
•   some events may only ﬁre when tapping away to another focusable
element (e.g. blur )
some browsers outright weird...

Browser/Android 4.3
mouseover > mousemove > touchstart > touchend >
mousedown > mouseup > click
Browser/Blackberry PlayBook 2.0
touchstart > mouseover > mousemove > mousedown >
touchend > mouseup > click
UC Browser 10.8/Android 6
mouseover > mousemove > touchstart > (touchmove)+ > touchend >
mousedown > focus > mouseup > click

shouldn't affect your code,
unless you're expecting a very
speciﬁc sequence...

touch events
vs
limitations/problems

why the delay?
double-tap to zoom
(mostly anyway)

patrickhlauke.github.io/touch/tests/event-listener.html

touch / mouse events delay
[300ms delay]

“how can we make it feel
responsive like a native app?”

react to events ﬁred before the
300ms delay...

touchstart for an
“immediate” control
(e.g. ﬁre/jump button on a game)

touchend for a control that
fires after finger lifted
(but this can result in events firing after zoom/scroll)

interlude: simple feature
detection for touch events

/* feature detection for touch events */
if ( 'ontouchstart' in window ) {
/* some clever stuff here */
}
/* older browsers have flaky support so more
hacky tests needed...use Modernizr.touch or similar */

/* common performance “trick”
conditional "touch OR mouse/keyboard" event binding */
if ('ontouchstart' in window) {
// set up event listeners for touch
foo.addEventListener('touchend', ...);
...
} else {
// set up event listeners for mouse/keyboard
foo.addEventListener('click', ...);
...
}
/* if touch events are supported,
only listen to touchend, otherwise listen to click */

/* common performance “trick” (condensed) */
var clickEvent =
( 'ontouchstart' in window ? 'touchend' : 'click');
foo.addEventListener( clickEvent , ...);
/* if touch events are supported,
only listen to touchend, otherwise listen to click */

hybrid devices
touch + mouse + keyboard

patrickhlauke.github.io/touch/tests/event-listener_naive-touch-or-mouse.html

@patrick_h_lauke showing a naive "touch or mouse" issue on Flickr
(which has since been ﬁxed)

Bug 888304 - touch-events on Firefox-desktop should be disabled until we can support them
properly

Issue 392584: TouchEvent API should be enabled consistently

even on "mobile" we can have
multiple inputs...

Android + mouse – like touch (mouse emulating touch)
touchstart > touchend > mouseover > mousemove > mousedown >

Blackberry PlayBook 2.0 + mouse – like desktop
mouseover > mousedown > (mousemove)+ > mouseup > click

Blackberry Leap (BBOS 10.1) + mouse – like desktop
mouseover > mousedown > (mousemove)+ > mouseup > click

Windows 10 Mobile/Microsoft Edge + mouse – like desktop
mouse events (+ pointer events) + click

Windows 10 Mobile/Microsoft Edge + keyboard – like desktop
focus > click

Android + keyboard – like desktop
focus > click

iOS keyboard only works in
same situations as on-screen
keyboard
(e.g. text inputs, URL entry)

VoiceOver enables full keyboard access on iOS

iOS + keyboard – similar to touch
focus / touchstart > touchend > (mouseenter) > mouseover >
mousemove > mousedown > blur > mouseup > click

mobile Assistive Technologies
(e.g. screen readers on touchscreen devices)

iOS + VoiceOver (with/without keyboard) – similar to touch
focus / touchstart > touchend > (mouseenter) > mouseover >
mousemove > mousedown > blur > mouseup > click

Android 4.3/Chrome + TalkBack – keyboard/mouse hybrid
focus / blur > mousedown > mouseup > click > focus

Android 6.1/Chrome + TalkBack – like touch
touchstart > touchend > mouseover > mouseenter > mousemove >
mousedown > focus > mouseup > click

Android 6.1/Firefox + TalkBack – similar to touch
touchstart > mousedown > focus > touchend > mouseup > click

further scenarios?
•   desktop with external touchscreen
•   touchscreen laptop with non-touch second screen
•   touchscreen laptop with trackpad/mouse
•   ...and other permutations?

note on trackpad gestures
•   trackpads that allow multi-finger gestures (e.g. recent MacBooks,
Magic Mouse/Trackpad, Windows precision trackpad, ASUS Smart
Gesture, Wacom Intuos Pro gestures etc) don't fire touch events /
expose touch points
•   these gestures are handled directly by the OS/driver and don't reach
the browser as raw touches
•   Wacom Intuos Pro touch input exposed as mouse to OS
•   OS X/Safari 9.1+ does fire Apple's proprietary gesture* events (for
pinch-to-zoom, rotation)

no way to detect these cases...

Modernizr.touch detects touch events not touch devices

Stu Cox: You can't detect a touchscreen

hacks.mozilla.org - Detecting touch [...]

/* feature detection for touch events */
/* browser supports touch events but user is
not necessarily using touch (exclusively) */
/* it could be a mobile, tablet, desktop, fridge ... */
}

/* DON'T DO THIS:
conditional "touch OR mouse/keyboard" event binding */
...
} else {
...
}

/* DO THIS:
doubled-up "touch AND mouse/keyboard" event binding */
/* but this would fire our function twice for touch? */
patrickhlauke.github.io/touch/tests/event-listener_naive-event-doubling.html

/* DO THIS:
doubled-up "touch AND mouse/keyboard" event binding */
foo.addEventListener('touchend', function(e) {
// prevent compatibility mouse events and click
e.preventDefault();
...
});
patrickhlauke.github.io/touch/tests/event-listener_naive-event-doubling-ﬁxed.html

preventDefault() kills
scrolling, pinch/zoom, etc

apply preventDefault()
carefully
(just on buttons/links, not entire page)

patrickhlauke.github.io/touch/fastclick
YouTube: iOS/Safari 300ms click delay: vanilla Bootstrap and using fastclick.js

patrickhlauke.github.io/touch/fastclick/fastclick.html
YouTube: iOS/Safari 300ms click delay: vanilla Bootstrap and using fastclick.js

browsers working to remove
double-tap to zoom delay
(when page not zoomable)

... content="minimum-scale=1, maximum-scale=1"
patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

Bug 922896 - Optimizations to remove 300ms [...] delay
[RESOLVED FIXED]

iOS/Safari designed themselves into a corner: “double-tap to scroll”
Bug 122212 - Optimizations to remove 300ms touch > mouse events delay

however, Apple did ﬁnally
change its mind...

Changeset 191072 - Web pages with unscalable viewports shouldn't have a single tap delay
(from iOS 9.3 onwards)

@thomasfuchs - Safari/iOS10beta3 ignores unscalable viewports
(no ofﬁcial Apple documentation about the change...but the 300ms delay is back)

"Force enable zoom" reintroduces delay – a fair compromise?

"mobile optimised" viewport
and
"double-tap to zoom"

Chrome 32+ / Android: content="width=device-width"
suppresses double-tap-to-zoom, still allows pinch zoom
Google Developers: 300ms tap delay, gone away

Bug 941995 - Remove 300ms [...] on "responsive" pages
[RESOLVED FIXED]

Bug 150604 - Implement viewport-width-based fast-click heuristic

Bug 151077 - Fast-clicking should trigger when scale is equal to explicitly set initial scale

WebKit blog: More Responsive Tapping on iOS

patrickhlauke.github.io/touch/tests/results/#suppressing-300ms-delay

no concept of hover
on touch devices
iOS fakes it, Samsung Galaxy Note II/Pro + stylus, ...

iOS "fakes" hover support: stops event sequence on content change
iOS Developer Library - Safari Web Content Guide - Handling Events

YouTube: Samsung Galaxy Note Pro 12.2 stylus hover
...but you can't rely on this, as most devices don't have it...

Modernizr: Detecting a mouse user

what about
CSS4 Media Queries?

W3C Media Queries Level 4 (Editor's Draft)

/* MQ4 Interaction Media Features*/
pointer: none | coarse | fine
hover: none | on-demand | hover
any-pointer: none | coarse | fine
any-hover: none | on-demand | hover

dev.opera - Interaction Media Features and their potential (for incorrect assumptions)

patrickhlauke.github.io/touch/pointer-hover-any-pointer-any-hover
primary input is unchanged (and Chrome/Android required a full restart)

/* Naive uses of Interaction Media Features */
@media (hover: hover) {
/* primary input has hover...so we can rely on it? */
}
@media (pointer: fine) {
/* primary input has fine pointer precision...
so make all buttons/controls small? */
}
/* hover and pointer only check "primary" input, but
what if there's a secondary input that lacks capabilities? */

/* Better uses of Interaction Media Features */
@media (any-hover: none) {
/* at least one input lacks hover capability...
don't rely on it or avoid altogether */
}
@media (any-pointer: coarse) {
/* at least one input has coarse pointer precision...
provide larger buttons/controls for touch */
}
/* test for presence of *any* "least capable"
input (primary or not) */

if in doubt, offer a way to switch interfaces...
or just always use a touch-optimised interface

detect which input the users is
using right now...

GitHub: ten1seven / what-input

avoid hover/mouseover
interfaces?

complement for
keyboard/touch!

patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard.html

patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard-touch.html

events ﬁred on tap
(mouseenter) > mouseover >
mousemove* > mousedown > (focus) >
mouseup > click
* mouse event emulation ﬁres only a single mousemove
too many touchmove events prevent mouse compatibility events after touchend

doubling up handling of
mousemove and touchmove

var posX, posY;
...
function positionHandler(e) {
posX = e.clientX ;
posY = e.clientY ;
}
...
canvas.addEventListener(' mousemove ', positionHandler, false);

var posX, posY;
...
posX = e.clientX ;
posY = e.clientY ;
}
...
canvas.addEventListener(' mousemove ', positionHandler, false);
canvas.addEventListener(' touchmove ', positionHandler, false);
/* but this won't work for touch... */

interface MouseEvent : UIEvent {
readonly attribute long screenX ;
readonly attribute long screenY ;
readonly attribute long clientX ;
readonly attribute long clientY ;
readonly attribute boolean ctrlKey;
readonly attribute boolean shiftKey;
readonly attribute boolean altKey;
readonly attribute boolean metaKey;
readonly attribute unsigned short button;
readonly attribute EventTarget relatedTarget;
// [DOM4] UI Events
readonly attribute unsigned short buttons;
};
www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent
www.w3.org/TR/uievents/#events-mouseevents

partial interface MouseEvent {
readonly attribute double screenX;
readonly attribute double screenY;
readonly attribute double pageX ;
readonly attribute double pageY ;
readonly attribute double clientX;
readonly attribute double clientY;
readonly attribute double x ;
readonly attribute double y ;
readonly attribute double offsetX ;
readonly attribute double offsetY ;
};
www.w3.org/TR/cssom-view/#extensions-to-the-mouseevent-
interface

interface TouchEvent : UIEvent {
readonly attribute TouchList touches ;
readonly attribute TouchList targetTouches ;
readonly attribute TouchList changedTouches ;
readonly attribute boolean altKey;
readonly attribute boolean metaKey;
readonly attribute boolean ctrlKey;
readonly attribute boolean shiftKey;
};
www.w3.org/TR/touch-events/#touchevent-interface

interface Touch {
readonly attribute long identifier;
readonly attribute EventTarget target;
readonly attribute long screenX ;
readonly attribute long screenY ;
readonly attribute long clientX ;
readonly attribute long clientY ;
readonly attribute long pageX ;
readonly attribute long pageY ;
};
www.w3.org/TR/touch-events/#touch-interface

TouchList differences
touches
all touch points on screen
targetTouches
all touch points that started on the element
changedTouches
touch points that caused the event to ﬁre

changedTouches depending on event:
•   for touchstart , all new touch points that became active
•   for touchmove , all touch points that changed/moved since last event
•   for touchend / touchcancel , touch points that were removed
(and are not active anymore at this point)

patrickhlauke.github.io/touch/touchlist-objects

var posX, posY;
...
if ((e.clientX)&&(e.clientY)) {
posX = e.clientX; posY = e.clientY;
} else if (e.targetTouches) {
posX = e.targetTouches[0].clientX;
posY = e.targetTouches[0].clientY;
e.preventDefault() ;
}
}
...
canvas.addEventListener('mousemove', positionHandler, false );
canvas.addEventListener('touchmove', positionHandler, false );

TouchList collections order
•   order of individual Touch objects in a TouchList can change
•   e.g. targetTouches[0] not guaranteed to always be the same finger
when dealing with multitouch
•   not problematic for single-finger interactions, but use identifier
property for each Touch to explicitly track a particular touch point /
finger in multitouch

patrickhlauke.github.io/touch/particle/4a

patrickhlauke.github.io/touch/paranoid_0.5
www.splintered.co.uk/experiments/archives/paranoid_0.5

patrickhlauke.github.io/touch/picture-slider

Basic mouse-driven fake slider
Exercise/demo: expand mouse-driven code to also work with touch

tracking ﬁnger movement over
time ... swipe gestures

patrickhlauke.github.io/touch/swipe

bradfrostweb.com/demo/mobile-ﬁrst

do absolute minimum on each
touchmove
(usually: store coordinates)

do heavy lifting (drawing etc.)
separately, avoid queueing
(e.g. using setTimeout and/or requestAnimationFrame )

Function.prototype.debounce = function (milliseconds, context) {
var baseFunction = this, timer = null, wait = milliseconds;
return function () {
var self = context || this, args = arguments;
function complete() {
baseFunction.apply(self, args);
timer = null;
}
if (timer) {
clearTimeout(timer);
}
timer = setTimeout(complete, wait);
};
};
window.addEventListener('touchmove', myFunction.debounce(250) );

Function.prototype.throttle = function (milliseconds, context) {
var baseFunction = this, lastEventTimestamp = null,
limit = milliseconds;
return function () {
var self = context || this, args = arguments, now = Date.now();
if (!lastEventTimestamp || now - lastEventTimestamp >= limit) {
lastEventTimestamp = now;
baseFunction.apply(self, args);
}
};
};
window.addEventListener('touchmove', myFunction.throttle(250) );

patrickhlauke.github.io/touch/touch-limit

why stop at a single point?
multitouch support

/* iterate over touch array */
for (i=0; i< e.targetTouches .length; i++) {
...
posX = e.targetTouches[i].clientX ;
posY = e.targetTouches[i].clientY ;
...
}

patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html

iOS/iPad preventDefault()
can't override 4-ﬁnger gestures

iOS7+ Safari/WebView
preventDefault() can't
override edge swipes

YouTube: iOS/Safari (iPhone) can't prevent edge swipe gestures

Chrome/iOS (WebView) custom swipe, also can't be prevented
patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html

/* iOS/Safari/WebView has gesture events for size/rotation,
not part of the W3C Touch Events spec. */
gesturestart / gesturechange / gestureend
function(e) {
/* e.scale
e.rotation */
/* values can be plugged directly into
CSS transforms etc */
}
/* not supported in Chrome/Firefox/Opera */
iOS Developer Library - Safari Web Content Guide - Handling gesture events

patrickhlauke.github.io/touch/iosgestures

patrickhlauke.github.io/touch/iosgestures/image.html

/* with some trigonometry we can replicate
these from basic principles using coordinates
from two touch points */
var distance = Math.sqrt(Math.pow(...)+Math.pow(...));
var angle = Math.atan2(...);

patrickhlauke.github.io/touch/pinch-zoom-rotate

patrickhlauke.github.io/touch/picture-organiser

not all old/cheap devices/OSs
support multitouch!

LG Optimus 2X – Android 2.3.7

Touch Events speciﬁcation
grey areas...

do touch events ﬁre during
scroll/zoom?

different models and behaviours, particularly in old browsers
YouTube: Touch Events and scrolling/zooming (playlist)

e.g. older versions of Chrome ﬁre touchcancel on scroll/zoom
YouTube: Google Developers - Mobile Web Thursdays: Performance on Mobile

not deﬁned in spec (yet), but de facto
yes in most modern browsers
patrickhlauke.github.io/touch/gesture-touch

Google Developers: A More Compatible, Smoother Touch
(lots more complexity / edge cases, particularly for old browsers/devices)

W3C Touch Events Extensions WG Note

/* Extension to touch objects */
partial interface Touch {
readonly attribute float radiusX;
readonly attribute float radiusY;
readonly attribute float rotationAngle;
readonly attribute float force;
};

Google Developers: Using rotationAngle and touchRadius

/* Touch Events – contact geometry */
readonly attribute float radiusX ;
readonly attribute float radiusY ;
readonly attribute float rotationAngle ;
readonly attribute float force;
};

patrickhlauke.github.io/touch/tracker/tracker-radius-rotationangle.html
YouTube: Touch Events: radiusX, radiusY and rotationAngle

Rick Byers - Paint (with rotationAngle and touchRadius)

/* Touch Events – force */
readonly attribute float radiusX;
readonly attribute float radiusY;
readonly attribute float rotationAngle;
readonly attribute float force ;
};
force : value in range 0 – 1 . if no hardware support 0
(some devices, e.g. Nexus 10, "fake" force based on radiusX / radiusY )

patrickhlauke.github.io/touch/tracker/tracker-force-pressure.html
iPhone 6s with 3D Touch supports force ...
(Safari and WKWebView, e.g. Chrome/iOS, but not UIWebView, e.g. Firefox/iOS)

patrickhlauke.github.io/touch/tracker/tracker-force-pressure.html
...while in non-3D Touch models (e.g. iPhone 5c) force == 0

3D Touch ≠ Force Touch
•   Force Touch is Apple's proprietary extension to mouse events
•   only aimed at force-enabled trackpads (new Apple models)
•   new events: webkitmouseforcewillbegin , webkitmouseforcedown ,
webkitmouseforceup , webkitmouseforcechanged
•   mouse events have additional webkitForce property
Safari Developer Library: Responding to Force Touch Events

Google Developers: Precision Touch for Precise Gestures

Bug 133180 - Touch events should allow for device-pixel accuracy

interface Touch {
readonly attribute long identifier;
readonly attribute EventTarget target;
readonly attribute long float screenX;
readonly attribute long float screenY;
readonly attribute long float clientX;
readonly attribute long float clientY;
readonly attribute long float pageX;
readonly attribute long float pageY;
};

W3C Touch Events Community Group

W3C Web Events WG - Touch Events errata
(early effort to clarify grey areas, simplify language used)

W3C Touch Events Level 2 (Editor's Draft)
(merges errata, touch events extensions, fractional touch coordinates)

up to IE9 (Win7 / WinPhone7.5)
only mouse events

in IE10 Microsoft introduced
Pointer Events

uniﬁes mouse, touch and pen input into a single event model

...and some new inputs (though currently mapped to mouse)
Building Xbox One Apps using HTML and JavaScript
YouTube: Xbox One Edge Browser sends Pointer Events

not just some
“not invented here”
technology

submitted by Microsoft as W3C Candidate REC 09 May 2013

Pointer Events - W3C REC 24 February 2015

work now continues to enhance/expand Pointer Events...
W3C Pointer Events Level 2 (Editor's Draft)

Bug 822898 - Implement pointer events

about:config / dom.w3c_pointer_events.enabled

Issue 162757: Implement pointer events in Chrome behind experimental ﬂag

YouTube: Google Developers - HTTP 203: Pointer Events
for a while, Chrome wanted to abandon Pointer Events...

Chrome 53 (stable) - enable pointer events
chrome://ﬂags/#enable-pointer-events
blink-dev › Intent to Ship: Pointer Events (Chrome 55 / December?)

caniuse.com: Can I use pointer events?

Microsoft submitted a proof of concept WebKit patch
html5labs.interoperabilitybridges.com/prototypes/...

Bug 105463 - Implement pointer events RESOLVED WONTFIX

Maciej Stachowiak - [webkit-dev] pointer events speciﬁcation - ﬁrst editors draft

@patrick_h_lauke paraphrasing Apple's stance on Pointer Events...

Apple Pencil currently
indistinguishable from touch
in browser / JavaScript
(no tilt information, ﬁres touch events)

the anatomy of Pointer Events
(sequence, event object, ...)

events fired on tap (IE10/IE11)
mousemove* >
pointerover > mouseover >
pointerenter > mouseenter >
pointerdown > mousedown >
gotpointercapture >
focus >
pointermove > mousemove >
pointerup > mouseup >
lostpointercapture >
pointerout > mouseout >
pointerleave > mouseleave >
click
mouse events fired “inline” with pointer events
(for a primary pointer, e.g. first finger on screen)

current IE10/11 quirk
...
click
the fact that click comes last is a current IE10/11 quirk – according
to spec it should come after pointerup > mouseup , and does already
when using touch-action
W3C Pointer Events WG mailing list - Jacob Rossi (Microsoft)

events ﬁred on tap (Edge)
mousemove* >
pointerover > mouseover >
pointerenter > mouseenter >
pointerdown > mousedown >
focus
gotpointercapture >
pointermove > mousemove >
click >
pointerleave > mouseleave
note click ﬁred correctly after pointerup in Microsoft Edge

vendor-prefixed in IE10
MSPointerDown etc
navigator.msMaxTouchPoints
-ms-touch-action
unprefixed in IE11 (but prefixed versions still mapped for compatibility)

/* Pointer Events extend Mouse Events
vs Touch Events and their completely new objects/arrays */
interface PointerEvent : MouseEvent {
readonly attribute long pointerId;
readonly attribute long width;
readonly attribute long height;
readonly attribute float pressure;
readonly attribute float tangentialPressure; /* Level 2 */
readonly attribute long tiltX;
readonly attribute long tiltY;
readonly attribute long twist; /* Level 2 */
readonly attribute DOMString pointerType;
readonly attribute boolean isPrimary;
}

handling mouse input is
exactly the same as traditional
mouse event
(only change pointer* instead of mouse* event names)

simple feature detection for
pointer events

/* detecting pointer events support */
if ( window.PointerEvent ) {
/* some clever stuff here but this covers
touch, stylus, mouse, etc */
}
/* still listen to click for keyboard! */

"don't forget about keyboard users" note in Pointer Events spec

/* detect maximum number of touch points */
if ( navigator.maxTouchPoints > 0 ) {
/* device with a touchscreen */
}
if ( navigator.maxTouchPoints > 1 ) {
/* multitouch-capable device */
}

pointer events
vs
limitations/problems of mouse
event emulation

(IE11/WinPhone 8.1 Update no optimization for width=device-width )

(IE/Win8 has double-tap to zoom, so problem on desktop too)

(Microsoft Edge/Win10 has double-tap to zoom, so problem on desktop too)

pointer / mouse events and delay
...
[300ms delay]
click
...
300ms delay just before click event

we could try a similar
approach to touch events...

• double-up pointerup and click listeners?
• prevent code ﬁring twice with preventDefault ?
won't work: preventDefault() stops mouse compatibility events, but
click is not considered mouse compatibility event

a more declarative approach
with touch-action

CSS property
what action should the browser handle?
touch-action: auto | none | [ pan-x || pan-y ] | manipulation
www.w3.org/TR/pointerevents/#the-touch-action-css-property
only determines default touch action, does not stop compatibility
mouse events nor click

Pointer Events Level 2
expanded set of values (useful for pull-to-refresh, carousels, etc)
touch-action: auto | none |
[ [ pan-x | pan-left | pan-right ] ||
[ pan-y | pan-up | pan-down ] ] |
manipulation
w3c.github.io/pointerevents/#the-touch-action-css-property

touch-action:none
(suppress all default browser behaviour)

patrickhlauke.github.io/touch/tests/event-listener_touch[...]

touch-action:none kills
scrolling, long-press,
pinch/zoom

touch-action:manipulation
(suppress double-tap-to-zoom)

Bug 979345 - Implement touch-action:manipulation [...]

Issue 349016: Add support for touch-action:manipulation

Bug 149854 - Implement touch-action: manipulation; for iOS

caniuse.com: Can I use touch-action?

"Allow zooming on all web content" reintroduces delay

"Allow zooming on all web content" reintroduces delay
patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

"Allow zooming..." option currently missing in Win 10 Mobile

width=device-width still has delay in IE11/WinPhone 8.1 Update
patrickhlauke.github.io/touch/tests/event-listener_width-device-width.html

Microsoft Edge removes delay for width=device-width
patrickhlauke.github.io/touch/tests/event-listener_width-device-width.html

MSDN: Using aria-haspopup to simulate hover on touch-enabled devices
dirty/inappropriate hack, but interesting experiment nonetheless

patrickhlauke.github.io/touch/css-dropdown/mouse-only-aria-haspopup.html

Channel9 MSDN - IE11: Using Hover Menus with Touch
tap-and-hold to trigger hover menus...

mousemove / pointermove ﬁre, but browser scroll action takes over

you can "fake" it with
touch-action:none
and listen for mouse events...

patrickhlauke.github.io/touch/particle/2a
(does not work in Microsoft Edge/Windows 10 Mobile
due to touch events support)

better: just listen to
pointermove...

no need for separate mouse or
touch event listeners

/* touch events: separate handling */
foo.addEventListener('touchmove', ... , false);
foo.addEventListener('mousemove', ... , false);
/* pointer events: single listener for mouse, stylus, touch */
foo.addEventListener(' pointermove ', ... , false);

no need for separate mouse or
touch code to get x / y coords

/* Pointer Events extend Mouse Events */
foo.addEventListener(' pointermove ', function(e) {
...
posX = e.clientX ;
posY = e.clientY ;
...
}, false);
www.w3.org/TR/pointerevents/#pointerevent-interface

3D Rotator by Creative Punch
coded to only use mouse events

3D Rotator modiﬁed to use Pointer Events
minimal code changes, as Pointer Events extend mouse events

Basic mouse-driven fake slider
Exercise/demo: convert mouse-driven code to use Pointer Events

but you can distinguish
mouse or touch or stylus

foo.addEventListener('pointermove', function(e) {
...
switch( e.pointerType ) {
case ' mouse ':
...
break;
case ' pen ':
...
break;
case ' touch ':
...
break;
default : /* future-proof */
}
...
} , false);

/* in IE11/Edge, pointerType returns a string
in IE10, the return type is long */
MSPOINTER_TYPE_TOUCH: 0x00000002
MSPOINTER_TYPE_PEN: 0x00000003
MSPOINTER_TYPE_MOUSE: 0x00000004
MSDN: IE Dev Center - API reference - pointerType property

/* PointerEvents don't have the handy TouchList objects,
so we have to replicate something similar... */
var points = [];
switch (e.type) {
case ' pointerdown ':
/* add to the array */
break;
case ' pointermove ':
/* update the relevant array entry's x and y */
break;
case ' pointerup ':
case ' pointercancel ':
/* remove the relevant array entry */
break;
}

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer.html
(note multiple isPrimary pointers, per pointer type)

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer-hud.html

simultaneous use of inputs is
hardware-dependent
(e.g. Surface 3 "palm blocking" prevents concurrent
touch/stylus/mouse, but not
touch/external mouse/external stylus)

YouTube: Pointer Events support multiple stylus/pen inputs simultaneously

/* like iOS/Safari, IE/Win has higher-level gestures ,
but these are not part of the W3C Pointer Events spec.
Replicate these from basic principles again... */
MSDN IE10 Developer Guide: Gesture object and events

extended capabilities
(if supported by hardware)

/* Pointer Events - pressure */
readonly attribute float pressure ;
readonly attribute float tangentialPressure;
readonly attribute long twist;
}
pressure : value in range 0 – 1 . if no hardware support,
0.5 in active button state, 0 otherwise

patrickhlauke.github.io/touch/tracker/...
YouTube: Touch tracker with Surface 3 pen

hovering stylus
•   hardware-dependant
•   pointermove ﬁres
•   pressure == 0 (non-active button state)
•   track pointerdown / pointerup to be safe

/* Pointer Events - contact geometry */
readonly attribute long width ;
readonly attribute long height ;
}
if hardware can't detect contact geometry, either 0 or "best guess"
(e.g. for mouse/stylus, return width / height of 1 )

patrickhlauke.github.io/touch/tracker/tracker-width-height.html
YouTube: Pointer Events width and height ...

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer-hud.html
(Nokia Lumia 520: pressure is 0.5 - active state - and width / height is 0 )

/* Pointer Events - tilt */
readonly attribute long tiltX ;
readonly attribute long tiltY ;
}
tiltX / tiltY : value in degrees -90 – 90 .
returns 0 if hardware does not support tilt

patrickhlauke.github.io/touch/pen-tracker
YouTube: ...pen with tilt, pressure and hover support

patrickhlauke.github.io/touch/pen-tracker
YouTube: Chrome on Samsung Galaxy Note 4 - Pointer Events and stylus

pointermove ﬁres if any
property changes,
not just x/y position
( width , height , tiltX , tiltY , pressure )

pointer capture
(implicit vs explicit)

• touch events have implicit capture: once you start a touch movement
on an element, events keep firing to the element even when moving
outside the element's boundaries
• pointer events behave like mouse events: only fire events to an
element while pointer (e.g. finger on touchscreen) inside element, but
you can explicitly capture pointers

/* events related to pointer capture */
gotpointercapture / lostpointercapture
/* example of how to capture a pointer explicitly */
element.addEventListener('pointerdown', function(e) {
this. setPointerCapture(e.pointerId) ;
}, false }
/* capture automatically released on pointerup / pointercancel
or explicitly with releasePointerCapture() */
patrickhlauke.github.io/touch/tests/pointercapture.html

setPointerCapture also
works for mouse input
can be used instead of traditional "bind mousemove to
document / body on mousedown " approach

MSDN IEBlog: The Mobile Web should just work for everyone
Windows Phone 8.1 Update now supports Pointer Events and Touch Events

MSDN IEBlog: Making the web “just work” with any input: Mouse, Touch, and Pointer Events

"frankensteining"
pointer and touch events

[300ms delay] >
mouseover > mousemove > mousedown > mouseup > click
vs
pointerover > mouseover > pointerdown > mousedown >
pointermove > mousemove > pointerup > mouseup >
[300ms delay] > click >
pointerout > mouseout > pointerleave > mouseleave

events ﬁred on tap (IE11)
mousemove* > pointerover > mouseover >
pointerenter > mouseenter > pointerdown >
touchstart >
mousedown > gotpointercapture > focus >
pointermove > touchmove > mousemove >
pointerup >
touchend >
mouseup > lostpointercapture > pointerout > mouseout >
click
IE11/Windows Phone 8.1u1
with "frankensteined" Pointer/Touch Events support
(but not on desktop)

events ﬁred on tap (Edge 13)
mousemove >
pointerover > mouseover > pointerenter > mouseenter >
pointerdown >
touchstart >
mousedown > focus
gotpointercapture
pointermove > mousemove > pointerup >
touchend >
mouseup > click >
pointerout > mouseout > pointerleave > mouseleave
essentially, relevant Touch Events after pointerdown and pointerup

correctly ﬁres touch events on scroll/zoom
patrickhlauke.github.io/touch/gesture-touch/

touch events only turned on
for mobile

W3C Touch Events WG mailing list
Jacob Rossi - Enabling Touch Events everywhere

about:flags in Microsoft Edge to turn on touch events on desktop
(e.g. touch-enabled laptops)

transitional event handling
(until all browsers support pointer events)

/* cover all cases (hat-tip Stu Cox) */
if (window.PointerEvent) {
/* bind to Pointer Events: pointerdown, pointerup, etc */
} else {
/* bind to mouse events: mousedown, mouseup, etc */
/* bind to Touch Events: touchstart, touchend, etc */
}
}
/* bind to keyboard / click */

polyﬁlls for pointer events
(code for tomorrow, today)

GitHub - jQuery Pointer Events Polyﬁll (PEP)

/* adding jQuery PEP */
<script src="https://code.jquery.com/pep/0.4.1/pep.js"></script>
/* need to use custom touch-action attribute, not CSS (yet) */
<button touch-action="none" >...</div>

patrickhlauke.github.io/touch/pep/listener/event-listener.html

events ﬁred on tap with PEP
pointerover > pointerenter > pointerdown >
touchstart >
pointerup > pointerout > pointerleave >
touchend >
mouseover > mouseenter > mousemove >
mousedown > mouseup >
click
essentially, relevant Pointer Events before touchstart and touchend

3D Rotator modiﬁed to use Pointer Events (and PEP)

Basic pointer events-driven fake slider with setPointerCapture
Exercise/demo: make slider work in non-Pointer-Events browsers

utility libraries
(because life is too short to hand-code gesture support)

/* Hammer's high-level events example */
var element = document.getElementById('test_el');
var hammertime = new Hammer(element);
hammertime.on("swipe",
function(event) {
/* handle horizontal swipes */
});

patrickhlauke.github.io/touch/hammer/swipe

nothing beats having real devices...

BrowserStack (includes Physical Devices)

Chrome DevTools / Remote Debugging Android Devices
Chrome DevTools / Using the Console / Monitor events

enable touch events even without a touchscreen
(to test "naive" 'ontouchstart' in window code)
chrome://ﬂags/#touch-events

Chrome DevTools / Device Mode & Mobile Emulation
correctly emulates touch events

correctly emulates pointer events

correctly emulates pointer events (with pointerType=="touch" )

emulate a touch interface (even without mobile emulation)

Firefox Developer Tools > Responsive Design Mode
correctly emulates touch events

no touch emulation, nor touch events + pointer events (like on real
Windows 10 Mobile) emulation, in Edge/F12 Tools

no touch emulation in Safari "Responsive Design Mode"

•   Matt Gaunt – Touch Feedback for Mobile Sites
•   Jonathan Stark – FastActive
•   Stephen Woods – HTML5 Touch Interfaces
•   YouTube: Stephen Woods – Responsive HTML5 Touch Interfaces
•   Chris Wilson + Paul Kinlan – Touch And Mouse: Together Again For
The First Time
•   Ryan Fioravanti – Creating Fast Buttons for Mobile Web Applications
•   Boris Smus – Multi-touch Web Development
•   Boris Smus – Generalized input on the cross-device web
•   Boris Smus – Interactive touch laptop experiments

•   Rick Byers + Boris Smus (Google I/O) – Point, Click, Tap, Touch -
Building Multi-Device Web Interfaces
•   Grant Goodale – Touch Events
•   W3C – Touch Events Extensions
•   Mozilla Developer Network – Touch Events
•   WebPlatform.org – Pointer Events
•   Rick Byers – The best way to avoid the dreaded 300ms click delay is
to disable double-tap zoom
•   Chromium Issue 152149: All touch-event related APIs should exist if
touch support is enabled at runtime

•   Tim Kadlec – Avoiding the 300ms Click Delay, Accessibly
•   David Rousset - Unifying touch and mouse [...]
•   Microsoft – Pointer events updates (differences between IE10-IE11)
•   Patrick H. Lauke – Webseiten zum Anfassen
•   Patrick H. Lauke – Drei unter einem Dach: Pointer-Events für Maus,
Stylus und Touchscreen
•   Patrick H. Lauke – Erweiterte Interaktionsmöglichkeiten mit
Touchscreens
•   Patrick H. Lauke – Make your site work on touch devices
•   Stu Cox – You can't detect a touchscreen

•   Microsoft – Hover touch support (IE10/IE11)
•   W3C Media Queries Level 4 – pointer
•   Stu Cox – The Good & Bad of Level 4 Media Queries
•   Peter-Paul Koch – Touch table
•   Peter-Paul Koch – Preventing the touch default
•   Peter-Paul Koch – Mouse event bubbling in iOS
•   YouTube: Edge Conference (Feb 2013 London) – Panel: Input
•   Edge Conference (Mar 2014 London) – Panel: Pointers and Interactions
•   Trent Walton – Device-Agnostic

•   Stu Cox – The Golden Pattern for Handling Touch Input
•   Matt Gaunt – ‘Focusing’ on the Web Today
•   Mobiscroll – Working with touch events
•   Peter-Paul Koch – The iOS event cascade and innerHTML
•   Patrick H. Lauke – Make your site work on touch devices
•   Scott Jehl (Filament Group) – Tappy: A custom tap event handler
•   Yehuda Katz – Touch events on the web are fundamentally unﬁnished

•   Andrea Giammarchi – PointerEvents No More
•   YouTube: Matt Gaunt (Google) - Touch in a Web App? No Problem
•   Luke Wroblewski – How to Communicate Hidden Gestures
•   David Washington - Designing custom touch interactions
•   David Washington - Building pull-to-refresh
•   Andy Peatling - JavaScript Pull to Refresh for the Web
•   Tilo Mitra – The State of Gestures
•   YouTube: Tilo Mitra (YUI) – The State of Gestures

•   Rob Larsen - The Uncertain Web: Pointer Event Polyﬁll and Chrome
Fragmentation
•   Detlev Fisher - Implications of new input modes (touch, speech,
gestures) for the Web Content Accessibility Guidelines
•   Ralph Thomas - Towards declarative touch interactions
•   Windows Dev Center: Windows desktop applications > Guidelines >
Interaction (touch, keyboard, mouse, pen)
•   Microsoft Open Technologies - jQuery Adopts Pointer Events

•   jQuery blog - Getting on Point
•   IEBlog - Pointer Events W3C Recommendation, Interoperable Touch,
and Removing the Dreaded 300ms Tap Delay
•   Microsoft Open Technologies - Pointer Events is now a W3C Standard
•   Patrick H. Lauke (The Paciello Group) - Pointer Events advance to W3C
Recommendation
•   Jacob Rossi - The Input Shouldn't Matter
•   hacks.mozilla.org - Pointer Events now in Firefox Nightly

•   Rick Byers – Interoperability Case Studies at BlinkOn 5 (touch-action)
•   Mozilla Developer Network – Pointer Events
•   Christian Saylor – A Touch of Brilliance: Why 3D Touch Matters
•   Lukas Mathis – Usability on Mobile Is Getting Worse
•   Rocket Insights – Initial thoughts on 3D Touch
•   Wiser Coder – Disable “pull to refresh” in Chrome for Android

•   Google Developers / Web Fundamentals: Add touch to your site
•   Alex Gibson – Different ways to trigger touchcancel in mobile
browsers
•   BBC Global Experience Language: How to design for touch
•   Google BlinkOn 5: Lightning talks / Mustaq Ahmed – Pointer Events
in Chrome implementation update
•   A Book Apart: Josh Clark – Designing for Touch

•   Ruadhán O'Donoghue – The HTML5 Pointer Events API: Combining
touch, mouse, and pen
•   Jordan Staniscia – Hover is dead. Long live hover.
•   Jason Grigsby – Four Truths About Input
•   Jason Grigsby – Adapting to Input
•   David Corbacho – Debouncing and Throttling Explained
•   YouTube: Pre-Touch Sensing for Mobile Interaction (Microsoft)

•   YouTube: HTML5DevConf: Jacob Rossi, "Pointer Events, Panning &
Zooming, and Gestures"
•   YouTube: Jacob Rossi, "Pointing Forward" at W3Conf 2013
•   YouTube: Tim Park: Pointing Forward (updated) – JSConf.Asia 2013
•   Flickity – touch, responsive, ﬂickable carousels

youtube.com/watch?v=AZKpByV5764

get in touch
@patrick_h_lauke
github.com/patrickhlauke/touch
patrickhlauke.github.io/getting-touchy-presentation
slideshare.net/redux
paciellogroup.com
splintered.co.uk

Getting touchy - an introduction to touch and pointer events (1 day workshop) / JavaScript Days 2016 / Berlin / 10 October 2016

Recommended

Recommended

More Related Content

What's hot

What's hot (13)

Similar to Getting touchy - an introduction to touch and pointer events (1 day workshop) / JavaScript Days 2016 / Berlin / 10 October 2016

Similar to Getting touchy - an introduction to touch and pointer events (1 day workshop) / JavaScript Days 2016 / Berlin / 10 October 2016 (20)

More from Patrick Lauke

More from Patrick Lauke (20)

Recently uploaded

Recently uploaded (20)

Getting touchy - an introduction to touch and pointer events (1 day workshop) / JavaScript Days 2016 / Berlin / 10 October 2016