SlideShare a Scribd company logo
1 of 174
Download to read offline
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Human-Computer Interaction
from natural interaction to ubicomp
cdn-images-1.medium.com/max/2000/1*YGT4bQea3spMLg2_tYkOjw.jpeg
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
“The details are not the details.
They make the design.”
Charles Eames
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
How about interacting with “special” devices
offering natural interfaces?
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
Sense Relevant information
Audio
Speech, Identity, Intonation,
Precision (timing)
Vision
Identity, Facial expression, Body language,
Gesture, Accuracy (spatial)
Tactile/
somesthetic
Tabs/pads/devices, Texture,
Precision (timing), Accuracy (spatial)
Scent Atmosphere, likability
Taste Clarification, enjoyment
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
The interpretation of sensorial data
depends on user expectations
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
M. Nielsen, T. Moeslund, M. Storring & E. Granum (2008)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
Also, the interpretation of sensorial data
depends on specific input devices (K. Hinckley, ‘08)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
Also, the interpretation of sensorial data
depends on specific input devices (K. Hinckley, ‘08)
mouse, trackball, isometric joystick,
touchpad, touchscreen, pen-operated device, watch,
camera, microphone, indirect table,…
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
Interaction modes
specify how an user interface responds to user actions
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
Interaction modes
specify how an user interface responds to user actions
for a particular manner of performing a task,
UI adopts a specific mode if the interpretation of this task
is always the same
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
Interaction modes
when a task has a different interpretation,
the UI offers another interaction mode
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
Interaction modes
when a task has a different interpretation,
the UI offers another interaction mode
modal vs. modeless interaction
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
Interaction modes
an interface could be modal for a given action,
and modeless for another (Jef Raskin, 2000)
if an UI is considered modeless,
then that UI must not be modal for any other action
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
Interaction modes
changing the interaction mode should have
a solid motivation
…even in the context of (mobile) Web
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Haptic  Gesture  Locomotion
Auditory & Voice  Tangible
natural interaction
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
(O’Malley & Gupta, 2008)
based on tactile sensations to provide information
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
(O’Malley & Gupta, 2008)
based on tactile sensations to provide information
typical examples:
haptic screen for “touching” objects
natural output – e.g., vibrations (via tactons)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
cutaneous touch versus kinesthetic touch
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
used in various contexts and domains:
CAD (Computer-Aided Design)
design prototyping
product evaluation
simulation of exceptional situations
remote task execution (tele-operation)
mobile & wearable computing
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
usually, a haptic environment is defined by a formal model
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
example:
virtual wall – used in 3D games/simulations
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
fixed haptic interfaces
haptic joysticks
pen-based haptic devices
floor- and ceiling-mounted interfaces
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
M. O’Malley, A. Gupta, “Haptic Interfaces”,
in P. Kortum (Ed.), HCI Beyond the GUI, Elsevier, 2008
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
portable haptic interfaces
exoscheletons
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Light Exoskeleton (L-Exos)
described by Frisoli et al. (2005)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
PowerArm HCI project: Arduino, Android & Unity3D
Dan-Gabriel Bratu, Daniel Vicol, Sergiu Recean (FII, 2013)
profs.info.uaic.ro/~stefan.negru/studentprojects/powerarm.html
case study
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
tactile interfaces
could detect heat, pressure, vibrations, pain
the feedback is usually provided by the user fingers
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
tactile interfaces
vibrotactile interfaces
wearable tactile interfaces
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
tactile interfaces
vibrotactile interfaces
wearable tactile interfaces
useful in the context of user accessibility
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Moose haptic interface (O’Modhrain & Gillespie, 1995)
ccrma.stanford.edu/files/papers/stanm95.pdf
case study
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
AHNE – Audio-Haptic Navigation Environment
(SOPI Research Group, Aalto University, 2011)
user tracking via Kinect + OpenNI & OSCeleton
vimeo.com/28447850
case study
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
aspects regarding human factors:
understanding bio-mechanical, sensorial-motor,
and cognitive abilities of users (Tan et al., 1994)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
accuracy of movement measurement
(position, velocity, acceleration)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
accuracy of movement measurement
(position, velocity, acceleration)
in the mobile Web context, see HTML5 APIs
e.g., Touch, Vibration, Screen Orientation, Gamepad, Media Capture
www.w3.org/2018/04/web-roadmaps/mobile/
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
design principles:
base your mechanical design
on the inherent capabilities of the human operator
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
design principles:
consider human sensitivity to tactile stimuli
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
design principles:
use active rather than passive movement
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
design principles:
minimize confusion and control instabilities
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
design principles:
ensure realistic display of environments
with tactile devices
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Haptic UI
specific uses:
(tele-)medicine
e.g., laparoscopic surgery simulator, palpation simulator
3D surface modeling
learning the interaction with touchscreen devices
digital art
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
using hand tracking data, VRgluv employs predictive
physics algorithms to calculate the size and stiffness of
virtual objects and recreates them in your hands
vrgluv.com
case study
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Pseudo-Haptic UI
(software) emulation of a haptic device
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Pseudo-Haptic UI
sensorial substitution (Lenay et al., 2003)
a haptic stimulus is substituted by another sense
(e.g., an audio signal simulating
the contact with virtual objects)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Pseudo-Haptic UI
offering pseudo-haptic feedback (Anatole Lecuyer, 2007)
simulation of stiffness via a combination of
the interaction with the input device & the visual feedback
pseudo-haptic simulation of textures
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Simulation-based Palpation using Pseudo-haptic Feedback
(Min Li et al., 2012)
www.youtube.com/watch?v=OcIczYCUkmc
case study
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
inspired by non-verbal interactions between people
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
inspired by non-verbal interactions between people
alternative or complementary to traditional interactions
“tap is the new click”
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
“While natural gestures are often subtle,
gesture interfaces rely on emphasized gestures.”
M. Nielsen et al., 2008
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
types:
hand gestures for pointing out objects
or controlling a mouse-like pointer
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Interacţiune neconvenţională
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
types:
motion-tracking interfaces
used by several game consoles or laptops
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
types:
facial gestures
useful in detection of reactions and emotions
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
usually, based on specific input devices
mechanical/magnetic – e.g., mouse, data glove, Wii,…
single/multi-point touch
examples: smart phones, tablets
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
could be “invisible” for the user
detecting gestures via computer vision:
segmentation – extracting the relevant fragments
tracking – detecting the movements of interest
classification – obtaining the desired information
(e.g., via machine learning, neural nets,…)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Interacţiune neconvenţională
general architecture of a gesture-based interactive system
(Moustakas et al., 2006)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
gesture classification:
semantic – the meaning of a gesture
functional – what the gesture does in an interface
descriptive – how the gesture is performed
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
functionally, gestures can signify:
command – Quit, Undo, Configure
pointing – e.g., selecting an item of interest
manipulation – example: scaling/rotating an image
control – emulates the control over an entity
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
dynamic + spatial-temporal “classical” gestures
performed via an input device: mouse, stylus,…
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interactionMaterial Design – types of gestures:
navigational
(tap, scroll & pan, drag, swipe, pinch)
action
(tap, long press, swipe)
transform
(double tap, pinch, compound gestures, pick up & move)
material.io/design/interaction/gestures.html
case study
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
case study
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
the interaction vocabulary should be limited
we must assure an optimal interaction space (spatial zone)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
“special” interactive zones having specific semantics
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
technology- versus human-based gestures
causes of discomfort:
stressful/fatigue producing for the user
nearly impossible for some people to perform
illogically imposed functionality
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
some gestures easily
recognizable by software
meanings/actions difficult
to be understood/performed
by the human users:
T0: without signification
(residue)
T1: select
T2: copy & paste
T3: delete
T4, T5: menu/release
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
a gesture-based UI must:
be intuitive
take into consideration the user mental model
be easy to learn & deployed (ergonomics)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
practical design advices:
be tolerant to human errors
beware of cultural factors
consider the context
provide positive feedback
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
patterns for touchscreens & interactive surfaces
(Dan Saffer, 2009)
tap to open/activate
tap to select
drag to move object
slide to scroll, spin to scroll
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
patterns for touchscreens & interactive surfaces
(Dan Saffer, 2009)
slide and hold for continuous scroll
tap to stop
pinch to shrink and spread to enlarge
two fingers to scroll
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
gesture notation (Luke Wroblewski, 2010)
www.lukew.com/ff/entry.asp?1071
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
patterns for free-form interactive gestures
(Dan Saffer, 2009)
move body to activate
point to select/activate
wave to activate
place hands inside to activate
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Gesture interaction
patterns for free-form interactive gestures
(Dan Saffer, 2009)
rotate to change state
step to activate
shake to change
tilt to move
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
device specific gesture notation – for example: Kinect
gesturecons.com
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
locomotion design
natural interaction
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
interfaces that both enable users to move around
in real or virtual spaces and make users feel
as if they are moving
Mary C. Whitton & Sharif Razzaque, 2008
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
could be virtual, not only physical
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
could be virtual, not only physical
main problem:
to capture the user’s intent using data that can be derived
from sensing the pose (position and orientation)
and movement of the user’s body
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
discussion
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
perceiving self-motion
each sensorial channel (visual, auditory, vestibular,…) is
used in detection of user or surrounded objects movement
Dichgans & Brandt, 1977
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
detection of pose and motion
artificial sensors (trackers)
optical motion capture
mechanical solutions
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
optical motion
capture
optical motion
capture
exoskeleton
trackers
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
case study
interactive climbs
project360.mammut.ch
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
providing feedback to user(s)
visual information:
head-mounted displays (HMDs)
surround-screen displays
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
walking simulation
treadmills (Brooks, 1998; Hollerbach et al., 2000)
walking-in-place (WIP) interfaces
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
design principles (Whitton & Razzaque, 2008):
making the interface feel natural
higher-fidelity reproduction of human way of walking
avoiding simulator sickness – cybersickness
dspace.dsto.defence.gov.au/dspace/bitstream/1947/4079/1/DSTO-TR-1419%20PR.pdf
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
a VR treadmill allowing users to maneuver
a scene using their feet (Nick Randolph, 2018)
medium.com/uxxr/the-user-experience-of-virtual-reality-c464762deb8e
case study
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
issues that can be detected in the testing phase:
realism + preservation of spatial understanding
cognitive distraction
break-in-presence (BIP) events
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
an action that causes break-in-presence
(virtual scenephysical location)
details in A. Steed, A. Brogni, V. Vinayagamoorthy,
“Breaks in Presence as Usability Criteria”, HCII 2005
www0.cs.ucl.ac.uk/staff/v.vinayagamoorthy/pdf/hcii2005.pdf
discussion
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
advice:
match the locomotion metaphor to the interface goals
walking metaphor versus vehicle metaphor
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
practical considerations:
Is viewing direction independent of motion direction?
Are hands used for locomotion?
Able to move in any direction and change direction easily?
Can walking speed or step length be set for each user?
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Locomotion interfaces
practical considerations:
Are tracker sensors immune to other signals in the room?
Are wireless trackers used?
What parts of body will have sensors/markers on them?
Cables – if any – interfere with user’s motion?
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
case study
motion studies in the context of modern art
Universal Everything – universaleverything.com
other examples: prostheticknowledge.tumblr.com/tagged/motion
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Interacţiune neconvenţională
auditory & vocal interactions
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Auditory interfaces
using non-speech audio information for interaction
for details, see (S. Camille Peres et al., 2008)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Auditory interfaces
motivations and uses:
presenting information to visually impaired people
providing an additional information channel
alerting people to error or emergency states of a system
providing information via devices with small screens
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Auditory interfaces
why an audio interaction?
reducing visual overload
reinforcing visual messages
when eyes are elsewhere
when audio is more informative
conveying emotion – e.g., for electronic games
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Auditory interfaces
sonification = using non-speech sound to render data
the simplest method is audification
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Auditory interfaces
representational
auditory icons
abstract
earcons
Stephen Brewster (2008)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Auditory interfaces
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Auditory interfaces
problems regarding the audio interaction:
annoyance
privacy
auditory overload
interference/masking
low resolution
lack of familiarity
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Auditory interfaces
aspect of interest:
the tasks performed by the user via an audio interface
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Auditory interfaces
aspect of interest:
what (kind of) information could be exclusively
represented by sounds?
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Auditory interfaces
aspect of interest:
the suitable format audio – lossless vs. lossy – to be used
FLAC, PCM, WAV versus AAC, OGG, MP3
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Auditory interfaces
aspect of interest:
producing the sound
thinking about sound as information
designing the sound – consider auditory content as
a kind of sound ecology (Walker & Kramer, 2004)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
audio-only games – www.audiogames.net
case study
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
facilitating the conversation between software and user
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
important aspect: speech recognition
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
user perceptions regarding a VUI are fragile
lack of trust (Susan Hura, 2008)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
users interact generally via prompts
“Hello, you have to choose from the following options…”
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
several discourse makers are needed for:
confirmation – “yes, of course”, “alright”
changing the topic of conversation
emphasizing a statement – “as we previously said…”
narrative sequences – “so,…”
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
problem: the lack of concision
“I can help you with the following five options. You can
interrupt me and speak your choice at any time. Please,
select one of the following: sign up for new service, add
features to my service, move my existing service, problems
with my satellite service, or ask a billing question.”
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
problem: dealing with errors
“I’m sorry. I did not understand your response.
I can help you with the following five options…”
inducing the feeling of user culpability
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
problem: increasing the user discomfort
“Please say or enter your account number!”
the user cannot control the interface
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
problem: using the same message (in different contexts)
“Attention! I don’t understand! Please, indicate the correct answer!...
Attention! I don’t understand! Please, indicate the correct answer!...
Attention! I don’t understand! Please, indicate the correct answer!...”
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
speech recognition – main phases (Cohen et al., 2004)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
design guidelines:
present menu options in a way
that makes them comprehensible and easy to retain
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
design guidelines:
consider error sources when
writing error-handling prompts
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
design guidelines:
provide context
VUI: You can say: account balances, last contribution, transfers,
allocations, or life insurance.
User: Transfers.
VUI: Transfers. You can say: hear pending transfers
or make a transfer.
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
design guidelines:
use hints and reminders
“Next time, you can say ‘order status’ at the main menu
to get here quicker.”
“You can speak your account number or key it in.”
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
design guidelines:
favor models over instructions
learning by doing
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
design guidelines:
use discourse markers liberally
using in conversation common expressions
like “all right”, “next”, “thanks”,…
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
design guidelines:
allow for conversational repair
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
design guidelines:
be careful with terminology
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
recent examples – mobile, Web & smart home contexts:
Amazon Alexa + Amazon Echo
developer.amazon.com/alexa
developer.amazon.com/echo
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
recent examples – mobile, Web & smart home contexts:
Apple Siri (iOS, watchOS)
developer.apple.com/sirikit/
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
recent examples – mobile, Web & smart home contexts:
Google Assistant
developers.google.com/assistant/sdk/
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Voice user interfaces
recent examples – mobile, Web & smart home contexts:
Microsoft Cortana in conjunction to Azure Bot
developer.microsoft.com/cortana
azure.microsoft.com/services/bot-service/
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
case study
A home assistant able to announce who is home
by using facial recognition
(Juan Perez & Robin Cole, 2018)
image data captured via Webcam and processed locally
with Facebox – machinebox.io/docs/facebox – or on cloud
by using Amazon Rekognition or Google Vision
announcement speaker uses Google Text-To-Speech
service, integrated into Home Assistant – an open-source
home automation platform: www.home-assistant.io
www.hackster.io/97766/announce-who-is-home-using-facial-recognition-dcc389
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
tangible interfaces
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Tangible interfaces
digital informationphysical form (representation)
Hiroshi Ishii
digital content could be directly perceived & manipulated
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Tangible interfaces
key components:
control – permits users to manipulate the content
external representations – perceived by human senses
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Tangible interfaces
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Tangible
Manipulation
haptic
direct
manipulation
lightweight
interaction
isomorph
effects
Spatial
Interaction
inhabited space
configurable
materials
visual access
full body
interaction
performative
action
Embodied
Facilitation
embodied
constraints
multiple access
points
tailored
(adaptive)
representations
Expressive
Represen-
tation
representation-
nal significance
externalization
perceived
coupling
Eva Hornecker (2010)
see also www.ehornecker.de/TangiblesFramework.html
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Tangible Interaction Conceptual Framework
responding to key design questions via cards
Can users grab, feel and move “the important stuff”?
Do people and objects meet and invite into interaction?
Can you create a meaningful place with atmosphere?
Are representations legible, meaningful, and expressive?
Does shifting stuff (or your own body)
around have meaning?
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Tangible interfaces
for other resources + demos, consult:
Eva Hornecker, Tangible Interaction (2015)
www.interaction-design.org/encyclopedia/tangible_interaction.html
Tangible Interfaces discipline – mas834.media.mit.edu
Tangible Media Group @ MIT – tangible.media.mit.edu
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
multimodal interaction
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
multimodal interaction
Processing two or many inputs – voice, gestures,
tactile, etc. – in a coordinated manner in order to
obtain a complex output (e.g., multimedia, 3D) or
to perform a task
Sharon Oviatt, 2008
Paulo Barthelmess & Sharon Oviatt, 2008
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
multimodal interaction
“The situation where the user is provided with
multiple modes for interacting with a system.”
Abdo El Ali, 2011
www.slideshare.net/Abd0/multimodal-interaction-an-introduction
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Interacţiune multimodală
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
multimodal interaction
Diverse contexts are considered
from mobile interfaces to collaborative systems
virtual and/or augmented reality
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
multimodal interaction
Advantages:
error reduction via mutual disambiguation
increasing user performance
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
multimodal interaction
Issues to be resolved:
linguistic factors
redundancy & complementarity
choosing the “right” multi-/mono-modal input
cognitive load
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
discussion
designing multimodal interaction for info kiosks
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
discussion
designing multimodal interaction for info kiosks
aspects of interest (M. C. Maguire, 1999):
choosing a suitable location
encouraging use
providing instructions for using the system
user privacy
multiple input methods: touch, speech, keypad
proper output: language, color, images, feedback,…
navigation possibilities
ui4all.ics.forth.gr/UI4ALL-97/maguire.pdf
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
www.macadamian.com/2015/03/20/design-iot-product/
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
“Invisibly enhancing the world that already exists.”
(Mark Weiser, 1988)
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
“Invisibly enhancing the world that already exists.”
(Mark Weiser, 1988)
ubicomp
pervasive computing
ambient intelligence (AmI)
physical computing
everyware
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
“Invisibly enhancing the world that already exists.”
(Mark Weiser, 1988)
can occur using any device,
in any location,
and in any format
computers disappear in the environment
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
Related topics of research:
context awareness
distributed & mobile computing
Internet of Things (IoT) + wireless sensor networks
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interactionUsing (semantic) Web technologies
to specify & interact with IoT components
describing IoT for machines/users via Web of Things
specifications (W3C, 2018): www.w3.org/WoT/
developing applications by using Things Framework
(Mozilla), a collection of re-usable software components
to build Web things exposing the Web Thing API:
iot.mozilla.org/things/
for further details, visit webofthings.org
discussion
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
Scales of devices
Mark Weiser’s initial proposals (1991)
tabs – centimeter-sized devices:
PDAs, voice recorders, smart-phones, smart-watches,…
vision: individuals own many of them and they can all
communicate with each other and environment
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
interacting with a smart-watch via widgets
UI design considerations in A. Komarov, How We Designed and
Built Our First Apple Watch App (2015)
www.smashingmagazine.com/2015/08/how-we-designed-and-built-our-first-apple-watch-app/
discussion
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
Scales of devices
Mark Weiser’s initial proposals (1991)
pads – handheld decimeter-sized devices:
tablets, laptops, e-readers, etc.
vision: an individual owns several,
but it is not assumed to be always with them
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
Scales of devices
Mark Weiser’s initial proposals (1991)
boards – meter-sized interactive devices:
mounted walls, e-tables, smart info kiosks,…
vision: buildings or institutions own them and
lots of people used them
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Virtual Reality
term coined by Jaron Lanier (1989)
a computer-based simulated 3D environment
intended for its users to inhabit and to interact via avatars
an interface between the “physical” reality
and a virtual environment
discussion
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Virtual Reality – desktop
1st wave (1995) – expensive hardware
HMD (Head Mounted Display)
2nd wave (since 2010)
MxR (Mixed Reality Lab), Oculus Rift
consumer-ready HMDs (2016)
Oculus Rift, Sony Morpheus, HTC/Valve Vive
for further information, study the Mark Billinghurst’s presentations:
www.slideshare.net/marknb00/presentations
discussion
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Augmented Reality
term coined by Thomas Caudell (1990)
the use of technology which allows the perception of
the physical world to be enhanced or modified by
computer-generated stimuli perceived
with the aid of special equipment
generation of virtual objects merged into real scenes
discussion
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
reality-virtuality continuum (Milgram & Kishino, 1994)
consult also www.slideshare.net/busaco/from-virtual-to-augmented-reality
discussion
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interactionMobile phone AR & VR
HMD = VR mount + smartphone
mobile phone AR
smartphone + live camera view + sensor input
(e.g., GPS, compass)
mobile phone VR
smartphone + sensor input + additional VR viewer
case study
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Mobile phone AR & VR
low-cost immersive viewers
case study
VR2GO (MxR Lab, 2013)
projects.ict.usc.edu/mxr/diy/vr2go/
Google Cardboard (2014)
g.co/cardboard
M. Billinghurst, Cardboard VR: Building Low Cost VR Experiences (2016)
www.slideshare.net/marknb00/cardboard-vr-building-low-cost-vr-experiences
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
Mobile phone AR & VR
popular software toolkits:
ARCore (Google) – developers.google.com/ar/
+
Google VR – developers.google.com/vr/
ARKit (Apple) – developer.apple.com/arkit/
case study
M. Billinghurst, Developing AR and VR Experiences with Unity (2017)
www.slideshare.net/marknb00/developing-ar-and-vr-experiences-with-unity
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
AR & VR in the modern Web context
3D content via WebGL
developer.mozilla.org/docs/Web/WebGL
github.com/sjfricke/awesome-webgl
WebXR (draft W3C specification, May 2018)
support for accessing VR + AR devices, including sensors
and head-mounted displays, on the Web
immersive-web.github.io/webxr/
for other details, consult webvr.info/developers/
case study
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
current support provided by
Web browsers: webvr.rocks
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
Scales of devices
new proposals (Stefan Poslad, 2009)
dust – miniaturized devices without visual output
smartdust: a system of many tiny micro-
electromechanical systems (MEMS) that can detect light,
temperature, vibration, magnetism, chemicals,…
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
Scales of devices
new proposals (Stefan Poslad, 2009)
skin – fabrics based upon light emitting, conductive,
polymers, organic computer devices
smart clotheswearable computing
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
case study
www.hackster.io/team-fluxx/fluxx-005ac5
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
Scales of devices
new proposals (Stefan Poslad, 2009)
clay – ensembles of MEMS that can be formed into
arbitrary 3D shapes, resembling physical objects
remember tangible
interfaces?
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing
M. Melnykowycz, Wearable Sensors and UX Development (2014)
www.slideshare.net/Boltzmann/wearable-sensors-and-ux-development
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing: future
Brain-Computer Interface
direct communication between the brain and
an external device
assisting, augmenting, or repairing human cognitive or
sensory-motor functions
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
ubiquitous computing: future
Alexis Ortiz-Rosario, Hojjat Adeli, “Brain-Computer
Interface Technologies: from Signal to Action”,
Reviews in the Neurosciences, 24 (5), 2013
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
natural interaction
case study
noninvasive brain–computer interface systems can restore
functions lost to disability, allowing for spontaneous,
direct brain control of external devices
S. Perdikis et al., “The Cybathlon BCI race: Successful longitudinal
mutual learning with two tetraplegic users”, PLoS Biol, 16(5), 2018
doi.org/10.1371/journal.pbio.2003787
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
“Conclusion”
“The most profound technologies are those
that disappear. They weave themselves into
the fabric of everyday life
until they are indistinguishable from it.”
Mark Weiser, “The Computer for the 21st Century”,
Scientific American, 265 (3), 1991
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
sensorial experience
natural user interfaces
multimodal interaction
ubiquitous computing
“Conclusion”
ukfiremag.mdmpublishing.com/how-augmented-reality-can-assist-fire-and-rescue-safety/
Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Good luck!

More Related Content

What's hot

HCI 2015 (2/10) Human Factor. Users
HCI 2015 (2/10) Human Factor. UsersHCI 2015 (2/10) Human Factor. Users
HCI 2015 (2/10) Human Factor. UsersSabin Buraga
 
HCI 2014 (1 of 10): Overview
HCI 2014 (1 of 10): OverviewHCI 2014 (1 of 10): Overview
HCI 2014 (1 of 10): OverviewSabin Buraga
 
HCI 2015 (6/10) Design Patterns: Social Interaction
HCI 2015 (6/10) Design Patterns: Social InteractionHCI 2015 (6/10) Design Patterns: Social Interaction
HCI 2015 (6/10) Design Patterns: Social InteractionSabin Buraga
 
HCI 2014 (3 of 10): Design Models and Methodologies
HCI 2014 (3 of 10): Design Models and MethodologiesHCI 2014 (3 of 10): Design Models and Methodologies
HCI 2014 (3 of 10): Design Models and MethodologiesSabin Buraga
 
HCI 2015 (7/10) HCI Engineering. UI Evaluation. Models
HCI 2015 (7/10) HCI Engineering. UI Evaluation. ModelsHCI 2015 (7/10) HCI Engineering. UI Evaluation. Models
HCI 2015 (7/10) HCI Engineering. UI Evaluation. ModelsSabin Buraga
 
HCI 2015 (8/10) Data Visualization (an introduction)
HCI 2015 (8/10) Data Visualization (an introduction)HCI 2015 (8/10) Data Visualization (an introduction)
HCI 2015 (8/10) Data Visualization (an introduction)Sabin Buraga
 
HCI 2015 (4/10) Visual Design. Information Architecture. Design Patterns
HCI 2015 (4/10) Visual Design. Information Architecture. Design PatternsHCI 2015 (4/10) Visual Design. Information Architecture. Design Patterns
HCI 2015 (4/10) Visual Design. Information Architecture. Design PatternsSabin Buraga
 
HCI 2015 (5/10) Information Architecture. Design Patterns. Layout, Actions, D...
HCI 2015 (5/10) Information Architecture. Design Patterns. Layout, Actions, D...HCI 2015 (5/10) Information Architecture. Design Patterns. Layout, Actions, D...
HCI 2015 (5/10) Information Architecture. Design Patterns. Layout, Actions, D...Sabin Buraga
 
HCI: Design Patterns for Social Web Interaction
HCI: Design Patterns for Social Web InteractionHCI: Design Patterns for Social Web Interaction
HCI: Design Patterns for Social Web InteractionSabin Buraga
 
HCI 2015 (3/10) Design Methodologies
HCI 2015 (3/10) Design MethodologiesHCI 2015 (3/10) Design Methodologies
HCI 2015 (3/10) Design MethodologiesSabin Buraga
 
HCI 2014 (2 of 10): Human Factor. Users
HCI 2014 (2 of 10): Human Factor. UsersHCI 2014 (2 of 10): Human Factor. Users
HCI 2014 (2 of 10): Human Factor. UsersSabin Buraga
 
HCI 2014 (4 of 10): From Information Architecture to Design Patterns
HCI 2014 (4 of 10): From Information Architecture to Design PatternsHCI 2014 (4 of 10): From Information Architecture to Design Patterns
HCI 2014 (4 of 10): From Information Architecture to Design PatternsSabin Buraga
 
HCI 2014 (10 of 10): Natural User Interfaces. Ubiquitous Computing
HCI 2014 (10 of 10): Natural User Interfaces. Ubiquitous ComputingHCI 2014 (10 of 10): Natural User Interfaces. Ubiquitous Computing
HCI 2014 (10 of 10): Natural User Interfaces. Ubiquitous ComputingSabin Buraga
 
HCI 2014 (8 of 10): Data Visualization
HCI 2014 (8 of 10): Data VisualizationHCI 2014 (8 of 10): Data Visualization
HCI 2014 (8 of 10): Data VisualizationSabin Buraga
 
HCI 2015 (9/10) Affective Factors
HCI 2015 (9/10) Affective FactorsHCI 2015 (9/10) Affective Factors
HCI 2015 (9/10) Affective FactorsSabin Buraga
 
Human-Computer Interaction: An Overview
Human-Computer Interaction: An OverviewHuman-Computer Interaction: An Overview
Human-Computer Interaction: An OverviewSabin Buraga
 
HCI 2014 (7 of 10): HCI Engineering. UI Evaluation. Models
HCI 2014 (7 of 10): HCI Engineering. UI Evaluation. ModelsHCI 2014 (7 of 10): HCI Engineering. UI Evaluation. Models
HCI 2014 (7 of 10): HCI Engineering. UI Evaluation. ModelsSabin Buraga
 
HCI 2014 (6 of 10): Social (Web) Interactions
HCI 2014 (6 of 10): Social (Web) InteractionsHCI 2014 (6 of 10): Social (Web) Interactions
HCI 2014 (6 of 10): Social (Web) InteractionsSabin Buraga
 
HCI 2014 (5 of 10): UI Design. From Design Patterns to Flow
HCI 2014 (5 of 10): UI Design. From Design Patterns to FlowHCI 2014 (5 of 10): UI Design. From Design Patterns to Flow
HCI 2014 (5 of 10): UI Design. From Design Patterns to FlowSabin Buraga
 

What's hot (19)

HCI 2015 (2/10) Human Factor. Users
HCI 2015 (2/10) Human Factor. UsersHCI 2015 (2/10) Human Factor. Users
HCI 2015 (2/10) Human Factor. Users
 
HCI 2014 (1 of 10): Overview
HCI 2014 (1 of 10): OverviewHCI 2014 (1 of 10): Overview
HCI 2014 (1 of 10): Overview
 
HCI 2015 (6/10) Design Patterns: Social Interaction
HCI 2015 (6/10) Design Patterns: Social InteractionHCI 2015 (6/10) Design Patterns: Social Interaction
HCI 2015 (6/10) Design Patterns: Social Interaction
 
HCI 2014 (3 of 10): Design Models and Methodologies
HCI 2014 (3 of 10): Design Models and MethodologiesHCI 2014 (3 of 10): Design Models and Methodologies
HCI 2014 (3 of 10): Design Models and Methodologies
 
HCI 2015 (7/10) HCI Engineering. UI Evaluation. Models
HCI 2015 (7/10) HCI Engineering. UI Evaluation. ModelsHCI 2015 (7/10) HCI Engineering. UI Evaluation. Models
HCI 2015 (7/10) HCI Engineering. UI Evaluation. Models
 
HCI 2015 (8/10) Data Visualization (an introduction)
HCI 2015 (8/10) Data Visualization (an introduction)HCI 2015 (8/10) Data Visualization (an introduction)
HCI 2015 (8/10) Data Visualization (an introduction)
 
HCI 2015 (4/10) Visual Design. Information Architecture. Design Patterns
HCI 2015 (4/10) Visual Design. Information Architecture. Design PatternsHCI 2015 (4/10) Visual Design. Information Architecture. Design Patterns
HCI 2015 (4/10) Visual Design. Information Architecture. Design Patterns
 
HCI 2015 (5/10) Information Architecture. Design Patterns. Layout, Actions, D...
HCI 2015 (5/10) Information Architecture. Design Patterns. Layout, Actions, D...HCI 2015 (5/10) Information Architecture. Design Patterns. Layout, Actions, D...
HCI 2015 (5/10) Information Architecture. Design Patterns. Layout, Actions, D...
 
HCI: Design Patterns for Social Web Interaction
HCI: Design Patterns for Social Web InteractionHCI: Design Patterns for Social Web Interaction
HCI: Design Patterns for Social Web Interaction
 
HCI 2015 (3/10) Design Methodologies
HCI 2015 (3/10) Design MethodologiesHCI 2015 (3/10) Design Methodologies
HCI 2015 (3/10) Design Methodologies
 
HCI 2014 (2 of 10): Human Factor. Users
HCI 2014 (2 of 10): Human Factor. UsersHCI 2014 (2 of 10): Human Factor. Users
HCI 2014 (2 of 10): Human Factor. Users
 
HCI 2014 (4 of 10): From Information Architecture to Design Patterns
HCI 2014 (4 of 10): From Information Architecture to Design PatternsHCI 2014 (4 of 10): From Information Architecture to Design Patterns
HCI 2014 (4 of 10): From Information Architecture to Design Patterns
 
HCI 2014 (10 of 10): Natural User Interfaces. Ubiquitous Computing
HCI 2014 (10 of 10): Natural User Interfaces. Ubiquitous ComputingHCI 2014 (10 of 10): Natural User Interfaces. Ubiquitous Computing
HCI 2014 (10 of 10): Natural User Interfaces. Ubiquitous Computing
 
HCI 2014 (8 of 10): Data Visualization
HCI 2014 (8 of 10): Data VisualizationHCI 2014 (8 of 10): Data Visualization
HCI 2014 (8 of 10): Data Visualization
 
HCI 2015 (9/10) Affective Factors
HCI 2015 (9/10) Affective FactorsHCI 2015 (9/10) Affective Factors
HCI 2015 (9/10) Affective Factors
 
Human-Computer Interaction: An Overview
Human-Computer Interaction: An OverviewHuman-Computer Interaction: An Overview
Human-Computer Interaction: An Overview
 
HCI 2014 (7 of 10): HCI Engineering. UI Evaluation. Models
HCI 2014 (7 of 10): HCI Engineering. UI Evaluation. ModelsHCI 2014 (7 of 10): HCI Engineering. UI Evaluation. Models
HCI 2014 (7 of 10): HCI Engineering. UI Evaluation. Models
 
HCI 2014 (6 of 10): Social (Web) Interactions
HCI 2014 (6 of 10): Social (Web) InteractionsHCI 2014 (6 of 10): Social (Web) Interactions
HCI 2014 (6 of 10): Social (Web) Interactions
 
HCI 2014 (5 of 10): UI Design. From Design Patterns to Flow
HCI 2014 (5 of 10): UI Design. From Design Patterns to FlowHCI 2014 (5 of 10): UI Design. From Design Patterns to Flow
HCI 2014 (5 of 10): UI Design. From Design Patterns to Flow
 

Similar to HCI 2018 (10/10) Natural User Interfaces. Ubiquitous Computing

HCI 2015 (10/10) Natural User Interfaces. Ubiquitous Computing
HCI 2015 (10/10) Natural User Interfaces. Ubiquitous ComputingHCI 2015 (10/10) Natural User Interfaces. Ubiquitous Computing
HCI 2015 (10/10) Natural User Interfaces. Ubiquitous ComputingSabin Buraga
 
Hci03 designmethodologies-150910103121-lva1-app6892
Hci03 designmethodologies-150910103121-lva1-app6892Hci03 designmethodologies-150910103121-lva1-app6892
Hci03 designmethodologies-150910103121-lva1-app6892Ștefan Gordîn
 
HCI 2014 (9 of 10): Affective Factors
HCI 2014 (9 of 10): Affective FactorsHCI 2014 (9 of 10): Affective Factors
HCI 2014 (9 of 10): Affective FactorsSabin Buraga
 
Sketch2presentation
Sketch2presentationSketch2presentation
Sketch2presentationjin.fan
 
Multimodal man machine interaction
Multimodal man machine interactionMultimodal man machine interaction
Multimodal man machine interactionDr. Rajesh P Barnwal
 
Mask
MaskMask
Maskhwh3
 
Web search algorithms and user interfaces
Web search algorithms and user interfacesWeb search algorithms and user interfaces
Web search algorithms and user interfacesStefanos Anastasiadis
 
Interactiune om-calculator (Factorul uman. Utilizabilitatea)
Interactiune om-calculator (Factorul uman. Utilizabilitatea)Interactiune om-calculator (Factorul uman. Utilizabilitatea)
Interactiune om-calculator (Factorul uman. Utilizabilitatea)Sabin Buraga
 
Ubiquitous computing abstract
Ubiquitous computing abstractUbiquitous computing abstract
Ubiquitous computing abstractPriti Punia
 
426 lecture6b: AR Interaction
426 lecture6b: AR Interaction426 lecture6b: AR Interaction
426 lecture6b: AR InteractionMark Billinghurst
 

Similar to HCI 2018 (10/10) Natural User Interfaces. Ubiquitous Computing (12)

HCI 2015 (10/10) Natural User Interfaces. Ubiquitous Computing
HCI 2015 (10/10) Natural User Interfaces. Ubiquitous ComputingHCI 2015 (10/10) Natural User Interfaces. Ubiquitous Computing
HCI 2015 (10/10) Natural User Interfaces. Ubiquitous Computing
 
Hci03 designmethodologies-150910103121-lva1-app6892
Hci03 designmethodologies-150910103121-lva1-app6892Hci03 designmethodologies-150910103121-lva1-app6892
Hci03 designmethodologies-150910103121-lva1-app6892
 
SS2
SS2SS2
SS2
 
HCI 2014 (9 of 10): Affective Factors
HCI 2014 (9 of 10): Affective FactorsHCI 2014 (9 of 10): Affective Factors
HCI 2014 (9 of 10): Affective Factors
 
Sketch2presentation
Sketch2presentationSketch2presentation
Sketch2presentation
 
Multimodal man machine interaction
Multimodal man machine interactionMultimodal man machine interaction
Multimodal man machine interaction
 
2 4-10
2 4-102 4-10
2 4-10
 
Mask
MaskMask
Mask
 
Web search algorithms and user interfaces
Web search algorithms and user interfacesWeb search algorithms and user interfaces
Web search algorithms and user interfaces
 
Interactiune om-calculator (Factorul uman. Utilizabilitatea)
Interactiune om-calculator (Factorul uman. Utilizabilitatea)Interactiune om-calculator (Factorul uman. Utilizabilitatea)
Interactiune om-calculator (Factorul uman. Utilizabilitatea)
 
Ubiquitous computing abstract
Ubiquitous computing abstractUbiquitous computing abstract
Ubiquitous computing abstract
 
426 lecture6b: AR Interaction
426 lecture6b: AR Interaction426 lecture6b: AR Interaction
426 lecture6b: AR Interaction
 

More from Sabin Buraga

Web 2020 01/12: World Wide Web – aspecte arhitecturale
Web 2020 01/12: World Wide Web – aspecte arhitecturaleWeb 2020 01/12: World Wide Web – aspecte arhitecturale
Web 2020 01/12: World Wide Web – aspecte arhitecturaleSabin Buraga
 
Web 2020 02/12: Programare Web – HTTP. Cookie-uri. Sesiuni Web
Web 2020 02/12: Programare Web – HTTP. Cookie-uri. Sesiuni WebWeb 2020 02/12: Programare Web – HTTP. Cookie-uri. Sesiuni Web
Web 2020 02/12: Programare Web – HTTP. Cookie-uri. Sesiuni WebSabin Buraga
 
Web 2020 03/12: Programare Web – Arhitectura aplicaţiilor Web. Inginerie Web
Web 2020 03/12: Programare Web – Arhitectura aplicaţiilor Web. Inginerie WebWeb 2020 03/12: Programare Web – Arhitectura aplicaţiilor Web. Inginerie Web
Web 2020 03/12: Programare Web – Arhitectura aplicaţiilor Web. Inginerie WebSabin Buraga
 
Web 2020 04/12: Programare Web – Dezvoltarea aplicaţiilor Web în PHP
Web 2020 04/12: Programare Web – Dezvoltarea aplicaţiilor Web în PHP Web 2020 04/12: Programare Web – Dezvoltarea aplicaţiilor Web în PHP
Web 2020 04/12: Programare Web – Dezvoltarea aplicaţiilor Web în PHP Sabin Buraga
 
Web 2020 05/12: Modelarea datelor. Familia XML. Extragerea datelor cu XPath. ...
Web 2020 05/12: Modelarea datelor. Familia XML. Extragerea datelor cu XPath. ...Web 2020 05/12: Modelarea datelor. Familia XML. Extragerea datelor cu XPath. ...
Web 2020 05/12: Modelarea datelor. Familia XML. Extragerea datelor cu XPath. ...Sabin Buraga
 
Web 2020 06/12: Procesarea datelor XML & HTML. Document Object Model
Web 2020 06/12: Procesarea datelor XML & HTML. Document Object ModelWeb 2020 06/12: Procesarea datelor XML & HTML. Document Object Model
Web 2020 06/12: Procesarea datelor XML & HTML. Document Object ModelSabin Buraga
 
Web 2020 07/12: Procesarea datelor XML & HTML – Simple API for XML. Procesări...
Web 2020 07/12: Procesarea datelor XML & HTML – Simple API for XML. Procesări...Web 2020 07/12: Procesarea datelor XML & HTML – Simple API for XML. Procesări...
Web 2020 07/12: Procesarea datelor XML & HTML – Simple API for XML. Procesări...Sabin Buraga
 
Web 2020 08/12: Servicii Web. De la arhitecturi orientate spre servicii la SO...
Web 2020 08/12: Servicii Web. De la arhitecturi orientate spre servicii la SO...Web 2020 08/12: Servicii Web. De la arhitecturi orientate spre servicii la SO...
Web 2020 08/12: Servicii Web. De la arhitecturi orientate spre servicii la SO...Sabin Buraga
 
Web 2020 09/12: Servicii Web. Paradigma REST
Web 2020 09/12: Servicii Web. Paradigma RESTWeb 2020 09/12: Servicii Web. Paradigma REST
Web 2020 09/12: Servicii Web. Paradigma RESTSabin Buraga
 
Web 2020 10/12: Servicii Web. Micro-servicii. Serverless. Specificarea API-ur...
Web 2020 10/12: Servicii Web. Micro-servicii. Serverless. Specificarea API-ur...Web 2020 10/12: Servicii Web. Micro-servicii. Serverless. Specificarea API-ur...
Web 2020 10/12: Servicii Web. Micro-servicii. Serverless. Specificarea API-ur...Sabin Buraga
 
Web 2020 11/12: Interacţiune Web asincronă. Aplicaţii Web de tip mash-up. JAM...
Web 2020 11/12: Interacţiune Web asincronă. Aplicaţii Web de tip mash-up. JAM...Web 2020 11/12: Interacţiune Web asincronă. Aplicaţii Web de tip mash-up. JAM...
Web 2020 11/12: Interacţiune Web asincronă. Aplicaţii Web de tip mash-up. JAM...Sabin Buraga
 
Web 2020 12/12: Securitatea aplicaţiilor Web. Aspecte esenţiale
Web 2020 12/12: Securitatea aplicaţiilor Web. Aspecte esenţialeWeb 2020 12/12: Securitatea aplicaţiilor Web. Aspecte esenţiale
Web 2020 12/12: Securitatea aplicaţiilor Web. Aspecte esenţialeSabin Buraga
 
STAW 01/12: Arhitectura aplicaţiilor Web
STAW 01/12: Arhitectura aplicaţiilor WebSTAW 01/12: Arhitectura aplicaţiilor Web
STAW 01/12: Arhitectura aplicaţiilor WebSabin Buraga
 
STAW 02/12: Programare Web: Limbajul JavaScript. Aspecte esenţiale
STAW 02/12: Programare Web: Limbajul JavaScript. Aspecte esenţialeSTAW 02/12: Programare Web: Limbajul JavaScript. Aspecte esenţiale
STAW 02/12: Programare Web: Limbajul JavaScript. Aspecte esenţialeSabin Buraga
 
STAW 03/12: Programare Web: Limbajul JavaScript. Aspecte moderne: ES6 et al.
STAW 03/12: Programare Web: Limbajul JavaScript. Aspecte moderne: ES6 et al.STAW 03/12: Programare Web: Limbajul JavaScript. Aspecte moderne: ES6 et al.
STAW 03/12: Programare Web: Limbajul JavaScript. Aspecte moderne: ES6 et al.Sabin Buraga
 
STAW 04/12: Programare Web: Node.js
STAW 04/12: Programare Web: Node.jsSTAW 04/12: Programare Web: Node.js
STAW 04/12: Programare Web: Node.jsSabin Buraga
 
STAW 05/12: Arhitectura navigatorului Web
STAW 05/12: Arhitectura navigatorului WebSTAW 05/12: Arhitectura navigatorului Web
STAW 05/12: Arhitectura navigatorului WebSabin Buraga
 
STAW 06/12: JavaScript în navigatorul Web. De la DOM la Ajax şi mash-up-uri
STAW 06/12: JavaScript în navigatorul Web. De la DOM la Ajax şi mash-up-uriSTAW 06/12: JavaScript în navigatorul Web. De la DOM la Ajax şi mash-up-uri
STAW 06/12: JavaScript în navigatorul Web. De la DOM la Ajax şi mash-up-uriSabin Buraga
 
STAW 07/12: Ingineria dezvoltării aplicaţiilor JavaScript
STAW 07/12: Ingineria dezvoltării aplicaţiilor JavaScriptSTAW 07/12: Ingineria dezvoltării aplicaţiilor JavaScript
STAW 07/12: Ingineria dezvoltării aplicaţiilor JavaScriptSabin Buraga
 
STAW 08/12: Programare Web. Suita de tehnologii HTML5
STAW 08/12: Programare Web. Suita de tehnologii HTML5STAW 08/12: Programare Web. Suita de tehnologii HTML5
STAW 08/12: Programare Web. Suita de tehnologii HTML5Sabin Buraga
 

More from Sabin Buraga (20)

Web 2020 01/12: World Wide Web – aspecte arhitecturale
Web 2020 01/12: World Wide Web – aspecte arhitecturaleWeb 2020 01/12: World Wide Web – aspecte arhitecturale
Web 2020 01/12: World Wide Web – aspecte arhitecturale
 
Web 2020 02/12: Programare Web – HTTP. Cookie-uri. Sesiuni Web
Web 2020 02/12: Programare Web – HTTP. Cookie-uri. Sesiuni WebWeb 2020 02/12: Programare Web – HTTP. Cookie-uri. Sesiuni Web
Web 2020 02/12: Programare Web – HTTP. Cookie-uri. Sesiuni Web
 
Web 2020 03/12: Programare Web – Arhitectura aplicaţiilor Web. Inginerie Web
Web 2020 03/12: Programare Web – Arhitectura aplicaţiilor Web. Inginerie WebWeb 2020 03/12: Programare Web – Arhitectura aplicaţiilor Web. Inginerie Web
Web 2020 03/12: Programare Web – Arhitectura aplicaţiilor Web. Inginerie Web
 
Web 2020 04/12: Programare Web – Dezvoltarea aplicaţiilor Web în PHP
Web 2020 04/12: Programare Web – Dezvoltarea aplicaţiilor Web în PHP Web 2020 04/12: Programare Web – Dezvoltarea aplicaţiilor Web în PHP
Web 2020 04/12: Programare Web – Dezvoltarea aplicaţiilor Web în PHP
 
Web 2020 05/12: Modelarea datelor. Familia XML. Extragerea datelor cu XPath. ...
Web 2020 05/12: Modelarea datelor. Familia XML. Extragerea datelor cu XPath. ...Web 2020 05/12: Modelarea datelor. Familia XML. Extragerea datelor cu XPath. ...
Web 2020 05/12: Modelarea datelor. Familia XML. Extragerea datelor cu XPath. ...
 
Web 2020 06/12: Procesarea datelor XML & HTML. Document Object Model
Web 2020 06/12: Procesarea datelor XML & HTML. Document Object ModelWeb 2020 06/12: Procesarea datelor XML & HTML. Document Object Model
Web 2020 06/12: Procesarea datelor XML & HTML. Document Object Model
 
Web 2020 07/12: Procesarea datelor XML & HTML – Simple API for XML. Procesări...
Web 2020 07/12: Procesarea datelor XML & HTML – Simple API for XML. Procesări...Web 2020 07/12: Procesarea datelor XML & HTML – Simple API for XML. Procesări...
Web 2020 07/12: Procesarea datelor XML & HTML – Simple API for XML. Procesări...
 
Web 2020 08/12: Servicii Web. De la arhitecturi orientate spre servicii la SO...
Web 2020 08/12: Servicii Web. De la arhitecturi orientate spre servicii la SO...Web 2020 08/12: Servicii Web. De la arhitecturi orientate spre servicii la SO...
Web 2020 08/12: Servicii Web. De la arhitecturi orientate spre servicii la SO...
 
Web 2020 09/12: Servicii Web. Paradigma REST
Web 2020 09/12: Servicii Web. Paradigma RESTWeb 2020 09/12: Servicii Web. Paradigma REST
Web 2020 09/12: Servicii Web. Paradigma REST
 
Web 2020 10/12: Servicii Web. Micro-servicii. Serverless. Specificarea API-ur...
Web 2020 10/12: Servicii Web. Micro-servicii. Serverless. Specificarea API-ur...Web 2020 10/12: Servicii Web. Micro-servicii. Serverless. Specificarea API-ur...
Web 2020 10/12: Servicii Web. Micro-servicii. Serverless. Specificarea API-ur...
 
Web 2020 11/12: Interacţiune Web asincronă. Aplicaţii Web de tip mash-up. JAM...
Web 2020 11/12: Interacţiune Web asincronă. Aplicaţii Web de tip mash-up. JAM...Web 2020 11/12: Interacţiune Web asincronă. Aplicaţii Web de tip mash-up. JAM...
Web 2020 11/12: Interacţiune Web asincronă. Aplicaţii Web de tip mash-up. JAM...
 
Web 2020 12/12: Securitatea aplicaţiilor Web. Aspecte esenţiale
Web 2020 12/12: Securitatea aplicaţiilor Web. Aspecte esenţialeWeb 2020 12/12: Securitatea aplicaţiilor Web. Aspecte esenţiale
Web 2020 12/12: Securitatea aplicaţiilor Web. Aspecte esenţiale
 
STAW 01/12: Arhitectura aplicaţiilor Web
STAW 01/12: Arhitectura aplicaţiilor WebSTAW 01/12: Arhitectura aplicaţiilor Web
STAW 01/12: Arhitectura aplicaţiilor Web
 
STAW 02/12: Programare Web: Limbajul JavaScript. Aspecte esenţiale
STAW 02/12: Programare Web: Limbajul JavaScript. Aspecte esenţialeSTAW 02/12: Programare Web: Limbajul JavaScript. Aspecte esenţiale
STAW 02/12: Programare Web: Limbajul JavaScript. Aspecte esenţiale
 
STAW 03/12: Programare Web: Limbajul JavaScript. Aspecte moderne: ES6 et al.
STAW 03/12: Programare Web: Limbajul JavaScript. Aspecte moderne: ES6 et al.STAW 03/12: Programare Web: Limbajul JavaScript. Aspecte moderne: ES6 et al.
STAW 03/12: Programare Web: Limbajul JavaScript. Aspecte moderne: ES6 et al.
 
STAW 04/12: Programare Web: Node.js
STAW 04/12: Programare Web: Node.jsSTAW 04/12: Programare Web: Node.js
STAW 04/12: Programare Web: Node.js
 
STAW 05/12: Arhitectura navigatorului Web
STAW 05/12: Arhitectura navigatorului WebSTAW 05/12: Arhitectura navigatorului Web
STAW 05/12: Arhitectura navigatorului Web
 
STAW 06/12: JavaScript în navigatorul Web. De la DOM la Ajax şi mash-up-uri
STAW 06/12: JavaScript în navigatorul Web. De la DOM la Ajax şi mash-up-uriSTAW 06/12: JavaScript în navigatorul Web. De la DOM la Ajax şi mash-up-uri
STAW 06/12: JavaScript în navigatorul Web. De la DOM la Ajax şi mash-up-uri
 
STAW 07/12: Ingineria dezvoltării aplicaţiilor JavaScript
STAW 07/12: Ingineria dezvoltării aplicaţiilor JavaScriptSTAW 07/12: Ingineria dezvoltării aplicaţiilor JavaScript
STAW 07/12: Ingineria dezvoltării aplicaţiilor JavaScript
 
STAW 08/12: Programare Web. Suita de tehnologii HTML5
STAW 08/12: Programare Web. Suita de tehnologii HTML5STAW 08/12: Programare Web. Suita de tehnologii HTML5
STAW 08/12: Programare Web. Suita de tehnologii HTML5
 

Recently uploaded

WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentationphoebematthew05
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 

Recently uploaded (20)

WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentation
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 

HCI 2018 (10/10) Natural User Interfaces. Ubiquitous Computing

  • 1. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ Human-Computer Interaction from natural interaction to ubicomp cdn-images-1.medium.com/max/2000/1*YGT4bQea3spMLg2_tYkOjw.jpeg
  • 2. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ “The details are not the details. They make the design.” Charles Eames
  • 3. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ How about interacting with “special” devices offering natural interfaces?
  • 4. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience Sense Relevant information Audio Speech, Identity, Intonation, Precision (timing) Vision Identity, Facial expression, Body language, Gesture, Accuracy (spatial) Tactile/ somesthetic Tabs/pads/devices, Texture, Precision (timing), Accuracy (spatial) Scent Atmosphere, likability Taste Clarification, enjoyment
  • 5. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience The interpretation of sensorial data depends on user expectations
  • 6. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience M. Nielsen, T. Moeslund, M. Storring & E. Granum (2008)
  • 7. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience Also, the interpretation of sensorial data depends on specific input devices (K. Hinckley, ‘08)
  • 8. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience Also, the interpretation of sensorial data depends on specific input devices (K. Hinckley, ‘08) mouse, trackball, isometric joystick, touchpad, touchscreen, pen-operated device, watch, camera, microphone, indirect table,…
  • 9. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience Interaction modes specify how an user interface responds to user actions
  • 10. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience Interaction modes specify how an user interface responds to user actions for a particular manner of performing a task, UI adopts a specific mode if the interpretation of this task is always the same
  • 11. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience Interaction modes when a task has a different interpretation, the UI offers another interaction mode
  • 12. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience Interaction modes when a task has a different interpretation, the UI offers another interaction mode modal vs. modeless interaction
  • 13. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience Interaction modes an interface could be modal for a given action, and modeless for another (Jef Raskin, 2000) if an UI is considered modeless, then that UI must not be modal for any other action
  • 14. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience Interaction modes changing the interaction mode should have a solid motivation …even in the context of (mobile) Web
  • 15. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ Haptic  Gesture  Locomotion Auditory & Voice  Tangible natural interaction
  • 16. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI (O’Malley & Gupta, 2008) based on tactile sensations to provide information
  • 17. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI (O’Malley & Gupta, 2008) based on tactile sensations to provide information typical examples: haptic screen for “touching” objects natural output – e.g., vibrations (via tactons)
  • 18. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI cutaneous touch versus kinesthetic touch
  • 19. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI used in various contexts and domains: CAD (Computer-Aided Design) design prototyping product evaluation simulation of exceptional situations remote task execution (tele-operation) mobile & wearable computing
  • 20. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI usually, a haptic environment is defined by a formal model
  • 21. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction example: virtual wall – used in 3D games/simulations
  • 22. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI fixed haptic interfaces haptic joysticks pen-based haptic devices floor- and ceiling-mounted interfaces
  • 23. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ M. O’Malley, A. Gupta, “Haptic Interfaces”, in P. Kortum (Ed.), HCI Beyond the GUI, Elsevier, 2008
  • 24. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI portable haptic interfaces exoscheletons
  • 25. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Light Exoskeleton (L-Exos) described by Frisoli et al. (2005)
  • 26. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction PowerArm HCI project: Arduino, Android & Unity3D Dan-Gabriel Bratu, Daniel Vicol, Sergiu Recean (FII, 2013) profs.info.uaic.ro/~stefan.negru/studentprojects/powerarm.html case study
  • 27. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI tactile interfaces could detect heat, pressure, vibrations, pain the feedback is usually provided by the user fingers
  • 28. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI tactile interfaces vibrotactile interfaces wearable tactile interfaces
  • 29. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI tactile interfaces vibrotactile interfaces wearable tactile interfaces useful in the context of user accessibility
  • 30. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Moose haptic interface (O’Modhrain & Gillespie, 1995) ccrma.stanford.edu/files/papers/stanm95.pdf case study
  • 31. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction AHNE – Audio-Haptic Navigation Environment (SOPI Research Group, Aalto University, 2011) user tracking via Kinect + OpenNI & OSCeleton vimeo.com/28447850 case study
  • 32. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI aspects regarding human factors: understanding bio-mechanical, sensorial-motor, and cognitive abilities of users (Tan et al., 1994)
  • 33. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI accuracy of movement measurement (position, velocity, acceleration)
  • 34. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI accuracy of movement measurement (position, velocity, acceleration) in the mobile Web context, see HTML5 APIs e.g., Touch, Vibration, Screen Orientation, Gamepad, Media Capture www.w3.org/2018/04/web-roadmaps/mobile/
  • 35. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI design principles: base your mechanical design on the inherent capabilities of the human operator
  • 36. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI design principles: consider human sensitivity to tactile stimuli
  • 37. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI design principles: use active rather than passive movement
  • 38. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI design principles: minimize confusion and control instabilities
  • 39. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI design principles: ensure realistic display of environments with tactile devices
  • 40. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Haptic UI specific uses: (tele-)medicine e.g., laparoscopic surgery simulator, palpation simulator 3D surface modeling learning the interaction with touchscreen devices digital art
  • 41. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction using hand tracking data, VRgluv employs predictive physics algorithms to calculate the size and stiffness of virtual objects and recreates them in your hands vrgluv.com case study
  • 42. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Pseudo-Haptic UI (software) emulation of a haptic device
  • 43. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Pseudo-Haptic UI sensorial substitution (Lenay et al., 2003) a haptic stimulus is substituted by another sense (e.g., an audio signal simulating the contact with virtual objects)
  • 44. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Pseudo-Haptic UI offering pseudo-haptic feedback (Anatole Lecuyer, 2007) simulation of stiffness via a combination of the interaction with the input device & the visual feedback pseudo-haptic simulation of textures
  • 45. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Simulation-based Palpation using Pseudo-haptic Feedback (Min Li et al., 2012) www.youtube.com/watch?v=OcIczYCUkmc case study
  • 46. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction inspired by non-verbal interactions between people
  • 47. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction inspired by non-verbal interactions between people alternative or complementary to traditional interactions “tap is the new click”
  • 48. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction “While natural gestures are often subtle, gesture interfaces rely on emphasized gestures.” M. Nielsen et al., 2008
  • 49. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction types: hand gestures for pointing out objects or controlling a mouse-like pointer
  • 50. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ Interacţiune neconvenţională
  • 51. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction types: motion-tracking interfaces used by several game consoles or laptops
  • 52. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction types: facial gestures useful in detection of reactions and emotions
  • 53. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction usually, based on specific input devices mechanical/magnetic – e.g., mouse, data glove, Wii,… single/multi-point touch examples: smart phones, tablets
  • 54. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction could be “invisible” for the user detecting gestures via computer vision: segmentation – extracting the relevant fragments tracking – detecting the movements of interest classification – obtaining the desired information (e.g., via machine learning, neural nets,…)
  • 55. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ Interacţiune neconvenţională general architecture of a gesture-based interactive system (Moustakas et al., 2006)
  • 56. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction gesture classification: semantic – the meaning of a gesture functional – what the gesture does in an interface descriptive – how the gesture is performed
  • 57. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction functionally, gestures can signify: command – Quit, Undo, Configure pointing – e.g., selecting an item of interest manipulation – example: scaling/rotating an image control – emulates the control over an entity
  • 58. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction dynamic + spatial-temporal “classical” gestures performed via an input device: mouse, stylus,…
  • 59. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interactionMaterial Design – types of gestures: navigational (tap, scroll & pan, drag, swipe, pinch) action (tap, long press, swipe) transform (double tap, pinch, compound gestures, pick up & move) material.io/design/interaction/gestures.html case study
  • 60. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction case study
  • 61. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction the interaction vocabulary should be limited we must assure an optimal interaction space (spatial zone)
  • 62. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction “special” interactive zones having specific semantics
  • 63. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction technology- versus human-based gestures causes of discomfort: stressful/fatigue producing for the user nearly impossible for some people to perform illogically imposed functionality
  • 64. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ some gestures easily recognizable by software meanings/actions difficult to be understood/performed by the human users: T0: without signification (residue) T1: select T2: copy & paste T3: delete T4, T5: menu/release
  • 65. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction a gesture-based UI must: be intuitive take into consideration the user mental model be easy to learn & deployed (ergonomics)
  • 66. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction practical design advices: be tolerant to human errors beware of cultural factors consider the context provide positive feedback
  • 67. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction patterns for touchscreens & interactive surfaces (Dan Saffer, 2009) tap to open/activate tap to select drag to move object slide to scroll, spin to scroll
  • 68. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction patterns for touchscreens & interactive surfaces (Dan Saffer, 2009) slide and hold for continuous scroll tap to stop pinch to shrink and spread to enlarge two fingers to scroll
  • 69. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction gesture notation (Luke Wroblewski, 2010) www.lukew.com/ff/entry.asp?1071
  • 70. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction patterns for free-form interactive gestures (Dan Saffer, 2009) move body to activate point to select/activate wave to activate place hands inside to activate
  • 71. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Gesture interaction patterns for free-form interactive gestures (Dan Saffer, 2009) rotate to change state step to activate shake to change tilt to move
  • 72. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction device specific gesture notation – for example: Kinect gesturecons.com
  • 73. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ locomotion design natural interaction
  • 74. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces interfaces that both enable users to move around in real or virtual spaces and make users feel as if they are moving Mary C. Whitton & Sharif Razzaque, 2008
  • 75. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces could be virtual, not only physical
  • 76. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces could be virtual, not only physical main problem: to capture the user’s intent using data that can be derived from sensing the pose (position and orientation) and movement of the user’s body
  • 77. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ discussion
  • 78. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces perceiving self-motion each sensorial channel (visual, auditory, vestibular,…) is used in detection of user or surrounded objects movement Dichgans & Brandt, 1977
  • 79. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces detection of pose and motion artificial sensors (trackers) optical motion capture mechanical solutions
  • 80. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ optical motion capture optical motion capture exoskeleton trackers
  • 81. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction case study interactive climbs project360.mammut.ch
  • 82. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces providing feedback to user(s) visual information: head-mounted displays (HMDs) surround-screen displays
  • 83. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces walking simulation treadmills (Brooks, 1998; Hollerbach et al., 2000) walking-in-place (WIP) interfaces
  • 84. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces design principles (Whitton & Razzaque, 2008): making the interface feel natural higher-fidelity reproduction of human way of walking avoiding simulator sickness – cybersickness dspace.dsto.defence.gov.au/dspace/bitstream/1947/4079/1/DSTO-TR-1419%20PR.pdf
  • 85. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction a VR treadmill allowing users to maneuver a scene using their feet (Nick Randolph, 2018) medium.com/uxxr/the-user-experience-of-virtual-reality-c464762deb8e case study
  • 86. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces issues that can be detected in the testing phase: realism + preservation of spatial understanding cognitive distraction break-in-presence (BIP) events
  • 87. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction an action that causes break-in-presence (virtual scenephysical location) details in A. Steed, A. Brogni, V. Vinayagamoorthy, “Breaks in Presence as Usability Criteria”, HCII 2005 www0.cs.ucl.ac.uk/staff/v.vinayagamoorthy/pdf/hcii2005.pdf discussion
  • 88. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces advice: match the locomotion metaphor to the interface goals walking metaphor versus vehicle metaphor
  • 89. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces practical considerations: Is viewing direction independent of motion direction? Are hands used for locomotion? Able to move in any direction and change direction easily? Can walking speed or step length be set for each user?
  • 90. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Locomotion interfaces practical considerations: Are tracker sensors immune to other signals in the room? Are wireless trackers used? What parts of body will have sensors/markers on them? Cables – if any – interfere with user’s motion?
  • 91. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction case study motion studies in the context of modern art Universal Everything – universaleverything.com other examples: prostheticknowledge.tumblr.com/tagged/motion
  • 92. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ Interacţiune neconvenţională auditory & vocal interactions
  • 93. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Auditory interfaces using non-speech audio information for interaction for details, see (S. Camille Peres et al., 2008)
  • 94. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Auditory interfaces motivations and uses: presenting information to visually impaired people providing an additional information channel alerting people to error or emergency states of a system providing information via devices with small screens
  • 95. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Auditory interfaces why an audio interaction? reducing visual overload reinforcing visual messages when eyes are elsewhere when audio is more informative conveying emotion – e.g., for electronic games
  • 96. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Auditory interfaces sonification = using non-speech sound to render data the simplest method is audification
  • 97. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Auditory interfaces representational auditory icons abstract earcons Stephen Brewster (2008)
  • 98. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Auditory interfaces
  • 99. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Auditory interfaces problems regarding the audio interaction: annoyance privacy auditory overload interference/masking low resolution lack of familiarity
  • 100. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Auditory interfaces aspect of interest: the tasks performed by the user via an audio interface
  • 101. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Auditory interfaces aspect of interest: what (kind of) information could be exclusively represented by sounds?
  • 102. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Auditory interfaces aspect of interest: the suitable format audio – lossless vs. lossy – to be used FLAC, PCM, WAV versus AAC, OGG, MP3
  • 103. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Auditory interfaces aspect of interest: producing the sound thinking about sound as information designing the sound – consider auditory content as a kind of sound ecology (Walker & Kramer, 2004)
  • 104. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction audio-only games – www.audiogames.net case study
  • 105. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces facilitating the conversation between software and user
  • 106. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces important aspect: speech recognition
  • 107. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces user perceptions regarding a VUI are fragile lack of trust (Susan Hura, 2008)
  • 108. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces users interact generally via prompts “Hello, you have to choose from the following options…”
  • 109. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces several discourse makers are needed for: confirmation – “yes, of course”, “alright” changing the topic of conversation emphasizing a statement – “as we previously said…” narrative sequences – “so,…”
  • 110. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces problem: the lack of concision “I can help you with the following five options. You can interrupt me and speak your choice at any time. Please, select one of the following: sign up for new service, add features to my service, move my existing service, problems with my satellite service, or ask a billing question.”
  • 111. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces problem: dealing with errors “I’m sorry. I did not understand your response. I can help you with the following five options…” inducing the feeling of user culpability
  • 112. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces problem: increasing the user discomfort “Please say or enter your account number!” the user cannot control the interface
  • 113. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces problem: using the same message (in different contexts) “Attention! I don’t understand! Please, indicate the correct answer!... Attention! I don’t understand! Please, indicate the correct answer!... Attention! I don’t understand! Please, indicate the correct answer!...”
  • 114. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces speech recognition – main phases (Cohen et al., 2004)
  • 115. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces design guidelines: present menu options in a way that makes them comprehensible and easy to retain
  • 116. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces design guidelines: consider error sources when writing error-handling prompts
  • 117. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces design guidelines: provide context VUI: You can say: account balances, last contribution, transfers, allocations, or life insurance. User: Transfers. VUI: Transfers. You can say: hear pending transfers or make a transfer.
  • 118. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces design guidelines: use hints and reminders “Next time, you can say ‘order status’ at the main menu to get here quicker.” “You can speak your account number or key it in.”
  • 119. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces design guidelines: favor models over instructions learning by doing
  • 120. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces design guidelines: use discourse markers liberally using in conversation common expressions like “all right”, “next”, “thanks”,…
  • 121. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces design guidelines: allow for conversational repair
  • 122. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces design guidelines: be careful with terminology
  • 123. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces recent examples – mobile, Web & smart home contexts: Amazon Alexa + Amazon Echo developer.amazon.com/alexa developer.amazon.com/echo
  • 124. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces recent examples – mobile, Web & smart home contexts: Apple Siri (iOS, watchOS) developer.apple.com/sirikit/
  • 125. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces recent examples – mobile, Web & smart home contexts: Google Assistant developers.google.com/assistant/sdk/
  • 126. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Voice user interfaces recent examples – mobile, Web & smart home contexts: Microsoft Cortana in conjunction to Azure Bot developer.microsoft.com/cortana azure.microsoft.com/services/bot-service/
  • 127. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction case study A home assistant able to announce who is home by using facial recognition (Juan Perez & Robin Cole, 2018) image data captured via Webcam and processed locally with Facebox – machinebox.io/docs/facebox – or on cloud by using Amazon Rekognition or Google Vision announcement speaker uses Google Text-To-Speech service, integrated into Home Assistant – an open-source home automation platform: www.home-assistant.io www.hackster.io/97766/announce-who-is-home-using-facial-recognition-dcc389
  • 128. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ tangible interfaces
  • 129. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Tangible interfaces digital informationphysical form (representation) Hiroshi Ishii digital content could be directly perceived & manipulated
  • 130. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Tangible interfaces key components: control – permits users to manipulate the content external representations – perceived by human senses
  • 131. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction
  • 132. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Tangible interfaces
  • 133. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Tangible Manipulation haptic direct manipulation lightweight interaction isomorph effects Spatial Interaction inhabited space configurable materials visual access full body interaction performative action Embodied Facilitation embodied constraints multiple access points tailored (adaptive) representations Expressive Represen- tation representation- nal significance externalization perceived coupling Eva Hornecker (2010) see also www.ehornecker.de/TangiblesFramework.html
  • 134. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Tangible Interaction Conceptual Framework responding to key design questions via cards Can users grab, feel and move “the important stuff”? Do people and objects meet and invite into interaction? Can you create a meaningful place with atmosphere? Are representations legible, meaningful, and expressive? Does shifting stuff (or your own body) around have meaning?
  • 135. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Tangible interfaces for other resources + demos, consult: Eva Hornecker, Tangible Interaction (2015) www.interaction-design.org/encyclopedia/tangible_interaction.html Tangible Interfaces discipline – mas834.media.mit.edu Tangible Media Group @ MIT – tangible.media.mit.edu
  • 136. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ multimodal interaction
  • 137. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ multimodal interaction Processing two or many inputs – voice, gestures, tactile, etc. – in a coordinated manner in order to obtain a complex output (e.g., multimedia, 3D) or to perform a task Sharon Oviatt, 2008 Paulo Barthelmess & Sharon Oviatt, 2008
  • 138. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ multimodal interaction “The situation where the user is provided with multiple modes for interacting with a system.” Abdo El Ali, 2011 www.slideshare.net/Abd0/multimodal-interaction-an-introduction
  • 139. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ Interacţiune multimodală
  • 140. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ multimodal interaction Diverse contexts are considered from mobile interfaces to collaborative systems virtual and/or augmented reality
  • 141. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ multimodal interaction Advantages: error reduction via mutual disambiguation increasing user performance
  • 142. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ multimodal interaction Issues to be resolved: linguistic factors redundancy & complementarity choosing the “right” multi-/mono-modal input cognitive load
  • 143. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction discussion designing multimodal interaction for info kiosks
  • 144. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction discussion designing multimodal interaction for info kiosks aspects of interest (M. C. Maguire, 1999): choosing a suitable location encouraging use providing instructions for using the system user privacy multiple input methods: touch, speech, keypad proper output: language, color, images, feedback,… navigation possibilities ui4all.ics.forth.gr/UI4ALL-97/maguire.pdf
  • 145. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing www.macadamian.com/2015/03/20/design-iot-product/
  • 146. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing “Invisibly enhancing the world that already exists.” (Mark Weiser, 1988)
  • 147. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing “Invisibly enhancing the world that already exists.” (Mark Weiser, 1988) ubicomp pervasive computing ambient intelligence (AmI) physical computing everyware
  • 148. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing “Invisibly enhancing the world that already exists.” (Mark Weiser, 1988) can occur using any device, in any location, and in any format computers disappear in the environment
  • 149. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing Related topics of research: context awareness distributed & mobile computing Internet of Things (IoT) + wireless sensor networks
  • 150. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interactionUsing (semantic) Web technologies to specify & interact with IoT components describing IoT for machines/users via Web of Things specifications (W3C, 2018): www.w3.org/WoT/ developing applications by using Things Framework (Mozilla), a collection of re-usable software components to build Web things exposing the Web Thing API: iot.mozilla.org/things/ for further details, visit webofthings.org discussion
  • 151. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing Scales of devices Mark Weiser’s initial proposals (1991) tabs – centimeter-sized devices: PDAs, voice recorders, smart-phones, smart-watches,… vision: individuals own many of them and they can all communicate with each other and environment
  • 152. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction interacting with a smart-watch via widgets UI design considerations in A. Komarov, How We Designed and Built Our First Apple Watch App (2015) www.smashingmagazine.com/2015/08/how-we-designed-and-built-our-first-apple-watch-app/ discussion
  • 153. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing Scales of devices Mark Weiser’s initial proposals (1991) pads – handheld decimeter-sized devices: tablets, laptops, e-readers, etc. vision: an individual owns several, but it is not assumed to be always with them
  • 154. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing Scales of devices Mark Weiser’s initial proposals (1991) boards – meter-sized interactive devices: mounted walls, e-tables, smart info kiosks,… vision: buildings or institutions own them and lots of people used them
  • 155. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Virtual Reality term coined by Jaron Lanier (1989) a computer-based simulated 3D environment intended for its users to inhabit and to interact via avatars an interface between the “physical” reality and a virtual environment discussion
  • 156. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Virtual Reality – desktop 1st wave (1995) – expensive hardware HMD (Head Mounted Display) 2nd wave (since 2010) MxR (Mixed Reality Lab), Oculus Rift consumer-ready HMDs (2016) Oculus Rift, Sony Morpheus, HTC/Valve Vive for further information, study the Mark Billinghurst’s presentations: www.slideshare.net/marknb00/presentations discussion
  • 157. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Augmented Reality term coined by Thomas Caudell (1990) the use of technology which allows the perception of the physical world to be enhanced or modified by computer-generated stimuli perceived with the aid of special equipment generation of virtual objects merged into real scenes discussion
  • 158. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction reality-virtuality continuum (Milgram & Kishino, 1994) consult also www.slideshare.net/busaco/from-virtual-to-augmented-reality discussion
  • 159. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interactionMobile phone AR & VR HMD = VR mount + smartphone mobile phone AR smartphone + live camera view + sensor input (e.g., GPS, compass) mobile phone VR smartphone + sensor input + additional VR viewer case study
  • 160. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Mobile phone AR & VR low-cost immersive viewers case study VR2GO (MxR Lab, 2013) projects.ict.usc.edu/mxr/diy/vr2go/ Google Cardboard (2014) g.co/cardboard M. Billinghurst, Cardboard VR: Building Low Cost VR Experiences (2016) www.slideshare.net/marknb00/cardboard-vr-building-low-cost-vr-experiences
  • 161. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction Mobile phone AR & VR popular software toolkits: ARCore (Google) – developers.google.com/ar/ + Google VR – developers.google.com/vr/ ARKit (Apple) – developer.apple.com/arkit/ case study M. Billinghurst, Developing AR and VR Experiences with Unity (2017) www.slideshare.net/marknb00/developing-ar-and-vr-experiences-with-unity
  • 162. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction AR & VR in the modern Web context 3D content via WebGL developer.mozilla.org/docs/Web/WebGL github.com/sjfricke/awesome-webgl WebXR (draft W3C specification, May 2018) support for accessing VR + AR devices, including sensors and head-mounted displays, on the Web immersive-web.github.io/webxr/ for other details, consult webvr.info/developers/ case study
  • 163. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction current support provided by Web browsers: webvr.rocks
  • 164. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing Scales of devices new proposals (Stefan Poslad, 2009) dust – miniaturized devices without visual output smartdust: a system of many tiny micro- electromechanical systems (MEMS) that can detect light, temperature, vibration, magnetism, chemicals,…
  • 165. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing Scales of devices new proposals (Stefan Poslad, 2009) skin – fabrics based upon light emitting, conductive, polymers, organic computer devices smart clotheswearable computing
  • 166. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction case study www.hackster.io/team-fluxx/fluxx-005ac5
  • 167. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing Scales of devices new proposals (Stefan Poslad, 2009) clay – ensembles of MEMS that can be formed into arbitrary 3D shapes, resembling physical objects remember tangible interfaces?
  • 168. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing M. Melnykowycz, Wearable Sensors and UX Development (2014) www.slideshare.net/Boltzmann/wearable-sensors-and-ux-development
  • 169. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing: future Brain-Computer Interface direct communication between the brain and an external device assisting, augmenting, or repairing human cognitive or sensory-motor functions
  • 170. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ ubiquitous computing: future Alexis Ortiz-Rosario, Hojjat Adeli, “Brain-Computer Interface Technologies: from Signal to Action”, Reviews in the Neurosciences, 24 (5), 2013
  • 171. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ natural interaction case study noninvasive brain–computer interface systems can restore functions lost to disability, allowing for spontaneous, direct brain control of external devices S. Perdikis et al., “The Cybathlon BCI race: Successful longitudinal mutual learning with two tetraplegic users”, PLoS Biol, 16(5), 2018 doi.org/10.1371/journal.pbio.2003787
  • 172. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ “Conclusion” “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” Mark Weiser, “The Computer for the 21st Century”, Scientific American, 265 (3), 1991
  • 173. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ sensorial experience natural user interfaces multimodal interaction ubiquitous computing “Conclusion” ukfiremag.mdmpublishing.com/how-augmented-reality-can-assist-fire-and-rescue-safety/
  • 174. Master on Software Engineering :: Human-Computer Interaction Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/ Good luck!