Examples
(1 through 8):
Example
1
File:
control-t.avi
Frame:
No gesture synchronized, with table
Example
2
File:
f37-g-t
Frame:
gesture synched with frame 37, with gesture, with table
Example
3
File:
f39-g-t
Frame:
gesture synched with frame 39, with gesture, with table
Example
4
File:
control-nt
Frame:
No gesture synchronized, no table
Example
5
File:
f29-g-t
Frame:
gesture synched with frame 29, with gesture, with table
Example
6
File:
f37-g-nt
Frame:
gesture synched with frame 37, with gesture, no table
Example
7
File:
f43-g-t
Frame:
gesture synched with frame 43, with gesture, with table
Example
8
File:
f33-g-nt
Frame:
gesture synched with frame 33, with gesture, no table
Note. This is only a few (8) of the examples used in the experiment. The full study used many more examples with the hand gesture synchronized at every frame along the sentence.
Results
1.
Speech signal and corresponding frame positions for the synchronisation
of the middle of the gesture (5 frames long):
2.
Results for categorization: Click here
Note
how “there” is perceived or categorized (50% or more of time) even
when the gesture is synchronized on frame 38 or 39 – “there”
is perceived even before the speech audio for
“there” has begun:
Gestures
influence what a listener perceives or hears as emphasised!
3.
Results for clarity ratings. Click here
Note
how “book” is perceived or categorized most clearly when the
gesture is synchronized with the centre of the speech and
that the affordance of a table influences clarity of perception (i.e., does it
make sense?).
Gestures
influence how clearly a listener perceives the word categorised
- and affordances or context changes the meaning of the perceived speech!
For
full explanation, download
the full publication.
