social linkedin box blue 32
social facebook box blue 32
social twitter box blue 32
social facebook box blue 32

iit-rbcs-logo-v3

Home RBCS ■ Dynamic Touch and Interaction

Dynamic Touch and Interaction

Object manipulation is a crucial skill in many goal-directed actions (e.g., opening a door, hammering, writing), as well as in many exploratory behaviors (e.g., feeling the weight of an object, finding an object in a bag, etc.). The Dynamic Touch and Interaction Lab was created in 2011 with the objective of advancing our understanding of human manipulatory skills. Specifically, the lab aims to understand how object manipulation develops from the first years of life to adulthood, and to suggest plausible model for the implementation of object manipulation tasks on humanoid robots.

Our research focuses on two aims: 1) investigating interaction and contact forces during the manipulation of objects, a dimension of the interaction that has been often overlooked, and 2) the use of custom and commercial haptic devices for control and manipulate the physical interaction with the subject.

Research topics:

The development of perceptual and motor skills Haptic perception Finger force synergies in multi-finger grasps
alt

This research topic groups ongoing studies on the development of the haptic sense and of the motor skills of children

alt

This research topic groups studies of the haptic sense, from the haptic perception of space and force to the haptic perception of object properties like their shape or weight.

alt

This research topic is focused on the analysis of finger forces during the grasp and manipulation of objects.

Low-cost assessment of the sensory-motor state
alt

The general objective of the studies presented here is to investigate the potential of commercially available cost-effective haptic devices for evaluating one's sensory-motor state.

EXTERNAL PROJECTS:

REWIRE
The goal of this project is to develop cost-effective rehabilitation systems that patients can use in their own homes. Our specific contribution is outlined in "Rehabilitative training using haptic devices" and is being conducted in collaboration with the University of Milano.

SELECTED PUBLICATIONS:


  • *Baud-Bovy G., Tatti F. and Borghese N. A. (2013)
    An evaluation of the effects on postural stability of a force feedback rendered by a low-cost haptic device in various tasks
    IEEE World Haptics Conference 2013, pp. 667-672, Daejeon, Korea, April 14-17, 2013
  • Santello M., *Baud-Bovy G. and Jorntell H. (2013)
    Neural bases of hand synergies
    Frontiers in Computational Neuroscience, vol. 7, (no. 23), 1662-5188
  • *Baud-Bovy G. and Gentaz E. (2012)
    The perception and representation of orientations: a study in the haptic modality
    Acta Psychologica, vol. 141, (no. 1), pp. 24-30
  • Pirovano M., Mainetti R., *Baud-Bovy G., Lanzi P. L. and Borghese N. A. (2012)
    Self-Adaptive Games for Rehabilitation at Home
    IEEE Conference on Computational Intelligence and Games (CIG 2012), pp. 179 - 186, Granada (Spain)
  • Parietti, *Baud-Bovy G., Gatti E., Riener, Guzzella and Vallery H. (2011)
    Series Viscoelastic Actuators Can Match Human Force Perception
    IEEE-ASME Transactions on Mechatronics, vol. 16, (no. 5), pp. 853-860, 1083-4435
  • F. Sarlegna, G. Baud-Bovy, and F. Danion
    Delayed visual feedback affects both manual tracking and grip force control when transporting a hand-held object
    Journal of Neurophysiology, vol. 104, pp. 641-653, 2010
  • D. Mèary, L. Leocani, R. Chieffo, M. Comola, G. Comi and G. Baud-Bovy
    Probing the Control Processes of the Motor System
    IEEE Transactions on Instrumentation & Measurement, vol. 59, no. 10, pp. 2488-2495, 2010
  • Baud-Bovy, G., Gatti, E. (2010)
    Hand-Held Object Force Direction Identification Thresholds at Rest and during Movement
    In A.M.L. Kappers et al. (Eds.): Haptics: Generating and Perceiving Tangible Sensations, Proceedings of the International EuroHaptics 2010 Conference, Lecture Notes in Computer Science (LNCS) 6192, Springer, pp. 231-236
  • L. Scocchia, R. Actis-Grosso, C. de' Sperati, N. Stucchi, and G. Baud-Bovy
    Observer's Control of the Moving Stimulus Increases the Flash-Lag Effect
    Vision Research, 49(19):2363-2370, 2009
  • Gentaz E., Baud-Bovy G, Luyat M. (2008)
    The haptic perception of spatial orientations
    Experimental Brain Research, 187:331-348
  • G. Baud-Bovy, D. Prattichizzo, and S. Rossi
    Contact forces evoked by transcranial magnetic stimulation of the motor cortex in a multi-finger grasp
    Brain Research Bulletin, vol. 75, no. 6, pp. 723-736, 2008
  • C. de Sperati, G. Baud-Bovy
    Blind Saccades: An Asynchrony between Seeing and Looking
    Journal of Neuroscience, 28(17):4317-4321, 2008
alt The development of perceptual and motor skills
People involved:

Gabriel Baud-Bovy, Netta Gurari, Monica Gori, Giulio Sandini

The development of object manipulation skills in children.

As adults, we are able to manipulate objects such as pressing the keys on a keyboard to write a report, strumming the wires on a guitar to play a song, and manipulating knitting needles to make a scarf. How does one develop this ability to so accurately and dexterously interact with his/her surrounding world? In this research, we are interested in better understanding how such object interaction skills develop in toddlers.

During the first year of one's life, gross motor skills advance to the point that crawling, sitting, standing, or even walking is possible. By the age of two and three, children have begun to more actively interact with their surrounding objects and are able to achieve tasks such as placing a peg in a hole, rotating a joystick, and opening a drawer (see Figure, right panel). We are investigating how the skills for interacting with such constrained objects develop in toddlers and young children.

To begin our research on this topic, we have developed the experimental apparatus shown in the Figure (right panel). The joystick is sensorized with an encoder and force/sensor so that it is possible to quantitatively describe the manner in which the lever is being interacted with. We are investigating whether young children are able to manipulate the joystick in the most efficient manner; that is, do children rotate the joystick in the non-constrained direction, or are they expending energy by pushing the joystick along the constraints (e.g., against the blue casing) which do not provide any added benefit for completing the task.

The development of object manipulation skills in children. Left: Constrained tasks that babies and young toddlers perform. Right: Experimental setup. A joystick that is sensorized with an encoder and force/torque sensor.

The combination of position and haptic cues in children

Both finger position and contact force provide information about the shape of a surface that is manually explored with the fingertip (Robles-de-la-Torre & Hayward, 2001). Previous studies have shown that adults integrate redundant information in various perceptual tasks in an optimal way, weighting each information according to its reliability (e.g. Ernst & Banks, 2000; Drewing & Ernst, 2006). In contrast, children do not appear to integrate information optimally until late childhood, both across sensory-modalities (Gori et al., 2008) and within the same sensory modality (Nardini et al., 2010). This study is the first one to investigate the development of sensory integration within the haptic modality. Preliminary results suggest that the relative weight of position and force does not changes with age despite the considerable increase in performance observed between 7 and 10 year old children (Baud-Bovy, Gori & Sandini, 2012).

The combination of position and haptic cues in children. Left panel: Experimental setup. Middle panel: Experimental conditions. Direction of the force (force cue, arrows) on the contact surface (position cue, solid line). Right panel: Weight of the position cue and performance across age groups.

Genetic component of sensory motor skills: a twin study

The general objective of this on-going research is to investigate the role played by genetics in the development of sensory-motor skills, and their relation to personality traits. A preliminary study suggested that siblings' performance in simple motor tasks are more similar than the performance between children of the same age (Bartoli et al., 2011). A further study based on a larger number of children (>100 8-to-12 year old twin pairs ) aims at disentangling ambient and genetic contributions to this result. This research is conducted in collaboration with Anna Ogliari and Marco Battaglia from the Academic Centre for the Study of Behavioral Plasticity of the Vita-Salute San Raffaele University, Milan.

Genetic component of sensory motor skills: a twin study. The motor performance of more than 200 twins was tested with a haptic device (Omni device, see middle panel) in a reaction time task, in a tracking task and in a Fitts' law task.  The average age-corrected reaction times,  as well as other measurements, were strongly correlated between siblings (right panel).

*Baud-Bovy G., *Sandini G. and *Gori M. (2012)
The development of position and force cues in haptic shape perception
European Conference on Visual Perception (ECVP 2012)

Bartoli E., Battaglia M., Ogliari A. and *Baud-Bovy G. (2011)
Sensorimotor Testing in Children
IEEE - World Haptics Conference 2011, pp. 1-6, June 21-24, 2011, Istanbul, Turkey

G. Baud-Bovy, E. Gentaz
The visual localization of the center of mass in adults and five-year old children
Current Psychology Letters: Behaviour, Brain & Cognition. 13(2):np, 2004

top
alt Haptic perception
People involved:

Gabriel Baud-Bovy, Netta Gurari

Touch and proprioception provide a wealth of information about the body movements and the geometric and material properties of the touched objects. Current research in this topic is focused on modeling sensory thresholds related to the perception of very weak forces (0-20 grams) and understanding the contribution of tactile and proprioceptive afferents. Numerous studies were also to investigate the haptic perception of various object properties (i.e. size, orientation, mass) and, more generally, how our body senses space and forces without vision.

Weak force perception

What is the minimum force transmitted by a hand-held object that can be perceived? To answer this question, we asked subjects holding the end effector of a customized high-precision haptic device to indicate the direction of a force produced by the device. We found that healthy adults can identify the direction of force of 0.05 N (5 grams) when allowed to move the arm and 0.1 N (10 grams) when the hand is stationary (Baud-Bovy & Gatti, 2010). Note that this type of research that requires a very accurate control of the force needs to be conducted with specially designed devices (Parietti et al., 2011) or with highly-customized force rendering devices (see Figure).

Weak force perception. Left: Customization of a high-precision force-feedback rendering device (Omega, Force dimension) used to render forces accurately (typical RMS < 1.5 g, see Bocca & Baud-Bovy, 2009). Note that a force sensor (Nano 17, ATI)is mounted to measure and control the force produced by the device. Right: Two flat resistive pressure sensors were recently mounted on a specially designed handle to monitor the grip force during these experiments


.Orientation perception

Reproducing a haptically perceived orientation can yield surprisingly large errors, in excess of 20 degrees in some conditions. The goal of this study is to investigate the origin of these errors and the nature of the underlying representations. To explain the error patterns observed in the reproduction of haptically perceived orientations, we suggested in a recent study that the orientations are represented categorically in terms of vertical, horizontal and oblique categories (Baud-Bovy & Gentaz, 2012). Moreover, the similarity of the error patterns across tasks and sensory modalities suggests that a common mechanism is at play in a large set of conditions (see also Gentaz et al., 2008; Baud-Bovy & Gentaz, 2006; Baud-Bovy & Viviani, 2004). These studies are conducted in collaboration with Edouard Gentaz from the Laboratoire de Psychologie et Neurocognition, Universitè Pierre Mendès-France, Grenoble, France.

The haptic perception of orientations. Left: Haptic device (Phantom 1.5, Sensable) used to present the orientation to the blindfolded participants. Right: The error patterns predicted by the extended category adjustment model (solid lines) corresponds to the systematic (solid squares) and variable (empty squares) errors observed in the experiment (adapted from Baud-Bovy & Gentaz, 2012).


Object length perception

What is the minimum transient change of length of a bimanually hand-held bar that can be detected? To answer this question, we used a bimanual haptic device ("braccio di ferro") developed by the Motor Learning and Rehab lab in the RBCS department to haptically render a rigid bar that could quickly change its dimension. The task of the subject was to report whether the length of the bar had increased or decreased. We found that the minimum change of length hat could that be detected was much larger during a discrete movement (> 10 mm) than in a static condition (< 2 mm). This finding suggests that our proprioceptive acuity markedly decreases during movement (Baud-Bovy, Squeri & Sanguinetti, 2010)

Object length perception. Left: A bimanual haptic device simulated a rigid bar, that can quickly change its length. Middle: In the static condition (ST), the subject maintained the bar (white rectangle) above a fixed position (grey bar). In the movement conditions, the subject had to move the bar forward (VR) or laterally (HR) while the bar length briefly changed, Right: The panel shows the increase of the thresholds during the movement conditions (VR and HR) relative to the static condition (ST).

Mass Perception

Does the manner in which we move an object change the perception of its mass? In this study, we compared the perceived mass of an object when it is moved different distances or at different speeds. We found that the perceived force was influenced by the movement characteristics (Baud-Bovy & Scocchia, 2009).

Mass perception. Left: The haptic device simulated objects with different masses (M) that the subject moved with different amplitudes (A) and/or durations (D). Right:  The table shows the masses (M) of objects that felt equally heavy for different movement conditions.

*Baud-Bovy G. and Gentaz E. (2012)
The perception and representation of orientations: a study in the haptic modality
Acta Psychologica, vol. 141, (no. 1), pp. 24-30

Parietti, *Baud-Bovy G., Gatti E., Riener, Guzzella and Vallery H. (2011)
Series Viscoelastic Actuators Can Match Human Force Perception
IEEE-ASME Transactions on Mechatronics, vol. 16, (no. 5), pp. 853-860, 1083-4435

Baud-Bovy, G., Gatti, E. (2010)
Hand-Held Object Force Direction Identification Thresholds at Rest and during Movement
In A.M.L. Kappers et al. (Eds.): Haptics: Generating and Perceiving Tangible Sensations, Proceedings of the International EuroHaptics 2010 Conference, Lecture Notes in Computer Science (LNCS) 6192, Springer, pp. 231-236

Baud-Bovy G., Squeri V. and Sanguineti V. (2010)
Size-Change Detection Thresholds of a Hand-Held Bar at Rest and during Movement
Haptics: Generating and Perceiving Tangible Sensations, vol. 6192, pp. 327-332, Amsterdam, July 8-10, 2010

Baud-Bovy G., Schochia L (2009)
Is mass invariant? Effects of movement amplitude and duration
Proceedings of the 25th Meeting of the International Society for Psychophysics, Fechner's Day, 21-24 October, Galway, Ireland, pp. 369-374

Gentaz E., Baud-Bovy G, Luyat M. (2008)
The haptic perception of spatial orientations
Experimental Brain Research, 187:331-348

Baud-Bovy G., Gentaz E. (2006)
The haptic reproduction of orientations in three-dimensional space
Experimental Brain Research, 172(3):283-300

Baud-Bovy G., Viviani P. (2004)
Amplitude and direction errors in kinesthetic pointings
Experimental Brain Research, 157(2):197-214

top
alt Finger force synergies in multi-finger grasps
People involved:

Gabriel Baud-Bovy, Ali Akhras

The general objective is to understand how the brain controls contact forces during the manipulation of objects. We are currently starting to investigate with Transcranial Magnetic Stimulation (TMS) whether motor imagery alone can evoke the complex force synergies typically observed in multi-digit grasps (see also Baud-Bovy et al., 2008). The research with TMS is conducted in collaboration with Alessandro d'Ausilio from the Mirror Neuron Lab.

Effect delayed visual feedback on the grip force

This study investigated the role of visual feedback in the control of the grip force (Sarlegna et al., 2010). Surprisingly, we found that grip force was influenced by a delay in the visual feedback. This result could be explained by assuming that the delay in the visual feedback induced an illusion that affected the perceived load of the hand-held object (see Figure).

Effect of visual delay on load force/grip force coupling. Left: The task consisted in moving an hand-held object attached with a string (load force, (FE)). Right: The delayed visual feedback gave the illusion that the cursor represented the movement of an object weighting approximately 500 grams attached to the hand with a second spring adding an illusory load (FM). The total load force ((FE) + (FM)) corresponding to this illusion could explain the observed Low Force/Grip Force coupling.


A theoretical study of the capacity of the human hand for efficiently producing a stable grasp

The following study investigated the existing trade-off between grasp stability (which increases when the object is squeezed) and energy expenditure. In particular, this study investigated what happens with respect to the stability of the grasp when the "effort" is minimized (Baud-Bovy et al., 2005).

Does torque minimization yield a stable human grasp? The study investigated whether grasping with the least effort possible would yield a stable grasp (Baud-Bovy et al., 2005). We found that a cost function minimizing energy expenditure could predict a stable grasp if the net force is zero. 


Left Panel: Kinematic model used in the study. Middle Panel: contour plot of the energy expenditure with null external force. Right Panel: Contact forces predicted by the model minimizing energy expenditure. Note that forces are oriented inwards even though this constraint was not included in the model.

The virtual finger hypothesis

This study investigates the effect that the shape of an object has on the direction of the contact forces used to grasp the object. This experiment yielded evidence that the control of finger forces was hierarchically organized (Baud-Bovy & Soechting, 2001).

The virtual finger hypothesis. This hypothesis states that contact forces are constrained by the position of virtual fingers, which are defined at a higher level in the  motor control hierarchy. Top Panels: Top view of the contact forces (thin solid lines) in the horizontal plane when lifting differently shaped objects with three digits (Baud-Bovy & Soechting,  2001). This study showed that the thumb  force (Th) was directed toward a point at a mid-distance between the index (I) and middle (M) fingers,  supporting  the notion that this position corresponded to a virtual finger (VF).

Santello M., *Baud-Bovy G. and Jorntell H. (2013)
Neural bases of hand synergies
Frontiers in Computational Neuroscience, vol. 7, (no. 23), 1662-5188

F. Sarlegna, G. Baud-Bovy, and F. Danion
Delayed visual feedback affects both manual tracking and grip force control when transporting a hand-held object
Journal of Neurophysiology, vol. 104, pp. 641-653, 2010

G. Baud-Bovy and G. Manca
Motor synergies in the control of a time-varying force with a two-and three-finger grasp
Progress in Motor Control VII, 2009

G. Baud-Bovy, D. Prattichizzo, and S. Rossi
Contact forces evoked by transcranial magnetic stimulation of the motor cortex in a multi-finger grasp
Brain Research Bulletin, vol. 75, no. 6, pp. 723-736, 2008

G. Baud-Bovy and B. F. M. Marino
Anticipatory Finger Synergies in the tripod grasp before an impact
Congress 2007 of the Italian Society of Neuroscience, 2007

G. Baud-Bovy, D. Prattichizzo, and N. Broggi
Does torque minimization yield a stable grasp?
In F. Barbagli, D. Prattichizzo, K. Salisbury (Eds.), Multi-point interaction in Robotics and Virtual Reality. Springer Tracts for Advanced Robotic (STAR), Springer, Berlin, pp. 21-40, 2005

G. Baud-Bovy and J. F. Soechting
Two Virtual Fingers in the Control of the Tripod Grasp
Journal of Neurophysiology, vol. 86, pp. 604-615, 2001

top
alt Low-cost assessment of the sensory-motor state
People involved:

Gabriel Baud-Bovy, Fabio Tatti

A haptic interface is a robot that can provide force feedback to the user. Nowadays, haptic interfaces encompass a wide variety of devices, from low-cost devices aimed at gamers to large, powerful and expensive systems used in some clinics for the rehabilitation of patients. The general objective of this research is to investigate the potential of low-cost haptic-based solutions to assess one's sensory-motor state.

At-home rehabilitation of posture and balance

The general aim of this project is to provide a low-cost and game-based system that could be used at home to patients who need additional physical therapy (Pirovano et al., 2012). Our main contribution to this project consists in developing and testing the effectiveness of a force-feedback device in assisting or perturbing the patient's equilibrium. To that end, we are currently characterizing the impact of various haptic effects on posture and balance (see Figure). Additionally, we are comparing low cost gaming devices (Falcon) to high-end haptic devices (Omega) to establish whether the impact on posture and balance significantly differs. This project is run in collaboration with the University of Milano as part of the European project REWIRE

Image title

Posture and balance study. The experimental setup used to evaluate the effect of force feedback assistance / disturbance produced by a hand-held haptic device (Omega device is represented in the figure) involves an infrared camera based motion capture system (Vicon) and a force platform (Kistler).

Process-specific assessment of the motor control system

In this project, we investigate the potential of commercially available force feedback devices to assess specific processes involved in the control of the hand in tracking tasks, both in normal population (Mèary & Baud-Bovy, 2009) and in a stroke patient (Leocani et al., 2011).

Upper limb function assessment. In this study, the visual and force feedbacks were manipulated to probe various control processes involved in various manual tasks (see Table; Mèary & Baud-Bovy, 2009). A study based on 80 adults showed a global decrease of performance with age, which was more marked in tasks relying more intensively on visuo-motor control processes (Mèary & Baud-Bovy, in preparation). The whole testing procedure lasts about 15-20 minutes per hand with the aim of being potentially usable by patients in a clinic (Mèary and Baud-Bovy, 2010)

*Baud-Bovy G., Tatti F. and Borghese N. A. (2013)
An evaluation of the effects on postural stability of a force feedback rendered by a low-cost haptic device in various tasks
IEEE World Haptics Conference 2013, pp. 667-672, Daejeon, Korea, April 14-17, 2013

Pirovano M., Mainetti R., *Baud-Bovy G., Lanzi P. L. and Borghese N. A. (2012)
Self-Adaptive Games for Rehabilitation at Home
IEEE Conference on Computational Intelligence and Games (CIG 2012), pp. 179 - 186, Granada (Spain)

D. Mèary, L. Leocani, R. Chieffo, M. Comola, G. Comi and G. Baud-Bovy
Probing the Control Processes of the Motor System
IEEE Transactions on Instrumentation & Measurement, vol. 59, no. 10, pp. 2488-2495, 2010

D. Mèary and G. Baud-Bovy
Toward a robot-assisted assessment of the control processes of the motor system
Proceedings of the third joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (World Haptics), pp. 368-373, 2009

top

INFORMATION NOTICE ON COOKIES

IIT's website uses the following types of cookies: browsing/session, analytics, functional and third party cookies. Users can choose whether or not to accept the use of cookies and access the website.
By clicking on further information, the full information notice on the types of cookies used will be displayed and you will be able to choose whether or not to accept cookies whilst browsing on the website.

Try our new site and tell us what you think
Take me there