"#$%&'$-The EU funded RTD project ~HAPTEX addresses
the challenge of developing a Virtual Reality (VR) system for the realistic and accurate rendering of the physical interactions of humans with textiles, through the real-time generation of artificial visual and haptic stimuli. This challenge concerns the development of both the SW and the HW components of the VR system, as well as it implies a substantial advancement in the understanding of the mechanisms underlying the human haptic perception of fine physical properties like those of textiles. This paper reports some important details relating to the technical implementation of the developed HW and SW components with special emphasis on the issues related to their integration into a single VR system. Furthermore some preliminary results relating to the functional tests carried out on the integrated system are also reported.
I. INTRODUCTION EXTILES are deIormable objects characterized by very Iine surIace and bulk physical properties, indicated with terms such as stiIIness, smoothness, soItness, Iullness, crispness, thickness, weight, etc. Taken as a whole they constitute the so called Fabric Hand (deIined in |1|) oI a speciIic Iabric, which is the basis Ior assessing its quality in relation to a given use (Ior example Ior realizing a man`s winter suit). These properties can be well distinguished and quantitatively evaluated by the human haptic sensorial system (related to sense oI touch), with an important contribution given by the sense oI sight. There is experimental evidence that the highly sophisticated mechanoreceptors located in the human skin have a predominant role in the evaluation, even iI the signals generated by these sensors are combined in the brain with those generated by the kinesthetic sensors located in the physiological articulations and in the muscles. For example, when gently stroking the Iingertip on a Iabric to evaluate its smoothness, the kinesthetic sensors give to the brain
* PERCRO Laboratory, Scuola Superiore Sant`Anna, Pisa, Italy Iontana, marcheschi, tarri, Iabio, bergamasco}sssup.it # WelIenlab, Institute oI Man-Machine Communication, Leibniz Universitt Hannover, boettcher, allerkamp, Iew}welIenlab.de Biomedical Physics Group, School oI Physics, University oI Exeter alan.c.brady, j.qu, i.r.summers}exeter.ac.uk inIormation about the Iingertip speed and the global Iorce exerted on the Iabric while the mechanoreceptors sense the small local Iluctuation oI the tangential Iorce due to Iriction. The EU Iunded RTD project 'HAPTEX addresses the challenge oI developing a VR system intended Ior the visual- haptic realistic and accurate rendering oI the complex physical interactions arising during the manipulation oI textiles. Due to the limitations oI the present technology, since the beginning it has been decided to Iocus the system simulation capability on the interactions that can be attained using only two Iingertips: the ones oI the index and the thumb (see Figure 1).
Figure 1 - Reference scenario of the HAPTEX project. Taking into account the above considerations, the reIerence conIiguration Ior the development oI the device responsible Ior generating the artiIicial mechanical stimuli to be delivered on the Iingertips(named the Whole Haptic Interface, WHI), has been conceived as the combination oI two independent Iorce-controlled manipulators (Force Feedback Device, FFD), and two arrays oI independently actuated pins (Tactile Actuator, TA). Each FFD is able to track the movements oI the Iingertip and to convey the global Iorce oI arbitrary direction on it, and each TA mounted on the end-eIIector oI the corresponding manipulator is able to deliver to the surIace oI the Iingertip skin speciIied spatial and temporal patterns, (see Figure 2). Integrating Force and Tactile Rendering Into a Single VR System Marco Fontana *, Simone Marcheschi*, Federico Tarri *, Fabio Salsedo *, Massimo Bergamasco *, Dennis Allerkamp #, Guido Bttcher #, Franz-Erich Wolter #, Alan C. Brady , Jianguo Qu , Ian R. Summers T 2007 International Conference on Cyberworlds 0-7695-3005-2/07 $25.00 2007 IEEE DOI 10.1109/CW.2007.37 277
Figure 2 - Reference configuration of the Whole Haptic Interface . II. DESCRIPTION OF THE SYSTEM A. Svstem Architecture Achieving a convincing virtual textile simulation requires a good compromise to be reached between the need oI accuracy in the material representation and the need oI speed Ior obtaining visually realistic simulation Irame rates compatible with real-time perception. These Iactors have to be considered both in the visual and the haptic Iields. However, the graphics rendering loop has diIIerent requirements compared to the haptic rendering loop in terms oI reIresh Irequencies. While in graphics a reIresh rate oI 30 Ips is quite acceptable, in haptics a response Irequency oI 300-1000 Hz is needed to ensure accurate interaction. A dedicated structure has thereIore been deIined Ior adapting the diIIerent Irame rates required by the mechanical simulation and the haptic rendering computations. Two separate computation threads were implemented: The Iirst is a low-Irequency thread running a complex large-scale simulation oI the whole cloth surIace achieving quantitative accuracy oI the diIIicult nonlinear anisotropic behavior oI cloth in real-time. This simulation has to use nonlinear strain- stress Iunctions related to the cloth being simulated. An eIIicient non-linear minimization method computing spline Iunctions approximating the aIorementioned non-linear strain- stress curves is presented in |20|. The second thread is a high- Irequency thread Ior computing the local data necessary Ior haptic rendering and Ior accurately sending haptic Iorces back to the mechanical simulation. Textile Simulation Thread Textile Simulation Thread Local Simulation Thread Local Simulation Thread Force Feedback Thread Force Feedback Thread running at 60Hz running at 300Hz running at 1kHz bordering geometry Local Geometry bordering geometry bordering geometry Local Geometry force generation Force Extrapolation Force Feedback Device Force Feedback Device forces positions Motion Estimation movements Visual Display Visual Display global mesh adaption Fingertip Model contact formulation deform refine Tactile Rendering Thread Tactile Rendering Thread Tactile Array Tactile Array contact area force distribution, velocity amplitudes
Figure 3 - Different threads within the HAPTEX system Although the threads allow parallelizing oI the computation, we also need some synchronization oI the threads. ThereIore, the synchronization works as Iollows. In the initial stage all threads are running at their dedicated update rates. The Iorce Ieedback thread is constantly Ietching new positions Irom the Iorce Ieedback device. These positions are processed to predict the user`s motion and to estimate the next position. At the same time the (global) textile simulation thread is computing the deIormations caused only by gravity, whereas the local simulation thread waits Ior any new local geometries to be simulated. At each simulation step oI the global thread the local thread receives Iingertip dimensions, the current and predicted position. The global thread analyses its underlying mesh with respect to potential collisions with the Iingertip Ior the next time step. These regions including their physical states are sent to the local thread in order to be geometrically reIined and inserted into the local simulation. AIterwards both simulation threads continue to run according to their data. With the newly added local mesh, the local thread checks iI any collision has taken place in between a local simulation time step. In case oI a contact the occurring deIormation oI the local part oI the textile is computed according to the Iingertip model being used. The Iorces at the Iingertip generated during the contact are sent to the Iorce Ieedback thread. The contact area estimated by the contact model is transmitted to the tactile renderer whereby Ior each pin a contact Iorce is computed. According to the deIined positions oI the pins on the Iinger the local velocity is also provided. The schematic in Figure 3 shows the separation oI the computational tasks into the diIIerent threads. B. Force Renderer The Iorce-Ieedback renderer is responsible Ior the modeling oI the interaction between the Iingertip and the Iabric. This implies the computation oI Iorces occurring at the contact, considering the physical properties oI the objects involved. In the haptics literature there exist several approaches to render contact Iorces. In |17| an eIIective point-based rendering algorithm was Iirstly introduced and constantly improved by others, i.e. |19||18|.Ruspini et al. |15||16|extended the algorithm to support contacts oI arbitrary shapes. In contrast to the well known proxy method a recent approach (see |14|) suggests to compute the contact Iorces by solving the Signorini contact problem employing Iinite elements. Although the latter method models the deIormation oI the Iingertip at the contact appropriately, it is demanding oI precious computation time because it requires iteratively solving non-linear equation systems. A good compromise between accuracy and computational eIIort is oIIered by a penetration- or penalty-Iorce based method. The penetration- or penalty-Iorce-model computes the Iorce as a result oI the contact proportional to the penetration depth or intersecting volume. The depth is given as the length oI the vector deIining the shortest translation oI the colliding bodies to a touch situation. For computational reasons we use the penetration depth Ior Iorce calculations. The mechanical equivalent to the aIorementioned situation can be described by a spring attached to both bodies enIorcing a repulsion in case oI a collision. The Iorces being applied on both bodies are computed by the length oI the depth vector and diIIer only in the direction. II no additional external Iorces are applied, then aIter several simulation steps the bodies will reach a Iorce equilibrium as depicted in Figure 4. 278
d d finge rt ip s phe re x d x v n x v n+ 1 F C n F C n+ 1 node s of t e xt ile d d node s of t e xt ile finge rt ip s phe re x d x v x v n n+ 1 F C F C n n+ 1 d d finge rt ip s phe re x d x v n x v n+ 1 F C n F C n+ 1 node s of t e xt ile d d node s of t e xt ile finge rt ip s phe re x d x v x v n n+ 1 F C F C n n+ 1
Figure 4 - Initial state (left) and final state (right) of the penetration algorithm At this stage the bodies are still intersecting. In this strategy the Iingertip is modeled as a rigid sphere being the Iirst contact body. Due to its shape, a Iast collision test with the textile can be made. The Iingertip collides with a textile particle n with position v n x iI d d v n d x x < holds. Both Iingers are in collision under the Iollowing condition: d d d d x x 2 1 0 < . To distinguish between static and kinetic Iriction we separate the Iorce resultant r v F into tangential T v f and normal N v f components applied to a textile particle v. The Iorce r v F is the sum oI the internal Iorce J v F , penetration Iorce C v F and Iriction Iorce. II both Iorces satisIy the stick condition equation Ior static Iriction as seen below. N v S t v f f < Then we have v N v r v T v r v v N v n f F f F n f = =
In this state the particles are not moving relatively to the Iingertip. The position is set according to the Iingertip movement. Forces oI the remaining Iree particles oI the textile have to be recomputed. Otherwise, iI the stick condition is not met, the kinetic Iriction is added to the particle. The Iorce sent to the Iorce Ieedback device is determined by the reaction Iorce oI the contacting particles and the second Iingertip D F . This leads to the Iollowing equation rel rel v N v v v rel rel N v S C v D d v v F v v f F F F K K S s
= where v indexes the particles being in contact with the Iingertip and v S and v K deIining the particles respective to their Iriction state.
C. Tactile Renderer The tactile renderer is based on vibrations that play an important role in the tactile exploration oI Iine surIaces. To produce appropriate excitation patterns we use an array oI vibrating contactor pins as described in Section II-E. The tactile renderer generates 24 drive signals Ior the individual contactors oI a single tactile stimulator, on the basis oI the Iollowing inputs: small scale description oI the textile surIace: a single 'tile oI the textile weave pattern, speciIied as a pseudo-topology at 0.1 mm resolution over the 2D surIace oI the tile (derived Irom Kawabata measurements oI roughness and/or Iriction); large-scale description oI the textile surIace: a representation oI the non-uniIormity oI the textile surIace, speciIied as a pseudo-amplitude at 1 mm resolution over the 2D surIace oI the 200 mm 200 mm sample oI virtual textile (derived Irom optical imaging); position and orientation oI the Iinger pad on the virtual textile; speed and direction oI the movement oI the Iinger pad over the virtual textile.
Figure 5 - The tactile renderer. Input and ouput data are specified in 25 ms timesteps. Figure 5 shows an overview oI the process using the input data are used to speciIy the output drive signals. Taking into account oI the direction oI the movement, a spatial-Irequency spectrum is calculated Irom the pseudo topology oI the small- scale description oI the virtual surIace. InIormation on the speed oI movement oI the Iinger pad is used to convert spatial- Irequency components into temporal-Irequency components. The resulting temporal- Irequency spectrum is reduced to only two amplitudes, A 40 and A 320 , by applying appropriate bandpass Iilter Iunctions (see |21|), corresponding to the 40- Hz and 320-Hz channels. Amplitudes Ior the 40- Hz 279
component in the drive signals Ior each oI the 24 channels are obtained Irom A 40 by weighting according to data Irom the large-scale description oI the virtual surIace, Ior the 24 locations on the Iinger where the contactors oI the tactile stimulator are positioned. Similarly, amplitudes Ior the 320-Hz component in the drive signals Ior each oI the 24 channels are obtained Irom A 320 by weighting according to data Irom the large-scale description oI the virtual surIace.
D. Force Feedback Device In the scope oI the HAPTEX project the Force Feedback Device (FFD) is responsible Ior tracking the global movements oI the user`s index and thumb Iingertips, delivering controlled Iorces on them as evaluated by the Force Renderer Module and holding the Tactile Actuators. In order to allow early integration oI the diIIerent components oI the HAPTEX system, it has been decided to develop two diIIerent types oI FFD, the Iirst one derived Irom an existing device (named 'FFD in conIiguration A) and the second one designed Irom scratch (named 'FFD in conIiguration B). Although the two conIigurations use very diIIerent basic solutions and hardware implementations, they have quite similar Iunctionalities, even iI the related perIormances are substantially diIIerent. In both cases, one oI the main requirements in their development has been the achievement oI a highly accurate Iorce Ieedback. This requirement derives directly Irom the stated goal oI allowing the discrimination oI the Iine mechanical properties oI the textiles. From the technical point oI view this implies the accurate generation and control oI Iorces that can be oI the order oI Iew grams (0.01 N), as they arise during the natural manipulation oI textiles. To address this challenge, a new explicit Iorce control has been developed making use oI purpose-designed highly sensitive multi-component Iorce sensors, placed directly in contact with the user`s Iingertips. The Iollowing paragraphs describe in detail the two conIigurations and report some experimental data relating to their perIormances. 1) FFD in Configuration A The FFD in conIiguration A has been derived Irom the GRAB system (a detailed description is reported in |2|), developed by the PERCRO Laboratory in the Iramework oI the homonymous European RTD Project. The system is composed oI two identical robotic arms, each having 6 DOF oI which the Iirst 3 translational DOF are instrumented and actuated, while the remaining 3 orientational DoF are only passive. Each robotic arm is Iunctionally equivalent to the well known Phantom |3| haptic interIace but has larger workspace.
Figure 6 - Picture of the sensor-equipped gimbal. With respect to the original GRAB system, the FFD in conIiguration A uses new gimbals (see Figure 6) mounted on the robotic arm, each equipped with 3 rotational position sensors (encoders) Ior the measurement oI the Iingertip orientation and with a 3 component highly sensitive Iorce sensor speciIically conceived Ior the application, and an explicit Iorce control making use oI the said Iorce sensor. Explicit Iorce control algorithm consists oI an inner velocity loop-outer Iorce loop scheme (see Figure 7). The mechanical model oI the device and the criteria Ior the dimensioning oI the controller are described in detail in |5| and in. |6| Due to the presence oI elasticity in the transmission oI the Iorce Irom the actuator to the moving mass, the bandwidth oI the Iorce controller has been limited to 3 Hz in order to ensure the stability oI the system. d F f K Kv V1 F (s) (s) F V2 m F s Xm Velocity Loop (s) F f Fu Force Loop Plant Force Loop Plant
Figure 7 - Scheme of the force control Furthermore, investigations on the attachments oI the device with the Iingertips have been carried out. Indeed, during the natural handling with the textiles the interaction Iorces can be oI the order oI Iew grams. This implies that the attachment oI the Iingertip with the device has to be so to not hinder the high sensitivity oI the human mechanoreceptors located in the skin. From this point oI view the thimble-like attachment, commonly used by the existing haptic interIace, is not well suited because it produces a pre-stress oI the skin reducing the sensitivity oI the mechanoreceptors. In order to address this issue, diIIerent alternative solutions Ior the Iingertip attachment have been investigated (a bare plate, a plate with elastic strip, a plate with eccentric thimble). As expected the experimental test demonstrated that the human sensitivity is at best when the bare plate is used, even iI this solution is the worst Ior the transmission oI the torques required Ior the orientation oI the gimbal. 280
Finally a Iield test investigating the system capability to track the Iinger motion with minimal resistance Iorce has been carried out. The subjects were asked to move their Iingers at constant velocity and the system was set to display no Iorces. The maximum module oI the acquired resistant Iorce was about one tenth oI a Newton (10 grams Iorce), as it can be observed in Figure 8. 2) FFD in configuration B. development of the hand exoskeleton The FFD in conIiguration B consists oI a Hand Exoskeleton (HE) expressly conceived Ior the accurate generation oI light Iorces. 0 2 4 6 8 10 12 14 16 -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 Time (s) F o r c e
( N )
Figure 8 - Plot of the resistant force versus time during finger tracking at constant velocity Several works (Ior example |4|, |7| and |8| can be Iound in the literature addressing the development oI HEs. According to their type oI Iunctionality, the existing HE can be grouped in two diIIerent categories,: - Multi-phalanx HEs: they can generate Iorces on each phalanx oI the Iinger along a Iixed direction with respect to the phalanx (e.g. normal to the phalanx); - Fingertip HEs: they can generate Iorces only on the (the Iingertips along arbitrary direction in 3 D space. Considering the application addressed by the HAPTEX project, the second type oI Iunctionality has been selected. A scheme oI quasi-anthropomorphic kinematics has been selected Ior the implementation oI the Iinger exoskeleton. This solution allows exploiting the beneIits oI anthropomorphic kinematics, like the maximum ratio oI the available over the needed workspaces and minimum encumbrance oI the linkages, while avoiding at the same time the singularity that would occurs when the Iinger is completely extended. In Figure 9 a CAD model oI the HE is shown. It can be noticed that the encumbrance oI the device has been mainly located in the dorsal side with the aim oI allowing the complete closing oI the hand. This has been achieved through the use oI Remote Centre oI Rotation Mechanisms that implement rotational joints having the axes located outside the linkages. The whole mechanism has 4 Degrees OI Freedom (DoF), even iI it is actuated with only three motors, thanks to the coupling oI the last DoF (end-eIIector Joint) with the previous one. The coupling is acceptable because also in the human hand the last phalanx can be rarely moved independently Irom the middle phalanx during natural movements. The HE is equipped with three electrical motors with low speed reduction ratios (1:14). The actuators are placed at the base oI the Iinger exoskeleton in proximity oI the dorsal side oI the palm. The joints are actuated through steel cables working in tension. For the position sensing common incremental encoders located on the axis oI the motors have been used while Ior the Iorce sensing a compact highly sensitive 3 component Iorce sensor, placed directly in contact with the user`s Iingertip, has been expressly conceived Ior the application. The purposely developed electronics Ior the sensor acquisition and the driving oI the motors have been located inside the motor box. The communication with the control PC takes place through standard RS-232 serial cable. The device is capable oI exerting a continuous Iorce on the Iingertips oI 5N with a resolution oI 0.005 N. At present, several parts oI the device have been realized and other ones are being manuIactured. A Iurther development will be the integration oI the dorsal tactile array described in Section III on the dorsal side oI the last phalanx oI the exoskeleton.
Figure 9 - CAD model of the Hand Exoskeleton. E. Tactile Actuator Two designs oI stimulator array have been developed as the tactile component oI the HAPTEX system, hereaIter reIerred to as conIigurations 'A and 'B. Both conIigurations use identical actuators and have the same contactor layout, and each has been designed to be used with the FFD. The conIiguration 'A actuator has been designed to display a 2D tactile pattern on a single Iingertip. In contrast, the type 'B actuator (pictured in Iigure 10) is suitable Ior use in a pair, displaying tactile inIormation to both the Iinger and thumb in a grasp conIiguration. In an ideal device, the Iingertip would be completely covered with contactors. The spacing oI these would be around 1 per square millimeter, matching the spatial acuity oI the mechanoreceptors in the skin |9|. In reality, the number oI actuators is limited by the space available and the complexity oI the drive electronics and the wiring. A pin spacing oI 2 mm has been used, as it is believed that Ior many tactile tasks this is indistinguishable Irom a 1 mm spacing |10|. 24 pins are arranged in 64 grid, with Iixed pins in interstitial positions to localise the stimulation sites on the skin, and to transIer Iorces Irom the FFD to the Iinger. The height oI the active pins is set such that they indent slightly into the skin when the Iinger pad is gently pressed into the Iixed pins and the surround. 281
Both designs are driven by piezo-electric bimorphs, chosen Ior their simplicity, ease oI assembly and low cost. The dimensions oI the bimorphs were selected by numerical modelling and experiment. The Iree-length oI the bimorphs was chosen such that their resonant Irequency (100 Hz, rising to 120Hz when loaded by the skin) lies between the two working Irequencies oI the renderer (40 Hz and 320 Hz), and such that the Iirst anti- resonance (450 Hz) is not too close to the upper working Irequency. The response is designed such that the roll-oII above resonance counteracts the increase in sensitivity oI the human Iingertip |11|. This balances the perceived intensity at a given drive signal, and reduces the dynamic range required Irom the drive electronics. The width oI each bimorph is a compromise must be struck between the amount oI Iorce the actuator could deliver and the overall size oI the device. A pair oI commercially available 2.1 mm wide actuators (APC International), arranged in parallel, was chosen. This gave a width oI 4.2 mm and a static blocked Iorce oI 0.5 mN/volt. The design oI the type 'A is a logical development oI earlier systems |12|,|13|, with the piezoelectric actuators located below the palmar surIace oI the Iingertip. It is suitable Ior use with a single Iinger in a 2D environment, and has been used to develop the drive electronics and soItware, and Ior testing the tactile renderer.
Figure 10 - Early prototype of the configuration ~B, showing the two different orientations of the piezo-beams The design oI the type 'B actuator TA was constrained by the need to move the actuators out Irom underneath the Iingertip to reduce the depth oI the device below the Iingertip, and so allow Ior a grasp conIiguration. In addition, the device had to accommodate and integrate with the Iorce sensor Ior the FFD, and to Iit the FFD mechanism. In the type 'A conIiguration, the Iingertip rests on a planar surIace. The contactor pins are arranged to be perpendicular to this plane. The type 'B conIiguration uses a curved contact plate which Iollows the shape oI the Iinger pad. Contactor pins are approximately normal to the contact plate (illustrated in Figure 11), with diIIerent orientations oI the drive mechanism Ior diIIerent rows oI pins. As a result, some oI the drive links have to Iollow relatively complex paths. Subjective tests oI this conIiguration have shown that the perIormance is good.
Figure 11 - This end on sketch of the fingertip shows the orientation of the contactor pins and the direction in which they are driven. III. SYSTEM INTEGRATION A. Mechanical Integration of TA with FFD The integration oI the FFD with the TA requires the resolution oI non-trivial problems that have implications Ior the most appropriate mutual mechanical arrangement oI the diIIerent parts constituting the two devices. This arrangement should be so to allow, on one hand, the delivery to the Iingertip oI the mechanical stimulations that each device can generate and on the other, to comply with the speciIic capabilities and requirements oI the devices. In particular the TA pins can produce dynamic indentation oI the skin, but the Iorces (normal to the surIace) that they can deliver are relatively small (in the range oI Iew grams).
Figure 12 - Scheme of the force transmission between fingertip and plate Furthermore they cannot withstand lateral Iorces acting on the pin because these Iorces would produce their deIlection. In short the mechanical arrangement has to be so to prevent the loading oI the pins with relatively high Iorces. Vice versa the FFD can deliver to the skin relatively high Iorce in every direction, but in order to guarantee a high accurate Iorce Ieedback these Iorces must be sensed by the 3 component Iorce sensor (see scheme in Figure 12).
Figure 13 - Mechanical Integration of FFD and TA
282
The FFD in conIiguration A and the TA in conIiguration B have been successIully integrated in order to realize the development level 4 (DL4) oI the HAPTEX system (see Figure 13). B. Integration of the Force and Tactile Renderer For the computation oI the Iorce-Ieedback the haptic renderer already requires among others the contact point, the velocity oI the Iingertip relative to the Iabric and the normal Iorce (cI. |20|). Instead oI a single contact point the tactile renderer requires a contact area to decide which contactor pins have to be activated. Also a single normal Iorce is not suIIicient Ior a convincing tactile simulation. Rather a distribution oI the normal Iorce over the whole contact area is needed by the tactile renderer. For each contactor pin, the geometry oI the tactile actuator determines the point on the Iingertip where the stimulation caused by that pin occurs. These points have been included in the model oI the Iingertip. Employing the contact model the position (in local coordinates oI the Iabric) and the normal Iorce at each point is computed. The velocity is computed by applying a discrete diIIerential operator. The tactile renderer is called every 25 ms, i.e. it runs in a 40 Hz loop. However, the contact data is computed by the haptic renderer every millisecond and the data needs to be resampled. To avoid aliasing the data is passed through a digital antialiasing Iilter that suppresses Irequency components above 20 Hz. In the stand-alone version oI the tactile renderer the Iabric was assumed to be planar. Furthermore the Iinger was assumed to always being in contact with the plane oI the Iabric (see |9|). However, in the integrated case the Iabric may have Iolds and wrinkles, resulting in a more complex contact geometry. The Iingertip deIorms due to the contact pressure, also inIluence the shape oI the contact area. As a consequence not all pins oI the tactile actuator should be activated, only those inside the contact area. The shape oI each Iinger`s contact area is provided by the Iorce-Ieedback renderer. Note that a contact area may not exist, i.e. in that case the Iinger does not touch the Iabric. While moving a Iingertip over a rough surIace we experience a tactile sensation. Its intensity also depends on the Iorce with which the Iinger is pressed against the Iabric. For relatively small Iorces a linear dependency between the Iorce and the sensation seems to be reasonable. ThereIore the amplitudes computed by the tactile renderer are multiplied by k F F beIore they are transmitted to the tactile actuator. F denotes the Iorce between the Iinger and the Iabric and k F is a constant relating the Iorce to the intensity. Previous to the integration there was no possibility to assess the contact pressure between the Iingertip and the Iabric. ThereIore a constant Iorce normal to the Iabric was assumed implicitly by the tactile renderer.
IV. PRELIMINARY TESTS Extensive tests have been perIormed to assess the mechanical and electrical disturbance induced by the vibration and the electrical noise generated by the TA on the measure oI the interaction Iorce, because oI its potential negative impact on the accuracy oI the Iorce Ieedback. The tests have been perIormed activating only one speciIic pin oI the TA array at a time and acquiring the resulting measured Iorce signal. The Irequency spectrum oI the signal has been then evaluated using the Discrete Fourier TransIormation (see Figure 14). The disturbances produced by 4 diIIerent pins, having diIIerent locations in the TA array and mechanical coupling conditions (Iree-moving or in contact with the Iorce plate hole) have been investigated. The tests evidenced that the disturbance produced by the vibration is predominant with respect to the electrical noise and that Iree-moving pins produce about 10 times less disturbance than produced by pins in contact. The magnitude oI the disturbance on the excitation Irequency (40Hz and 320 Hz) is about 200 times greater Ior the Iree-moving pin with respect to the base noise than when the pin is not activated. Furthermore a system test has been also perIormed in order to assess the global consequence oI the induced noise on the accuracy oI the Iorce Ieedback. No meaningIul eIIects have been detected due to the low bandwidth oI the Iorce Ieedback (about 3 Hz).
Figure 14 - Typical Discrete Fourier Transform of the disturbance produced by one TA pin vibrating at 40Hz on the force measure. V. CONCLUSIONS AND FUTURE WORK In this paper we have reported the preliminary work perIormed towards the realization oI a system able to simulate the visual and haptic interaction with Iabrics. The manipulation oI Iabrics is a very complex perceptual experience that is associated with a lot oI stimuli that are very hard to simulate and to reproduce virtually. The system that has been realized integrates two kinds oI stimulation devices: a Tactile Array (TA) Ior the tactile stimulation oI the Iingertip and a Force Feedback Device (FFD) Ior delivering the interaction Iorces. In this work we Iocused on the issue oI the integration oI these two devices Irom the soItware and the hardware point oI views. From the soItware point oI view the main issue is to generate simultaneously the signals to be provided to the FFD and the TA complying with real time speciIications. To achieve this two separate soItware components called the Force Feedback Renderer and the Tactile Renderer, were realized and integrated. 283
For what concerns the hardware, the real challenges consist oI realizing a device able to convey both the delicate and Iine interaction Iorces that are involved with the manipulation oI textiles and deliver the associated tactile stimulus. It has been shown that the integration oI these two devices can raise several issues like the electrical and mechanical cross-talk between the two Ieedback devices and the issue oI how to deliver the Iorce to the user`s Iingertip whilst not transmitting it to the pins oI the array. This has lead to the design oI a purposely developed haptic-tactile integrated device. The preliminary tests on the integrated system have shown that the global perIormances oI the device are acceptable and the major issues associated with the integration have been successIully overcome. In the near Iuture we will set up the complete system Ior two Iinger interaction and perIorm several tests. The next step will be the realization oI a portable wearable FFD and to integrate it with the tactile array. ACKNOWLEDGMENT The project HAPtic sensing oI virtual TEXtiles (HAPTEX) is a research project Iunded under the Sixth Framework Programme (FP6) oI the European Union (Contract No. IST- 6549). The Iunding is provided by the Future and Emerging Technologies (FET) Programme, which is part oI the InIormation Society Technologies (IST) programme and Iocuses on novel and emerging scientiIic ideas. Its mission is to promote research that is oI a long-term nature or involves particularly high risks, compensated by the potential oI a signiIicant societal or industrial impact. REFERENCES |1| H. Behery, EIIect oI Mechanical and Physical Properties on Fabric Hand, Publisher: Taylor & Francis Ltd, 2005. |2| Avizzano C. A., Marcheschi S., Angerilli M., Fontana M., Bergamasco M.: 'A Multi-Finger Haptic InterIace Ior Visually Impaired People. In Proceedings oI the 2003 IEEE International Workshop on Robot and Human Interactive Communication, Vol. 31, (2003), 165- 170. |3| Thomas H. Massie and J. K. Salisbury., 'The PHANTOM Haptic InterIace: A Device Ior Probing Virtual Objects Proceedings oI the ASME Winter Annual Meeting, Symposium on Haptic InterIaces Ior Virtual Environment and Teleoperator Systems, Chicago, IL, Nov. 1994. |4| Bouzit M., Popescyu G., Burdea G., Bioan R.: 'The Rutger Master II-nd Force Feedback Glove, In Proc. IEEE Vr. Haptic Sysmposium, Vol.7 (March 2002) 256-263. |5| Eppinger S., Seering, W.: 'Understanding bandwidth limitations in robot Iorce control. In Proc.Robotics and Automation ConIerence, 1987 IEEE International ConIerence on , Vol.4 (1987), 904- 909. |6| Volpe R., Khosla P., 'A theoretical and experimental investigation oI explicit Iorce control strategies Ior manipulators. In IEEE Transactions on Automatic Control, vol.38, 11 (1993), 1634-1650. |7| Wege, A.; Kondak, K.; Hommel, G., Mechanical design and motion control oI a hand exoskeleton Ior rehabilitation Mechatronics and Automation, 2005 IEEE International ConIerence, Vol.1, Iss., 29 July-1 Aug. 2005, Pages: 155- 159 Vol. 1 |8| Virtual Technologies Inc., Cyber Grasp User Guide, 1999.. |9| K.O. Johnson, T. Yoshioke, F. Vega-Bermudez, 'Tactile Iunctions oI mechanoreceptive aIIerents innervating the hand, J. Clin. Neurophysiol., vol. 17, 2000, pp. 539558.
|10| I.R. Summers, C.M. Chanter, A.L. Southall and A.C. Brady, 'Results Irom a Tactile Array on the Fingertip in Proc. Eurohaptics 01, Birmingham, 2001, pp. 2628 |11| R.T. Verillo, A .J. Fraioli, R. L. Smith, 'Sensation magnitude oI vibrotactile stimuli, Perception & Psychophysics, vol. 6, pp. 366372, 1969. |12| I.R. Summers, C.M. Chanter, 'A broadband tactile array on the Iingertip, J. Acoust. Soc. Am., vol. 112, pp. 21182126, 2002. |13| I.R. Summers, A.C. Brady, M. Syed, C.M. Chanter, 'Design oI Array Stimulators Ior Synthetic Tactile Sensations, in Proc. World Haptics`05, Pisa, 2005, pp. 586587. |14| Duriez C. et al.: Realistic Haptic Rendering oI Interacting DeIormable Objects in Virtual Environments, IEEE Transactions on Visualization and Computer Graphics (2006), pp. 36-47 |15| Ruspini D.C. et al.: Haptic interaction in virtual environments, Proc. IEEE/RSJ int. conI. on Intel. Robots & Systems (1997), Vol.3 |16| Ruspini D.C. et al. : The haptic display oI complex grapical environments, Proc. 24th annual. ConI. on Computer Graphics and Interactive Techniques (1997) |17| Zilles C.B., Salisbury J.K.: A constraint-based god-object method Ior haptic display, Proc. IEE/RSJ Int. conI. on Intel. Robots & Systems (1995), Vol.3, pp. 146-151 |18| Ho C.H., Basdogan C., Srinivasan M.A.: EIIicient point-based rendering techniques Ior haptic display oI virtual objects, Presence: Teleoperators and Virtual Environments (1999), Vol. 8 |19| Morgenbesser, H., and Srinivasan, M. Force shading Ior haptic shape perception. In Proceedings oI the ASME Dynamics Systems and Control Division (1996), vol. 58, pp. 407--412. |20| Volino P. et al.: From measured physical parameters to the haptic Ieeling oI Iabric, Proc. OI HAPTEX`05 |21| Dennis Allerkamp, Guido Bttcher, Franz-Erich Wolter, Alan C. Brady, Jianguo Qu, and Ian R. Summers. A vibrotactile approach to tactile rendering. Vis. Comput., 23(2):97108, 2007