Over the past decade, developments in the global economy have driven major changes in the way industries produce their goods. Today, there is increasingly varied demand from consumers, requiring robots and other manufacturing systems to deal with a larger variety of products, which also tend to be produced in smaller quantities than before. Consequently, instead of performing a small set of tasks in a repetitive manner as they tended to do in the past, today’s robots are rather asked to execute increasingly complex and varied operations. Therefore, in order to adapt today’s robots to modern manufacturing environments, current perception technologies and control algorithms must evolve significantly. In this perspective, tactile perception and dexterity have been identified as critical elements for the future of robotics (Georgia Institute of Technology, 2013; University of California San Diego, 2016). They are currently a bottleneck that hinders robots’ ability to manipulate and interact with their environment (University of California San Diego, 2016), and advances are necessary to meet current and future automation needs.

Robots in the agile manufacturing paradigm

Different factors explain why consumers demand is changing more quickly today than it was fifty years ago. One factor is the globalization of markets, which provides consumers easy access to a wide variety of products from around the world. This allows consumers to discover new products more easily. The ease and speed with which we can now communicate information from a source to a destination located almost anywhere around the world have also contributed to this phenomenon.

In this agile manufacturing ideology, traditional automation is not suited to smaller quantitybut-higher-variety needs, since changes to products or processes generally require costly upgrades and significant downtime. Instead, the ideal manufacturing systems is flexible enough to accommodate a larger number of configurations. For robots to comply with this new context where mass customization is key, both their perception and control algorithms must be improved (University of California San Diego, 2016). Besides, the recent growth in adoption of collaborative robots in factories adds its share of challenges too, since a growing number of robots operate in the same workspace as humans. During the past few decades, research in robotic perception has mostly been focused on providing robots with accurate and reliable artificial vision. Progress in this field has successfully enabled robots to automate a large number of industrial processes. Yet despite significant improvements in artificial vision, many tasks are still difficult or impossible to automate with the current methods of perception employed by industry .

Dexterous manipulation: bridging the gap between humans and robots

The sense of touch helps humans achieve many tasks. Indeed, the somatosensory system (i.e.: touch), with its four main types of mechanoreceptors, gives us sensations that are essential to the execution of a large number of tasks that we often taken for granted. However, people who suffer from hypoesthesia (a reduced sense of touch) have a very hard time doing their everyday activities (Johansson & Flanagan, 2009), highlighting how important this sense really is. These same tasks that are generally easy for humans are frequently difficult to achieve from a robotics perspective. For example, the simple act of lacing a pair of shoes, taking an egg out of its box, or folding a stack of laundry is still quite challenging for robots.

On assembly lines, this gap between robotic and human dexterity severely limits the progression of automation in industries. One of the major limitations affecting industry is the difficulty these robots have in carrying out tasks that require a high level of dexterity, such as assemblies that are highly constrained in force and torque, and particular insertion tasks (Roberge & Duchaine, 2017). This explains why the technical challenge at the 2018 World Robot Summit (WRS) was a complex assembly task that included several insertion steps and aimed at assembling a complete belt drive unit (WRS, 2018): a relatively easy task for human workers, but a very difficult one for robots. Even the simple task of connecting a USB key to a computer port often requires advanced manipulation algorithms to be put in place (Li et al., 2014), whereas humans are able to do this blindfolded using solely their sense of touch.

E-commerce giants also suffer from this lack of advanced dexterous manipulation skills in robots, which limits the automation level in their warehouses. A concrete example of this limitation can be found, for example, in the material handling chains of e-commerce companies such as Amazon, Alibaba, Ebay and many others. These companies have to deal with a wide variety of items and even today, the automation of the packaging step, as well as the handling of items purchased by customers, still remain problematic. These actions are therefore still carried out manually to a large extent, although robotics is already widely spread inside warehouses (Ackerman, 2019; Ibbotson, 2018). For example, Walmart uses Alphabot robots (Lert, 2020) inside highly automated warehouses to move items purchased by customers, but grabbing the items, aggregating them into orders and placing them into boxes are still performed manually by employees (Ibbotson, 2018). One particular problem with having robots do this task is to implement a control strategy that allows the stable grasping and handling of a particular object without damaging it. Multiple difficulties have prompted a significant proportion of these companies to pay large sums of money to finance research dedicated to the development of technological solutions adapted to these problems. One of the best-known examples in the field is Amazon, which has catalyzed advanced and dexterous manipulation research through the popular Amazon Picking Challenge (Lawrence, 2020). Clearly, industry is motivated to solve the problems brought about by robots’ lack of sense of touch.

Tactile sensing transduction techniques

There are many ways to build a tactile sensor, but they all share one point: they try to replicate, at different levels, at least one function of a biological tactile sensory system. In humans, functions of our biological tactile sensory system are carried out by four distinct types of mechanoreceptors: Merkel’s disks, Meissner’s corpuscles, Pacinian corpuscles, and Ruffini endings. What we consider our sense of touch is the combined action of these mechanoreceptors, which are individually responsible for a distinct perceptual function (Johnson, 2001). Together, they allow humans to feel light pressure, deep pressure, vibrations, shear, torsion and temperature (Iwamura, 2009). This section presents different transduction technologies that are commonly mentioned in the literature and used to build sensors that can reproduce some of these sensing capabilities. Some of the most famous sensor implementations for each transduction method will be presented and discussed at the same time.

Optical and vision-based sensors

ptical tactile sensors generally consist of light-emitting diodes (LEDs)—a transduction medium which also often acts as the contact interface—and at least one photodetector, such as a charged coupled device-based (CCD-based) camera or a photodiode. Depending on the implementation, a geometrical change in the transduction medium will change the way light is transmitted, for example by modulating the reflection intensity, altering the received light spectrum, or changing its wavelength, phase, or polarization (Xie et al., 2013). Some optical tactile sensors are based on frustrated total internal reflection (FTIR) (Lavatelli et al., 2019). Another related technique is based on the visual tracking of some known features embedded in the sensor’s material while external forces and moments are applied . These sensors have some advantages over sensors based on other transduction technologies: their spatial resolution is generally high, and they are unaffected by electromagnetic noise. However, optical tactile sensors tend to be bulky, to consume a significant amount of power, and to require more computing power to process their data (Kappassov et al., 2015).

Early examples of optical tactile sensor implementations were suggested by Begej (1988). The principle was relatively straightforward and consisted of using a camera to measure the frustration of the total internal reflection (TIR) happening on an elastomer interface. While this specific sensing method is relatively simple to achieve, it still requires a bulky implementation and can mostly only be used to localize and to quantify contact area.

Today, many researchers are choosing vision-based tactile sensors. A good example of such sensors are those from the TacTip family (Ward-Cherrier et al., 2018), where a camera is used to track a group of pins embedded within a silicon material with skin-like smoothness. These specific sensors are able to accurately localize contact points with an average error ranging from 0.16−0.20 mm. However, the fact that these sensors measure from 85 to 161 mm (from the base to the sensing pad) could make it difficult to integrate them into a robotic gripper, because they could severely limit the stroke of most grippers. Another well-known example of a vision-based sensor is the GelSight tactile sensor, which is one of the tactile sensors with the highest spatial resolution (Johnson & Adelson, 2009). It has even been shown that this sensor can detect the difference in height caused by the ink on the surface of a twenty-dollar bill. Although this sensor has been integrated to a robotic gripper (Li et al., 2014), further concerns about its size have pushed researchers to develop a revised and more compact version called GelSlim (Donlon et al., 2018). This version uses a mirror to change the camera’s field of view, which enabled significant reductions to the sensor’s thickness. However, the sensor must now be embedded into a whole finger instead of just a fingertip. Both GelSight and GelSlim rely on three types of illumination (red, blue, and green) at three different locations and use photometric stereo to convert the images to 2.5D data, which requires more computing power than most transduction technology implementations.