Computational Design of Skinned Quad-Robots

We present a computational design system that assists users to model, optimize, and fabricate quad-robots with soft skins. Our system addresses the challenging task of predicting their physical behavior by fully integrating the multibody dynamics of the mechanical skeleton and the elastic behavior of the soft skin. The developed motion control strategy uses an alternating optimization scheme to avoid expensive full space time-optimization, interleaving space-time optimization for the skeleton, and frame-by-frame optimization for the full dynamics. The output are motor torques to drive the robot to achieve a user prescribed motion trajectory. We also provide a collection of convenient engineering tools and empirical manufacturing guidance to support the fabrication of the designed quad-robot. We validate the feasibility of designs generated with our system through physics simulations and with a physically-fabricated prototype.

SECTION 1 Introduction

The design and construction of robots that can execute compelling motions is a challenging task. It requires careful geometric planning of robotic mechanisms and professional knowledge of the kinematic and dynamic behavior of the robot. Embedding such knowledge into procedures of computational robot design [1], [2] in conjunction with rapid prototyping techniques, such as 3D printing technology, bears tremendous potential to accelerate the construction of personalized robots. For instance, Megaro et al. [3] used a kinematic optimization algorithm for the design of multilegged robots consisting of rigid links. However, real-world creatures are not merely rigid skeleton rigs. The muscle and flesh surrounding the skeleton provides diverse morphologies, enriched expressivity. For instance, people wearing a prosthetic limb often prefer a highly realistic rubbery artificial arm over a more functional mechanical one [4]. Moreover, skins and muscles might be essential for facilitating challenging tasks, such as reproducing compliant grasping of a hand or swimming motions of a fish [5], [6].

Different to the simulation of the robots with rigid links only, the influence of the soft skin on the control torques at actuators and on the contact forces at the ground should be simulated and fully taken care of to judge the plausibility of a designed soft robot. This imposes a significant computational challenge for the motion design and fabrication of skinned robots. In this paper, our goal is to address this problem by integrating both dynamic simulation and kinematic optimization into the motion design of quad-robot systems, with soft skins attached as their organic embodiments. Its kernel is an optimization problem that integrates both user-provided kinematic preference and physical constraints of the robot to obtain a dynamically feasible motion plan. The primary physical constraint is the dynamic viability of the skeleton trajectory when a soft skin is attached. The dynamics of the robot is formulated as two-way coupled subsystems of the rigid multibody system and the deformable skin. In addition, our system also incorporates motor constraints and a stability constraint. The motor constraints ensure that the calculated joint torques are within the physical limits of the installed servo motors, whereas the stability constraint requires the center of projection (COP) of the robot structure to fall inside the supporting polygon. As a result, given the surface mesh and desired kinematic trajectories of the robot, our pipeline generates a physically-valid motion plan that can be realized under the given hardware constraints. The robot itself can be fabricated using rapid prototyping technology.

While space-time optimization is widely used in long-horizon motion design problem, it is computationally prohibitive if directly applied to our case of two-way coupled system, mostly due to the large number of DOFs of the soft skin mesh coupled with the skeleton and various physcial constraints. To this end, we propose an alternating optimization algorithm of two optimization steps to mitigate the computational cost: (1) Space-time optimization with repect to the DOFs of the rigid skeleton while assuming the deformation of the soft skin remains the same as the previous iteration. (2) Frame-by-frame optimization as in [7] to further optimize the motion plan obtained in step 1 according to all the physical constraints at each frame. To this end, we non-trivially extend the spring-based control force formulation in [7] to handle the full simulation of the two-way coupled rigid skeleton and soft skin. Our solver can efficiently handle the Lagrange multipliers introduced by the coupling constraints and the collisions between the skeleton and skin. With a proper initialization, these two steps are alternatively executed until the convergence. In the space-time optimization step, the influence of the soft skin to the rigid skeleton is treated as known quantity and simplified to be the coupling forces and the influence of the center of mass (COM) at each link in the skeleton due to the skin deformation. This setting makes the space-time optimization computationally efficient through decoupling the skin mesh DOFs, which is critical to achieve global effect in the motion design. The skin deformation is updated after each frame-by-frame optimization.

To ease fabrication, we provide a convenient workflow with tailored engineering tools and empirical fabrication guidance embedded in a standard CAD software, empowering regular users to design quad-robots. The modular design scheme allows the user to quickly start from a design template of the mechanical skeleton in SolidWorks and adapt it to the body shape. The rigid skeleton is fabricated via 3D printing, and the skin is separately fabricated using injection molding by pieces. We tested the optimization algorithm on skinned quad-robots with varying body-to-leg ratios and different mechanical skeletons. Both physical and numerical experiments show that the proposed algorithm is an effective means of obtaining physically valid motions of the skinned quad-robots.

SECTION 2 Related Work

Computational fabrication aims at designing and creating physical artifacts with the help of computational methods. A large class of methods addresses inverse design problems by incorporating fabrication limitations in geometric design algorithms via constrained optimization or the integration of fast simulation techniques [8], [9]. This line of research enables the design of objects with a wide range of controllable physical and mechanical properties, such as appearance [10], [11], [12], [13], deformation [14], [15], [16], [17], articulation [18], [19], [20], and mechanical motion [21], [22], [23], [24], [25]. Some existing contributions also investigated how to instantiate virtual characters as 3D-printable physical entities like mechanical robots [3], [21]. Yet, these methods merely focus on robots consisting of rigid links and basic balancing constraints and/or velocity limits.

Bickel et al. [26] proposed a process for designing synthetic skin and actuation parameters for animatronic characters that mimic facial expressions of a given subject. Skouras et al. [15] optimized the internal material distribution so that the resulting character exhibits the desired deformation behavior. Focusing on actuation, Bern et al. [27] computed the layout of winch-tendon networks to animate plush toys, and Ma et al. [28] optimized the chamber structure and material distribution for designing soft pneumatic objects.

Our work shares some of these goals but takes a significantly different approach. Instead of relying on a rigid or quasi-static underlying simulation of the skin deformation in our case, the influence of the soft skin on the embedded moving mechanical skeleton makes us face a dynamic two-way coupled multibody-elastic problem. It is much more challenging to accurately simulate and optimize due to the drastically increased system complexity, nonlinearity and discontinuity, i.e., due to the complimentary constraints enforced at the contact vertices.

Physics-based character motion generation has vast applications in both graphics and robotics. Algorithmic approaches include leveraging space-time optimization with necessary physical constraints and developing controllers to drive forward simulations.

The seminal work by Witkin and Kass [29] generated motion trajectories by optimizing physical constraints and animator controls at key frames, a well-known space-time constraints framework for animation. With proper motion data, the space-time optimization produces realistic articulated motions for bipedal or multilegged characters through different physical properties [30], [31], [32], [33], [34], [35], [36], [37], [38]. It can be used to transform motion capture data into physically plausible motions [39].

The locomotion controller aims to compute joint torques or control forces to drive the locomotion behaviors of articulated figures. The joint torques are usually calculated via the proportional and derivative (PD) controller such that the rigid skeleton of a character follows designated joint angle trajectories [40], [41], [42]. Balance control strategies, such as the swing foot placement or zero moment point, are essential to generating stable locomotions. [40], [41], [43], [44], [45], [46], [47], [48]. Continuous adaptation of the target joint trajectory for balancing a walking human was developed in [49]. Controllers that produce highly dynamic skills for human animation were suggested in [50], [51], [52]. The joint torques can also be computed via optimal control to approximate the motion capture data or motion data from kinematic simulators [53], [54].

Our work is inspired by studies on how to drive the soft skin deformation with the underyling rigid skeletons or pseudo muscle force [7], [55], [56]. Two-way coupling of rigid bodies and elastic bodies was considered in [57]. Fast simulation and control of soft robots of various configurations and actuations has also been studied using finite element method and the reduced formulation of compliance matrix [58], [59], [60], [61], [62]. The adjoint method is adopted to solve the space-time motion optimization problem for soft robots in two recent contributions [63], [64]. For the space-time motion constrained by the equation of motion, the adjoint method is efficient in computing the gradients of the motion with respect to control parameters. Since the method requires the constraints to be known ahead of the simulation, the contacts between the robot and the environment should be fixed as in [63] or approximated as a one-sided energy function in [64]. In contrast, our alternative optimization algorithm can handle dynamic contact points with LCP constraints, which is beneficial when the contacts need to be carefully modelled.

Elastic body simulation focuses on the formulation of an elastic deformation energy and the proper handling of contact constraints to simulate realistic deformations of soft bodies [65], [66], [67], [68]. A comprehensive survey of physics-based elastic deformation models can be found in [69].

Space-time optimization techniques can also be applied to control the motion of elastic bodies that are represented by volumetric meshes. To reduce the number of variables used to control the vertex positions in the optimization, model reduction techniques are frequently used [70], [71], [72]. Barbič et al. [70] imposed the equation of motion constraint in elastic body deformation, using the discrete adjoint method to compute the gradients of control forces. Pan et al. [73] integrated the contact forces as additional variables to handle environment interactions and solved the space-time objective with alternating optimization, but did not handle the two-way coupling problem we want to solve.

SECTION 3 Overview

Given an input surface mesh of a quad-robot, we first design mechanical skeleton and skin mesh (Section 6.1) for the robot, and then use the proposed two-step alternating algorithm to optimize for a physically plausible motion plan (Section 5). The overall system flowchart is illustrated in Fig. 1.

Fig. 1.

We propose a computational fabrication system for designing and fabricating skinned quad-robots. Given an input mesh representing the shape of a quad-robot such as this beetle-like robot, we design its mechanical skeleton with motors to drive its locomotion. The motion plan is generated by an optimization algorithm with kinematic trajectories as the user input. The trajectories consist of the foot swing trajectories (red curves), center of mass (COM) trajectory (blue curve) and foot contact plan (yellow bars) input by the user. Our optimization fully takes account of all the physical and dynamical constraints. By fabricating this robot design, it can be verified that our algorithm is able to generate plausible and physically feasible motion plans for quad-robots, and the simulated results well match the physical experimental results. Note that we only render the transparent input surface without thickness to demonstrate the surface-structure coupling geometry.

During the alternating optimization, the first space-time optimization step outputs joint angle trajectories for the design according to the user-specified end-effector and COM trajectories (Section 5.1). This step is made possible by only considering the approximated skin deformation. The second frame-by-frame optimization step improves the physical plausibility of the joint angle trajectories with full simulation and various physical constraints(Section 5.2), such as physical torque limits for the selected motors in the design. Two-way coupled multibody-elastic dynamics (Section 4) are adopted in this step for the full simulation.

Finally, the designed robot is fabricated by fast prototyping methods for a physical validation (Section 6.2), and stepper motors are mounted to drive the skeleton and the attached soft skin to realize the motion plan.

SECTION 4 Two-way Coupled Multibody-Elastic Dynamics

A core ingredient of our system is the dynamic simulation of the robot using a self-actuated rigid skeleton with a soft skin attached. We are inspired by existing coupled simulation systems in graphics [74], [75], [76] and exploit the Lagrange multipliers method to enforce the two-way coupling between the skeleton and the soft skin, which can be naturally integrated into the subsequent locomotion optimization. The Lagrange multipliers are used to guarantee that the skeleton and skin are attached to each other at prescribed locations, and we solve for all the unknown DOFs from both subsystems simultaneously.

4.1 Two-Way Skeleton-Skin Coupling

We use Lagrangian mechanics for both the rigid skeleton and soft skin and obtain a symmetric formulation for these two subsystems. To ease the formulation of motion contraints, we use the generalized coordinates $\mathbf {q}$ to parameterize the entire skeleton, where $\mathbf {q} = \lbrace c_x, c_y,c_z,q_1, \ldots, q_m\rbrace$. The vector $\mathbf {q}$ is composed of the Cartesian coordinates of the center of mass (COM) at the root link $\lbrace c_x,c_y,c_z\rbrace$ and the three Euler angles at each joint $\lbrace q_1,\ldots,q_m\rbrace$. We opt to model the soft skin using the neo-Hookean material model [77] because it has been demonstrated to be well suited for robotic skins made out of silicone [78] and can handle large local deformations induced by joint rotation observed in our examples. If necessary, however, our approach should be easily extensible to more sophisticated material models such as Mooney-Rivlin and Ogden. The DOFs representing the vertex displacements of the tetrahedral skin mesh are denoted by the vector $\mathbf {u}$. Though numerical discretization, the equations of the forward simulation of rigid skeleton and soft skin at each time step can be written into a linear system $\mathbf {A}x = \mathbf {b}$, where $\mathbf {A}$ is the system matrix and $\mathbf {b}$ is a vector of the sum of constant terms and external forces. The detailed derivation of two linear systems, $\mathbf {A}_r$, $\mathbf {b}_r$ for rigid skeleton and $\mathbf {A}_d$, $\mathbf {b}_d$ for soft skin, are elaborated in Appendix A, which can be found on the Computer Society Digital Library at http://doi.ieeecomputersociety.org/10.1109/TVCG.2019.2957218.

The two subsystems are coupled by attaching the soft skin to the underlying skeleton at prescribed glue vertices. Mathematically, this straightforward treatment leads to a set of nonlinear position constraints: \begin{equation*} \mathscr {C}(\mathbf {q},\mathbf {u})=\mathscr {R}(\mathbf {q})\mathbf {r}+\mathbf {t}-\mathbf {x}_c=0. \tag{1} \end{equation*} The matrix $\mathcal{R}$ converts the positions on the rigid links where the skin is attached, $\mathbf {r}$, from local to world coordinates. This operation can be easily expressed as a rotation chain from the links all the way back to the root. Meanwhile, $\mathbf {t}$ concatenates the root translations of the rigid links, and $\mathbf {x}_c$ denotes the positions of the glue vertices on the skin mesh $\mathbf {x}_c = \mathbf {S}_c \mathbf {u}$, where $\mathbf {S}_c$ is a selection matrix.

Using the Lagrange multipliers method, we obtain the coupled multibody-elastic system: \begin{equation*} \begin{bmatrix}\mathbf {A}_r & \mathbf {0} & \nabla _q\mathscr {C}^\top \\ \mathbf {0} & \mathbf {A}_d & \nabla _u\mathscr {C}^\top \\ \nabla _q\mathscr {C} & \nabla _u\mathscr {C} & \mathbf {0} \end{bmatrix} \begin{bmatrix}\Delta \mathbf {q}\\ \Delta \mathbf {u}\\ \pmb {\lambda }\end{bmatrix} = \begin{bmatrix}\mathbf {b}_r\\ \mathbf {b}_d\\ \mathbf {0} \end{bmatrix}. \tag{2} \end{equation*} The coupling constraint is linearized via $\nabla \mathscr {C}$. In each time step, we solve for the changes of the system DOFs in Eq. (2). Thus, we have $\mathbf {q}^{i} = \mathbf {q}^{i-1} + \Delta \mathbf {q}$ and $\mathbf {u}^{i} = \mathbf {u}^{i-1} + \Delta \mathbf {u}$, where the superscript $[\cdot ]^{i}$ indicates the frame index.

4.2 Collision and Contact Handling

There are two types of collisions/contacts we need to take care of in our simulation. The first is the collision between the robot’s feet and the ground surface, which provides necessary support and friction forces to the robot. As will be detailed in Section 5.2, we resolve them using linear complementary constraints (LCP) to guarantee the physical feasibility of the locomotion.

The second is the self-collision of the soft skin; rotating joints tend to compress the inward skin and induce self-collisions. Moreover, skin-skeleton collisions¹ could also occur when the skeleton is being articulated. Because such collisions typically take place under low relative velocities, we handle them using the explicit penalty force method [79], [80].

SECTION 5 Motion Plan Optimization

As the system input, the trajectories of the COM, end effectors, and the footfall pattern are provided by the user. Our system generates a dynamically feasible motion plan for the robot that resembles as much as possible the one prescribed by the user. A motion plan, defined as $\mathcal {P}=\lbrace \mathbf {q}^i, i=1...N; \Delta t \rbrace$, includes the skeleton configuration $\mathbf {q}^i$ of the $i$th frame for $i=1$ to $N$, and the time step size $\Delta t$.

Previous works, e.g., [Megaro et al. 15], solved the motion design problem by simultaneously finding the optimal skeleton configuration for all the frames in $\mathcal {P}$. However, this problem becomes much more challenging in the case of a multibody-elastic system, due to the large number of DoFs and associated physical constraints. Thus, the global approach with full DOFs is infeasible in our case. In this section, we elaborate the details of our two-step alternating motion plan optimization algorithm which uses approximated skin deformation to significantly improve the efficiency of the global space-time optimization. Specifically, the influence of the skin deformation on the skeleton is approximated as the coupling forces at glue vertices and the influenced COM position of each link due to the skin deformation. The convergence of the algorithm is tested considering the difference between the space-time optimized one and the optimized one after the frame-by-frame optimization step. Fig. 2 illustrates the algorithm flowchart.

Fig. 2.

Alternating algorithm flowchart.

5.1 Space-Time Optimization With Skin Deformation Approximation

Similar to the formulation in [36], for each $i$th frame, we consider the set of generalized coordinates of the rigid skeleton, $\mathbf {q}^i$, together with the contact forces $\mathcal {F}^i_{c,j}$ and torques $\pmb{\tau }^i_c,j$ that are exerted on the $j$th end effector in contact with the ground. The influence of the skin deformation to the skeleton at $i$th frame is simplified to be the coupling forces $\pmb {\lambda }^i$ at glue vertices and the skin deformation $\mathbf {u}^i$. They are obtained from the previous frame-by-frame optimization and not optimized in this step. The displacements in $\mathbf {u}^i$ are represented into the local coordinate frames of links at each frame in order to compute the influence of the skin deformation to the COM of the robot.

Given these quantities, the optimization objective is defined as the weighted sum of six terms that balance the user-specified end effector and COM trajectories and the smoothness of the optimized motion: \begin{equation*} E_A = \min \sum _i\left(\alpha _tE_{\pmb {\tau }}^i+\alpha _sE_S^i+\alpha _cE_{COM}^i+\alpha _eE_{EE}^i+\alpha _fE_F^i+\alpha _oE_O^i\right).\ \tag{3} \end{equation*} The first term $E_{\pmb {\tau }}^i$ is standard in space-time optimization to minimize the torques $\pmb {\tau }^i$ exerted at the joints: \begin{equation*} \displaystyle E_{\pmb {\tau }}^i = {\frac{1}{m}}^2\left\Vert \pmb {\tau }^i(\mathbf {q}^i, \pmb {\lambda }^i, \mathcal {F}^i_c, \pmb {\tau }^i_c)\right\Vert ^2. \end{equation*} The torques $\pmb {\tau }^i$ are computed using inverse dynamics, and the coupling forces $\pmb {\lambda }^i$ are integrated as the external forces exerted by the elastic skin at the coupling points.

The second term $E_{S}^i$ encourages the smoothness of the optimized motion, which is: \begin{equation*} E_{S}^i = \left\Vert \mathbf {q}^{i+1} - 2\mathbf {q}^{i}+\mathbf {q}^{i-1}\right\Vert ^2. \end{equation*}

The two terms, $E_{EE}^i$ and $E_{COM}^i$, enforce the optimized motion to follow the user-specified end-effector and COM trajectories respectively: \begin{equation*} \displaystyle E_{EE}^i = \left\Vert \phi _{EE}^i(\mathbf {q}^i,\mathbf {u}^i) - \mathbf {e}^i\right\Vert ^2,\quad \displaystyle E_{COM}^i = \left\Vert \phi ^i_{COM}(\mathbf {q}^i,\mathbf {u}^i) - \mathbf {g}_i\right\Vert ^2, \end{equation*} where the functions $\phi _{EE}^i(\mathbf {q}^i,\mathbf {u}^i)$ and $\phi ^i_{COM}(\mathbf {q}^i,\mathbf {u}^i)$ define how to compute the end effector and COM positions, given the generalized coordinates of the skeleton and the deformation of the skin mesh. For each end effector, we select one vertex closest to the end effector of the rigid skeleton and represent this vertex into the local coordinate system of the end effector to compute $\phi _{EE}$ (see Fig. 3).

Fig. 3.

Foot contacts.

The term $E_F^i$ penalizes the deviation from the motion $\lbrace \tilde{\mathbf {q}}^i, i=1,..,N\rbrace$ generated in the previous frame-by-frame optimization or the initialized motion plan at the first iteration: \begin{equation*} E_F^i = \left\Vert \mathbf {q}^i - \tilde{\mathbf {q}}^i\right\Vert ^2. \end{equation*} After the frame-by-frame optimization step, the weight of the $E_F$ term to follow the motion plan of the previous iteration in the space-time optimization is increased by 10 percent with respect to its value in the previous iteration, which means the algorithm leans toward physical plausibility.

The last term $E_O$ enforces that the end-effectors in the contact are flat: \begin{equation*} \displaystyle E_{O}^i = \left\Vert \psi ^i_{EE}(\mathbf {q}^i) - \hat{\mathbf {n}}\right\Vert ^2, \end{equation*} where $\psi ^i_{EE}(\mathbf {q}^i)$ is a function to compute the orientation of the end effector and $\hat{\mathbf {n}}$ is set to be (0,1,0), the normal of the support plane. The formulation of this term is motivated by the fact our skinned robot is soft and hence a contact area appears whenever feet contact the ground. We try to maximize the contact area at the moment of contact because it is important to achieve a stable motion. Similarly, when the foot is about to hover, we want to clear as many contact vertices as possible. Therefore, when an end effector is close to these important moments, for example, the foot is on the ground, 5 frames before it hits the ground or after it starts to leave the ground, we add $E_{O}$ to the objective function, which try to make the end effector face the upright direction of the ground.

We also impose hard kinematic and dynamic constraints on the optimized variables to obtain a stable motion plan. The kinematic constraints include the contact constraint, the center of pressure (COP) constraint and the optional periodic constraint, while the dynamic constraints include the momentum and the friction force constraints as in [36].

Contact Constraint. In the optimization, we need to enforce the footfall pattern that is specified by the user to encode when the foot should leave or touch the ground. This constraint can be written into: \begin{align*} & \displaystyle c^i_j{\phi _{EE,j}^i(\mathbf {q}^i,\mathbf {u}^i)}_y=0,\qquad \qquad \qquad \qquad \qquad \qquad \qquad \forall i,j, \\ & \displaystyle c^{i-1}_jc^i_j(\phi _{EE,j}^{i}(\mathbf {q}^i,\mathbf {u}^i) - \phi _{EE,j}^{i-1}(\mathbf {q}^i,\mathbf {u}^i)) = 0,\qquad \qquad \qquad \forall i,j, \tag{4} \end{align*} where $c^i_j$ is a binary variable set to 1 if the $j$th end effector is in contact with the ground at the $i$th frame. This variable can be directly derived from the footfall pattern. The $y$ coordinate of the end effector, denoted by ${\phi _{EE,j}^i(\mathbf {q}^i,\mathbf {u}^i)}_y$, should be 0 since the ground is set to be $y=0$.

COP Constraint. The COP should be inside the supporting polygon for a stable motion plan. This constraint can be written into: \begin{equation*} \displaystyle \mathbf {P}\cdot \phi ^i_{COP} \leq \mathbf {0}, \qquad \forall i, \tag{5} \end{equation*}

where the funtion $\phi ^i_{COP}$ computes the COP position using the same method as in [3]. The rows in $\mathbf {P}$ represent edges of the supporting polygon. Since the space-time optimization requires the foot to be flat on the supporting plane, the supporting polygon is formed by the convex hull of the vertices that represent the sole meshes of the end-effectors in contact. However, the contact forces and torques at a sole are simplified to be exerted on a single point of the end effector to ease the optimization. Fig. 3 illustrates the contact points (red balls) selected as the positions of the end-effect for the beetle-like robot.

Periodic Constraint. When the user desires a periodic motion, the joint angles are expected to be the same in the first and the last frame of the optimized motion plan: \begin{equation*} J(q^1) = J(q^N), \tag{6} \end{equation*} where $J$ extracts the joint angles from the generalized coordinates.

Momentum Constraint. The change of the linear and angular momentum of the robot should be determined by the external contact forces and torques, which can be formulated into the following equations: \begin{align*} & \displaystyle \dot{\mathbf {R}}^i = mg + \sum _jc^i_j\mathcal {F}^i_{c,j}, \qquad \qquad \qquad \qquad \qquad \qquad \;\;\quad \forall i,j, \\ & \displaystyle \dot{\mathbf {L}}^i = \sum _j c^i_j\left(\left(\mathbf {p}_{c,j}^i - \phi ^i_{COM}(\mathbf {q}^i,\mathbf {u}^i)\right)\times \mathcal {F}^i_{c,j}+ \pmb {\tau }_{c,j}^i\right), \qquad \forall i,j, \tag{7} \end{align*} where $\mathbf {R}^i$ and $\mathbf {L}^i$ are, respectively, the total linear and angular momentum of the robot at $i$th frame, and $\mathbf {p}_{c,j}^i$ gives the contact position of the $j$th end effector.

Friction Force Constraint. The contact force should be inside the friction cone to satisfy the Coulomb model of friction. Specifically, we have: \begin{align*} & \displaystyle \frac{1}{m}(\mu {\mathcal {F}^i_{c,j}}_\perp - \Vert {\mathcal {F}^i_{c,j}}_{\Vert }\Vert) \geq 0, \qquad \qquad \qquad \qquad \forall c^i_j=1, \\ & \displaystyle \frac{1}{m}^2(\Vert {\mathcal {F}^i_{c,j}}\Vert ^2 - \Vert \frac{{\pmb {\tau }^i_j}_{\Vert }}{\nu _b}\Vert ^2 - \frac{{\pmb {\tau }^i_j}_{\perp }}{\nu _t}^2) \geq 0, \qquad \qquad \quad \forall c^i_j=1, \tag{8} \end{align*} where ${\mathcal {F}^i_{c,j}}_\perp$ and ${\mathcal {F}^i_{c,j}}_\Vert$ represent the components of the contact force perpendicular and parallel to the ground respectively. The first equation requires the contact force to be inside a friction cone. The second equation relates the contact force and the contact torque, where $\nu _b$ and $\nu _t$ are set to be the radius of the circumcircle of the sole mesh.

Optimization. With the defined objective function and constraints, the space-time optimization step can be written into: \begin{align*} & \min _{\mathbf {q}^i,\mathcal {F}^i_{c,j}, \pmb {\tau }^i_c,j, i=1,..N} \qquad E_A \\ & \text{st.}: \ \boldsymbol{\phi}_e = 0, \boldsymbol{\phi}_g \leq 0, \tag{9} \end{align*} where $\boldsymbol{\phi}_e$ and $\boldsymbol{\phi}_g$ represent the equality and inequality constraints respectively. We solve this sequential quadratic programming problem using the Gauss-Newton algorithm. The weights in the objective function are specified as follows: $\alpha _t = 1e-2, \alpha _s = 0.5, \alpha _c = \alpha _e = 1$, $\alpha _f = 1$ and $\alpha _o = 10$.

5.2 Frame-by-Frame Optimization With Full Dynamics

In the frame-by-frame optimization, we drop the simplification of the previous step and consider the full dynamics formulated in Eq. (2) in a similar way as in [7]. This allows us to further improve the physical plausibility of the motion. In practice, this requires the solution of a difficult quadratic programming problem with complementarity constraints (QPCC) to handle contact and friction. Different to the soft body only formulation in [7], the coupling constraints between multi-body skeletion and elastic skin introduce a large number of Lagrange multiplier variables and largely increase the complexity of the solver. To speed up the solution, we follow the condensation technique widely used in physical simulation [81], [82]. Specifically, we select the driving torques at joints and foot contact forces as optimization variables and lump the DOFs of mesh vertices and rigid skeletons to these variables through the condensation of the system matrix in Eq. (2). This choice facilitates the formulation of the physical torque limit constraints of motors as well. In this section, we describe the details of the matrix condensation and the per-frame optimization problem.

System Matrix Condensation. With the coupled multibody-elastic system defined in Eq. (2), the nonlinear relation between the simulated DOFs $\Delta \mathbf {u}$ and $\Delta \mathbf {q}$ and the vector $\mathbf {b}_r$ and $\mathbf {b}_d$ can be revealed by eliminating the unknown Lagrange multipliers $\pmb {\lambda }$ in the matrix condensation (see Appendix B, available in the online supplemental material, for the detailed derivation): \begin{align*} \Delta \mathbf {u}=\mathbf {A}_d^{-1}\mathbf {b}_d-\mathbf {A}_d^{-1}\nabla _u\mathscr{C}^\top \mathbf {A}_C^{-1}\left(\nabla _q\mathscr {C}\mathbf {A}_r^{-1}\mathbf {b}_r+\nabla _u\mathscr {C}\mathbf {A}_d^{-1}\mathbf {b}_d\right), \\ \Delta \mathbf {q}=\mathbf {A}_r^{-1}\mathbf {b}_r-\mathbf {A}_r^{-1}\nabla _q\mathscr {C}^\top \mathbf {A}_C^{-1}\left(\nabla _q\mathscr {C}\mathbf {A}_r^{-1}\mathbf {b}_r+\nabla _u\mathscr {C}\mathbf {A}_d^{-1}\mathbf {b}_d\right), \tag{10} \end{align*} where $\mathbf {A}_C=\nabla _q\mathscr {C}\mathbf {A}_r^{-1}\nabla _q\mathscr {C}^\top +\nabla _u\mathscr {C}\mathbf {A}_d^{-1}\nabla _u\mathscr {C}^\top$.

According to the formulation of $\mathbf {b}_r$ and $\mathbf {b}_d$ in Appendix A, available in the online supplemental material, the joint torques and the contact forces exerted on the soft skin are absorbed into the vector $\mathbf {g}_r$ and $\mathbf {g}_d$. Since the rest terms in $\mathbf {b}_r$ and $\mathbf {b}_d$ are constant, we can use $\phi _{\Delta \mathbf {q}}$ and $\phi _{\Delta \mathbf {u}}$ to represent Eq. (10) more precisely: \begin{equation*} \Delta \mathbf {q}=\phi _{\Delta \mathbf {q}}\big (\pmb {\tau },\mathcal {F}_{\perp },\mathcal {F}_{\Vert }\big),\quad \text{and}\quad \Delta \mathbf {u}=\phi _{\Delta \mathbf {u}}\big (\pmb {\tau },\mathcal {F}_{\perp },\mathcal {F}_{\Vert }\big). \tag{11} \end{equation*} Here, $\mathcal {F}_{\perp }$ and $\mathcal {F}_{\Vert }$ are magnitudes of normal and tangent forces at all the contact vertices on the skin mesh, and $\pmb {\tau }$ represnets the join torques. To handle LCP constraints, the contact force at a contact vertex is modeled as $\mathcal {F}=\hat{\mathbf {n}}\mathcal {F}_{\perp }+\mathbf {D}\mathcal {F}_{\Vert }$ instead of the 3D vector representation in Section 5.1, where $\hat{\mathbf {n}}\mathcal {F}_{\perp }$ represents the upright supporting force along the contact normal $\hat{\mathbf {n}}=[0,1,0]^\top$ and $\mathcal {F}_{\perp }\in \mathbb {R}$ is the force magnitude. $\mathbf {D}\in \mathbb {R}^{3\times 4}$ is a matrix and its columns are the vectors that span the contact plane. In our system, we use four directions to form the friction cone [83] and $\mathcal {F}_{\Vert }\in \mathbb {R}^4$ is the tangent magnitude. The explicit penalty forces for resolving the collision between the skeleton and skin are pre-determined quantities and merged into the constant terms in $\mathbf {b}_r$ and $\mathbf {b}_d$.

Given $\mathbf {q}^{i-1}$ and $\mathbf {u}^{i-1}$, $\Delta \mathbf {q}^i$ and $\Delta \mathbf {u}^i$ determine the positions and orientations of COM, COP, and end effectors at the $i$th frame. Thanks to Eq. (11), these kinematics variables are now functions of $\pmb {\tau }$, $\mathcal {F}_{\perp }$, and $\mathcal {F}_{\Vert }$. Therefore, we can derive the functions required in the computation of kinematic information, namely $\phi ^i_{EE}, \psi ^i_{EE}, \phi ^i_{COM}$ and $\phi ^i_{COP}$, if external forces $\pmb {\tau }^i$, $\mathcal {F}_{\perp }^i$, and $\mathcal {F}_{\Vert }^i$ are given. Note that the purpose of these functions are explained in the space-time optimization (see Section 5.1).

Optimization. We follow the control strategy used in [7] to optimize the input motion plan on a frame-by-frame basis. It is used to make sure that the output joint angle trajectories of the space-time optimization step are physically feasible, which is achieved using the two-way coupled multibody-elastic dynamics as constraints. At each frame, it can be formulated as a quadratic programming problem with complementarity constraints.

Specifically, we seek for joint torques ($\pmb {\tau }^i$) and contact forces ($\mathcal {F}_{\perp }^i$, $\mathcal {F}_{\Vert }^i$) such that the corresponding $\Delta \mathbf {q}^i=\phi ^i_{\Delta \mathbf {q}}$ and $\Delta \mathbf {u}^i=\phi ^i_{\Delta \mathbf {u}}$ satisfy necessary hard constraints and the resulting locomotion matches the input locomotion as much as possible. Mathematically, it is formulated as \begin{align*} & \displaystyle \min _{\pmb {\tau }^i,\mathcal {F}_{\perp }^i,\mathcal {F}_{\Vert }^i,\pmb {\lambda }_{\Vert }^i} E_G\left(\pmb {\tau }^i,\mathcal {F}^i_{\perp },\mathcal {F}^i_{\Vert }\right) & \\ & \displaystyle \text{subject to:} \\ & \displaystyle \Vert \pmb {\tau }^i_m\Vert < U_m, m = 1,\ldots,M \\ & \displaystyle \mathbf {P}\cdot \phi ^i_{COP}(\pmb {\tau }^i,\mathcal {F}_{\perp }^i,\mathcal {F}_{\Vert }^i) \leq \mathbf {0} \\ & \displaystyle \mathbf {0} \leq \begin{bmatrix}\mathcal {F}^i_{\perp } \\ \mathcal {F}^i_{\Vert } \\ \lambda ^i_{\Vert } \end{bmatrix} \perp \begin{bmatrix}\displaystyle \hat{\mathbf {n}}^\top \frac{\Delta \mathbf {u}_c^i}{\Delta t} \\ \displaystyle \mathbf {D}^{\top }\frac{\Delta \mathbf {u}_c^i}{\Delta t} + \displaystyle \mathbf {1}\lambda _{\Vert }\\ \mu \mathcal {F}^i_{\perp } - \mathbf {1}^{\top }\mathcal {F}^i_{\Vert } \end{bmatrix} \geq \mathbf {0}. \tag{12} \end{align*}

In Eq. (12), the first hard inequality constraint of $\Vert \pmb {\tau }^i_m\Vert < U_m$ is the motor constraint requiring for all the $M$ motors that the computed torque magnitude is within its physical limit $U_m$. The second inequality constraint $\mathbf {P}\cdot \phi _{COP} \leq \mathbf {0}$ requires the position of the COP to be within the supporting polygon as in Section 5.1. The last complementary constraint is enforced at each individual contact vertex. It characterizes the contact mechanism such that when normal force exists, the relative velocity between the ground and the contact vertex along the contact normal should be zero, etc. Here, $\Delta \mathbf {u}^i_c\in \mathbb {R}^3$ is the incremental displacement of a contact vertex. The auxiliary vector $\lambda _{\Vert }$ is related to the tangent velocity of a sliding contact; $\mu$ is the friction coefficient; and $\mathbf {1}$ is a vector of ones, that is, $\mathbf {1}=[1,1,1,1]^\top$.

The objective function $E_G$ has four terms: \begin{equation*} E_G= \alpha _S E_S + \alpha _F E_F + \alpha _{O} E_{O} + \alpha _{\pmb {\tau }}E_{\pmb {\tau }} +\alpha _C E_C, \tag{13} \end{equation*} where: \begin{align*} & \displaystyle E_S = \left\Vert \phi ^i_{\Delta \mathbf {q}} - \phi ^{i-1}_{\Delta \mathbf {q}}\right\Vert ^2, \qquad \qquad \displaystyle E_F = \left\Vert \mathbf {q}^{i} + \phi ^i_{\Delta \mathbf {q}} - \bar{\mathbf {q}}^{i+1}\right\Vert ^2, \\ & \displaystyle E_{O} = \left\Vert \psi ^i_{EE} - \hat{\mathbf {n}}\right\Vert ^2, \qquad \qquad \displaystyle E_{\pmb {\tau }} = \left\Vert \pmb {\tau }_i - \pmb {\tau }_{i-1}\right|^2 \\ & \displaystyle E_{C} = \left\Vert \mathcal{E}\left(\phi _{EE}^i - \phi _{EE}^{i-1}\right)\right\Vert ^2. \tag{14} \end{align*}

The first energy term $E_S$ is the smoothness penalty, which favors motions with consistent velocities. The term $E_F^i$ penalizes the deviation from the motion $\lbrace \bar{\mathbf {q}}^i, i=1,..,N\rbrace$ generated in the previous space-time optimization. $E_{O}$ is the same soft constraint on the orientation of an end effector as in Eq. (3), which can maximize the contact area for a stable motion. $E_{\pmb {\tau }}$ is used to penalize the large variation of the control torques at joints between frames. The last term $E_{C}$ imposes a penalty to moving end effectors who are responsible for supporting feet. In other words, if a foot is in contact with the ground and supporting the body, we use $E_{C}$ to reduce the risk of its possible tangent sliding. Here, $\mathcal {E}$ is an elementary matrix that picks positions of supporting end effectors. The weighting constants for each of these penalty terms are as follows: $\alpha _{S}=1$, $\alpha _{F}=10$, $\alpha _{O}=2$, $\alpha _{\pmb {\tau }} = 0.5$ and $\alpha _{C}=10$.

The contact forces are applied to the skin then affect the skeletal motion through the coupling. However, for the foot contacts between the robot and the ground, the contact forces are applied to the foot link using the soft contact method proposed in [55] for the fast response to the contacts and stable balance control. The computed contact forces applied at foot links are kept the same in the space-time optimization with a rigid-skeleton only.

5.3 Solving the QPCC

The key to solving the QPCC problem of Eq. (12) is to have a feasible configuration for all the contact vertices. Our strategy is similar to that of [7]: We flip complementary constraints when the inequality constraint reaches the boundary. Specifically, contact vertices fall into one of the three following categories:

• Contact breakage means that the contact vertex will leave the ground plane in the next frame, and the complementary constraints should be lifted.

• Sliding indicates that the contact vertex is moving within the ground plane. In this situation, the complementary constraint for its normal force $\mathcal {F}_{\perp }$ becomes: \begin{equation*} \mathcal {F}_{\perp } > 0,\quad \hat{\mathbf {n}}^\top \frac{\Delta \mathbf {u}_c^i}{\Delta t} = 0, \tag{15} \end{equation*} and the complementary constraints for the tangent force $\mathcal {F}_\parallel$ are: \begin{align*} \displaystyle \mathcal {F}_\parallel \geq \mathbf {0}, & \displaystyle \mathbf {D}^{\top }\frac{\Delta \mathbf {u}_c^i}{\Delta t} + \mathbf {1}\lambda _{\Vert } = \mathbf {0}; \\ \displaystyle \lambda _{\Vert } \geq 0, & \displaystyle \mu \mathcal {F}_{\perp } - \mathbf {1}^{\top }\mathcal {F}_{\Vert } = 0. \tag{16} \end{align*}

• Static friction implies that the contact vertex is fixed on the contact plane. In this case, the complementary constraint for its normal force is the same as Eq. (15). The constraints for the tangent force are: \begin{align*} \displaystyle \mathcal {F}_\parallel \geq \mathbf {0}, & \displaystyle \mathbf {D}^{\top }\frac{\Delta \mathbf {u}_c^i}{\Delta t} + \mathbf {1}\lambda _{\Vert } = \mathbf {0}; \\ \displaystyle \lambda _{\Vert } = 0, & \displaystyle \mu \mathcal {F}_{\perp } - \mathbf {1}^{\top }\mathcal {F}_{\Vert } \geq 0. \tag{17} \end{align*} The inequality constraint of $\mu \mathcal {F}_{\perp } - \mathbf {1}^{\top }\mathcal {F}_{\Vert } \geq 0$ specifies the friction cone constraint in the case of static friction. d

We begin solving Eq. (12) by assuming all the contact vertices are fixed, which simplifies the original problem to \begin{align*} & \displaystyle \min _{\pmb {\tau }^i,\pmb {\gamma }_c} E_G\left(\pmb {\tau }^i,\pmb {\gamma }_c\right) & \\ & \displaystyle \text{subject to:} \\ & \displaystyle \Vert \pmb {\tau }^i_m\Vert < U_m, m = 1,\ldots,M \\ & \displaystyle \mathbf {P}\cdot \phi ^i_{COP} \leq \mathbf {0} \\ & \displaystyle \left[\phi ^i_{\Delta \mathbf {u}}\right]_c=\mathbf {0}, \tag{18} \end{align*} where $\left[\phi ^i_{\Delta \mathbf {u}}\right]_c$ returns the incremental displacements of all the contact vertices. This assumption of fixing all the contact vertices is realized via the Lagrange multipliers method, and the resulting multipliers $\pmb {\gamma }_c$ correspond to the constraint forces at these vertices. Now, let $\pmb {\gamma }_c\in \mathbb {R}^3$ be the constraint force at one of the contact vertices. It can be decomposed along normal and tangent directions as: \begin{equation*} \gamma _{\perp } = \hat{\mathbf {n}}^\top \pmb {\gamma }_c,\quad \text{and}\quad \pmb {\gamma }_{\Vert } = \big (\mathbf {I}-\hat{\mathbf {n}}\hat{\mathbf {n}}^\top \big)\pmb {\gamma }_c. \tag{19} \end{equation*} We label all the contact vertices as contact breakage, static friction, or sliding by checking $\gamma _{\perp }$ and $\pmb {\gamma }_{\Vert }$. If $\gamma _{\perp }\leq 0$, which indicates a contact breakage, we remove the constraint at the vertex in the next iteration. If $\gamma _\perp > 0$, we further examine the magnitudes of $\mu \gamma _{\perp }$ and $\Vert \pmb {\gamma }_{\Vert }\Vert$. If $\mu \gamma _{\perp }>\Vert \pmb {\gamma }_{\Vert }\Vert$, the vertex falls into the static friction category, otherwise the vertex is considered sliding. After all the contact vertices are labelled, we can convert the complementary constraints into a set of equality or inequality constraints, as explained in Eqs. (15), (16), and (17), and re-solve the QP optimization. It is known that QPCC is NP-complete, and few contact vertices could make the optimization procedure computationally intractable. Therefore, we simplify this procedure by grouping vertices on the planar surface of the foot mesh into five patches similar to [7]. In our experiments, we found that such initial vertex grouping often provide a good start for the QPCC solver. We observe that the condensed QPCC solver is around 50x faster than the QPCC without condensation.

Typical converging curves are plotted in Fig. 4, and we stop the optimization after 10 iterations. The convergence of the QPCC solver means that it finds the contact category of each contact point that leads to minimal energy. It tries to switch the contact category of a contact point in the direction of lower energy, but occasional increased energy can still be observed.

Fig. 4.

Our QPCC solver converges quickly in most cases. The left plot is the converging curve of a frame when the front left leg of the monster-like robot leaves the ground. The middle plot is the converging curve of a frame when this leg is in the air (i.e., other three feet are on the ground). The right plot is the converging curve of a frame when this leg hits ground again.

5.4 Initialization

Given the mechanical skeleton and the skin mesh of a robot, we first associate the mesh vertices to the links of the skeleton to obtain its skinning information. Hence, the mesh vertices can be deformed with the skeleton in the space-time optimization step, while the local coordinates of the vertices should be computed using their deformed positions and the local frame of the links at each frame. The initial motion plan are computed through the space-time optimization step without the trajectory following terms $E_F^i$ . In this step, the skin deformation is assumed to be static and each link has additional weights from its associated vertices. Afterwards, the initial skin mesh deformation is simulated by imposing the coupling constraints in the elastic simulation of the skin, and the initial coupling force is then obtained according to the deformation of the tetrahedra connected to the coupling points [56].

SECTION 6 Design and Fabrication

Designing and fabricating a quad-robot is a challenging task. We facilitate the design by using a set of mechanical skeleton templates, and narrow the gap between professional and regular users by creating several SolidWorks scripts. This allows even an inexperienced user to tweak high-level semantic parameters. Fig. 6 shows three built-in mechanical skeleton templates provided in our system for quad-robots. Each template is built of modularized CAD parts to ease the fabrication cost. The first one is the design used in the beetle-like robot, which consists of a torso structure and four limb structures. Their exploded views are detailed in the figure as well. The torso structure has four shoulder joints that connect to its four limbs. A microcontroller board sending trigger signals to the motors is mounted inside of the torso. The limb structure includes linkage parts of an upper leg, a lower leg, and a foot. On each limb, two uniaxial motors are mounted to provide necessary rotational freedoms at the knee and the ankle. The other two templates vary in different initial poses and foot link geometries.

In the following, we describe the details of the design pipeline and the fabrication procedure respectively.

6.1 Design and Editing of Mechanical Skeleton

The design starts with a given 3D model that corresponds to the appearance of the robot. Our system extracts an initial skeletal line using the mesh contraction method [84] as shown in Fig. 5. This skeleton is actually an approximation of the medial axis of the model, and it is used as a general guide for the follow-up template embedding and editing. We employed the modular design idea so that the user can edit the geometry of a template mechanical component to obtain a customized mechanical skeleton for quad-robots of various morphologies. To this end, several SolidWorks scripts are developed to assign semantic parameters of a link, such as the link length, motor mount size, etc., and the user only needs to tweak these intuitive parameters to obtain a personalized design without creating one from scratch. The size of pilot holes on the link for screw installation remains unchanged under such edits. An example is given in Fig. 7, where the lengths of the link and the motor bay are increased. Although a few iterations may be necessary during embedding, the developed SolidWorks scripts greatly accelerate the procedure.

Fig. 5.

The initial skeletal line of the beetle-like robot.

Typically, given a new surface model of a quad-robot, we embed limbs first and then adjust the geometry of the torso to make sure it fits the exterior skin. Specifically, a global scale of the mechanical structure template and local rotations of the links are first performed so that the template can be inside the input surface mesh. Then, the user can select the start and end points of a link on the extracted skeletal line and trigger the designed script to edit the link geometry to match the specified length and adjust the width of the link. Finally, the bracing unit of the torso is generated in a similar way as the skin creation(the details follows shortly), and we dig out holes to reduce its weight, for instance, the bracing unit for the torso of the beetle-like robot shown in the first picture in the second row of Fig. 1.

Skin and Folding Regions Creation. The exterior skin of the robot is designed to be 8 mm thick at the foot and 4 mm thick at other parts by default, and it is created by the mesh hollow operation in Materialize Magics. This operation treats the space surrounded by the input 3D surface model as a solid and hollows the interior space of the solid to match the specified thickness parameters to create the skin that is amenable to fabrication. When it is being bent, the skin can yield rather large resisting forces under stretching deformations. Regular commercial motors may not possess sufficient power to overwhelm the internal stretching. To resolve this practical issue, we add a few folding regions on the original skin mesh, as shown in Fig. 8. The folding region is created by applying a sweeping cut operation in Solidworks over the original skin surface where the motor is installed. This small treatment increases the skin area where substantial bending occurs and effectively reduces the resulting stretching force. Note that the hollow operation is performed after the creation of the folding regions. Fortunately, the two software are compatible in mesh file format.

6.2 Fabrication

The mechanical structure of the robot is 3D printed with polypropylene-like stereolithography (SLA) resin, which is a widely used material for fabricating joints and low-friction moving parts. The exterior skin of the robot is made of a layer of soft rubbery material and fabricated via injection molding. To reduce the effort and cost of creating the skin molds, we fabricate the skin on a piece-by-piece basis: one limb has one skin piece, and the torso has two pieces as shown in Fig. 1 (Skin pieces). The skin-skeleton attachment is physically realized by another 3D printed bracing unit between the skin and the skeleton. This bracing unit is attached to each link on the skeleton and serves as a supporting structure between the rubbery skin and the mechanical skeleton (see Figs. 1 and 9). The purpose of this design is to expect that the friction between the skin and the printed parts can disable the relative motion between skin and skeleton at these parts, which is verified in the physical validation. We thus select the glue vertices in Fig. 9 according to the position of skin-skeleton attachment parts so that the coupling constraints can reflect this physical setting. The mass matrix and the inertia tensor of this bracing unit are integrated in our multibody subsystem dynamics. Finally, skin pieces are glued together after all skin pieces are installed using nonreactive PVA adhesive.

Motor Specification. We usually use the MG995R servo motor to drive the motion of the skinned robot. The motor’s size is $40.8\times 20\times 38$ mm with the maximum torque of $20~$kg$\cdot$cm under 6.4 V (i.e., $U_m=1.96$ N$\cdot$m in Eq. (12)). In total, 12 motors are installed in the beetle-like robot. All the motors are controlled with an Arduino board, which supports up to 32 motors.

SECTION 7 Experimental Results

In this section, we report motion design results generated by our algorithm and two fabricated robots for the physical validation and then proceed to the torque limits and folding region experiments conducted on the beetle-like robot. A comparison to kinematic optimization only is also provided for this robot. The performance of our optimization algorithm depends on the number of vertices on the skin mesh, the number of glue vertices, and the number of joints of the mechanical skeleton. The frame interval $\Delta t$ is 0.005 second, and we employ the discrete collision detection algorithm to handle self-collisions and foot-ground collisions. Normally a motion cycle has around 500 frames. Our optimization algorithm was implemented on a desktop PC with an intel i7-7700 CPU and 16 GB memory. The mechanical skeleton of the fabricated robots are designed based on the templates shown in Fig. 6. The soft skin is made of isotropic rubber material whose Young’s modulus is $0.09~GPa$, and Poisson’s ratio is 0.46. Table 1 reports some essential physical and simulation statistics of these three examples. The space-time optimization step with approximated skin deformation for a skeleton is around 40 seconds. For the slow-walking motions as shown in Figs. 1 and 14, we only need one iteration to converge, since the frame-by-frame optimization can reproduce the motion from space-time optimization step well with physical constraints. For the relatively fast trotting motion in Fig. 13, the algorithm converges after two iterations.

Fig. 6.

Three mechanical structure templates and the exploded views of the torso and limb structures of the first template.

Fig. 7.

With the assistance of the developed Solidwords scripts, the user only needs to tweak semantic parameters like the link length, motor mount size, etc. to obtain a customized link. The size of all the pilot holes for screw installation remains unchanged under such edits.

Fig. 8.

We add folding regions to facilitate the stretching deformation of the skin. The template folding region is similar to gear teeth, and it is formed by a sweeping cut operation, that is, the CSG difference between the volume surrounded by the original skin surface and the volume formed by rotating the sweeping contour along a central axis.

Fig. 9.

Glue vertices.

TABLE 1 Physical and Simulation Statistics of Three Tested Robots

Motion Design Results. Fig. 10 illustrates the simple foot lifting motions designed by our system for the beetle-like robot. These two motions, i.e., single-foot lifting and double-foot lifting, are also used to show the COP is constrained to be inside the support polygon with our optimization algorithm. Please see the accompanying video for the animation.

Fig. 10.

The foot lifting motion for the Beetle-like robot. (a) Single-foot lifting. (b) double-foot lifting. The green balls indicate the COP positions and gray lines the support polygons. Our optimization algorithm can constrain the COP to be inside the support polygon.

The embedded skeleton of the monster-like robot shown in Fig. 6 is designed using the third template. The weight of this robot is $11.87~\mathrm {kg}$, and its size is 48.5 cm × 64.6 cm × 27 cm. The Young modulus at the tail and belly of the monster is reduced by 85 percent to demonstrate the dynamics of the skin. Two different input foot trajectories are used to generate the walking motions for the robot. As shown in the leftmost column in Fig. 12, the first trajectory has a longer stride length but lower step, while the second one has shorter stride and higher step. Our system is able to accommodate such variations and produces a smooth and physically correct motion plan. The walking speed for these two walking motions are 0.11 meter/second and 0.06 meter/second.

We generate a trotting motion of 0.45 meter/second speed for the monster-like robot to demonstrate that our system can support fast motion. In this example, the trajectories of joint positions are labelled using the horse motion pictures photographed by Eadweard Muybridge, a famous photographer for his work on motions. The joint positions are mapped to a horse motion with the specified speed using the method in [85] and re-targeted to the skeleton of our monster. This initial kinematic motion (please see the accompanying video for the motion) is then optimized using our alternating motion optimization algorithm to turn it into a physically feasible motion.

We notice that the flight phase of the initial kinematic motion is not consistent with the foot contact plan. To be more specific, the time of the flighting phase is not enough for the monster to return back to the ground. Thus, the space-time optimization with physical constraints, especially the momentum constraint, is necessary to eliminate such inconsistency, which is critical to the success of the alternating optimization. Fig. 11 illustrates the effect of the COP optimization. The red balls indicate that the COP positions at two frames in the initialized trotting motion plan are outside of supporting polygon after the first space-time optimization that does not account for the deformation of the skin mesh. Thus, the frame-by-frame simulation fails to produce a stable trotting motion with its initially optimized motion plan, as shown in the second row of Fig. 13. The COP constraint is turned off to produce this failed example once this constraint can not be satisfied by the solver. After the second iteration, the COPs are moved into supporting polygons, as indicated by the green balls. A smooth trotting motion can then be generated as shown in the first row on the right of Fig. 13. The comparisons of the optimized joint angles and the $z$ components of COP are illustrated in Fig. 20. The variation of $z$ components indicates the COP moves from left to right so that it is inside the supporting region.

Fig. 11.

The effect of the COP constraints in the trotting motion plan. Red balls: The COP positions computed using the motion plan after initialization. Green balls: The COP positions optimized with the skin mesh deformation. The supporting polygons are in cyran.

Fig. 12.

The monster-like robot takes two different input foot trajectories, and our system computes natural and physically correct motions for both inputs.

Fig. 13.

Trotting motion for the monster-like robot. (a) Marked joint positions. The color of a dot indicates to which part of the horse skeleton it belongs, and the positions are mapped to our monster skeleton joint angles using the space-time optimization with only kinematic constraints. (b) The designed trotting motion with our alternating algorithm. (c) The unstable motion simulated by the frame-by-frame optimization when the skin deformation is not considered in the space-time optimization.

Another example is reported in Fig. 14. The mechanical skeleton of this robot is further edited based on the second template of Fig. 6. We lengthened the torso and shortened the limbs to fit this template into the input model (125 cm × 47 cm × 46 cm). Our system also produces plausible motion plans for this quad-robot.

Fig. 14.

The motion of a lizard-like robot. We edit the second template in Fig. 6 with SolidWorks scripts to create the design of its mechanical skeleton. With user provided inputs, our system generates plausible motions of this robot.

Physical Validation. We fabricate two robots to validate the physical feasibility of our algorithm. The dimension of the fabricated beetle-like robot shown in Fig. 1 is 47.62 cm length × 33.66 cm width × 37.29 cm height, and its skeleton weight and skin weight are 2.8 kg and 1.1 kg respectively. Its motion is designed to spread its feet on the ground to enlarge the supporting area. Another robot, cow-like robot, is designed to be similar to the monster robot, but with the modified 3D shape. Its dimension is 55.71 cm length × 22.86 cm width × 47.72 cm height, and its skeleton and skin weights are 4.3 kg and 1.65 kg respectively. Since the robot is much higher than the beetle-like robot and thinner in the width, the balance control of the robot is more difficult. To deal with this larger robot, we select the DS3218 servo motor (dimension: 44 mm length × 20 mm width × 64 mm height, Voltage 6.0V) with maximal torque 3.13 N$\cdot$M. Given the input contact plan and guide trajectories, our algorithm managed to produce a stable walking motion for it. Due to the possible configuration deviation, such as screws installing deviation, in the installation of the physical robot, there exist small vibrations during the phase of foot lifting in the walking motion, which is not fully captured in the simulation. However, the generated motion plan of our algorithm is still physically plausible, as shown in Fig. 15. The first and the last templates in Fig. 6 are edited to design the mechanical skeletons for the two robots respectively and then the skeletons are fabricated using 3D printing.

Torque Limit. The torque limit in the motion optimization (i.e., Eq. (12)) of the beetle-like robot is set as 1.96 N$\cdot$m to match the physical torque limit of the used MG995R motor. To verify if this hard constraint is faithfully enforced during the optimization, we examine torque values that are calculated by our motion design system after per-frame optimization. The result is reported in Fig. 16, where torque curves at an ankle joint and a shoulder joint are plotted. It can be seen that the imposed motor constraint successfully bounds the torque magnitude to be within the limit to ensure that the designed motion is physically possible.

Folding Regions. Adding folding regions to the robot’s skin is an effective fabrication artisanry to enable the robot assembly using off-the-shelf servo motors and lower the fabrication cost. To demonstrate its advantage, we compare the skin deformation under the joint rotation when the robot is attached to a regular soft skin and a folded skin. Both skins are fabricated using the same materials. It can be seen from Fig. 17 that rotating joints yield large stretching stress over the skin, which could easily go beyond the physical capacity of many commercial servo motors. In this test, we follow the aforementioned motor specification by setting the maximal torque to 1.96 N$\cdot$m and test if this power is sufficient to generate the necessary skin deformation. We set our target bending angle to $\pm 70^\circ$, which is a common value in many walking gaits. The motor is programmed to reach this target in 2 seconds. Our simulation shows that the smooth robot skin without folding region prevents the motor from producing sufficient joint rotation and the maximum angle that can be reached is only about $\pm 20^\circ$. With the folding region, swept by an 8 mm-depth tooth over the smooth skin, our simulation predicts that the motor is able to generate the desired rotation. The physical experiment results are quite consistent with our simulation prediction as reported side by side in Fig. 17.

VS. Kinematic-Only Optimization. In contrast to robots with only rigid mechanical skeletons, skinned robots exhibit a much more complicated dynamic behavior, which should be fully incorporated during the motion design. To illustrate the necessity of incorporating influences of the soft skin, we compare the motion plans generated using our method and the one by Megaro et al. [3]. Because the primary focus in [3] is to design robot creatures with only rigid links, Megaro and colleagues used a kinematic-based optimization strategy, which includes the trajectories of COM, COP, end effectors, and the footfall pattern. Based on the resulting motion plan, we compute the corresponding driving torques at joints using inverse dynamics. Specifically, the driving torques are computed by imposing another set of rotation constraints over the skeleton in Eq. (2) using the Lagrange multipliers method (with necessary complimentary constraints and inequality constraints to handle the ground contact and motor torque limit). The constrained joint rotation corresponds to the one obtained from the kinematic-based motion plan, and the multipliers represent required generalized constraint force, which are converted to joint torque via $\mathbf {J}_{\omega k}$ to achieve the target joint rotation. Moreover, the contraint force can not be enforced at the root joint since motor actuators are installed at the link joints. As shown in Fig. 18, the physical simulation results suggest that a kinematically valid joint trajectory does not guarantee a smooth walking cycle of the skinned robot, even though the constraints of COM/COP are also specified in the kinematic optimization without skin information. The coincidence, for the knee joint, of the joint angle curves of kinematic and inverse dynamics shows that our inverse dynamics computation can track the joint angle trajectories well due to the constraint forces, while the roll angle of the root link experiences a larger variation in the inverse dynamics simulation. It indicates a physically unstable walking motion. This is also verified in the physical experiment as indicated by the green lines shown in Fig. 19: the motion plan obtained using only kinematic optimization leads to a shaky motion. We also observe backward motions as highlighted in the figure. Our method, because it fully considers various physics conditions and constraints, yields a much better result.

Failure Case. While our system is stable in the generation of slow walking motion, we find the generation of fast trot motion is sensitive to the physical parameters. When the mass of the monster is increased to two times, its influence on the mass center cannot be balanced in the optimization and the COP constraint is violated in the space-time optimization result, possibly due to its conflict between the foot contact constraint. Hence, the frame-by-frame optimization will fail to produce a stable motion. Such situation might be handled through the integration of foot plan sampling step in [36].

Fig. 15.

The fabricated cow-like robot. The motion plan is generated according to the contact plan (yellow bars) shown on the second picture in the first row.

Fig. 16.

We plot the torque values of the optimized motion planning of the beetle-like robot. Orange curve: the torque curve at an ankle joint (at the orange box). Blue curve: the torque curve at a shoulder joint (at the orange box).

Fig. 17.

Adding folding regions significantly relieves the stretching stress over the skin. We program the motor at an ankle joint of the beetle-like robot to rotate $\pm 70^\circ$ within 2 seconds in this experiment. A smooth skin can only be bent around $\pm 20^\circ$, while the folded skin is able to reach the desired deformation. The physical experiment (black skins) results are consistent with simulation results (yellow skins).

Fig. 18.

The motion plan generated by the kinematic-only optimization [3] leads to unstable walking sequences. Left: Selected frames. Right: The joint angle curves. Kinematic: the kinematic optimization result. Inverse dynamics: the simulation result by following the kinematic optimization result. Our: our optimization result. The large roll angles of the root link is due to the unstable pose using kinematic-only optimization. Please see the accompanying video for the full comparison.

Fig. 19.

Physical experiments show that kinematic-only optimization is not a feasible solution for skinned robots – there are noticeable back steps (highlighted with a red box) in a motion cycle as the driving torques, after damped by the skin deformation, are not strong enough to produce necessary normal contact forces. Please refer to the accompanied video for a clearer comparison.

Fig. 20.

The comparison of joint angle curves and COM positions. With skin weight: the curves are from the initial space-time optimization result with only skin weight. With skin deformation: curves from the space-time optimization result with skin deformation simulated by frame-by-frame optimization. COM Z: the $z$ component of the COM, representing the COM movement from left to right during the motion.

SECTION 8 Conclusion and Future Work

In this paper, we have presented a fabrication-oriented motion planing algorithm and detailed design/fabrication procedures for personalized skinned quad-robots. The physical constraints, such as the equations of motion of the skinned robot and the motor constraints, are integrated into the motion planning such that the resulting motion plan is physically and dynamically feasible. The condensation formulation allows us to conveniently establish the nonlinear relationship between external forces and the target kinematic parameters of the locomotion and to reach a QPCC formulation for the motion design. Our experiments show that the system is able to assist regular users to obtain natural and smooth motions designed for skinned quad-robots.

In the future, we want to explore a gait synthesis algorithm to generate the motion plan from high-level parameters, such as velocity and turning angles. Combining captured gait data and optimization with dynamic constraints has the potential to significantly reduce users’ labor efforts of creating such motion planning. Currently, the coupling between the FEM simulation of the soft skin and the rigid body dynamics of the mechanical structure is not fast enough for a closed-loop control of skinned quad-robots. We want to explore model reduction or homogenization techniques to reduce the computational cost of FEM simulation and produce interactive feedback to control skinned robots online.

ACKNOWLEDGMENTS

The authors would like to thank anonymous reviewers for their constructive comments. Weiwei Xu is partially supported by Zhejiang Lab. Yin Yang is partially spported by NSF under Grant Nos. CHS 1845024 and 1717972. Weiwei Xu and Hujun Bao are supported by Fundamental Research Funds for the Central Universities. This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (Grant agreement No 715767).