-
Figure 1.
Aerial manipulator performing contact-based surface interaction.
-
Figure 2.
Optimal adaptive impedance control framework.
-
Figure 3.
Off-policy IRL convergence: per-iteration Critic-update norm
vs policy-iteration index (logarithmic axis); the algorithm stops when this quantity falls below$ \|\Delta{\boldsymbol{W}}_{c}\|_{2} $ .$ \varepsilon=10^{-6} $ -
Figure 4.
Normal position error
: IRL policy vs CasADi optimum, with the noisy baseline shown across the full horizon for context. The inset shows the pointwise IRL$ \tilde p_{n}(t) $ CasADi difference over the regulated contact interval.$ - $ -
Figure 5.
Normal contact force
: IRL policy vs CasADi optimum, both regulating around the$ f_{c,n}(t) $ setpoint indicated by the dotted reference line. The inset shows the pointwise IRL$ 10\,\mathrm{N} $ CasADi difference over the regulated contact interval.$ - $ -
Figure 6.
Normal contact force
for the three deformation exponents$ f_{c,n}(t) $ , all overlaid on a single axis. Method is encoded by line style (LC-LQR dash-dot, IRL dashed), and$ n_{p}\in\{1.0,1.5,2.0\} $ by colour shade (lighter$ n_{p} $ darker as$ \rightarrow $ grows). The shared noisy baseline is faded after the switch instant and the desired setpoint$ n_{p} $ is the dotted reference. Background shading marks the Approach/Data-collection/IRL-applied phases. The inset zooms the$ f_{d,n}=10\;\mathrm{N} $ –$ 8 $ band around the setpoint, exposing the near-setpoint tracking that the full vertical range compresses.$ 14\;\mathrm{N} $ -
Figure 7.
End-effector path in the contact plane under the three weight configurations of Table 2, with the noisy baseline shown in light blue for reference.
-
Figure 8.
Normal position error
for the three weight configurations across the full simulation horizon. The shaded background marks the three pipeline phases.$ \tilde p_{n}(t) $ -
Figure 9.
Normal contact force
for the three weight configurations across the full simulation horizon, regulated against the$ f_{c,n}(t) $ desired setpoint.$ 10\,\mathrm{N} $ -
1: Choose discount factor , weights$ \alpha $ , basis$ {\boldsymbol{Q}},{\boldsymbol{R}} $ , and integration window$ {\boldsymbol{\sigma}}({\boldsymbol{X}}) $ . Initialize Critic weights$ T_{\mathrm{int}} $ .$ {\boldsymbol{W}}_{c}^{(0)} $ 2: Apply a stabilizing behavior policy , where$ {\boldsymbol{u}}_{b}={\boldsymbol{u}}_{0}+{\boldsymbol{e}} $ corresponds to the baseline impedance gains and$ {\boldsymbol{u}}_{0} $ is exploration noise; collect closed-loop trajectory$ {\boldsymbol{e}} $ .$ \{{\boldsymbol{X}}(t),{\boldsymbol{u}}_{b}(t)\} $ 3:for each trajectory segment ,$ [t_{k},t_{k}+T_{\mathrm{int}}] $ do$ k=1,\dots,N $ 4: Form and$ {\boldsymbol{\phi}}_{k} $ via (33).$ \Delta V_{k}^{(i)} $ 5: end for 6: Assemble and$ {\boldsymbol{\Phi}}_{N} $ as in (34) and update the Critic via least squares (35).$ {\boldsymbol{\Delta}}_{N}^{(i)} $ 7: Update the Actor policy via (31). 8: Repeat steps 3–7 (re-forming the iteration-dependent targets along the same recorded trajectory) until .$ \|{\boldsymbol{W}}_{c}^{(i+1)}-{\boldsymbol{W}}_{c}^{(i)}\|<\varepsilon $ 9: Deploy along the contact normal; by (28), the components of$ \hat{{\boldsymbol{u}}}^{\star}({\boldsymbol{X}}) $ are the optimal time-varying impedance gains$ \hat{{\boldsymbol{u}}}^{\star} $ directly. The tangential channels retain their nominal gains, with predicted behaviour given by Proposition 1.$ \bigl(D_{n}^{\star}({\boldsymbol{X}}),K_{n}^{\star}({\boldsymbol{X}})\bigr) $ Table 1.
Off-policy IRL for optimal adaptive impedance control.
-
Parameter Value Description Simulation $ T $ $ 200\;\mathrm{s} $ Total simulation horizon $ \Delta t $ $ 0.01\;\mathrm{s} $ Sampling interval $ T_{\text{coll}} $ $ 5.01\;\mathrm{s} $ Exploration-noise window length End-effector initial condition $ {\boldsymbol{x}}_{0} $ $ [-3,\;3,\;0]^{\mathrm{T}}\;\mathrm{m} $ Initial end-effector position Reference trajectory generator (20) $ q $ $ 3 $ Auxiliary-state dimension $ {\boldsymbol{A}}_{d} $ $ \bigl[\begin{smallmatrix} 0 &\omega&0\\ -\omega&0&0\\ 0&0&0 \end{smallmatrix}\bigr],\;\omega=\pi/100\;\mathrm{rad/s} $ Exosystem dynamics matrix $ {\boldsymbol{C}}_{d} $ $ {\boldsymbol{I}}_{3} $ Exosystem output matrix $ {\boldsymbol{r}}_{d} $ $ [-0.066,\;-0.091,\;0]^{\mathrm{T}}\;\mathrm{m\;s^{-1}} $ Constant exosystem input $ {\boldsymbol{\zeta}}(0) $ $ [-2.9,\;3.1,\;0]^{\mathrm{T}}\;\mathrm{m} $ Initial auxiliary state Hunt–Crossley contact $ k_{c} $ $ 10^{2}\;\mathrm{N\;m^{-n_{p}}} $ Contact stiffness $ n_{p} $ $ 1.5 $ Deformation exponent, dimensionless (default) $ b_{c} $ $ 30\;\mathrm{s\;m^{-1}} $ Viscoelastic damping coefficient $ f_{d,n} $ $ 10\;\mathrm{N} $ Desired normal contact force $ {\boldsymbol{n}} $ $ [2,\;2,\;1]^{\mathrm{T}}/3 $ Outward surface normal (unit vector, tilted) $ {\boldsymbol{p}}_{e} $ $ [-3.5,\;1.5,\;0]^{\mathrm{T}}\;\mathrm{m} $ A point on the contact surface Baseline impedance (in the contact-attached frame) $ M_{n} $ $ 100\;\mathrm{kg} $ Baseline normal inertia $ D_{n}^{(0)} $ $ 200\;\mathrm{N\;s\;m^{-1}} $ Baseline normal damping $ K_{n}^{(0)} $ $ 100\;\mathrm{N\;m^{-1}} $ Baseline normal stiffness $ {\boldsymbol{M}}_{t} $ $ {\boldsymbol{I}}_{2}\;\mathrm{kg} $ Baseline tangential inertia $ {\boldsymbol{D}}_{t}^{(0)} $ $ 2{\boldsymbol{I}}_{2}\;\mathrm{N\;s\;m^{-1}} $ Baseline tangential damping $ {\boldsymbol{K}}_{t}^{(0)} $ $ {\boldsymbol{I}}_{2}\;\mathrm{N\;m^{-1}} $ Baseline tangential stiffness Off-policy IRL $ \alpha $ $ 0.01 $ Discount factor $ \deg({\boldsymbol{\sigma}}) $ $ 4 $ Polynomial critic basis order $ \sigma_{e} $ $ 0.1 $ Exploration-noise std (multiplicative on )$ {\boldsymbol{u}} $ $ {\boldsymbol{Q}} $ $ \mathrm{diag}(5\times 10^{4},\;5\times 10^{4},\;0) $ State-cost weight (default) $ {\boldsymbol{R}} $ $ \mathrm{diag}(10^{-1},\;10^{-1}) $ Gain-cost weight (default) Table 1.
Simulation parameters used in the "Numerical simulation" section, grouped by category.
-
Case $ {\boldsymbol{Q}} $ $ {\boldsymbol{R}} $ (Aggressive)$ 1 $ $ \mathrm{diag}(5\times 10^{4},\,5\times 10^{4},\,0) $ $ \mathrm{diag}(10^{-2},\,10^{-2}) $ (Intermediate)$ 2 $ $ \mathrm{diag}(5\times 10^{4},\,5\times 10^{4},\,0) $ $ \mathrm{diag}(10^{-1},\,10^{-1}) $ (Compliant)$ 3 $ $ \mathrm{diag}(5\times 10^{-5},\,5\times 10^{-5},\,0) $ $ \mathrm{diag}(1,\,1) $ Cases are ordered by decreasing ratio, i.e. from most aggressive to most compliant.$ {\boldsymbol{Q}}/{\boldsymbol{R}} $ Table 2.
Cost-weight configurations for the sensitivity study.
Figures
(9)
Tables
(3)