Introduction
Thyroid cancer is common: 18000 people per year are diagnosed in Japan and 567000 people per a year worldwide with thyroid cancer [1]. Current thyroid guideline recommendations call for examinations with ultrasound (US) imaging in all suspicious patients [2]. A combination of clinical factors and US features will determine whether the clinician should perform further confirmatory testing or determine whether to perform periodic US follow-up. Within this diagnostic scheme, US evaluation is widely accepted as an important diagnostic step for the screening of malignancy. Meanwhile, the US imaging is known as sophisticated task as it demands accurate positioning of the US probe and a constant contact force between the probe and the patient, which significantly affects the quality of the US images. Additionally, clinicians need to understand internal three-dimensional (3D) condition with two-dimensional (2D) images, which requires clinicians to cognitively maintain the spatial relationships between the US probe and visualized target as well as manage hand-eye coordination and be very operator dependent [3], [4]. Especially, the thyroid US procedure demands an extremely cautious approach from the operator, as the area being scanned at the neck is highly sensitive. This requires the operator to apply a minimal amount of contact force. Excessive contact force doesn't only lead to be uncomfortable for patients, but also causes to degrade the quality of acquired image such as a distortion of organ [5]. The distortion of thyroid in the acquired image may cause an error of thyroid volume measurement and result in inaccurate diagnosis [6]. Therefore, there is a critical demand for maintaining the minimal contact force which satisfies both the patient's comfortability and image quality.
A. Related Works
1) Robotic US System
Over the past two decades, researchers have extensively studied the application of a robot-assisted US system in various areas such as the upper limb [7], jugular vein [8], liver [9], [10], fetus [11], [12], [13], lung [14], [15] and heart [16], [17], recognizing its significant potential for addressing the related concerns. Several researches proposed the robotic US system targeting to the thyroid US [18], [19], [20]. Kojcev et al. [18] developed a tele-operative robotic US system with 3D camera that enabled to place the US probe to the region of interest on the neck via the captured point cloud data and compared the thyroid length measurements between the robotic acquisition and a manual expert measurement. The contact force during the scan was controlled based on the impedance control. Maximum applied contact force was described as 5 N but there was no further details. Kaminski et al. [19] also performed a phantom study with the similar configuration system for the assessment of thyroid diseases. They implemented an admittance control as the force-feedback control and maintained the contact force at about 7 N. Zielke et al. proposed a 3D thyroid volume estimation method with the robotic US system [20]. The US probe position was controlled to maintain the constant contact force with a built-in real-time impedance control of a commercial robot arm. The actual applied contact force was not described. Kim et al. developed a unique 6-degree-of-freedom (DOF) manipulator-type US scanning robot and implemented a hybrid position/force control [21]. The developed system was evaluated with a thyroid phantom and applying the contact force between 2 N and 15 N. Although those researchers could demonstrate successfully the clear advantages of using robotic US system to thyroid US, we concerned about its safety during the generation of contact force. All above systems rely on the impedance or admittance control with built-in or external force and torque sensor. The impedance control is better suited for interaction with stiff environments and the admittance control is better suited for interaction with soft environments, but both controls become potentially unstable due to the unexpected contact [22], [23]. In additions, the desired standard contact force in robotic thyroid US has not been investigated well.
2) Constant Force Generation for US Scan
In most research of robotic US system, the contact force is adjusted by controlling the position of the robot arm holding the US probe directly [19], [20], [21], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33]. Meanwhile, there is a concern for the stability to maintain the contact force with the force feedback of the robot arm. To address the concern, several research reported unique mechanisms to generate the constant contact force without the force feedback control of robot arm. Our previous works proposed a passive mechanism for fetal robotic US system with the combination of constant spring and counterweight. Meanwhile, the output force was fixed by the specification of utilized spring and then could not be adjusted finely depending on the individual differences [11], [13], [15]. For addressing the issue, Bao et al. developed a parallel, motor-spring-based end-effector for robotic US system which enables to adjust the contact force online [34]. The proposed end-effector could provide the contact force of 4-12 N with the US probe displacements of 0-8 mm. The proposed end-effector was designed for the application of fetal ultrasonography but in thyroid US, even lower contact force may be required as described in Section II-A. In addition, the applicable displacement of the US probe is limited and may not be able to accommodate the uncertain body motion or respiratory motion.
B. Contribution
To address the aforementioned limitations of the state-of-the-art, this study aims to analyze the contact force in terms of the safety and image quality and develop a robotic US system to generate the adequate contact force constantly and safely. First, the desired contact force in thyroid US is determined based on the experimental analysis in collaboration with thyroid US experts. Based on the preliminary data, we develop a series-elastic actuated constant force end-effector which enables to apply constant contact force passively and safely. We believe that this is the first study focusing on the analysis of the safety and image quality in the robotic thyroid US and conducting human trials. The main contribution of this work can be summarized as follows.
We analyzed the contact force during the thyroid US performed by experts and determined the standard contact force generated by the robotic US.
We developed a distinctive end-effector incorporating a series-elastic actuated mechanism that enables to generate arbitrarily low constant contact force semi-passively.
We integrated the proposed end-effector to a robot arm and performed validations with a thyroid phantom and seven healthy male subjects evaluating the image quality and safety with the integrated system.
Method
A. Experimental Analysis
First, we investigate the standard contact force in thyroid US through human trials in collaboration with thyroid US experts. A wireless linear US probe (Vscan Air, GE Healthcare, USA) is set at a 3D-printed holder incorporating a 6-axes force/torque sensor (Nano 17, ATI Industrial Automation, USA). We recruited 5 experts, and each of 5 experts performed a manual scan as an operator to other 4 experts as a subject; thus, 20 trials in total were performed. Following the actual clinical protocol of thyroid screening, the recruited experts performed a same comprehensive scan on the center, right and left lines of the neck along craniocaudal direction. The contact force applied perpendicular to the US probe face during scanning is measured. Table I represents the average and standard deviation of the contact force in all combinations of the operator and subject. The average contact force in total trials was 2.17 ± 1.11 N. Fig. 1 shows the box charts comparing the contact forces between operators and between subjects. There was no significant difference of the contact force both between the operators and between the subjects. Since they scanned each other, bias potentially occur due to feedback from their experience as the subject but contact forces between 1 N and 4 N generated by the robot would be acceptable.
Preliminary results of the contact force during manual scanning with healthy subjects; (top) comparison between operators, (bottom) comparison between subjects.
B. Overall System
Fig. 2 shows the overview of our robotic US system comprised of the series-elastic actuated constant force end-effector, a corporative 6-DOF robot arm (UR5e, Universal Robotics, Denmark) and a 3D camera (Torobo Eye SL40, Tokyo Robotics, Japan). The end-effector hold the wireless US probe and is mounted on the end of robot arm. The role of 3D camera is to capture the surface shape of neck as point cloud data, which is used for the scan path planning of the US probe mounted on the robot arm. The robot arm can move the end-effector following the predetermined path while the end-effector adjust the position of US probe for maintaining the contact force locally. This configuration ensures the patients’ safety because the robot arm never deviate the predetermined path and push the end-effector to the body surface.
Overview of developed robot system comprised of cooperative robot arm, 3D camera, US probe, and end-effector.
C. Series-Elastic Actuated Constant Force Control
As shown in Fig. 3(a), the proposed end-effector consists of several components including a micro electric linear actuator (L12-20PT, MigthyZap, Korea Republic), a force/torque sensor (Nano17, ATI Industrial Automation, USA), two linear springs, a constant force spring, a moving base and a probe hold unit, which is similar to a configuration of “series-elastic actuator” [35], [36]. The moving base is connected to the linear servo actuator and pushes the probe hold unit via the linear springs. The probe hold unit can move freely along two shafts and be connected to the constant spring. The force/torque sensor is embedded into the probe hold unit and capable of measuring the force applied to the US probe directly.
(a) Mechanism and (b) control scheme of the series-elastic actuated constant force end-effector.
The basic mechanism to generate the constant contact force is to control the amount of compression in the linear springs with the linear servo actuator. The compliance provided by the linear springs allows the linear servo actuator to absorb shocks, adapt to varying contact force, and provide a safer interaction with the environment when dynamic position changes occur, such as scanning on uneven tissue surfaces or respiratory motion. The output contact force Fout is defined as below.
\begin{equation*}
{F}_{\text {out}} = {K}_l\ \left({{x}_{hold} - {x}_{base}} \right) + {M}_{hold}g\cos \theta - {F}_c \tag{1}
\end{equation*}
An Arduino-based controller (IR-STS01, MigthyZap, Korea Republic) is employed to control the linear servo actuator. The resolution of the linear servo actuator is 6.6 μm, the maximum range is 27 mm, and practical limit of output force is 12 N. The liner spring constant is 0.45 N/mm. A custom software application was developed with Python to coordinate the control of the linear servo actuator with the readings from the force sensor. The update rate of the system is about 2 ms.
D. System Integration
The position of the US probe held by the proposed series-elastic actuated constant force end-effector is controlled by the robot arm. The position is calculated based on the point cloud data of neck surface captured by the 3D camera. Since the 3D camera is attached to the end-effector, the position of the captured point cloud data is linked to the frame of the robot arm. Fig. 4 illustrates the sequential transformations within the integrated system, comprising the robot arm, end-effector, 3D camera, and US probe. The interconnected components allow for the conversion of acquired points in the 3D camera frame (
The graphic user interface (GUI) to draw the scan path manually on the captured neck surface is developed with Open3D library as shown in Fig. 5. The rotation angle of the US probe during scanning is set at the normal on the position on the generated scan path. Noted that the center of rotational motion of the robot arm is set at the tip of the US probe. As the tip position of the US probe is dynamically changed during scanning, the center of rotation is calibrated online based on the amount of spring compression. The amount of spring compression is measured by an optical distance sensor (ZX‐LD100L, Omron, Japan).
E. Experimental Protocol and Setup
For validating the proposed system, three experiments are performed. First, we verify the contact force generated by the developed end-effector in several scanning speed with a thyroid phantom (Model 074, CIRS, USA). Four-way force (1, 2, 3, 4 N) is set as the targeted contact force under fixing the scanning speed at 2 mm/s. The scanning path is the center, left and right lines of thyroid phantom and its distance is 80 mm. To demonstrate the effectiveness of force-feedback control, a scan without force-feedback control was also performed, in which the probe position was only passively adjusted by linear springs. In addition, four-way scanning speed (1, 2, 5, 10 mm/s) is set under fixing the contact force at 2 N. To focus on the effect of the scanning speed, this experiment is performed only on the center path. The scanning distance is 80 mm.
In the second experiment, we validate the acquired image quality with the developed end-effector compared to the manual scan using the thyroid phantom. The US probe is set at the position where both the thyroid gland and cyst inside the thyroid gland are clearly visible. At the positions, the contact force is changed from 1 N to 4 N, and the US image is obtained in each of the targeted forces. For evaluating the acquired image quality quantitively, we calculate the contrast-to-noise ratio (CNR) between the cross-sections of cyst and thyroid gland. The CNR is defined as:
\begin{equation*}
CNR\ = \frac{{\left| {{\mu }_c - {\mu }_t} \right|}}{{\sqrt {\sigma _c^2 - \sigma _t^2} }}\ \tag{2}
\end{equation*}
Third, we demonstrate the thyroid US scan with the robot system to healthy subjects. In this human trial, we perform the subjective evaluations regarding the discomfort and pain during the scanning for patients and the acquired image visibility for physicians. The US scan was performed with the subjects seated in a chair as shown in Fig. 5. The Visual Analog Scale (VAS) is utilized to assess discomfort levels reported by the subjects. The VAS follows a 5-point scale, ranging from 1 to 5, where 1 indicates no discomfort and 5 denotes a sensation of being suffocated. The scanning path is manually determined around the area including their left thyroid gland based on the captured 3D point cloud data introduced in Section II-D. At the positions, the contact force is changed from 1 N to 4 N, and the US image is obtained in each of the targeted forces. Also, CNR between the cross-sections of carotid artery and thyroid gland and the diameter of the carotid artery are measured for evaluating the image quality in each condition. Additionally, the area of thyroid gland varying on the applied contact force is measured in order to evaluate the degree of the distortion due to the pressure. Thyroid gland boundary is marked manually and its area is calculated with an image editor of GIMP (ver. 2.10). The degree of distortion was evaluated by comparing the area obtained with the contact force of 1 N. The ratio of the area between 1N and other applied forces was calculated.
This study was authorized by the Institutional Research Ethics Committee of the National Institute of Advanced Industrial Science and Technology (Approval No. 2022-1299), and informed written consent was obtained from all participants prior to the commencement of the study. A total of seven male subjects were included in the study.
Results
A. Verification of Contact Force Generation
Fig. 6. represents the results of generated time-series contact force when the target contact force is varied from 1 N to 4 N and when no force feedback control is applied. Also, Table II shows the error between the target and actual contact forces in each the scanning path. Regardless of the choice of the scan path, the generated contact force was stable and maintained around the targeted force. The mean error of generated contact force was under 0.1 N in all cases.
Result of time-series contact force during scanning on three paths of thyroid phantom when the target contact force is varied from 1 N to 4 N. NC represents the result without force feedback control (adjusted only with linear spring).
Fig. 7 shows the result of generated contact force when the scanning speed is varied from 1 mm/s to 10 mm/s under fixing the contact force at 2 N. Although the deviation was increased when the scanning speed was increased, the maximum error was less than 0.3 N for scanning at 10 mm/s. The results suggest that the developed end-effector can safely generate the constant contact force required for the thyroid US imaging.
Result of generated contact force when the scanning speed is varied from 1 mm/s to 10 mm/s under fixing the contact force at 2 N.
B. Validation of Acquired Image Quality
Fig. 8 shows the obtained US images of the thyroid phantom varied on the contact force from 1N to 4N, and Table III shows the quantitative result of US image quality. At the contact force of 1 N, both CNR and cyst diameter were maximal, although shadow artifacts occurred slightly on the left side of the image. When increasing the contact force, the visibility of the cyst was decreased because the cyst was shifted out-of-plane due to the compression. Also, the position of organ in the US image was shifted upward overall, which contributed to improve the contrast in the lower part of the thyroid gland. Comparing the ground truth, the image quality at the contact force between 1 N and 2 N was most similar to that of the manual scan.
C. Validation in Human Trials
Fig. 9 shows the results of the VAS score from seven subjects. The result indicated that the score was increased when the contact force was increased. In order to compare the difference of the score between each contact force statistically, we used Tukey-Kramer as the statistical analysis. There were significant differences of the score between 1 N and 4 N (p = 0.0078) and between 2 N and 4 N (p = 0.025).
Result of subjective evaluation of discomfort when varying the contact force from 1 N to 4 N.
Fig. 10 shows the representative US images of one subject, with the contact force varied from 1 N to 4 N, and Table IV shows the result of the CNR, the diameter of the carotid artery and the observed area of the thyroid gland. As same as the phantom study, the position of organs in the US image was changed due to the compression. Regarding the image quality, there was no significant difference of the CNR and the diameter of the carotid artery between each contact force, but the observed area was decreased when increasing the contact force. Focusing on surrounding blood vessels, an internal jugular vein, which is located beside a carotid artery, was fully compressed when applying the contact force of 4 N as shown in Fig. 11. Note that a supplemental video shows the human trial demonstration of the US scan with the developed robot system.
Discussion
We presented a robotic thyroid US system that enables to scan with a low constant contact force, which was designed based on the analysis of physician's operation. To the best of the authors’ knowledge, this is the first study focusing on the analysis of the safety and image quality in the robotic thyroid US and conducting human trials. The results of phantom study showed that the developed robot system enabled to generate the intended contact force constantly in the range of 1 N and 4 N, required for thyroid US imaging as introduced in Section II-A, regardless the selected scan path route. In addition, the generated contact force was maintained in the certain range when increasing the scan velocity, although the deviation was large in the high velocity. Those results indicated that the developed end-effector could adjust the position of the US probe locally corresponding to the neck surface shape under maintaining the intended contact force. Although the relationship of the scan velocity to the generated contact force was evaluated up to 10 mm/s, the velocity above it may not be beneficial in terms of the patient's safety. Whereas, scanning at too slow velocity requires very long examination time, which can be uncomfortable for patients. Then, it is crucial to find an optimum scan velocity that satisfies the requirements for the safe scanning, the acquired image quality and acceptable examination time. As the limitation of the phantom study, the degradation of the image quality due to the scanning speed was not evaluated. Also, the experiment varying on the scanning speed was performed only on the center path of the phantom.
Focusing on the US images of thyroid phantom varied on the contact force, the visibility was slightly changed due to the applied contact force. The cyst inside the thyroid gland was clearly visualized in the contact force of 1 N but gradually disappeared when increasing the contact force. This may be because the cyst was compressed by the excessive force, or the cyst moved along the direction of out-of-plane due to the compression.
The results of the human trial demonstrated that the developed robot system enabled to acquire the US images with the diagnostic quality. Meanwhile, the results of the subjective evaluation indicated that the discomfort for the subjects was increased depending on the contact force. Even at the contact force of 4 N, subjects sometimes felt suffocating. Previous literatures [18], [19], [20] that didn't perform human trials reported that applied contact force was more than 5 N, but the lower contact force may be suitable in terms of the patient's safety and comfortability. Excessive contact force may not only cause pain or discomfort for patients but also compress the internal organ which potentially leads misdiagnosis. Focusing on the image quality with increasing the contact force, there was no significant difference of the CNR and the diameter of the carotid artery, but the thyroid distortion was increased. The result of the thyroid distortion, while predictable, may support maintaining the low contact force. On the other hand, the CNR and the diameter of the carotid artery were not directly affected due to the increase of the contact force, which indicates those metrics may not be suitable for evaluating the image quality in the thyroid US imaging. It is necessary for the further investigation for the diagnosis-related metric evaluating the image quality. In additions, we observed the internal jugular vein was compressed due to the excessive force. The internal jugular vein is used as the landmark for an image evaluation in the diagnosis. The visualization of internal jugular vein is essential to observe invasion of thyroid cancer and lymph node metastasis. In addition, prolonged compression of the internal jugular vein may cause stasis. Then, the low contact force of about 1 N should be applied to the thyroid US scanning. Meanwhile, the adjustment of contact force is possibly required for the image acquisition compensating individual differences of neck shape. For example, the fatty patient may require the contact force of 1 N or more to compress the subcutaneous fat. Also, even with the same contact force applied, the structural deformation would be different for different individuals. Then, it is necessary to develop a control architecture of online force adjustment based on the quality of acquired US image or visibility of specific anatomical features in thyroid US. Also, it may be necessary to investigate the relationship between the optimal force and the patient's condition such as body mass index (BMI). In summary, although the current results cannot substantiate that the compression affects the clinical image quality, it is desirable to maintain the low contact force in terms of the patient's comfortability during the scan.
While this study primarily focused on US probe contact in pre-clinical trials, several outstanding issues must be addressed to enable the application of the developed robotic system in clinical settings. In this study, the scanning path was determined manually based on external neck shape information. However, for clinical implementation, it will be necessary to adapt the scanning path in real-time, considering factors like thyroid gland misalignment or shadow artifacts that may degrade the acquired US images. Additionally, adjustments to the scan position must accommodate patient movements, such as breathing and swallowing, which can significantly affect both contact force and image quality. These movements can be detected by monitoring drastic changes in contact force and the position of the target organ within the acquired US image.
To meet these requirements, we aim to integrate scanning path planning based on individual 3D neck surface information and compensation control for sudden patient movements, ultimately achieving autonomous robotic thyroid US imaging without manual intervention. Furthermore, for clinical feasibility assessment, we plan to employ this system for thyroid screening, measuring thyroid size and operation time according to conventional protocols and comparing it with manual scanning.
Regarding the cost of the developed end-effector, there may be more cost-effective configurations available. The force/torque sensor used in this study provided precise contact force measurements but was relatively expensive. If a standardized contact force of approximately 1-2 N can be determined during thyroid scanning, it may be possible to forgo the need for a force feedback system by implementing a mechanism to maintain the contact force within a specific range, such as using a constant force spring.
Conclusion
This manuscript presents a robotic thyroid US system which can maintain the low constant contact force for satisfying the patient safety and acquired image quality. Based on the analysis of the physician's operation of thyroid US scan, we designed the novel end-effector which can generate the arbitral pressure between the US probe and the body surface by adjusting the level of spring compression real-time. The developed system was validated through the phantom and human studies in terms of the patient safety and acquired image quality. The results demonstrated that the developed system enables the safe and stable contact and revealed low contact force of about 1-2 N could be applicable in terms of the patient's comfortability but did not substantiate that the excessive compression affects the clinical image quality. This system can also be used in other US areas besides thyroid, such as abdomen, breast and lung. Our future work will involve developing an autonomous control system that can search for optimal US images while maintaining a minimal contact force and perform the large-scale studies on patients.
ACKNOWLEDGMENT
The authors would like to thank doctors and co-medical staffs in National Cancer Center Hospital East for helping to collect the preliminary data regarding the US scanning.