Real-Time Segmentation of Surgical Tools and Needle Using a Mobile-U-Net

Publikation: Kapitel i bog/rapport/konference-proceedingKonferencebidrag i proceedingsForskningpeer review


This paper presents a CNN-based method for markerless segmentation of surgical tools and suture needles for surgical task automation. The proposed CNN, which we refer to as Mobile-U-Net, is based on the classic U-net encoder-decoder architecture and uses a lightweight MobileNet encoder backbone. This enables the network to perform real-time segmentation at ∼36 and ∼90 frames per second on 672×1120 and 448×448 pixel images. The network is equipped with a multi-kernel output segmentation layer with three k×k×N softmax kernels with k = 1, 3, and 5. On a proprietary dataset from a surgical robot laboratory setup, the multi-kernel Mobile-U-Net achieves intersection over union scores of 0.9495 for surgical tool shafts, 0.8631 for tool end-effectors, 0.9350 for phantom tissue suture pad, 0.8531 for marked needle insertion points, and 0.7524 for suture needles. The method is validated on a second set of images achieving intersection over union scores of 0.9515, 0.8225, and 0.5638 for tool shafts, end-effectors and suture needles. Using multiple kernels improve needle segmentation by 3.11% and 7.00% on the two datasets compared to the baseline of using a single 1×1×N filter and 10.97% and 1.72% for overall mean intersection over union.
Titel2021 20th International Conference on Advanced Robotics (ICAR)
Publikationsdatodec. 2021
ISBN (Elektronisk)978-1-6654-3684-7
StatusUdgivet - dec. 2021
Begivenhed2021 20th International Conference on Advanced Robotics (ICAR) - Congress Centre Cankarjev dom, Ljubljana, Slovenien
Varighed: 7. dec. 202110. dec. 2021
Konferencens nummer: 20


Konference2021 20th International Conference on Advanced Robotics (ICAR)
LokationCongress Centre Cankarjev dom


Dyk ned i forskningsemnerne om 'Real-Time Segmentation of Surgical Tools and Needle Using a Mobile-U-Net'. Sammen danner de et unikt fingeraftryk.