-
摘要:
为实现颅缝早闭手术方案制定的规范化，提出一种结合深度学习、立体视觉和点云处理技术的颅骨切割轨迹生成方法，用于建立切割方案模板库和生成新的病例切割方案。该方法将深度学习应用于颅骨外表面的实例分割中，利用Mask R-CNN对手术区域进行检测和分割，利用简化轮廓提取算法提取切割轨迹，并结合点云处理技术将切割轨迹坐标进行2D-3D映射，实现切割轨迹自动化提取；在此基础上建立典型病例模板库，通过模板匹配方法自动生成新病例的切割方案。实验证明：该轨迹提取方法可以准确高效地检测出颅骨切割轨迹，并将轨迹坐标进行3D映射，点云深度测量误差小于3 mm，达到临床可用标准；模板匹配方法也有效生成新病例的切割轨迹，符合资深医生的手术方案。
Abstract:To standardize the plan generation of craniosynostosis surgery, we propose a skull cutting trajectory generation method which combines deep learning, stereo vision and point cloud processing technology to establish cutting plan template library and generate cutting trajectory for new cases. The proposed method, for the first time, uses the deep learning to segment the external surface of skull. First, it takes the advantage of Mask R-CNN to detect and segment the surgical cutting area. Then, a simplified contour extraction algorithm is explored to extract the surgical cutting trajectory. After that, the point cloud processing technology is used to map the surgical cutting trajectory to three-dimensional coordinates and realize the automatic extraction of cutting trajectory. Finally, a template library with typical cases is established and the cutting plans of new cases are automatically generated by template matching method. The experiment shows that the proposed trajectory extraction method can detect the skull surgical cutting trajectory accurately and efficiently and conduct three-dimensional mapping of trajectory coordinate.The depth measurement error of point cloud is less than 3 mm, which meets the clinical available standard. Using the template matching method, the new case's cutting trajectory can be generated effectively, which conforms to surgical plan of senior doctors.
-
Key words:
- craniosynostosis /
- trajectory extraction /
- deep learning /
- depth camera /
- point cloud processing
-
图 8 冠状缝早闭简化轮廓提取算法提取结果
Figure 8. Extraction result of simplified contour extraction algorithm for coronary craniosynostosis
图 9 矢状缝早闭简化轮廓提取算法提取结果
Figure 9. Extraction result of simplified contour extraction algorithm for sagittal craniosynostosis
图 10 矢状缝早闭OpenCV轮廓提取算法提取结果
Figure 10. Extraction result of OpenCV contour extraction algorithm for sagittal craniosynostosis
图 11 轮廓提取算法简化前后效率对比
Figure 11. Efficiency comparison between original and simplified contour extraction algorithm
图 13 冠状缝早闭手术方案的2D-3D映射效果
Figure 13. 2D-3D mapping result of coronary craniosynostosis surgery plan
图 14 矢状缝早闭手术方案的2D-3D映射效果
Figure 14. 2D-3D mapping result of sagittal craniosynostosis surgery plan
图 15 矢状缝早闭模板病例颅骨外观
Figure 15. Skull appearance of sagittal craniosynostosis template case
图 16 冠状缝早闭模板病例颅骨外观
Figure 16. Skull appearance of coronary craniosynostosis template case
图 17 矢状缝早闭测试病例颅骨点云与模板颅骨点云配准结果
Figure 17. Registration result of skull point cloud of sagittal craniosynostosis testing case with that of templates
图 18 矢状缝早闭测试病例基于模板匹配生成的初步切割方案
Figure 18. Preliminary cutting plan of sagittal craniosynostosis testing case based on template matching
图 19 冠状缝早闭测试病例颅骨点云与模板颅骨点云配准结果
Figure 19. Registration result of skull point cloud of coronary craniosynostosis testing case with that of templates
图 20 冠状缝早闭测试病例基于模板匹配生成的初步切割方案
Figure 20. Preliminary cutting plan of coronary craniosynostosis testing case based on template matching
表 1 模型精度
Table 1. Model precision
模型 backbone 精度/% 召回率/% mAP/% Mask R-CNN ResNet-101 84.44 58.25 99.68 
下载: 导出CSV
-
[1] 沈卫民, 王刚, 吴玉新, 等. 先天性颅缝早闭的颅骨成形——附6例报道[J]. 实用美容整形外科杂志, 2002, 13(4): 180-183. doi: 10.3969/j.issn.1673-7040.2002.04.006SHEN W M, WANG G, WU Y X, et al. Plastic skull for craniosynostosis: A report of 6 cases[J]. Journal of Practical Aesthetic and Plastic Surgery, 2002, 13(4): 180-183(in Chinese). doi: 10.3969/j.issn.1673-7040.2002.04.006 [2] 冯胜之, 张涤生. 先天性颅缝早闭症的治疗[J]. 中华整形烧伤外科杂志, 1995, 11(6): 406-411. https://www.cnki.com.cn/Article/CJFDTOTAL-ZHSA199506001.htmFENG S Z, ZHANG D S. Surgical treatment of craniosynostosis[J]. Chinese Journal of Plastic Surgery, 1995, 11(6): 406-411(in Chinese). https://www.cnki.com.cn/Article/CJFDTOTAL-ZHSA199506001.htm [3] KLEESIEK J, URBAN G, HUBERT A, et al. Deep MRI brain extraction: A 3D convolutional neural network for skull stripping[J]. NeuroImage, 2016, 129: 460-469. doi: 10.1016/j.neuroimage.2016.01.024 [4] CHILAMKURTHY S, GHOSH R, TANAMALA S, et al. Deep learning algorithms for detection of critical findings in head CT scans: A retrospective study[J]. Lancet, 2018, 392(10162): 2388-2396. doi: 10.1016/S0140-6736(18)31645-3 [5] HE K, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 2961-2969. [6] HE K, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 770-778. [7] LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 936-944. [8] REN S, HE K M, GIRSHICKR R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031 [9] GIRSHICK R. Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2015: 1440-1448. [10] ROSENFELD A, KAK A C. Digital picture processing[M]. New York: Academic Press, 1976: 113-116. [11] ROSENFELD A. Connectivity in digital pictures[J]. Journal of the ACM, 1970, 17(1): 146-160. doi: 10.1145/321556.321570 [12] SUZUKI S, BE K. Topological structural analysis of digitized binary images by border following[J]. Computer Vision Graphics and Image Processing, 1985, 30(1): 32-46. doi: 10.1016/0734-189X(85)90016-7 [13] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. doi: 10.1023/B:VISI.0000029664.99615.94 [14] MARR D, POGGIO T. A computational theory of human stereo vision[J]. Proceedings of the Royal Society of London, 1991, 204(1156): 301-328. [15] HEIKO H. Stereo processing by semi-global matching and mutual information[J]. IEEE Transactions on Pattern Analysis and Machine Intelligenc, 2008, 30(2): 328-341. doi: 10.1109/TPAMI.2007.1166 -
下载:
点击查看大图
计量
- 文章访问数: 457
- HTML全文浏览量: 212
- PDF下载量: 80
- 被引次数: 0
