3D surface reconstruction based on image stitching from gastric endoscopic video sequence

Mengyao Duan, Rong Xu, Jun Ohya

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    1 Citation (Scopus)

    Abstract

    This paper proposes a method for reconstructing 3D detailed structures of internal organs such as gastric wall from endoscopic video sequences. The proposed method consists of the four major steps: Feature-point-based 3D reconstruction, 3D point cloud stitching, dense point cloud creation and Poisson surface reconstruction. Before the first step, we partition one video sequence into groups, where each group consists of two successive frames (image pairs), and each pair in each group contains one overlapping part, which is used as a stitching region. Fist, the 3D point cloud of each group is reconstructed by utilizing structure from motion (SFM). Secondly, a scheme based on SIFT features registers and stitches the obtained 3D point clouds, by estimating the transformation matrix of the overlapping part between different groups with high accuracy and efficiency. Thirdly, we select the most robust SIFT feature points as the seed points, and then obtain the dense point cloud from sparse point cloud via a depth testing method presented by Furukawa. Finally, by utilizing Poisson surface reconstruction, polygonal patches for the internal organs are obtained. Experimental results demonstrate that the proposed method achieves a high accuracy and efficiency for 3D reconstruction of gastric surface from an endoscopic video sequence.

    Original languageEnglish
    Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
    Volume8856
    DOIs
    Publication statusPublished - 2013
    EventApplications of Digital Image Processing XXXVI - San Diego, CA
    Duration: 2013 Aug 262013 Aug 29

    Other

    OtherApplications of Digital Image Processing XXXVI
    CitySan Diego, CA
    Period13/8/2613/8/29

    Fingerprint

    Stitching
    Surface Reconstruction
    Surface reconstruction
    Point Cloud
    3D Reconstruction
    Seed
    Scale Invariant Feature Transform
    Feature Point
    organs
    High Efficiency
    Overlapping
    Siméon Denis Poisson
    High Accuracy
    Testing
    Structure from Motion
    Internal
    Transformation Matrix
    registers
    Patch
    seeds

    Keywords

    • 3D point cloud stitching
    • depth testing
    • Poisson surface reconstruction
    • SIFT
    • Structure-from-motion (SFM)

    ASJC Scopus subject areas

    • Applied Mathematics
    • Computer Science Applications
    • Electrical and Electronic Engineering
    • Electronic, Optical and Magnetic Materials
    • Condensed Matter Physics

    Cite this

    Duan, M., Xu, R., & Ohya, J. (2013). 3D surface reconstruction based on image stitching from gastric endoscopic video sequence. In Proceedings of SPIE - The International Society for Optical Engineering (Vol. 8856). [88561I] https://doi.org/10.1117/12.2022531

    3D surface reconstruction based on image stitching from gastric endoscopic video sequence. / Duan, Mengyao; Xu, Rong; Ohya, Jun.

    Proceedings of SPIE - The International Society for Optical Engineering. Vol. 8856 2013. 88561I.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Duan, M, Xu, R & Ohya, J 2013, 3D surface reconstruction based on image stitching from gastric endoscopic video sequence. in Proceedings of SPIE - The International Society for Optical Engineering. vol. 8856, 88561I, Applications of Digital Image Processing XXXVI, San Diego, CA, 13/8/26. https://doi.org/10.1117/12.2022531
    Duan M, Xu R, Ohya J. 3D surface reconstruction based on image stitching from gastric endoscopic video sequence. In Proceedings of SPIE - The International Society for Optical Engineering. Vol. 8856. 2013. 88561I https://doi.org/10.1117/12.2022531
    Duan, Mengyao ; Xu, Rong ; Ohya, Jun. / 3D surface reconstruction based on image stitching from gastric endoscopic video sequence. Proceedings of SPIE - The International Society for Optical Engineering. Vol. 8856 2013.
    @inproceedings{db580737684043fa8e449e98b87494fd,
    title = "3D surface reconstruction based on image stitching from gastric endoscopic video sequence",
    abstract = "This paper proposes a method for reconstructing 3D detailed structures of internal organs such as gastric wall from endoscopic video sequences. The proposed method consists of the four major steps: Feature-point-based 3D reconstruction, 3D point cloud stitching, dense point cloud creation and Poisson surface reconstruction. Before the first step, we partition one video sequence into groups, where each group consists of two successive frames (image pairs), and each pair in each group contains one overlapping part, which is used as a stitching region. Fist, the 3D point cloud of each group is reconstructed by utilizing structure from motion (SFM). Secondly, a scheme based on SIFT features registers and stitches the obtained 3D point clouds, by estimating the transformation matrix of the overlapping part between different groups with high accuracy and efficiency. Thirdly, we select the most robust SIFT feature points as the seed points, and then obtain the dense point cloud from sparse point cloud via a depth testing method presented by Furukawa. Finally, by utilizing Poisson surface reconstruction, polygonal patches for the internal organs are obtained. Experimental results demonstrate that the proposed method achieves a high accuracy and efficiency for 3D reconstruction of gastric surface from an endoscopic video sequence.",
    keywords = "3D point cloud stitching, depth testing, Poisson surface reconstruction, SIFT, Structure-from-motion (SFM)",
    author = "Mengyao Duan and Rong Xu and Jun Ohya",
    year = "2013",
    doi = "10.1117/12.2022531",
    language = "English",
    isbn = "9780819497062",
    volume = "8856",
    booktitle = "Proceedings of SPIE - The International Society for Optical Engineering",

    }

    TY - GEN

    T1 - 3D surface reconstruction based on image stitching from gastric endoscopic video sequence

    AU - Duan, Mengyao

    AU - Xu, Rong

    AU - Ohya, Jun

    PY - 2013

    Y1 - 2013

    N2 - This paper proposes a method for reconstructing 3D detailed structures of internal organs such as gastric wall from endoscopic video sequences. The proposed method consists of the four major steps: Feature-point-based 3D reconstruction, 3D point cloud stitching, dense point cloud creation and Poisson surface reconstruction. Before the first step, we partition one video sequence into groups, where each group consists of two successive frames (image pairs), and each pair in each group contains one overlapping part, which is used as a stitching region. Fist, the 3D point cloud of each group is reconstructed by utilizing structure from motion (SFM). Secondly, a scheme based on SIFT features registers and stitches the obtained 3D point clouds, by estimating the transformation matrix of the overlapping part between different groups with high accuracy and efficiency. Thirdly, we select the most robust SIFT feature points as the seed points, and then obtain the dense point cloud from sparse point cloud via a depth testing method presented by Furukawa. Finally, by utilizing Poisson surface reconstruction, polygonal patches for the internal organs are obtained. Experimental results demonstrate that the proposed method achieves a high accuracy and efficiency for 3D reconstruction of gastric surface from an endoscopic video sequence.

    AB - This paper proposes a method for reconstructing 3D detailed structures of internal organs such as gastric wall from endoscopic video sequences. The proposed method consists of the four major steps: Feature-point-based 3D reconstruction, 3D point cloud stitching, dense point cloud creation and Poisson surface reconstruction. Before the first step, we partition one video sequence into groups, where each group consists of two successive frames (image pairs), and each pair in each group contains one overlapping part, which is used as a stitching region. Fist, the 3D point cloud of each group is reconstructed by utilizing structure from motion (SFM). Secondly, a scheme based on SIFT features registers and stitches the obtained 3D point clouds, by estimating the transformation matrix of the overlapping part between different groups with high accuracy and efficiency. Thirdly, we select the most robust SIFT feature points as the seed points, and then obtain the dense point cloud from sparse point cloud via a depth testing method presented by Furukawa. Finally, by utilizing Poisson surface reconstruction, polygonal patches for the internal organs are obtained. Experimental results demonstrate that the proposed method achieves a high accuracy and efficiency for 3D reconstruction of gastric surface from an endoscopic video sequence.

    KW - 3D point cloud stitching

    KW - depth testing

    KW - Poisson surface reconstruction

    KW - SIFT

    KW - Structure-from-motion (SFM)

    UR - http://www.scopus.com/inward/record.url?scp=84886998620&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84886998620&partnerID=8YFLogxK

    U2 - 10.1117/12.2022531

    DO - 10.1117/12.2022531

    M3 - Conference contribution

    AN - SCOPUS:84886998620

    SN - 9780819497062

    VL - 8856

    BT - Proceedings of SPIE - The International Society for Optical Engineering

    ER -