Study of Temporal Modified-RANSAC based method for the extraction and 3D shape reconstruction of moving objects from dynamic stereo images and for estimating the camera pose

Naotomo Tatematsu, Jun Ohya

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    3 Citations (Scopus)

    Abstract

    This paper proposes a Temporal Modified-RANSAC based method that can discriminate each moving object from the still background in the stereo video sequences acquired by moving stereo cameras, can compute the stereo cameras' egomotion, and can reconstruct the 3D structure of each moving object and the background. We compute 3D optical flows from the depth map and results of tracking feature points. We define "3D flow region" as a set of connected pixels whose 3D optical flows have a common rotation matrix and translation vector. Our Temporal Modified-RANSAC segments the detected 3D optical flows into 3D flow regions and computes the rotation matrix and translation vector for each 3D flow region. As opposed to the conventional Modified-RANSAC for only two frames, The Temporal Modified-RANSAC can handle temporal images with arbitrary length by performing the Modified-RANSAC to the set of a 3D flow region that classified in the latest frame and new 3D optical flows detected in the current frame iteratively. Finally, the 3D points computed from the depth map in all the frames are registered using each 3D flow region's matrix to the initial positions in the initial frame so that the 3D structures of the moving objects and still background are reconstructed. Experiments using multiple moving objects and real stereo sequences demonstrate promising results of our proposed method.

    Original languageEnglish
    Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
    Volume7878
    DOIs
    Publication statusPublished - 2011
    EventIntelligent Robots and Computer Vision XXVIII: Algorithms and Techniques - San Francisco, CA
    Duration: 2011 Jan 242011 Jan 25

    Other

    OtherIntelligent Robots and Computer Vision XXVIII: Algorithms and Techniques
    CitySan Francisco, CA
    Period11/1/2411/1/25

    Fingerprint

    RANSAC
    Shape Reconstruction
    3D shape
    Optical flows
    3D Reconstruction
    Moving Objects
    Optical Flow
    estimating
    Camera
    Cameras
    cameras
    Rotation matrix
    Depth Map
    matrices
    Feature Point
    Pixels
    Pixel
    pixels
    Arbitrary
    Experiments

    Keywords

    • 3d-reconstruction
    • Detect multiple moving objects
    • Egomotion
    • Temporal modified-RANSAC

    ASJC Scopus subject areas

    • Applied Mathematics
    • Computer Science Applications
    • Electrical and Electronic Engineering
    • Electronic, Optical and Magnetic Materials
    • Condensed Matter Physics

    Cite this

    Study of Temporal Modified-RANSAC based method for the extraction and 3D shape reconstruction of moving objects from dynamic stereo images and for estimating the camera pose. / Tatematsu, Naotomo; Ohya, Jun.

    Proceedings of SPIE - The International Society for Optical Engineering. Vol. 7878 2011. 78780E.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Tatematsu, N & Ohya, J 2011, Study of Temporal Modified-RANSAC based method for the extraction and 3D shape reconstruction of moving objects from dynamic stereo images and for estimating the camera pose. in Proceedings of SPIE - The International Society for Optical Engineering. vol. 7878, 78780E, Intelligent Robots and Computer Vision XXVIII: Algorithms and Techniques, San Francisco, CA, 11/1/24. https://doi.org/10.1117/12.872277
    @inproceedings{cb4dbb0e63c0418282ddc04f512af2bb,
    title = "Study of Temporal Modified-RANSAC based method for the extraction and 3D shape reconstruction of moving objects from dynamic stereo images and for estimating the camera pose",
    abstract = "This paper proposes a Temporal Modified-RANSAC based method that can discriminate each moving object from the still background in the stereo video sequences acquired by moving stereo cameras, can compute the stereo cameras' egomotion, and can reconstruct the 3D structure of each moving object and the background. We compute 3D optical flows from the depth map and results of tracking feature points. We define {"}3D flow region{"} as a set of connected pixels whose 3D optical flows have a common rotation matrix and translation vector. Our Temporal Modified-RANSAC segments the detected 3D optical flows into 3D flow regions and computes the rotation matrix and translation vector for each 3D flow region. As opposed to the conventional Modified-RANSAC for only two frames, The Temporal Modified-RANSAC can handle temporal images with arbitrary length by performing the Modified-RANSAC to the set of a 3D flow region that classified in the latest frame and new 3D optical flows detected in the current frame iteratively. Finally, the 3D points computed from the depth map in all the frames are registered using each 3D flow region's matrix to the initial positions in the initial frame so that the 3D structures of the moving objects and still background are reconstructed. Experiments using multiple moving objects and real stereo sequences demonstrate promising results of our proposed method.",
    keywords = "3d-reconstruction, Detect multiple moving objects, Egomotion, Temporal modified-RANSAC",
    author = "Naotomo Tatematsu and Jun Ohya",
    year = "2011",
    doi = "10.1117/12.872277",
    language = "English",
    isbn = "9780819484154",
    volume = "7878",
    booktitle = "Proceedings of SPIE - The International Society for Optical Engineering",

    }

    TY - GEN

    T1 - Study of Temporal Modified-RANSAC based method for the extraction and 3D shape reconstruction of moving objects from dynamic stereo images and for estimating the camera pose

    AU - Tatematsu, Naotomo

    AU - Ohya, Jun

    PY - 2011

    Y1 - 2011

    N2 - This paper proposes a Temporal Modified-RANSAC based method that can discriminate each moving object from the still background in the stereo video sequences acquired by moving stereo cameras, can compute the stereo cameras' egomotion, and can reconstruct the 3D structure of each moving object and the background. We compute 3D optical flows from the depth map and results of tracking feature points. We define "3D flow region" as a set of connected pixels whose 3D optical flows have a common rotation matrix and translation vector. Our Temporal Modified-RANSAC segments the detected 3D optical flows into 3D flow regions and computes the rotation matrix and translation vector for each 3D flow region. As opposed to the conventional Modified-RANSAC for only two frames, The Temporal Modified-RANSAC can handle temporal images with arbitrary length by performing the Modified-RANSAC to the set of a 3D flow region that classified in the latest frame and new 3D optical flows detected in the current frame iteratively. Finally, the 3D points computed from the depth map in all the frames are registered using each 3D flow region's matrix to the initial positions in the initial frame so that the 3D structures of the moving objects and still background are reconstructed. Experiments using multiple moving objects and real stereo sequences demonstrate promising results of our proposed method.

    AB - This paper proposes a Temporal Modified-RANSAC based method that can discriminate each moving object from the still background in the stereo video sequences acquired by moving stereo cameras, can compute the stereo cameras' egomotion, and can reconstruct the 3D structure of each moving object and the background. We compute 3D optical flows from the depth map and results of tracking feature points. We define "3D flow region" as a set of connected pixels whose 3D optical flows have a common rotation matrix and translation vector. Our Temporal Modified-RANSAC segments the detected 3D optical flows into 3D flow regions and computes the rotation matrix and translation vector for each 3D flow region. As opposed to the conventional Modified-RANSAC for only two frames, The Temporal Modified-RANSAC can handle temporal images with arbitrary length by performing the Modified-RANSAC to the set of a 3D flow region that classified in the latest frame and new 3D optical flows detected in the current frame iteratively. Finally, the 3D points computed from the depth map in all the frames are registered using each 3D flow region's matrix to the initial positions in the initial frame so that the 3D structures of the moving objects and still background are reconstructed. Experiments using multiple moving objects and real stereo sequences demonstrate promising results of our proposed method.

    KW - 3d-reconstruction

    KW - Detect multiple moving objects

    KW - Egomotion

    KW - Temporal modified-RANSAC

    UR - http://www.scopus.com/inward/record.url?scp=79951878948&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=79951878948&partnerID=8YFLogxK

    U2 - 10.1117/12.872277

    DO - 10.1117/12.872277

    M3 - Conference contribution

    AN - SCOPUS:79951878948

    SN - 9780819484154

    VL - 7878

    BT - Proceedings of SPIE - The International Society for Optical Engineering

    ER -