The spacecraft SELENE has discovered a huge vertical hole named Marius Hills Hole on the moon and the possibility of an underground lava tube is being considered. It is expected to find an important scientific trace such as exposed geological structures, shape of rocks, and appearance of lave tubes. Further, the lava tube can be considered for use as a lunar base because the temperature is maintained at a certain level, and the tubes block the ultraviolet and cosmic ray radiation that destroys organic matter. Such vertical holes exist not only on the surface of the moon but also on Mars. Therefore, we developed a small mobile robot system named WAseda PIt exploring roboT (WAPIT) to explore vertical holes. WAPIT is supported by a single tether and can ascend and descend with a winch. The robot can drive on the wall surface and overcome obstacles by jumping over them with the help of two drop-shaped eccentric wheels. As the existing system cannot control the attitude of the robot in its non-grounded suspended state, the exploration can be restricted. To stabilize the rotation at the tether, small vibrations at the winch, and large vibrations at the fulcrum, we adopted a reaction wheel, damper mechanism, and winch control, respectively. This paper reports the overview of WAPIT and stabilization method of the robots along with the experimental results. As this small mobile robot can perform several functions with few mechanisms, it can explore a larger area despite the limited size and weight conditions of the robot.