While visual observation is an ideal method for detailed survey of seafloors, it is still difficult for an Autonomous Underwater Vehicle (AUV) to track rugged terrains at low altitudes of ideal 1 to 2 meters. This paper proposes a real-time navigation architecture for hovering class AUVs equipped with perceptional sensors looking forwards, sideways and downward, to operate under such conditions. The method controls vertical motion while placing almost no restrictions on motion in the horizontal plane, achieving high compatibility with existing navigation scheme. The algorithm is a hybrid of sensor and map based approaches where the vehicle continually updates a local geometrical map of hazard areas using its perceptional measurements while performing conventional auto-altitude tracking. If the vehicle enters any of the hazard areas, sensor based control is invoked with depth as the control reference to elevate the AUV until it passes the hazard area. The method was verified through tank experiments using the AUV Tri-Dog 1. The AUV successfully followed a lawn-mower trajectory over a rugged terrain consisting of knolls and slopes while maintaining an extremely low altitude of 1.2 meters.