Straight lines are important geometric features for aerial image understanding tasks like man-made object detection. As image scene becomes more complex, traditional method like Hough Transform may produce false detections and cannot work efficiently. In this paper, we propose a new multi-scale line detection approach that can efficiently detect semantic lines in aerial image with complex scene. Firstly, a method called "Trichotomy Line Extraction" detects reliable line segments locally. Then multi-scale image system is constructed by wavelet decomposition, from which global information is obtained to detect semantic lines. Experimental results show that proposed method can extract accurate linear features on complex scene aerial images in a robust and efficient way.