Fractional Motion Estimation (FME) on sub-pixels will occupy almost over 45% of the computation complexity of H.264 encoding process. Therefore a high performance VLSI architecture of FME is described in this paper to achieve the capacity of encoding the high-resolution real-time video stream for HDTV. Our design is improved from an existing work by involving a pipeline strategy in sub-pixel interpolation unit which can avoid the long delay paths in 6-tap ID FIR so as to increase the clock frequency up to 200MHz. Moreover, a 16-pixel search engine is adopted to remove the redundant interpolation area and parallelize the various block size search which can save more than half of the clock cycles in processing a macro block. Our design is implemented with only 189K gates at operating frequency of 200MHz in worst case(285MHz in typical case). It can provide the processing capacity of more than 250K MB/sec which is enough for 1080HD (1920×1088) video streams at frame rate of 30fps. It is a useful Intellectual Property (IP) design for multimedia system.