Edge gradient based fast inter motion estimation for H.264/AVC

H.264/AVC coding standard processes variable block size (VBS) motion estimation(ME) on multiple reference frames (MRF) to improve the coding gain. However, the computation is also increased in proportion to the product of the reference frame number and the inter modes. Through mathematical analysis, it is illustrated that the motion-compensated prediction error is mainly determined by the edge gradient amplitude. The image block being rich of texture and sharp edges contains much high frequency signal and this nature makes VBS MRF ME essential. The spatial homogeneity of a block is decided by its maximum edge gradient and its threshold for the edge gradient detection, which is 5 times quantization parameter (QP), is derived from rate-distortion theory. For the homogenous block, its ME computation for the futile reference frames and inter modes can be saved. Through evaluating the edge gradient, the block motion speed can be estimated to reduce the search area of integer ME. Based on the homogeneity of one 8×8 block, its Lagrangian Multiplier is also optimized to improve the coding quality. Moreover, we also provide a fast 'skip_mb' detection algorithm based on Hadamard co-efficients and the current QP. Experimental results show that 61.4%-88.9% computation can be saved by our approaches with the competitive coding quality as the exhaustive search algorithm.