This paper proposes a novel computational structure for Sum of Absolute Difference (SAD) in Fast Full Search (FFS) algorithm to ellipsis part of the SAD value dependency. This structure adopts an SAD Accumulating Termination Algorithm (SAD-ATA) and an Early Loop Termination Decision (ELTD). SAD-ATA is used to analyze and determine whether to skip the following operation of successive SAD accumulation or not, and ELTD is for the control and reduction of SAD comparison loops' degree in FFS ME. The Ratio Distortion (RD) based simulation results show that the proposed algorithm achieves an acceleration in ME processing speed by averagely more than 92% and 20% reduction compared with Full Search (FS) and Fast Full Search (FFS) in H.264 JM12.4, respectively. Meanwhile, the impact on video quality is negligible.