In this paper, a difference detection algorithm is proposed to reduce the computational complexity and power consumption in surveillance video compression. The content differences of the input video data are automatically detected by analyzing the color and moving correlation features. Macroblocks without content differences are directly distributed to the bitstream writer of the H.264/AVC encoder. Both the computational complexity and the power consumption are significantly reduced by skipping the entire encoding process. An average of over 84% of overall encoding complexity can be reduced. No loss is observed in both of subjective and objective video quality. Without any requirement in changing the encoder hardware, the proposed algorithm provides high adaptability to be integrated into the existing H.264/AVC video encoders.