Compared with previous standards, H.264/AVC adopts variable block size motion estimation (VBSME) and multiple reference frames (MRF) to improve the video quality. So full search motion estimation algorithm (FS) has to consume massive computation, to calculate every search point in the search window for every block type with multiple reference frame. Some mathematical analyses on edge block are presented in this paper. Moreover in most cases, search range for FS is fixed and large, which is not necessary for most cases. An adaptive search range scheme is provided in this paper. In summary, two fast VBSME algorithms are proposed in this paper: 1) subsampling method based on the block edge information is applied 2) adaptive search range based on motion vector information is utilized. Experiments demonstrate that combing our proposed algorithms with UMHS almost saves 92%-98% Integer Motion Estimation (IME) time compared with exhaustive search with only less than 0.5% bitrate increase or 0.04 db PSNR decline.