A QP and partition-size statistic based fuzzy algorithm for fast inter&intra mode decision in video coding

The inter motion estimation and intra motion estimation are most important and widely used prediction techniques in various kinds of video standards. However, inter and intra predictions are rarely considered together since the contents used for reference by these 2 predictions are different. As most of the previous works focus on either inter or intra or reluctantly combining inter/intra together, this paper proposes a fast mode decision algorithms that considers both inter and intra organically, by taking advantage of that both inter and intra have hierarchical partition sizes. Moreover, in contrast to traditional mode decision algorithms that either enable or disable one specific mode, this paper proposes a fuzzy variable called "effort" which can adaptively adjust the computation paid to each inter or intra mode. This "effort" is defined as a fractional value ranges from 0 to 1. In this paper, firstly the correlation between partition-size selection and QP is related, then by describing how to calculate and implement this "effort" the proposed algorithm is present. Experimental result shows that about 42 67% computational complexity can be saved by adopting the proposed fast mode decision algorithm.