With the increase of video applications, the security of video data becomes more and more important. In this paper, we propose a new video encryption scheme for H.264/AVC video coding standard. We define Unequal Secure Encryption (USE) as an approach that applies different cryptographic algorithms (with different security strength) to different partitions of video data. The USE scheme includes two parts: video data classification and unequal secure video data encryption. For data classification, we propose 3 data classification methods and define 5 security levels in our scheme. For encryption, we propose a new stream cipher algorithm FLEX and XOR method to reduce computational cost. In this way, our scheme can achieve both high security and low computational cost. The experimental results show that the computational cost of the USE scheme is very low. In security level 0, the computational cost is about 18% of naive encryption. The USE scheme is very suitable for high security and low cost video encryption systems.