There are many encryption algorithms for general data such as text data. But they are unsuitable for the encryption of the video data because of the real time constraint of the video applications. In this paper, a partial scramble scheme is proposed. The main feature of this scheme is making use of the characteristics of a H.264/AVC video. In stead of encrypting all the video data, some important parameters of the H.264/AVC video such as motion vector difference (MVD) and trailing ones (T1) of CAVLC module are scrambled in this scheme. There are 3 scramble levels in this scheme. For different scramble levels, MVDs and T1s of different MBs in a frame are scrambled according to different part of the key stream generated by a stream cipher. The advantage of this scheme is that it adds very small computional overhead to H.264/AVC coding process. Hence it is fast enough to be used for real time video applications. This scheme can be utilized in mobile and entertainment applications.