This paper presents a novel algorithm for face recognition. Local Quaternion Patters (LQP) is proposed for presenting the feature parts in the face. To keep the spatial feature of the face, an asymmetric similarity measure Weighted Spatially constrained Earth Mover's Distance (WSEMD) is studied for classification. In this step, the source image is partitioned into non overlapping local patches while the destination image is represented as a set of overlapping local patches at different positions and Gaussian Kernel is used. Finally, local and global weighting is applied to get a more accurate classifier. To evaluate the proposed method and its performance, three well-known and challenge face databases - ORL, Yale and FERET are used in our study. The experimental results show that the proposed method has higher accuracy than some other classic methods.