Some human brain functions are known to be lateralized to the left or right hemisphere. Although the perception of face has been shown to be mediated by the right hemisphere, whether the discrimination of eye gaze is also mediated by the right hemisphere remains uncertain. In the present study, we attempt to investigate whether the discrimination of human gaze direction is different between the left and right hemispheres. Participants performed a discrimination task using face images in which the face image with gaze was briefly displayed either in the left or right visual hemifield. We found that the discrimination performance of direct gaze was superior to that of averted gaze in the right hemifield when the face image without gaze (control stimulus) was presented in the opposite hemifield. However, the discrimination performance of direct and averted gaze was comparable in the left hemifield. Our findings may reflect a humans' implicit expectation that gaze is directed toward them when the perception of gaze is difficult, which stems from relative difficulty in processing gaze in the non-dominant left hemisphere.