Person re-identification is a task that aims to recognize a person of interest between two non-overlapping cameras. However, because of different camera angles and quality of the camera, person re-identification is becoming challenging task until now. Many existing methods have been proposed to solve these kinds of problems. Metric learning is already becoming a hot research topic especially for the effectiveness in matching person images problem. For metric learning process, most of them utilize all sample pairs without considering the ratio between distance of each pair. However, not all pairs are useful for the training process. We consider that there are some outliers that can not give good effect in the learning process. For example, a distance too far or too close can influence other samples and mislead the training process, resulting in longer training process time, less accuracy, and other bad effects. In this paper, we propose a new method based on eliminating the outliers, which is called Outlier Sample Elimination. Our method divides negative pairs into three groups using some thresholds to find the proper sample for learning process. During the learning process, we only use the best sample pairs to take into account in our loss function. We eliminate other samples that are considered as outliers.We try to evaluate our proposed method using the challenging VIPeR dataset. Our experiment shows that our method achieves a competitive performance.