It has been known remembered locations of visual stimuli are systematically distorted in a two-dimensional (i.e., retinal) coordinate, which includes the foveal bias, memory averaging, and landmark effect. The present study aimed at examining how the remembered depth position of a visual target would be distorted. Also, we examined whether depth distortion would differ for perceptual and motor-related tasks. In the experiments, a visual target was presented for one second at various distances from the observers by using a 3D projector. The fixation distance was also varied. After the disappearance of the target, observers performed either perceptual judgment (the method of constant to estimate the point of psychological subjective equality) or motor response (by using a 3D mouse) for the remembered target distance. The results showed that the remembered depth of the target was compressed toward the averaged depth of the possible depth range for both perceptual and motor responses. In addition, however, the perceptual responses tended to overestimate the depth when the fixation depth was closer than or the same as the nearest possible depth of the target. No bias was observed when the fixation depth was farther than the nearest possible depth. For the motor responses, there was a general underestimation irrespective of fixation depth. These results suggest that spatial memory of depth for visual stimuli may be qualitatively different between perceptual and motor responses.