We have developed a novel relative-story displacement sensor capable of measuring the 5-degree-of-freedom movement of building layers for structural health monitoring. Three pairs of infrared-light emitting diode arrays and positionsensitive detector units were used for simultaneously measuring the relative-story displacement, the inclination angle of the lower layer, and the torsion angle between two adjacent layers. For verification, laboratory tests were carried out using a shaking table, a motorized micrometer and a rotation stage. In the static experiment, it is verified that the local inclination angle and the torsion angle can be measured as well as the relative-story displacement using the sensor system. The resolution of the sensor system in the displacement measurement, that in the inclination angle measurement, and that in the torsion angle measurement were evaluated to be 0.10 mm, 34.4 μrad, and 14.6 μrad, respectively. In the dynamic response experiment, the accuracy of the sensor system was experimentally evaluated to be 0.20 mm in the relative-displacement measurement, 110 μrad in the inclination angle measurement, and 90 μrad in the torsion angle measurement, respectively. These results indicate that the developed sensor system has a sufficient accuracy for the structural health diagnostics of buildings.