ASTRO-H is the next generation JAXA X-ray satellite, intended to carry instruments with broad energy coverage and exquisite energy resolution. The Soft Gamma-ray Detector (SGD) is one of ASTRO-H instruments and will feature wide energy band (60-600 keV) at a background level 10 times better than the current instruments on orbit. The SGD is complimentary to ASTRO-H's Hard X-ray Imager covering the energy range of 5-80 keV. The SGD achieves low background by combining a Compton camera scheme with a narrow field-of-view active shield where Compton kinematics is utilized to reject backgrounds. The Compton camera in the SGD is realized as a hybrid semiconductor detector system which consists of silicon and CdTe (cadmium telluride) sensors. Good energy resolution is afforded by semiconductor sensors, and it results in good background rejection capability due to better constraints on Compton kinematics. Utilization of Compton kinematics also makes the SGD sensitive to the gamma-ray polarization, opening up a new window to study properties of gamma-ray emission processes. In this paper, we will present the detailed design of the SGD and the results of the final prototype developments and evaluations. Moreover, we will also present expected performance based on the measurements with prototypes.