The restructuring of today's computer memory hierarchies that are caught in a dilemma between performance gain and power reduction is one of the most promising ways to making the computers much more efficient with much less power. To this end, several possibilities of using NV memories and NV logic with spin-transfer-torque magnetic tunnel junction (STT-MTJ) as levels in new hierarchies are discussed. A new NV-SRAM cell consisting of four transistors and two MTJs (4T-2MTJ) is shown to be a promising candidate for future NV-cache memories. For NV-main memories, we propose a PFET-based 1T-1MTJ cell combined with a new sense amplifier (S/A). A new NV-latch that can be constructed in flip-flops of synchronous core circuits is proposed and the world's fastest 600MHz operation is experimentally demonstrated.