SIFT is regarded as one of the most powerful feature point detection algorithms in the world. The feature point detection part, allocating final positions of all feature points, majorly defines the accuracy and stability of the whole system. In this paper, we propose an FPGA-implementable hardware accelerator for this part. By introducing dual-pixel processing and the 3-stage-interpolation pipelined architecture with use of dual-port DDR2 memory access, we achieve to further improve process speed, meanwhile keeping high accuracy. By experiment, our system proves to reach Max Clock Frequency of 145.0 MHz, processing up to 40 VGA images including memory operations. Compared with conventional work, hardware cost is slightly increased as trade-off for accelerated speed. High efficiency as 98.72% and high cover rate as 92.85% is kept by our proposal. Our proposal is suitable as a real-time SIFT system structure.