Scale Invariant Feature Transform (SIFT) is well accepted as a robust feature point detection algorithm, which is invariant to rotation, scaling, illumination and viewpoint changes. Though powerful, high computation complexity acts as a bottleneck of the real-time systems. It is not until recently that the only hardware implementation scheme is proposed to reach real-time processing. In this paper, we propose a hardware accelerator structure of the Feature Point Detection part in SIFT which is possible to implement on FPGA. We apply integer-based Variable Pixel Representation which represents a pixel with variable number of registers in different computational stages to reduce redundant register consumption. Also, we introduce Skip Mode Prediction into the system, eliminating redundant computation, so as to shorten averaged computation time per pixel. Our work proves to speed up Max Clock Frequency for 75.0%, lower Register Consumption for 13.6%, and achieve higher Accuracy for 10-20% and Efficiency for 10.4% over conventional work. The proposal is more suitable for realtime system design of SIFT.