Energy conservation in building fields is one of the key issues in environmental point of view as well as that of industrial, transportation and residential fields. The half of the total energy consumption in a building is occupied by HVAC (Heating, Ventilating and Air Conditioning) systems. In order to realize energy conservation of HVAC system, this paper focuses on its operation optimizations, and a hybrid modeling technique with physical and Just-in-Time (JIT) model is introduced for building thermal load prediction. The load prediction is necessary to optimize operation of HVAC system in real-time. The proposed method is evaluated with real operation data of an existing building, and the improvement of load prediction performance is illustrated.