Real-time segmentation of moving objects in video sequences is a fundamental step for surveillance systems. One of successful methods for complex background is to use a multi-color background model per pixel. However, Common problem for this approach is that it suffers from illumination changing environment, in addition, it is incapable of removing shadows of moving objects. This paper proposed an effective scheme to improve the adaptive background model for each pixel by introducing a background training parameter into every Gaussian model, and region-based scheme is applied to judgment by utilizing both spatial and temporal information. Experimental results will be presented to validate proposed algorithm keep robustness in the situation of illumination changes, shadow can be removed in foreground mask, results shows False Alarm Rate can be reduced from 34.9% to 35.8% while the overlap varies within normal range from 0.4 to 0.6 compared with conventional Gaussian mixture model.