The large-scale introduction of renewable energy resources will cause instability in the power supply. Residential energy management systems will be even more important in the near future. An important function of such systems is visualization of appliance-wise energy consumption; residents will be able to consciously avoid unnecessary consumption behavior. However, visualization requires sensors to measure appliance-wise energy consumption and is generally a costly task. In this paper, an unsupervised method for nonintrusive appliance load monitoring based on a semi-binary non-negative matrix factorization model is proposed. This framework utilizes the total power consumption patterns measured at the circuit breaker panel in a house, and derives disaggregated appliance-wise energy consumption. In the proposed approach, the energy consumption of individual appliances is estimated by considering the appliance-specific variances based on an aggregated energy consumption data set. The authors implement the proposed method and evaluate disaggregation accuracy using real world data sets.