Markov-decision-process-assisted consumer scheduling in a networked smart grid

Zhi Liu, Cheng Zhang, Mianxiong Dong*, Bo Gu, Yusheng Ji, Yoshiaki Tanaka

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

32 Citations (Scopus)


Many recently built residential houses and factories are equipped with facilities for converting energy from green sources, such as solar energy, into electricity. Electricity consumers may input the extra electricity that they do not consume into the smart grid for sale, which is allowed by law in countries such as Japan. To reduce peak-time electricity usage, time-varying pricing schemes are usually adopted in smart grids, for both the electricity sold to consumers and the electricity purchased from consumers. Thanks to the development of cyber-physical systems and advanced technologies for communication and computation, current smart grids are typically networked, and it is possible to integrate information such as weather forecasts into such a networked smart grid. Thus, we can predict future levels of electricity generation (e.g., the energy from solar and wind sources, whose generation is predominantly affected by the weather) with high accuracy using this information and historical data. The key problem for consumers then becomes how to schedule their purchases from and sales to the networked smart grid to maximize their benefits by jointly considering the current storage status, time-varying pricing, and future electricity consumption and generation. This problem is non-trivial and is vitally important for improving smart grid utilization and attracting consumer investment in new energy generation systems, among other purposes. In this paper, we target such a networked smart grid system, in which future electricity generation is predicted with reasonable accuracy based on weather forecasts. We schedule consumers' behaviors using a Markov decision process model to optimize the consumers' net benefits. The results of extensive simulations show that the proposed scheme significantly outperforms the baseline competing scheme.

Original languageEnglish
Article number7739991
Pages (from-to)2448-2458
Number of pages11
JournalIEEE Access
Publication statusPublished - 2017
Externally publishedYes


  • Markov decision process
  • Smart grids
  • scheduling algorithms
  • time-varying systems

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)
  • Electrical and Electronic Engineering


Dive into the research topics of 'Markov-decision-process-assisted consumer scheduling in a networked smart grid'. Together they form a unique fingerprint.

Cite this