Model parameter learning using Kullback–Leibler divergence

Chungwei Lin, Tim K. Marks, Milutin Pajovic, Shinji Watanabe, Chih kuan Tung

Research output: Contribution to journalArticle

Abstract

In this paper, we address the following problem: For a given set of spin configurations whose probability distribution is of the Boltzmann type, how do we determine the model coupling parameters? We demonstrate that directly minimizing the Kullback–Leibler divergence is an efficient method. We test this method against the Ising and XY models on the one-dimensional (1D) and two-dimensional (2D) lattices, and provide two estimators to quantify the model quality. We apply this method to two types of problems. First, we apply it to the real-space renormalization group (RG). We find that the obtained RG flow is sufficiently good for determining the phase boundary (within 1% of the exact result) and the critical point, but not accurate enough for critical exponents. The proposed method provides a simple way to numerically estimate amplitudes of the interactions typically truncated in the real-space RG procedure. Second, we apply this method to the dynamical system composed of self-propelled particles, where we extract the parameter of a statistical model (a generalized XY model) from a dynamical system described by the Viscek model. We are able to obtain reasonable coupling values corresponding to different noise strengths of the Viscek model. Our method is thus able to provide quantitative analysis of dynamical systems composed of self-propelled particles.

Original languageEnglish
Pages (from-to)549-559
Number of pages11
JournalPhysica A: Statistical Mechanics and its Applications
Volume491
DOIs
Publication statusPublished - 2018 Feb 1
Externally publishedYes

Fingerprint

Parameter Learning
Kullback-Leibler Divergence
learning
divergence
Renormalization Group
XY Model
dynamical systems
Dynamical system
Model
Exact Results
Quantitative Analysis
Ludwig Boltzmann
Critical Exponents
Ising Model
Statistical Model
Critical point
estimators
Quantify
Probability Distribution
quantitative analysis

Keywords

  • Kullback–Leibler divergence
  • Model learning
  • Real-space renormalization

ASJC Scopus subject areas

  • Statistics and Probability
  • Condensed Matter Physics

Cite this

Model parameter learning using Kullback–Leibler divergence. / Lin, Chungwei; Marks, Tim K.; Pajovic, Milutin; Watanabe, Shinji; Tung, Chih kuan.

In: Physica A: Statistical Mechanics and its Applications, Vol. 491, 01.02.2018, p. 549-559.

Research output: Contribution to journalArticle

Lin, Chungwei ; Marks, Tim K. ; Pajovic, Milutin ; Watanabe, Shinji ; Tung, Chih kuan. / Model parameter learning using Kullback–Leibler divergence. In: Physica A: Statistical Mechanics and its Applications. 2018 ; Vol. 491. pp. 549-559.
@article{a19780795f0340a08cb29fffeb015456,
title = "Model parameter learning using Kullback–Leibler divergence",
abstract = "In this paper, we address the following problem: For a given set of spin configurations whose probability distribution is of the Boltzmann type, how do we determine the model coupling parameters? We demonstrate that directly minimizing the Kullback–Leibler divergence is an efficient method. We test this method against the Ising and XY models on the one-dimensional (1D) and two-dimensional (2D) lattices, and provide two estimators to quantify the model quality. We apply this method to two types of problems. First, we apply it to the real-space renormalization group (RG). We find that the obtained RG flow is sufficiently good for determining the phase boundary (within 1{\%} of the exact result) and the critical point, but not accurate enough for critical exponents. The proposed method provides a simple way to numerically estimate amplitudes of the interactions typically truncated in the real-space RG procedure. Second, we apply this method to the dynamical system composed of self-propelled particles, where we extract the parameter of a statistical model (a generalized XY model) from a dynamical system described by the Viscek model. We are able to obtain reasonable coupling values corresponding to different noise strengths of the Viscek model. Our method is thus able to provide quantitative analysis of dynamical systems composed of self-propelled particles.",
keywords = "Kullback–Leibler divergence, Model learning, Real-space renormalization",
author = "Chungwei Lin and Marks, {Tim K.} and Milutin Pajovic and Shinji Watanabe and Tung, {Chih kuan}",
year = "2018",
month = "2",
day = "1",
doi = "10.1016/j.physa.2017.09.018",
language = "English",
volume = "491",
pages = "549--559",
journal = "Physica A: Statistical Mechanics and its Applications",
issn = "0378-4371",
publisher = "Elsevier",

}

TY - JOUR

T1 - Model parameter learning using Kullback–Leibler divergence

AU - Lin, Chungwei

AU - Marks, Tim K.

AU - Pajovic, Milutin

AU - Watanabe, Shinji

AU - Tung, Chih kuan

PY - 2018/2/1

Y1 - 2018/2/1

N2 - In this paper, we address the following problem: For a given set of spin configurations whose probability distribution is of the Boltzmann type, how do we determine the model coupling parameters? We demonstrate that directly minimizing the Kullback–Leibler divergence is an efficient method. We test this method against the Ising and XY models on the one-dimensional (1D) and two-dimensional (2D) lattices, and provide two estimators to quantify the model quality. We apply this method to two types of problems. First, we apply it to the real-space renormalization group (RG). We find that the obtained RG flow is sufficiently good for determining the phase boundary (within 1% of the exact result) and the critical point, but not accurate enough for critical exponents. The proposed method provides a simple way to numerically estimate amplitudes of the interactions typically truncated in the real-space RG procedure. Second, we apply this method to the dynamical system composed of self-propelled particles, where we extract the parameter of a statistical model (a generalized XY model) from a dynamical system described by the Viscek model. We are able to obtain reasonable coupling values corresponding to different noise strengths of the Viscek model. Our method is thus able to provide quantitative analysis of dynamical systems composed of self-propelled particles.

AB - In this paper, we address the following problem: For a given set of spin configurations whose probability distribution is of the Boltzmann type, how do we determine the model coupling parameters? We demonstrate that directly minimizing the Kullback–Leibler divergence is an efficient method. We test this method against the Ising and XY models on the one-dimensional (1D) and two-dimensional (2D) lattices, and provide two estimators to quantify the model quality. We apply this method to two types of problems. First, we apply it to the real-space renormalization group (RG). We find that the obtained RG flow is sufficiently good for determining the phase boundary (within 1% of the exact result) and the critical point, but not accurate enough for critical exponents. The proposed method provides a simple way to numerically estimate amplitudes of the interactions typically truncated in the real-space RG procedure. Second, we apply this method to the dynamical system composed of self-propelled particles, where we extract the parameter of a statistical model (a generalized XY model) from a dynamical system described by the Viscek model. We are able to obtain reasonable coupling values corresponding to different noise strengths of the Viscek model. Our method is thus able to provide quantitative analysis of dynamical systems composed of self-propelled particles.

KW - Kullback–Leibler divergence

KW - Model learning

KW - Real-space renormalization

UR - http://www.scopus.com/inward/record.url?scp=85030865372&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85030865372&partnerID=8YFLogxK

U2 - 10.1016/j.physa.2017.09.018

DO - 10.1016/j.physa.2017.09.018

M3 - Article

AN - SCOPUS:85030865372

VL - 491

SP - 549

EP - 559

JO - Physica A: Statistical Mechanics and its Applications

JF - Physica A: Statistical Mechanics and its Applications

SN - 0378-4371

ER -