Representing the Twittersphere: Archiving a representative sample of Twitter data under resource constraints

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

The rising popularity of social media posts, most notably Twitter posts, as a data source for social science research poses significant problems with regard to access to representative, high-quality data for analysis. Cheap, publicly available data such as that obtained from Twitter's public application programming interfaces is often of low quality, while high-quality data is expensive both financially and computationally. Moreover, data is often available only in real-time, making post-hoc analysis difficult or impossible. We propose and test a methodology for inexpensively creating an archive of Twitter data through population sampling, yielding a database that is highly representative of the targeted user population (in this test case, the entire population of Japanese-language Twitter users). Comparing the tweet volume, keywords, and topics found in our sample data set with the ground truth of Twitter's full data feed confirmed a very high degree of representativeness in the sample. We conclude that this approach yields a data set that is suitable for a wide range of post-hoc analyses, while remaining cost effective and accessible to a wide range of researchers.

Original languageEnglish
Pages (from-to)175-184
Number of pages10
JournalInternational Journal of Information Management
Volume48
DOIs
Publication statusPublished - 2019 Oct 1

Keywords

  • Data collection
  • Representativeness
  • Sampling
  • Social media
  • Twitter

ASJC Scopus subject areas

  • Information Systems
  • Computer Networks and Communications
  • Library and Information Sciences

Fingerprint Dive into the research topics of 'Representing the Twittersphere: Archiving a representative sample of Twitter data under resource constraints'. Together they form a unique fingerprint.

  • Cite this