AutoBLG: Automatic URL blacklist generator using search space expansion and filters

Bo Sun, Mitsuaki Akiyama, Takeshi Yagi, Mitsuhiro Hatada, Tatsuya Mori

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    2 Citations (Scopus)

    Abstract

    Modern web users are exposed to a browser security threat called drive-by-download attacks that occur by simply visiting a malicious Uniform Resource Locator (URL) that embeds code to exploit web browser vulnerabilities. Many web users tend to click such URLs without considering the underlying threats. URL blacklists are an effective countermeasure to such browser-Targeted attacks. URLs are frequently updated; therefore, collecting fresh malicious URLs is essential to ensure the effectiveness of a URL blacklist. We propose a framework called automatic blacklist generator (AutoBLG) that automatically identifies new malicious URLs using a given existing URL blacklist. The key idea of AutoBLG is expanding the search space of web pages while reducing the amount of URLs to be analyzed by applying several pre-filters to accelerate the process of generating blacklists. Auto-BLG comprises three primary primitives: URL expansion, URL filtration, and URL verification. Through extensive analysis using a high-performance web client honeypot, we demonstrate that AutoBLG can successfully extract new and previously unknown drive-by-download URLs.

    Original languageEnglish
    Title of host publicationProceedings - IEEE Symposium on Computers and Communications
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages625-631
    Number of pages7
    Volume2016-February
    ISBN (Print)9781467371940
    DOIs
    Publication statusPublished - 2016 Feb 11
    Event20th IEEE Symposium on Computers and Communication, ISCC 2015 - Larnaca, Cyprus
    Duration: 2015 Jul 62015 Jul 9

    Other

    Other20th IEEE Symposium on Computers and Communication, ISCC 2015
    CountryCyprus
    CityLarnaca
    Period15/7/615/7/9

    Fingerprint

    Search Space
    Websites
    Generator
    Filter
    Resources
    Attack
    Honeypot
    Countermeasures
    Vulnerability
    Filtration
    Accelerate
    High Performance
    Tend
    Unknown
    Web browsers
    World Wide Web
    Demonstrate

    Keywords

    • Computers
    • Crawlers
    • Databases
    • Feature extraction
    • IP networks
    • Search engines
    • Uniform resource locators

    ASJC Scopus subject areas

    • Computer Networks and Communications
    • Computer Science Applications
    • Software
    • Mathematics(all)
    • Signal Processing

    Cite this

    Sun, B., Akiyama, M., Yagi, T., Hatada, M., & Mori, T. (2016). AutoBLG: Automatic URL blacklist generator using search space expansion and filters. In Proceedings - IEEE Symposium on Computers and Communications (Vol. 2016-February, pp. 625-631). [7405584] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISCC.2015.7405584

    AutoBLG : Automatic URL blacklist generator using search space expansion and filters. / Sun, Bo; Akiyama, Mitsuaki; Yagi, Takeshi; Hatada, Mitsuhiro; Mori, Tatsuya.

    Proceedings - IEEE Symposium on Computers and Communications. Vol. 2016-February Institute of Electrical and Electronics Engineers Inc., 2016. p. 625-631 7405584.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Sun, B, Akiyama, M, Yagi, T, Hatada, M & Mori, T 2016, AutoBLG: Automatic URL blacklist generator using search space expansion and filters. in Proceedings - IEEE Symposium on Computers and Communications. vol. 2016-February, 7405584, Institute of Electrical and Electronics Engineers Inc., pp. 625-631, 20th IEEE Symposium on Computers and Communication, ISCC 2015, Larnaca, Cyprus, 15/7/6. https://doi.org/10.1109/ISCC.2015.7405584
    Sun B, Akiyama M, Yagi T, Hatada M, Mori T. AutoBLG: Automatic URL blacklist generator using search space expansion and filters. In Proceedings - IEEE Symposium on Computers and Communications. Vol. 2016-February. Institute of Electrical and Electronics Engineers Inc. 2016. p. 625-631. 7405584 https://doi.org/10.1109/ISCC.2015.7405584
    Sun, Bo ; Akiyama, Mitsuaki ; Yagi, Takeshi ; Hatada, Mitsuhiro ; Mori, Tatsuya. / AutoBLG : Automatic URL blacklist generator using search space expansion and filters. Proceedings - IEEE Symposium on Computers and Communications. Vol. 2016-February Institute of Electrical and Electronics Engineers Inc., 2016. pp. 625-631
    @inproceedings{cc7890519b284431bc0f75dd69892428,
    title = "AutoBLG: Automatic URL blacklist generator using search space expansion and filters",
    abstract = "Modern web users are exposed to a browser security threat called drive-by-download attacks that occur by simply visiting a malicious Uniform Resource Locator (URL) that embeds code to exploit web browser vulnerabilities. Many web users tend to click such URLs without considering the underlying threats. URL blacklists are an effective countermeasure to such browser-Targeted attacks. URLs are frequently updated; therefore, collecting fresh malicious URLs is essential to ensure the effectiveness of a URL blacklist. We propose a framework called automatic blacklist generator (AutoBLG) that automatically identifies new malicious URLs using a given existing URL blacklist. The key idea of AutoBLG is expanding the search space of web pages while reducing the amount of URLs to be analyzed by applying several pre-filters to accelerate the process of generating blacklists. Auto-BLG comprises three primary primitives: URL expansion, URL filtration, and URL verification. Through extensive analysis using a high-performance web client honeypot, we demonstrate that AutoBLG can successfully extract new and previously unknown drive-by-download URLs.",
    keywords = "Computers, Crawlers, Databases, Feature extraction, IP networks, Search engines, Uniform resource locators",
    author = "Bo Sun and Mitsuaki Akiyama and Takeshi Yagi and Mitsuhiro Hatada and Tatsuya Mori",
    year = "2016",
    month = "2",
    day = "11",
    doi = "10.1109/ISCC.2015.7405584",
    language = "English",
    isbn = "9781467371940",
    volume = "2016-February",
    pages = "625--631",
    booktitle = "Proceedings - IEEE Symposium on Computers and Communications",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",

    }

    TY - GEN

    T1 - AutoBLG

    T2 - Automatic URL blacklist generator using search space expansion and filters

    AU - Sun, Bo

    AU - Akiyama, Mitsuaki

    AU - Yagi, Takeshi

    AU - Hatada, Mitsuhiro

    AU - Mori, Tatsuya

    PY - 2016/2/11

    Y1 - 2016/2/11

    N2 - Modern web users are exposed to a browser security threat called drive-by-download attacks that occur by simply visiting a malicious Uniform Resource Locator (URL) that embeds code to exploit web browser vulnerabilities. Many web users tend to click such URLs without considering the underlying threats. URL blacklists are an effective countermeasure to such browser-Targeted attacks. URLs are frequently updated; therefore, collecting fresh malicious URLs is essential to ensure the effectiveness of a URL blacklist. We propose a framework called automatic blacklist generator (AutoBLG) that automatically identifies new malicious URLs using a given existing URL blacklist. The key idea of AutoBLG is expanding the search space of web pages while reducing the amount of URLs to be analyzed by applying several pre-filters to accelerate the process of generating blacklists. Auto-BLG comprises three primary primitives: URL expansion, URL filtration, and URL verification. Through extensive analysis using a high-performance web client honeypot, we demonstrate that AutoBLG can successfully extract new and previously unknown drive-by-download URLs.

    AB - Modern web users are exposed to a browser security threat called drive-by-download attacks that occur by simply visiting a malicious Uniform Resource Locator (URL) that embeds code to exploit web browser vulnerabilities. Many web users tend to click such URLs without considering the underlying threats. URL blacklists are an effective countermeasure to such browser-Targeted attacks. URLs are frequently updated; therefore, collecting fresh malicious URLs is essential to ensure the effectiveness of a URL blacklist. We propose a framework called automatic blacklist generator (AutoBLG) that automatically identifies new malicious URLs using a given existing URL blacklist. The key idea of AutoBLG is expanding the search space of web pages while reducing the amount of URLs to be analyzed by applying several pre-filters to accelerate the process of generating blacklists. Auto-BLG comprises three primary primitives: URL expansion, URL filtration, and URL verification. Through extensive analysis using a high-performance web client honeypot, we demonstrate that AutoBLG can successfully extract new and previously unknown drive-by-download URLs.

    KW - Computers

    KW - Crawlers

    KW - Databases

    KW - Feature extraction

    KW - IP networks

    KW - Search engines

    KW - Uniform resource locators

    UR - http://www.scopus.com/inward/record.url?scp=84961918675&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84961918675&partnerID=8YFLogxK

    U2 - 10.1109/ISCC.2015.7405584

    DO - 10.1109/ISCC.2015.7405584

    M3 - Conference contribution

    AN - SCOPUS:84961918675

    SN - 9781467371940

    VL - 2016-February

    SP - 625

    EP - 631

    BT - Proceedings - IEEE Symposium on Computers and Communications

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -