Publication detail

Scraping Data from Web Pages using SPARQL Queries

BURGET, R.

Original Title

Scraping Data from Web Pages using SPARQL Queries

Type

article in a collection out of WoS and Scopus

Language

English

Original Abstract

Despite the increasing use of semantic data, plain old HTML web pages often provide a unique interface for accessing data from many domains. To use this data in computer applications or to integrate it with other data sources, it must be extracted from the HTML code. Currently, this is typically done by single-purpose programs called scrapers. For each data source, specific scrapers must be created, which requires a thorough analysis of the source page's implementation in HTML. This makes writing and maintaining a set of scrapers a complex and time-consuming task. In this paper, we present an alternative approach that allows defining scrapers based on visual properties of the presented content instead of the HTML code structure. First, we render the source page and create an RDF graph that describes the visual properties of every piece of the displayed content. Next, we use SPARQL to query the model and extract the data. As we demonstrate with real-world examples, this approach allows us to easily define more robust scrapers that can be used across multiple web sites and that that better cope with changes in the source documents.

Keywords

Web scraping, Page rendering, Data extraction, RDF, SPARQL

Authors

BURGET, R.

Released

16. 6. 2023

Publisher

Springer Nature Switzerland AG

Location

Alicante

ISBN

978-3-031-34443-5

Book

Web Engineering - 23rd International Conference, ICWE 2023

Edition

Lecture Notes in Computer Science

Pages from

293

Pages to

300

Pages count

8

URL

https://link.springer.com/chapter/10.1007/978-3-031-34444-2_21

BibTex

@inproceedings{BUT183806,
  author="Radek {Burget}",
  title="Scraping Data from Web Pages using SPARQL Queries",
  booktitle="Web Engineering - 23rd International Conference, ICWE 2023",
  year="2023",
  series="Lecture Notes in Computer Science",
  pages="293--300",
  publisher="Springer Nature Switzerland AG",
  address="Alicante",
  doi="10.1007/978-3-031-34444-2\{_}21",
  isbn="978-3-031-34443-5",
  url="https://link.springer.com/chapter/10.1007/978-3-031-34444-2_21"
}

Documents

icwe_scraping_draft.pdf