Implementation of a Data-Testid Attribute-Based Web Scraping Method for Accommodation Data Extraction from a Dynamic E-Commerce Website (Case Study: Traveloka)

Abstract

With the growing dominance of Online Travel Agent (OTA) platforms in the digital tourism industry, the data presented therein has become a crucial asset for market analysis, competitive intelligence, and academic research. However, automatic data extraction (web scraping) from these modern platforms faces significant challenges due to the use of JavaScript frameworks that generate dynamic HTML structures with non-semantic and frequently changing CSS class names. This study proposes and validates a robust web scraping methodology to address these issues. Using Traveloka's website as a case study, this research leverages the data-testid attribute—a stable marker deliberately implemented by developers for automated testing purposes—as the primary selector for data extraction. The implementation was carried out using the Python programming language, with Selenium for browser automation and Beautiful Soup for HTML parsing. The results demonstrate that this method successfully and consistently extracts accommodation data, including hotel names, locations, rating scores, number of reviews, and pricing structures. The study concludes that utilizing the data-testid attribute offers a superior and more stable alternative to traditional scraping techniques that rely on CSS selectors or DOM structures, thereby providing an effective blueprint for data acquisition from contemporary dynamic web applications.