Data Mining in R: Web Scraping and Text Analysis

Researchers often require access to large datasets from various sources, such as biodiversity databases, environmental monitoring websites, or online repositories. Manually collecting such data is time-intensive and inefficient. Web scraping can automate the extraction of scientific data from public databases, scientific publications, and organisational websites. For instance, one can scrape weather data or geological survey results to analyse trends or share findings with collaborators.

Establishing networks and promoting lifelong scientific learning often require analysing large volumes of text, such as conference abstracts, published papers, or grant announcements, to identify trends, common research interests, or potential collaborators. Text mining and Natural Language Processing (NLP) techniques can process large text datasets to identify key topics or perform Sentiment Analysis to assess public or academic opinions on certain scientific issues.

This course leverages the power of the R programming language, known for its extensive library of over 20,000 packages for statistical analysis, visualisation, text analysis, and machine learning. It focuses on teaching participants how to collect, process, and analyse web data effectively.

Key Highlights:

• Web Scraping Techniques: Learn how to extract data from websites using R packages designed for processing HTML.
• Natural Language Processing (NLP): Gain an introduction to NLP concepts, including text mining, tokenization, and sentiment analysis.
• Hands-On Learning: Engage in practical exercises to scrape review data from online platforms and perform advanced text manipulations.

This course includes a range of activities such as web-scraping demos, live-coding sessions, interactive quizzes, and practical exercises to work individually or in a group. Active participation and contribution are recommended.

In this course we highlight the importance of prioritising the use of official APIs first. Web scraping is covered only when no suitable or dedicated API exists and more specifically for scientific, research and educational purposes. We also cover the areas of responsible web scraping such as respecting terms of service, copyright and server overload.

Places are limited to 15 participants.