Scraping Websites with R

Students working

 

In Person  

This intermediate course will teach you how to scrape user-generated content from the internet using R. On the first day, the course will start with a theoretical introduction to web scraping and specific approaches to scraping forums. We will then select two web forums and go through the process of scraping them. On the second day, we will discuss different kinds of websites that present problems for web scraping. One example of these is websites with dynamically generated content. We will select one such website and walk through the process of collecting data from it. 

This is an intermediate-level course. Students must have a basic background in R. This includes, at least the basic data types in R, how to install and load packages, and how to use functions, pipes, and apply/map functions. It will be sufficient for students to have taken the Introduction to R and RStudio course

Those who have registered to take part will receive an email with full details on how to get ready for the course. 

After taking part in this event, you may decide that you need some further help in applying what you have learnt to your research. If so, you can book a Data Surgery meeting with one of our training fellows. 

More details about Data Surgeries. 

If you’re new to this training event format, or to CDCS training events in general, read more on what to expect from CDCS training. Here you will also find details of our cancellation and no-show policy, which applies to this event. 

This training is connected to the Legal and Ethical Issues in Web scraping training taking place on the 02nd of October.   

 

If you're interested in other training on web scraping and text analysis you can have a look at the following: 

 

Return to the Training Homepage to see other available events.

Digital Scholarship Centre

Digital Scholarship Centre, 6th floor

Main Library 

University of Edinburgh 

Edinburgh EH8 9LJ

You might be interested in

Graphic for a workshop titled ‘Text Classification in Practice: From Topic Models to Transformers.’ The background shows handwritten historical letters. A large green ampersand featuring an illustration of Ada Lovelace is placed on the left. The logo of the Centre for Data, Culture & Society (DCS) appears in the top right corner.

Text Classification in Practice: From Topic Models to Transformers

Graphic for a workshop titled ‘Getting Started with Inferential Statistics.’ The background is a black-and-white photograph of people studying in a library with partitioned desks. A large teal ampersand featuring an illustration of Ada Lovelace is placed on the left. The logo of the Centre for Data, Culture & Society (DCS) appears in the top right corner.

Getting Started with Inferential Statistics

an old map of Acotland with the text "Jennifer Smith & Brian Aitken, Project deep Dive"

Who Speaks Scots Where: What Crowdsourcing Reveals

Graphic for a workshop titled ‘Introduction to Geographical Data with QGIS.’ The background shows an old map of the world with detailed illustrations. A large teal ampersand featuring an illustration of Ada Lovelace is placed on the left. The logo of the Centre for Data, Culture & Society (DCS) appears in the top right corner.

Intro to Geographical Data with QGIS

Graphic for a workshop titled ‘Foundations of Webscraping.’ The background is a black-and-white photograph of students working together in a design studio with maps and models. A large teal ampersand featuring an illustration of Ada Lovelace is placed on the left. The logo of the Centre for Data, Culture & Society (DCS) appears in the top right corner.

Collecting Data from the Web: Foundation of Webscraping

Graphic for a workshop titled ‘Getting Started with Descriptive Statistics.’ The background is a black-and-white photograph of people reading and working in a library. A large magenta ampersand featuring an illustration of Ada Lovelace is placed on the left. The logo of the Centre for Data, Culture & Society (DCS) appears in the top right corner.

Getting Started with Descriptive Statistics

Graphic for a workshop titled ‘Data Viscualisation’ The background is a collage of historical printed text with an overlaid image of a wolf. A large green ampersand featuring an illustration of Ada Lovelace is placed on the left. The logo of the Centre for Data, Culture & Society (DCS) appears in the top right corner

Digital Method of the Month: Data Visualisation

Graphic for a workshop titled ‘Using Prompting Efficiently for Research.’ The background shows an aged, torn book page with visible text. A large green ampersand featuring an illustration of Ada Lovelace is placed on the left. The logo of the Centre for Data, Culture & Society (DCS) appears in the top right corner.

Using Prompting Efficiently for Research

Graphic for a workshop titled ‘Using API for Research.’ The background is a black-and-white photograph of people working with printing equipment and patterned sheets. A large magenta ampersand featuring an illustration of Ada Lovelace is placed on the left. The logo of the Centre for Data, Culture & Society (DCS) appears in the top right corner.

Using API for Research