Using Local Language Models for Research
In person
Large Language Models (LLMs) such as ChatGPT, Claude, and Gemini are powerful AI tools capable of generating text that closely resembles human writing. However, they also have significant limitations. These models are susceptible to hallucinations, meaning they can produce inaccurate or misleading information. Additionally, they often lack access to the most current information and operate behind paywalls, relying on computational resources that are beyond our control. Collectively, these limitations raise important concerns about the veracity of their outputs, the reproducibility of their results, data storage protocols that we cannot oversee, and energy resources that we cannot monitor.
To address some of these limitations and leverage LLMs for research in a responsible manner, we need greater control and oversight of them. To this end, this short course will introduce participants to the ecosystem of open-weight models. It will cover how to run them on your own laptop, how to augment them for small research tasks, and how to link them to external sources of information.
Each session in this course will feature hands-on exercises, so you will need to bring a laptop to the sessions. Do not worry about the capacity of the laptop you bring; part of the course is about learning what you can run on the computational resources you have access to.
This course will be taught by Martin Disley.
After taking part in this event, you may decide that you need some further help in applying what you have learnt to your research. If so, you can book a Data Surgery meeting with one of our training fellows.
More details about Data Surgeries.
Those who have registered to take part will receive an email with full details on how to get ready for this course.
If you’re new to this training event format, or to CDCS training events in general, read more on what to expect from CDCS training. Here you will also find details of our cancellation and no-show policy, which applies to this event.
Level
This workshop requires the following pre-knowledge:
- Basic familiarity with command line interfaces, either Unix (Mac/Linux) or PowerShell (Windows)
- Familiarity with coding with Python (applying functions/importing libraries)
Knowledge refreshment pre-reading suggestions:
Learning Outcomes
- Understand the limitations of LLM knowledge
- Compare and contrast approaches to LLM knowledge integration
- Construct a working Retrieval Augmented Generation (RAG) system
Skills
- Understand the naming conventions and model cards of open weight models
- Run open weight models on your own computer
- Augment these models with system prompts and simple retrieval systems
Explore More Training:
- Silent Disco: Hackers' Guide to Language Models
- Foundations of Machine Learning
- Explainable Machine Learning (XAI)
Return to the Training Homepage to see other available events
Room 4.35, Edinburgh Futures Institute
This room is on Level 4, in the North East side of the building.
When you enter via the level 2 East entrance on Middle Meadow Walk, the room will be on the 4th floor straight ahead.
When you enter via the level 2 North entrance on Lauriston Place underneath the clock tower, the room will be on the 4th floor to your left.
When you enter via the level 0 South entrance on Porters Walk (opposite Tribe Yoga), the room will be on the 4th floor to your right.
