BEGIN:VCALENDAR
VERSION:2.0
PRODID:talks.ox.ac.uk
BEGIN:VEVENT
SUMMARY:Generative Large Language Models: Performance\, Measurement and Bi
 ases - Joan Timoneda (Purdue University)
DTSTART;VALUE=DATE-TIME:20250527T123000
DTEND;VALUE=DATE-TIME:20250527T140000
UID:https://talks.ox.ac.uk/talks/id/6daf44d4-a651-46b2-a8b4-2c66cc555d21/
DESCRIPTION:Zoom: Join Zoom Meeting: https://zoom.us/j/99057170141?pwd=H6j
 ZR72T3cJPLOU8iq5jSWNxz8YbBV.1\nMeeting ID: 990 5717 0141\;   Passcode: 421
 752\n\nAbstract: Generative large language models (LLMs) are increasingly 
 used in the social sciences for data generation and text annotation\, yet 
 concerns remain about their biases and performance. This talk addresses th
 ese issues in two parts. First\, we examine political biases in LLM output
  by analyzing responses to sensitive political questions across languages 
 spoken in politically divergent societies. Focusing on OpenAI’s GPT-3.5 
 and GPT-4\, we find that model outputs are more conservative in languages 
 associated with conservative societies\, and that GPT-4 tends to produce m
 ore left-leaning responses than GPT-3.5. Second\, we evaluate LLM performa
 nce on complex annotation tasks using specialized political science texts.
  We propose a memory-based annotation approach\, where the model retains i
 ts own prior classifications. This method significantly outperforms few-sh
 ot chain-of-thought prompting\, suggesting a new direction for improving L
 LM-based annotation tasks.\nSpeakers:\nJoan Timoneda (Purdue University)
LOCATION:Nuffield College (SCR (A staircase))\, New Road OX1 1NF
TZID:Europe/London
URL:https://talks.ox.ac.uk/talks/id/6daf44d4-a651-46b2-a8b4-2c66cc555d21/
BEGIN:VALARM
ACTION:display
DESCRIPTION:Talk:Generative Large Language Models: Performance\, Measureme
 nt and Biases - Joan Timoneda (Purdue University)
TRIGGER:-PT1H
END:VALARM
END:VEVENT
END:VCALENDAR