OxTalks is Changing
OxTalks will soon move to the new Halo platform and will become 'Oxford Events.' There will be a need for an OxTalks freeze. This was previously planned for Friday 14th November – a new date will be shared as soon as it is available (full details will be available on the Staff Gateway).
In the meantime, the OxTalks site will remain active and events will continue to be published.
If staff have any questions about the Oxford Events launch, please contact halo@digital.ox.ac.uk
Benchmarking Intelligence Better - Ethics in AI Lunchtime Research Seminars
This talk concerns the challenge of evaluating intelligence in artificial systems such as GPT. While contemporary methods provide fine-grained assessments of task performance, they often fail to distinguish genuine intelligence from sophisticated mimicry, leading to familiar, long-standing debates about what to say about “Block Heads” and the Chinese Room. I think that we can make headway on this stubborn issue by thinking more carefully about how various activities come to be performed rather than focusing only on outward performance. In the context of LLMs, this suggests a path towards “deep benchmarking” which requires attending to the mechanisms AI systems use to complete tasks along with careful thinking about the conditions under which those mechanisms underwrite intelligent activity. Once we do that, how do things look with respect to chatbot intelligence? In my view, there are plausible mechanisms present in contemporary LLMs that underwrite intelligent activities, but the activities in question are a good way off from anything like semantic understanding let alone AGI.
Date:
5 February 2025, 12:30
Venue:
Please register to receive venue details
Speaker:
Dr Alex Grzankowski (University of London)
Organiser contact email address:
aiethicsevents@philosophy.ox.ac.uk
Hosts:
Dr Thomas Mitchell (Philosophy, Oxford),
Dr David Storrs-Fox (Philosophy, Oxford)
Part of:
Ethics in AI Lunchtime Seminars
Booking required?:
Required
Booking url:
https://www.oxford-aiethics.ox.ac.uk/benchmarking-intelligence-better
Booking email:
aiethicsevents@philosophy.ox.ac.uk
Cost:
Free
Audience:
Public
Editor:
Marie Watson
