Talks

The Semantic Competence of Large Language Models

Raphaël Millière

Over the past decade, artificial intelligence has made remarkable advancements, largely due to sophisticated neural networks capable of learning from vast amounts of data. One area where these advancements have been particularly evident is in natural language processing, as demonstrated by the impressive capabilities of Large Language Models (LLMs) like GPT-4 and ChatGPT. LLMs possess the astonishing ability to produce well-structured and coherent paragraphs on a wide array of subjects. Moreover, they can perform a diverse set of tasks, such as summarizing extensive articles, translating languages, answering complex questions, solving elementary problems, generating code for rudimentary programs, and even elucidating jokes. As AI becomes increasingly proficient in language-related tasks, it prompts the question of whether these models truly comprehend the language they process. This inquiry has reignited long-standing philosophical debates concerning the nature of language understanding. However, defining 'understanding' in this context and distinguishing it from consciousness remains a challenge. In this presentation, I will propose to focus on the more specific notion of 'semantic competence,' which refers to the ability to comprehend the meanings of words and phrases and to employ them effectively. I will explore two facets of semantic competence: inferential, which pertains to connecting linguistic expressions with other expressions, and referential, which relates expressions to the real world. Drawing from philosophy, linguistics, and recent research on LLMs, I will argue that these models possess considerable inferential competence and even a limited degree of referential competence. In conclusion, I will contemplate the elements still absent from AI models and consider what steps are necessary for them to achieve a level of semantic competence comparable to that of humans.

Encoding & Decoding Natural Language in fMRI

Alex Huth

The meaning, or semantic content, of natural speech is represented in highly specific patterns of brain activity across a large portion of the human cortex. Using recently developed machine learning methods and very large fMRI datasets collected from single subjects, we can construct models that predict brain responses with high accuracy. Interrogating these models enables us to map language selectivity and potentially uncover organizing principles. The same techniques also enable us to construct surprisingly effective decoding models, which predict language stimuli from brain activations recorded using fMRI. Using these models we are able to decode language while subjects imagine telling a story, and while subjects watch silent films with no explicit language content.

Data governance and transparency for Large Language Models: lessons from the BigScience Workshop

Anna Rogers

The continued growth of LLMs and their wide-scale adoption in commercial applications such as chatGPT make it increasingly important to (a) develop ways to source their training data in a more transparent way, and (b) to investigate it, both for research and for ethical issues. This talk will discuss the current state of affairs and some data governance lessons learned from Big Science, an open-source effort to train a multilingual LLM - including an ongoing effort for investigating the 1.6 Tb multilingual ROOTS corpus.