Google researchers can create an AI that thinks a lot like you after just a two-hour interview

Researchers at Stanford University paid 1,052 people $60 to read the first two lines of the book The Great Gatsby To the application. Next, the AI, which resembled a 2D sprite from a SNES-era Final Fantasy game, asked participants to tell their life story. Scientists took those interviews and turned them into artificial intelligence that they say replicates participants’ behavior with up to 85% accuracy.

The study is titled Generative agent simulation for 1000 peopleis a joint project between Stanford University and scientists working at Google’s DeepMind AI research lab. The idea is that creating AI agents based on random people can help policymakers and business people better understand the public. Why use focus groups or poll your audience when you can talk to them once, create an LLM based on that conversation, and then get their thoughts and opinions forever? Or, at least, as close as possible to recreating these thoughts and feelings.

“This work provides a basis for new tools that can help investigate individual and group behavior,” the research summary said.

“How, for example, might a diverse group of individuals respond to new public health policies and messages, respond to product launches, or respond to major shocks?” The paper continued. “When simulated individuals are combined into groups, these simulations can aid experimental interventions, develop complex theories that capture nuanced causal and contextual interactions, and expand our understanding of structures such as institutions and networks across fields such as economics, sociology, organizations, and political science.”

All of these possibilities based on a two-hour interview fed into the LLM’s message, which mostly answered questions like its real-life counterparts.

Much of the process has been automated. The researchers contracted with Bovitz, a market research firm, to gather participants. The goal was to obtain a broad sample of the US population, as broad as possible when limited to 1,000 people. To complete the study, users signed up for an account in a dedicated interface, created a 2D avatar, and began talking to an AI.

The questions and interview style are a modified version of those used before American Voices Project, A joint project between Stanford and Princeton University to interview people across the country.

Each interview began with participants reading the first two lines of the story The Great Gatsby (“In my younger and more vulnerable years, my father gave me some advice that I have kept over in my mind ever since. He said to me: ‘Whenever you feel like criticizing anyone, just remember that not everyone in this world has had the advantages that you have had.’ “) as a means of calibrating sound.

According to the paper, “The interview interface displayed a 2D avatar representing the interview agent in the center, with the participant’s avatar at the bottom, walking toward a goal column to indicate progress. When the AI ​​interview agent spoke, it was indicated by A pulsating animation of the central circle with the interviewer’s avatar.

The two-hour interviews produced, on average, transcripts 6,491 words long. Questions were asked about race, gender, politics, income, social media use, stressors they experience in their jobs, and raising their families. The researchers published the transcript of the interview and the questions asked by the artificial intelligence.

Those texts, each of less than 10,000 words, were then fed into another MA that the researchers used to spin generative operators intended to replicate participants. Next, the researchers subjected the participants and the AI ​​clones to more questions and economic games to see how they compared. “When an agent is queried, the entire interview transcript is entered into the claim form, directing the model to mimic the person based on their interview data,” the newspaper said.

This part of the process was as close to controlled as possible. Researchers used General Social Survey (GSS) and The Big Five Personality Inventory (BFI) to test how well LLMs align with their inspiration. She then ran participants and LLM holders through five economic games to see how they compared.

The results were mixed. The AI ​​agents answered about 85% of the questions in the same way as real-world participants answered the GSS. They reached 80% on the BFI. However, the numbers declined when agents started playing economic games. Researchers offered real-life participants cash prizes for playing games such as Prisoner’s dilemma and Dictator game.

In a prisoner’s dilemma, participants can choose to work together and succeed or take down their partner for a chance at a big win. In the dictator game, participants have to choose how to allocate resources to other participants. Real people have been paid more than the original $60 for playing these games.

In the face of these economic games, AI versions of humans haven’t replicated their real-world counterparts either. “On average, the generated factors achieved a normalized correlation of 0.66,” or about 60%.

The entire document is worth reading if you are interested in how academics think about AI agents and the public. It didn’t take long for researchers to summarize the human personality into a similarly behaving LLM. With time and energy, they will likely be able to bring the two closer together.

This is worrying to me. Not because I don’t want to see the indescribable human spirit turned into a spreadsheet, but because I know this kind of technology will be used for evil. We’ve already seen the dumbest public records-trained LLM students trick grannies into giving banking information to an AI relative after a quick phone call. What happens when these machines have a script? What happens when they have access to targeted personas based on social media activity and other publicly available information?

What happens when a company or politician decides that the public wants and needs something based on their spoken will, but on an approximation of it?