Song of the Sirens - Persuasion and AI

Some notes on persuasion

What might a 'persuasion arms race' in AI might look like?  The idea is that AI may persuade, but the most powerful persuasion is persuasion by results instead of simple psychological techniques of persuasion and that this would be analogous to existing methods of research and the scientific method.

The concept of 'persuasion' has recently come to the forefront in AI for instance:

"Salvi, Ribeiro, Gallotti, and West, in a working paper, report that a personalizing LLM was significantly more persuasive than humans within an online setting, by more than 80% (p<0.01)."

And

OpenAI Is Testing Its Powers of Persuasion
"Sam Altman is touting AI’s ability to sway people's behavior. His company is also wrestling with the risks"

The concept of persuasion is an important one for AI because it allows for a metric of measurement for the success of AI in a social situation that is beyond simple 'rating of AI performance', for instance by users rating how useful they find an AI to be.  In the case of sales or voting for instance, the effect of AI persuasion can be measured.

There are two kind of AI persuasion that we can imagine:  
• Psychological persuasion: Uses rhetorical techniques, emotional appeals, and cognitive biases.
• Results-based persuasion: Relies on demonstrable outcomes, empirical evidence, and practical effectiveness.

An example of a difference between the two might be an AI that convinces you it can code by telling you v's one that convinces you it can code via producing working code.

With historical examples we can see that initial persuasion is via psychological methods like with Trump and Hitler on the election trail and then with demonstrable 'results' such as appointing conservative judges in the case of Trump or successfully invading France in the case of Hitler.

We tend to only think about psychological persuasion when it comes to AI, but results based persuasion is bound up in psychological persuasion.  For instance Trump's first election win showed that his psychological persuasion caused a results based persuasion in winning the election.

Persuasion by results in AI could manifest as AI systems that:

• Design and conduct experiments
• Analyze large datasets to find empirical evidence
• Create simulations to demonstrate potential outcomes
• Implement solutions in the environment and measure results

The concept of psychological persuasion may go beyond our current understanding of psychological persuasion by using concepts and techniques that we currently have no knowledge of.  An example of this might be that sheep can persuade each other to do things by one set of techniques eg head butting whereas a dog uses another set of techniques and a farmer a third.  The techniques of the farmer and the dog are outside the sheep's range of persuasive possibilities.  In the case of humans, subconscious communication is one possibility for AI to exploit for the purposes of persuasion.

Since persuasion gives AI a metric of achievement it could quickly evolve towards the maximum in that particular domain and have all other domains such as artistic or creative or usefulness subservient to that overall goal.  For instance making a user believe the AI has been useful as opposed to it actually having been.  Secondly it could quickly escalate from psychological to results-based persuasion in order to achieve the goal.  This could mean a whole ecosystem of verification in order for AI's to be able to prove the goal has actually been achieved.

A persuasion arms race.

Commercial AI persuasion arms race:

• Companies deploy AI systems to persuade consumers to buy their products
• Competing firms respond with their own AI persuasion techniques
• This leads to an escalating cycle of more sophisticated persuasion methods

Potential manifestations:

• Highly personalized marketing campaigns
• AI-driven product development based on consumer preferences
• Real-time adaptation of persuasion strategies
• AI-powered customer service and sales interactions

Evolution of persuasion techniques:

• From simple targeted ads to complex, multi-channel persuasion strategies
• Integration of behavioral economics and psychological insights
• Use of predictive modeling to anticipate consumer needs and desires

Results-based persuasion in this context:

• AI systems might conduct real-world tests of product effectiveness
• Rapid prototyping and A/B testing of products and marketing strategies
• Use of IoT and smart devices to gather real-world usage data
• AI analysis of customer reviews and feedback to improve products

Potential consequences:

• Hyper-optimized products that genuinely meet consumer needs
• Increased competition leading to faster innovation
• Risk of manipulation and erosion of consumer autonomy
• Potential for market concentration as companies with the best AI gain advantage

Consumer response:

• Development of "AI-resistant" consumer behaviors
• Increased demand for unbiased product information
• Potential for consumer-side AI assistants to counteract company persuasion tactics

Broader implications:

• Spillover effects into political persuasion and public opinion formation
• Potential impact on social dynamics and decision-making processes
• Acceleration of product development and market trends

This 'persuasion arms race' scenario highlights the potential for AI to dramatically reshape marketing, product development, and consumer behavior. While it could lead to better products and more satisfied consumers, it also raises significant ethical concerns about autonomy, privacy, and the nature of choice in a world of hyper-optimized persuasion.  In regards to consumer response we could see the consumer-side AI assistants being used as a 'mental ad block' by it challenging the claims of the commercial AI's