A policy paper in Science warns that coordinated AI personas can infiltrate online groups, adapt their messages, and create the appearance of grassroots support. The authors propose platform defenses, model safeguards, and an international observatory for AI influence operations.

The threat deserves attention. Cheap, persistent agents make identity forgery and coordinated persuasion easier to run at scale.

The paper also leans heavily on “synthetic consensus,” a phrase that can blur two different questions. A swarm can manipulate what people see. The volume of social posts still gives us weak evidence about what the broader population believes.

Keeping those questions separate leads to better measurement and narrower policy.

Social activity measures attention⌗

Political conversation online comes from a self-selected group. Pew’s work on Twitter users found demographic and behavioral differences from the wider U.S. public. A later Pew analysis of political tweets found that a small share of users produced a large share of political content.

Trending topics and reply counts reveal activity on the platform. They show which ideas attract attention, which groups coordinate effectively, and which accounts post at high volume. Population opinion requires representative sampling and a clear definition of who is being measured.

Polling has known weaknesses, but it attempts that job directly. The AAPOR review of 2024 pre-election polling reported average error of 2.1 percentage points for national presidential polls and 3.0 points for state polls. Social engagement has no comparable error bound because the participants were never sampled to represent the electorate.

An AI swarm can push a topic onto millions of screens and win the competition for attention. Measuring the resulting belief change requires separate research.

AI changes the cost and concealment⌗

Coordinated messaging predates language models. Political parties, advocacy groups, public relations firms, governments, and ordinary activists all organize people around shared language and timing.

AI changes the operating cost. One organizer can maintain many personas, test variations quickly, respond around the clock, and hide the common source behind apparently independent accounts. The swarm can create social proof for a reader who assumes account count reflects person count.

That combination creates several concrete risks:

  • fabricated identities can enter private or semi-private communities;
  • high-volume replies can crowd out real participants;
  • adaptive messages can target different fears across groups;
  • coordinated harassment can become cheap and persistent;
  • operators can obscure the country, organization, or funder behind a campaign.

Each risk concerns behavior, attribution, and reach. A detection system can produce evidence about those properties. It cannot determine whether the promoted political claim is true or how many people would support it after an informed survey.

Foreign operations create a clearer case⌗

A foreign state running concealed personas inside a domestic election presents a defined counter-intelligence problem. The actor, jurisdiction, and strategic incentive all matter. Platforms and governments have a legitimate reason to detect, attribute, and disrupt that operation.

Domestic coordination is harder to classify. Anonymous speech protects whistleblowers, dissidents, and people discussing sensitive subjects. Grassroots movements often repeat shared language and organize bursts of activity. Automated detection can mistake effective human coordination for a synthetic campaign.

Policy needs to preserve that distinction. A label such as “coordinated behavior” should describe the evidence observed: common infrastructure, synchronized activity, shared control, or forged identity. Claims about public support require representative data.

Observatories need limits too⌗

An AI Influence Observatory could give researchers shared methods and faster visibility into cross-platform campaigns. It would also become a gatekeeper whose reports influence moderation, media coverage, and public trust.

That power calls for constraints:

  • publish detection methods and known error rates;
  • separate automated signals from confirmed attribution;
  • provide an appeal and correction process;
  • disclose funding and institutional relationships;
  • restrict conclusions to behavior the evidence supports;
  • preserve access for independent researchers with competing hypotheses.

Proof-of-human systems create another tradeoff. Strong identity checks can reduce account farms while increasing surveillance and excluding people who need anonymity. Platforms should explain which problem each credential solves and what personal data the solution collects.

Use the right instrument⌗

AI swarms make social platforms easier to manipulate and harder to interpret. They can manufacture the appearance of independent participation, dominate attention, and conceal the operator behind a campaign.

Those capabilities strengthen the case for provenance, platform transparency, and careful attribution. They also make engagement counts even less useful as a proxy for public opinion.

Use network analysis to investigate coordination. Use security work to identify operators. Use representative surveys and behavioral research to measure belief. Combining those jobs into a single idea of online consensus gives both the swarm and the detector more power than the evidence supports.

The public square was always a poor polling instrument. AI now makes that measurement error impossible to ignore.