comment image

What gave them away was a sentence anyone who has used a chatbot will recognize: "I can't write a message aimed at insulting a specific political party. I can help draft a neutral summary of the proposal instead." The exact same reply turned up on Instagram, under Italian Deputy Prime Minister Matteo Salvini's post marking the eleventh anniversary of the Bataclan attack, illustrated with a photo of Valeria Solesin, the young Italian woman killed that night in Paris. Italian media reported it on November 14. Similar replies were spotted on the profiles of Carlo Fidanza, of Brothers of Italy, and Dario Nardella, of the Democratic Party. In other cases the wording barely changed: the model declined to write a message "aimed at persuading specific MEPs" to back a legislative proposal.

Who is behind it, and who benefits, is a question for those with the authority to investigate. We are interested in something else. How is a machine like this actually built? And why does that refusal tell us a great deal, yet not the thing that matters most?

No model posts anything on its own

A language model produces text. That is all it does. It has no account, it doesn't open Facebook, it doesn't press "Post". For a generated sentence to end up under someone's post, you need an application running the show: it reads the content to comment on, asks the model for text, receives it and publishes it through an authenticated social media identity.

What the refusal shows us, then, is only the first link in the chain. The second one, the one that leads to whoever runs the network, is the system that controls the accounts.

There is another detail, and it says a lot about the quality of the engineering. In any reasonably well-built pipeline, the model's output is checked before it is used: if it's empty, off-topic or a refusal, it gets thrown away. Here there was no such check. The system took the output and posted it, as it was, from several accounts.

il rifiuto ha svelato il modello, non chi controlla l'account

The official route is closed

For a developer, the easiest path would be Meta's official APIs. But the documentation leaves little room for doubt. Commenting through the Graph API requires a Page access token, the pages_manage_engagement, pages_read_engagement and pages_read_user_engagement permissions, and a user authorized to moderate the Page and create content on it. The comment is published as a Page. Try it as a regular user and you get an error.

The permission that once let apps publish on behalf of personal profiles, publish_actions, was withdrawn from around 60,000 apps on August 1, 2018; the last extensions ran out in 2020. In plain terms: with the official tools, you cannot build a network that comments on other people's posts while impersonating hundreds of users.

On Instagram, where the comments under Salvini's post were captured in screenshots, the picture is the same. The official APIs only work with professional accounts, business or creator, connected to an authorized app. The comments, however, came from personal profiles with generic usernames, the word-plus-number kind. The explanation lies outside the APIs.

Puppeteered browsers and accounts of mixed origin

The most common technique is browser automation. Tools originally built for software testing, such as Playwright, Selenium or Puppeteer, let a program open a page, click, type and submit exactly as a person would. The software works inside an already authenticated session and needs no API permission at all, because as far as the platform is concerned it's just a user browsing.

That leaves the question of identities. Whoever runs a network like this doesn't have to own the accounts, but does have to control them. They may be profiles created for the purpose, bought from third parties, handed over willingly by their owners, or stolen through password or session-cookie theft. That last scenario would explain why profiles with years of history, real photos and real friends might suddenly start commenting in bulk. To be clear: in this case there is, so far, no evidence of compromised accounts.

Meta bans this kind of activity under its rules on inauthentic behavior and runs systems designed to detect it. The fact that identical comments stayed up long enough to be screenshotted says something about how well those filters work.

Why the model refused now

Security expert Alessandro Orlowski, speaking to the Italian news agency Adnkronos, pointed to an API update as the most likely cause: the operators would have been using a more permissive model, later replaced by the provider with a version that has stricter rules. Orlowski also described as weak the link, floated in the first hours, to a Microsoft outage around the same time.

It's a reasonable hypothesis, but not the only one. When a provider retires a model, code that calls it by name usually gets an error back, not a refusal. A refusal shows up if the application uses an alias that always points to the latest version, or if calls are being redirected. Nor can a self-hosted open-source model be ruled out. News outlets mentioned OpenAI and Anthropic as examples, not as established fact.

A prompt written before the post

One inconsistency deserves attention. Salvini's post was about a terrorist attack, not a piece of legislation. Yet the model offers "a neutral summary of the proposal", and in another variant talks about MEPs to be persuaded. That suggests the instructions were written upstream, in a generic prompt template, rather than generated from reading each individual post.

The language is a clue too. A refusal in English, under a post in Italian aimed at an Italian audience, suggests the instructions to the model were written in English, or that the platform used to manage the network was configured in that language. It may point to a generic automation tool, designed to serve several campaigns rather than built specifically for Italy.

The comments appeared on the profiles of politicians from three different parties, so "the party to insult" may well have been a variable in the prompt rather than a fixed target. That remains a hypothesis. Statements by the politicians involved about who was being attacked cannot be verified. To find out whether those accounts took sides, and whose, you would need to analyze the comments they posted in the past. The accounts are still online, so that work is possible, and an analysis of their history would say far more than any statement.

Where the answer lies

If the goal is to understand who runs the network, asking AI providers who generated the text won't get you very far. The right questions are for Meta. Which devices and IP addresses did the comments come from? Did the accounts share infrastructure? Did the sessions show the timing and rhythm of automation? Were the comments posted through the website or through an API? And how many other accounts were linked to the same devices?

Those answers exist, and they can be obtained. Meta logs IP addresses, devices, times and access methods, and hands them over to authorities upon formal request. For users in Europe, the entity to contact is Meta Platforms Ireland, in Dublin. The company says it will preserve an account's data for 90 days in connection with a criminal investigation if it receives a preservation request pending formal legal process. Timing is therefore what matters: if the authorities act quickly, the data needed to trace whoever runs the network can be recovered. A private individual, on the other hand, cannot obtain it, and can only report the accounts and file a complaint.

The refusal message switched on a light over the machine's engine. The steering wheel, the system that controls the social media identities, is still in the dark. And that is where the trail leads: if not to whoever may have commissioned the operation, at least to whoever keeps the machine running.

Sources