More than three hundred representatives of media companies walked into Capitol Hill last month to lobby for the Stealth Bot Prohibition Act. Their complaint is real: AI companies are sucking their sites dry, and the referring visitors stay away.

The solution the bill chooses is strange. Anyone who visits a site automatically must identify themselves and report the purpose of the collection. The law regulates identity and leaves behaviour untouched.

Three misunderstandings

1. Identity says nothing about harm. Under the New York variant, a publisher can get a subpoena to unmask a crawler without any indication of harm. A company that dutifully gives its name and makes ten thousand requests per minute is fine. The journalist who anonymously investigates whether tech companies favour their own products is not.

2. Anonymity is sometimes the method. Anyone who is observed and knows by whom behaves differently. That is why research into discrimination, deception or pricing behaviour must be able to happen anonymously. The Washington Post analysis of 31,000 financial disclosures, or the Internet Archive's work with more than a trillion pages, would not exist without a crawler.

3. The bill lands on the wrong party. The proposal mentions fines of up to $53,000 for behaviour that "reasonably likely" burdens a site. That standard is vague, and that works as a deterrent: archivists, researchers and accessibility researchers drop out before a judge is ever involved.

What would work

The authors point to an obvious alternative: look at the behaviour that causes harm. Too many requests, fraud, collecting data for prohibited purposes: you can set a standard for that and then enforce it.

Meanwhile, at the IETF there are initiatives for standards that let sites manage automated traffic, including anonymous attestations that a bot behaves properly. That is slower than a law, but it addresses the cause.

The core

Behind the debate is the question of whether access to public information requires an ID.

That is an old question. In 1965 the US Supreme Court ruled that you do not have to give your name first to receive a political publication. The principle has remained the same: receiving information is something different from signing in at a counter.

A crawler that misbehaves, you deal with. Someone who does not identify themselves is not thereby the problem.

Source: Breaking the Open Web Won't Stop the Bad Bots — Lila Bailey and Tess Vartanian, TechPolicy.Press.