How Italy's Ban on ChatGPT Revealed That Coders Already Rely On It

The ban created the perfect conditions to test how extensively computer programmers are using generative AI. The answer: probably more than you thought.

The Physics arXiv Blog iconThe Physics arXiv Blog
By The Physics arXiv Blog
Apr 26, 2023 5:45 PMApr 26, 2023 5:48 PM
Back behind view photo of dark skin programmer lady look big monitor check id-address work overtime check debugging system wear specs casual shirt sit table late night office indoors
(Credit:Roman Samborskyi/Shutterstock)

Newsletter

Sign up for our email newsletter for the latest science news
 

On 1 April, the Italian government banned ChatGPT, an artificial intelligence system that generates human-like text and computer code. ChatGPT is trained on massive amounts of data scraped off the internet and the Italian authorities were concerned that this constituted a breach of privacy for those who owned the data. The ban continues as it investigates further.

But this ban has had an unintended consequence. One of the big questions about ChatGPT and other so-called large language models, is how humans are beginning to use them to improve their productivity (or to replace other humans entirely). The potential seems clear but nobody knows how ChatGPT is already filtering into the workplace.

Enter David Kreitmeir and Paul Raschky at Monash University in Australia, who realized that the Italian ban inadvertently created the perfect conditions to find out by looking at work patterns before and afterwards. It turns out, they say, that ChatGPT is not just being used extensively in at least one profession, but that productivity in this industry has come to depend on it.

A Question of Coding

The area of work in question is computer coding. Anecdotal evidence abounds that coders extensively use ChatGPT to develop, write and correct code but evidence beyond this is hard to get.

But Kreitmeir and Raschky realized that if programmers in Italy had begun to rely on ChatGPT, the signs should quickly become apparent when access was switched off. It turns out that the working patterns of coders is recorded in various ways by GitHub, a widely used online software repository that also manages version control.

So they downloaded data from GitHub showing the hourly coding output of 8000 programmers in Italy and in two other European countries from before and after the ban.

The results are something of a surprise. “We find that the ban of ChatGPT decreased output of Italian GitHub users by around 50% in the first two business days after the initiation of the ban,” say Kreitmeir and Raschky. “Output levels returned to normal levels after that.”

The researchers say the return to normal was probably the result of coders finding ways round the ban, which is easy to circumvent using a virtual private network or Tor Relay to access ChatGPT in other countries.

Decline and Fall

Indeed, the researchers noted an immediate increase in Google searches for “virtual private networks” the day after the ban.

“Our findings provide support for the notion that the sizable but short-lived decline in output after the ban is the result of Italian users looking for and successfully finding ways to circumvent the blocked access,” they say.

That’s interesting work suggesting that ChatGPT is much more widely used than anticipated. “We interpret these findings as first evidence that ChatGPT is already actively used for high-skilled tasks in the economy,” conclude Kreitmeir and Raschky.

Given that ChatGPT was launched only in November last year, this may just be the first trickle in a tsunami of evidence to come.


Ref: The Unintended Consequences of Censoring Digital Technology – Evidence from Italy’s ChatGPT Ban : arxiv.org/abs/2304.09339

1 free article left
Want More? Get unlimited access for as low as $1.99/month

Already a subscriber?

Register or Log In

1 free articleSubscribe
Discover Magazine Logo
Want more?

Keep reading for as low as $1.99!

Subscribe

Already a subscriber?

Register or Log In

More From Discover
Recommendations From Our Store
Shop Now
Stay Curious
Join
Our List

Sign up for our weekly science updates.

 
Subscribe
To The Magazine

Save up to 40% off the cover price when you subscribe to Discover magazine.

Copyright © 2024 Kalmbach Media Co.