Incensed by a change in Stack Overflow's policy, which effectively leaves moderators unable to combat misleading AI-generated answers, volunteer moderators have gone on strike. While only 13% of the 600 moderators across the whole Stack Exchange network are striking, 15 of the 24 Stack Overflow moderators are participating.

As previously reported Stack Overflow's initial response to the flood of misleading and inaccurate answers being generated by ChatGPT was a ban on its use. This was introduced in December 2022 and subsequently extended to other Stack Exchange sites. In February 2023 the policy across all sites was:

If a user is believed to have used ChatGPT after the posting of this temporary policy, sanctions will be imposed to prevent them from continuing to post such content, even if the posts would otherwise be acceptable.

This policy was rescinded at the end on May with a new policy unilaterally imposed by Stack Exchange staff. According to the moderators, the new policy as implemented on May 29 in private, required:

“an immediate cessation of issuing suspensions for AI-generated content and to stop moderating AI-generated content on that basis alone.”

The following day, a slightly different version of the policy, with the status of an Official Post was released to the public, without the language requiring moderators to stop restricting all AI content. It defended the new policy on the basis of having conducted research into how the ban on AI-generated content was being implemented:

We recently performed a set of analyses on the current approach to AI-generated content moderation. The conclusions of these analyses strongly indicate to us that AI-generated content is not being properly identified across the network, and that the potential for false-positives is very high. Through no fault of moderators' own, we also suspect that there have been biases for or against residents of specific countries as a potential result of the heuristics being applied to these posts. Finally, internal evidence strongly suggests that the overapplication of suspensions for AI-generated content may be turning away a large number of legitimate contributors to the site.

It is both the new policy and the manner in which it has been introduced with no room for discussion or compromise that has led to strike action in which community members will refrain from moderating and curating content, including casting flags, and critical community-driven anti-spam and quality control infrastructure will be shut down. This will have a detrimental impact on Stack Overflow, but does seem to be necessary in order to avoid the even bigger detrimental effect of Stack Overflow's latest policy concerning AI-generated answers.

The principal rationale for the strike was outlined in an Open Letter sent on June 5, 2023 to Stack Exchange: 

Stack Overflow, Inc. has decreed a near-total prohibition on moderating AI-generated content in the wake of a flood of such content being posted to and subsequently removed from the Stack Exchange network, tacitly allowing the proliferation of incorrect information ("hallucinations") and unfettered plagiarism on the Stack Exchange network. This poses a major threat to the integrity and trustworthiness of the platform and its content.

The letter goes on to detail two complaints. The firstly is the new policy itself:

Specifically, moderators are no longer allowed to remove AI-generated answers on the basis of being AI-generated, outside of exceedingly narrow circumstances. This results in effectively permitting nearly all AI-generated answers to be freely posted, regardless of established community consensus on such content.

The moderator's second complaint is that:

details of the policies issued directly to moderators differ substantially from the guidelines outlined publicly, with moderators barred from publicly sharing the details.

The letter concludes:

Until Stack Overflow, Inc. retracts this policy change to a degree that addresses the concerns of the moderators, and allows moderators to effectively enforce established policies against AI-generated answers, we are calling for a general moderation strike, as a last-resort effort to protect the Stack Exchange platform and users from a total loss in value. We would also like to remind Stack Overflow, Inc. that a network that entirely relies on volunteers for its moderation model cannot then consistently ignore, mistreat, and malign those same volunteers.

A post about the strike on Meta Stack Exchange has the title "Moderation Strike: Stack Overflow, Inc. cannot consistently ignore, mistreat, and malign its volunteers" which echos the letter's final sentence and makes explict that the  signatories to the Open Letter have wider concerns than the AI policy:

We deserve much more than just retracting the AI policy. Stack Exchange already made promises after the 2019 debacle that they have since failed to meet. We are worried that Stack Exchange will continue down the same path once the situation calms down.

UPDATE: 

Stack Overflow has reached out to us with a statement from Philippe Beaudette, Vice President of Community at Stack Overflow who confirms that the strike action is in response to the company's position on detection tools regarding AI-generated content and explains:

Stack Overflow ran an analysis and the ChatGPT detection tools that moderators were previously using have an alarmingly high rate of false positives. Usage of these tools correlated to a dramatic upswing in suspensions of users with little or no prior content contributions; people with original questions and answers were summarily suspended from participating on the platform. These unnecessary suspensions and their outsize impact on new users run counter to our mission and have a negative impact on our community.

Beaudette continues:

We stand by our decision to require that moderators stop using the tools previously used. We will continue to look for alternatives and are committed to rapid testing of those tools.

He concludes: 

Our moderators have served this community for many years, and we appreciate their collective decades of service. We are confident that we will find a path forward. We regret that actions have progressed to this point, and the Community Management team is evaluating the current situation as we work hard to stabilize things in the short term.

These concilliatory sentiments, together with the fact that the post about the strike includes the words:

we do not think that our relationship is beyond repair

suggest that there may be room to resolve the situation.

Personally, and looking to the longer term, I am optimistic that combining AI with Stack Overflow's store of wisdom, could make it even more useful to developers than it is at present. Meanwhile there is a huge risk from AI's ability to provide misinformation attributed to non-existent but plausible sources in such a convincing manner that users are hoodwinked and tempted not to double check reality. On the other hand looking for AI-powered assistance in improving the syntax and grammer of poorly constructed English sentences by non-native speaker would seem to be perfectly acceptable.   

More Information

May 30 - Policy Regarding AI-Generated Content

Moderation Strike: Stack Overflow, Inc. cannot consistently ignore, mistreat, and malign its volunteers

Open letter to Stack Overflow

Related Articles

Runaway Success Of ChatGPT

Stack Overflow Traffic Slumps As Devs Turn to ChatGTP

Chat GPT 4 - Still Not Telling The Whole Truth

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info