Now. I know next to nothing about that subculture and even less about that specific community. But I doubt very much that this individual is motivated by genuine concern over “cultural appropriation”, especially when there is little or no evidence to support their accusation. Far more likely to be a case of clinical-level personality disorder, and has appropriated the rhetoric of “Tumblr social justice” because that’s a known weak spot in some communities’ immune systems. In other words, it’s pure trolling in the original sense of the word.

There are plenty of reasons to criticise the memes of Tumblr-style social justice, and one of those is the way it creates such a rich environment for trolls. It can be hard to tell the different between trolls and zealots with no clue about boundaries, and that allows trolls to thrive.

Trolls are parasites. They suck up attention and energy that could have been used in sharing information or building community, while contributing nothing in return. Their “free speech” comes with too high a price tag for anyone else.

Aside from obvious conclusion that there are some awful people on 4chan, what can the testing community learn from this?

One seems to be that if you carry out testing in a very public space, any testing failures will be very public as well. An artificial intelligence turning into a noxious racist is a pretty spectacular fail in anyone’s books. Given the well-known nature of the bottom half of Twitter, it’s also an all-too-predictable failure; people in my own Twitter feed express very little surprise over what happened. It’s not as if anyone is unaware of the trolls of 4chan and the sorts of things they do.

What they should have done is another question. Tay was a self-learning algorithm that merely repeated the things she’d been told, without any understanding of their social contexts or emotional meanings. She’s like a parrot that overhears too much swearing. It meant that if she fell in with bad company at the start, she’d inevitably go bad.

The most important lesson, perhaps, is that both software designers and testers need to consider evil. Not to be evil, of course, but to think of what evil might do, and how it might be stopped.

It’s a reminder of just how much of the toxicity of internet discussions is the work of a tiny number of people. It’s also a reminder that many of the worst trolls aren’t true believers in a cause, but just delight in causing mayhem and damage for their own entertainment.

Most of those groups accepted Goldberg as one of their own, since he reliably repeated their memes and talking points. Which makes the “Hurr, hurr, my outgroup fell for him” I’m hearing sound a bit hollow. Your own sect probably fell for him too. As I’ve said before, if your rhetoric so predictable that an outsider can fake it without being immediately recognisable, you have a problem.

Has a successful troll ever passed themselves off as a pragmatic, principled moderate? It’s difficult to imagine, because they would involve laying themselves bare and expressing doubts, something that’s orders of magnitude harder to fake than fanaticism.

A few years back, spam threatened to overwhelm the internet. Our email inboxes were getting flooded with fake Viagra and make-money-fast schemes that drowned out legitimate communications. Likewise bot-generated comment spam meant that any blogger that wanted to enable comments either needed to spend vast amounts of time hand-moderating comments or see their comment sections flooded with garbage.

The spammers and their apologists used to say “Just delete it”, and then whined about freedom of speech every time anyone proposed anti-spam solutions.

We didn’t let the spammers win. Instead we built reputation systems like Akismet, and we added Bayesian filtering to our email, and it turned the tide. They weren’t 100% effective, and did generate the occasional false positive, but they have reduced spam to a manageable problem.

Today we’ve got a huge problem with trolls. They reduce the signal-to-noise ratio across so many sites that “don’t read the comments” and “bottom half of the internet” are commonly used phrases. They harass people online to the extent that far too many people with something worthwhile to say end up being hounded off social media.

Trolls can kill productive conversation. “Just ignore them” is equivalent to “Just delete it”.

Dealing with trolls is a hard problem. Trolling is vastly more subjective and context-dependent that spam. Building an equivalent reputation system based upon who’s favourited or blocked blog comments and social media posts won’t be an easy task. Building one that reduces the impact of bad behaviour without creating dangerous echo-chambers may prove even harder. But it can’t be an impossible task either.

I’m not the only one who thinks this:

Dave Rickey writing in Zen of Design

Twitter is a breeding ground for social dysfunction, where you are lulled into a sense of community and comradery because everyone you follow and everyone that follows you are basically in agreement. The only things that can penetrate the bubble are “Outrage Porn” being retweeted into it, and attacks responding to outrage porn that is being passed around other bubbles.

There’s no room for nuance or in-depth discussion, and anyone who makes the mistake of trying will see their lengthy and thoughtful think-piece distilled down to a barely-true (if that) 140 character sound bite that will be used as a new piece of outrage porn.

David Auerbach writing in Slate:

People are accustomed to being irreverent in conversations with friends, but on Twitter, anyone who might take offense is likely to overhear (unless your tweets are protected, but why be on Twitter in that case?). At least you can go on Reddit without having the repugnant Philosophy of Rape subreddit being shoved in your face; Twitter drags everyone down to the bottom. No matter whom you unfollow, mute, or block, someone you do follow will sooner or later draw your attention to an outrage and encourage you to join the condemnation. On Twitter, negativity is viral.

Twitter didn’t used to be like this. I can remember the times when it was the virtual equivalent of the friendly local pub where all your mates hung out and you swapped joles and stories. I remember reading Robert Scoble’s blog post from five years ago claiming Twitter didn’t suffer from the “forum/chatroom problem” because your feed showed only people you’d invited to join the conversation.

We’ve lost that somewhere along the line.

Maybe it was when Twitter gave greater prominence to the notifications tab. Maybe it all went pear-shaped when they introduced the retweet, something Robert Scoble raised as a concern. Or maybe it was just that, like so many other places, Twitter was better in the early days before the rabble arrived, when most people were enthusiastic early adopters.

Twitter at it’s best can still be great fun; I love the rapid-fire exchanges between one particular group of friends who managed a mashup of The Shipping Forecast and Bruce Forsythe’s Generation Game (“…set of matching luggage 4, becoming 5 later…“)

But I can’t help feeling that either we all need to be smarter in our use of Twitter, with a little less “outrage porn”, ot Twitter needs to rethink some aspects of how the service works, so it amplifies the loudest voices a little less.

This is what “privilege” means.

The terrible thing is that this isn’t restricted to internet blowhards. Our government is made up of people like this. As the gap between the rich and everyone else grows ever larger in English-speaking world, we can only expect this to get worse.

First, How to Improve Feminism in 2014 in Vice (Warning! Do not read the comments unless you want to gawp at sexist idiots). Second, and far more important, this very well-written piece On Toxicity and Abuse in Online Activism, talking about online anger, and why the whole social-justice call-out culture has turned toxic.

It’s long, but it’s worth reading the whole thing. A central point is when the zealots coming from within the social justice movement start to look like the trolls from places like 4chan, something has gone horribly wrong.

I’ve always believed that tribalism is a Very Bad Thing, and one aspect of tribalism is when you turn a blind eye to behaviour from within your own “tribe” that’s just as bad as anything from the “enemy camp”. Sadly, good causes sometimes attract horrible people, you only have to look at all the atrocities perpetrated in the name of religion to recognise that.

What particularly disturbs me is the way in which sections of the mainstream media and others in positions of power use the worst of what happens online to condemn all that happens online. One manifestation of this is the way in which the word “troll” has been appropriated by sections of the mainstream and redefined.

The word once had quite a specialised meaning limited to a particular sort of disruptive behaviour, but it has now become a catch-all term to describe any behaviour that some journalists and editors deem inappropriate. Their responses to what they call “trolling” often seem less about combating abuse than reasserting their role as gatekeeper, to restore to themselves the right to decide who gets to speak in public and who doesn’t. It is what US academic Susan Herbst calls “the strategic use of civility”.

I think he makes some good points here. On the one hand, when game designers get death threats for making minor changes to weapon statistics in a game, something is very, very wrong. But that’s  a completely different thing from someone like Suzanne Moore not being able to express rather bigoted comments in a newspaper column without being called out on it.

You only have to mention names like “Jan Moir”, “Brendan O’Neill”, “James Delingpole” or “Julie Birchill” to recognise that some scribblers in the mainstream media are trolls in the original sense of the word, writing link-bait that deliberately pushes people’s buttons in order to get more pageviews for advertisers.

The power of the internet is that it gives the voice to those who don’t have big media soapboxes, and allows the expression of ideas and opinions that are marginalised by those who control the media. The fact that some of those ideas and opinions are bad ones doesn’t change this. We should not let what amounts to an old-fashioned moral panic let those in power take that away.

Commenter EpistocracyNow makes another very good point about the way the word “troll” gets misused to mean “Anyone not on my side”.

… there are also ideologically biased people who viciously pursue “trolls” who forcefully express competing views, but give a pass to genuine trolls or abusers on their own side. It’s a form of dissonance avoidance – if someone is a “troll”, you don’t have to acknowledge the uncomfortable, dissonance-inducing things he or she might be saying.

I’ve seen a lot of that of late, especially in the Great Geek Culture Sexism Wars. I guess it’s inevitable when opposing camps get so entrenched that “Then and Us” trumps “Right and Wrong”.

Think of playground bullying, for example – there’s a massive difference between a child calling another child a dick and a hundred children standing around one child shouting, “You’re a dick!”

To be blunt, Twitter doesn’t scale. It wasn’t designed for people to make tens of thousands of connections, and I’m not entirely convinced that the humans using it were either – not without some strategy to cope with it all.

There isn’t an easy solution, and I hope that Twitter will find away to prevent harassment of individuals without removing the ability of ordinary people to speak truth to power. We should not allow trolls to be used as a stalking horse for much broader restrictions on political dissent. This is especially pertinent once David Cameron and the UK tabloid press jump on the bandwagon. That’s the point where we need to be extra vigilant about the direction in which the bandwagon starts heading.

There are wider social issues as well.

It also runs afoul of the completely ****ed up relationship our society has with celebrity. I’ve lost count of the number of times I’ve seen it argued that people with a decent follower account should be expected to “take it”, as a sort of penalty for being popular.

Which also make me wonder how much reality TV such as The Apprentice or X-Factor, or the cruelty-based nature of some so-called “comedy” (I’m thinking of that Russell Brand prank phone call incident a while ago) feeds the idea that it’s acceptable to be abusive to complete strangers.