Cherry-picking is an extremely common fallacy which, as Wikipedia points out, is "a major problem in public debate". That being the case, it is extremely important to establish what cherry-picking is, and what it is not.

So first of all we need to be clear that cherry-picking is not the same as finding evidence to support your position. There is absolutely nothing wrong with "pointing to individual cases or data that seem to confirm a particular position," and I'll go into some examples of this later. If you claim that merely identifying evidence which supports your position is cherry-picking, then all evidence ever produced for anything would be cherry-picking. You would be undermining the very idea of evidence.

The important distinction with cherry-picking is the second part of the definition from Wikipedia: "while ignoring a significant portion of related cases or data that may contradict that position." The main focus of cherry-picking is not what you say, but what you don't say. It is a fallacy of omission. But it is also important to note that not all omissions will be significant, relevant, or amount to a contradiction.

All cherry-picking involves omission of data, but not all omission of data involves cherry-picking.

One black swan

In Europe during the middle ages, the proposition 'All swans are white' was considered to be more or less an absolute truth. Every swan which had ever been seen had been white. This was so self-evident that the very idea of a black swan was a metaphor for something which could not exist.

Then, in 1697, the Dutch explorer Willem de Vlamingh became the first European to see what we would now call a black swan, whilst sailing around the Western coast of Australia. Thirty years later, two specimens were caught, which proved that they existed; but it was not until 1790 that John Latham scientifically classified black swans as actually being swans. For a hundred years after their first sighting by a European, the proposition 'All swans are white' was so powerful that black swans were not considered to be swans at all, but rather a separate species which looked similar to a swan, but was black.

I mention this partly because it clarifies the what is meant by 'significant' and 'contradiction'. The black swan has, for many years, been the typical illustration of Karl Popper's concept of falsification, which is (essentially) the idea that science progresses through disproof of theories, rather than attempting to prove them. No matter how many white swans you show me as evidence for the proposition 'All swans are white', a single black swan is enough to disprove it.

Of course, we first have to accept that the black swan is actually a swan; but once we agree upon that, a single specimen is significant enough to contradict the theory. Similarly, before the black swan was scientifically recognised as being a swan, its existence was irrelevant to the proposition 'All swans are white'. It wasn't a swan, so it had no bearing on the theory at all.

Significance, then, does not necessarily involve quantity. A single black swan is significant, if it contradicts the theory. Likewise, all of the white swans ever seen by human eyes are insufficient to prove the proposition 'All swans are white'.

Returning to the topic of cherry-picking, what I'm getting at is that how much evidence is presented or omitted is actually irrelevant. A single relevant piece of evidence can be significant enough to contradict a proposition.

At the same time, the relevance and significance of the evidence is determined by the proposition itself. 'All swans are white' posits precisely that every single swan is white; because it is so universal, the proposition can be undermined by a single piece of contradictory evidence.

But what if the proposition were not so universal? What if the proposition were instead, 'Most swans are white'? In that case, a single black swan is insufficient to contradict the theory. In that case, a single black swan is relevant, but insignificant. In order to contradict this particular proposition, we would need 50%+1 black swans. If three out of ten swans are black, the proposition is still true; if six out of ten are black, the proposition is false. In this case, because of the way the proposition is formulated, quantity is important.

We might also make another proposition about swans: 'Black swans exist'. In this case, we only need a single black swan to prove the proposition, no matter how many white swans there are out there. Notice that presenting a black swan in support of this proposition is not cherry-picking. Ignoring all the white swans in the world is absolutely fine, because the proposition says nothing, and implies nothing, about white swans. We might say that the white swans are relevant to the proposition—because they're swans—but no matter how many there are, they are not significant. In this case, only black swans are relevant and significant.

To summarise what I've said so far, significance and relevance are determined by what the proposition is trying to claim, and neither necessarily have anything to do with quantity. In terms of cherry-picking, it is cherry-picking to ignore relevant and significant evidence; but it is not cherry-picking to ignore evidence which is not relevant or not significant. Some further examples:

• It is cherry-picking to say 'All swans are black', and only provide black swans as evidence.
• It is cherry-picking to say 'Most swans are black', and manipulate your evidence so that this appears to be the case.
• It is not cherry-picking to say 'Some swans are black', and only provide black swans as evidence.

As I said at the start, all cherry-picking involves omission of evidence, but not all omission of evidence is cherry-picking. Sometimes, evidence is omitted simply because it is irrelevant. The mere fact that there is evidence which is not being discussed is not enough to justify the charge of cherry-picking. That evidence must first be shown to be relevant and significant, and the only way to do that is to pay careful attention to exactly what the theory is claiming.

If all you're interested in is the theoretical part, that's over now. I'm going to turn to something a bit more controversial in the next section.

Feminism

If you've been wondering what this has to do with feminism, I'll explain. Ever since Anita Sarkeesian released her first video on Tropes vs. Women in Video Games, one of the most common criticisms of her work is that she cherry-picks examples of sexism in games which support her argument. She thinks video games are sexist; she looks for, and finds evidence of this; but she only does so because she is cherry-picking, and ignoring all of the games which don't support her argument. I want to discuss this in the rest of this article.

As I said above, not all omissions of evidence are cherry-picking. To determine whether someone is cherry-picking, we need to look closely at exactly what the proposotion they are trying to support is, and judge on a case-by-case basis. While I don't want to go through every claim Sarkeesian has made (that would entire another article, or perhaps a series), I do want to examine the general criticism.

Note also that this discussion is entirely separate from any of the broader points Sarkeesian makes. Here, at least, I am not interested her in her analysis of the social implications of sexism in video games. The only thing I am addressing in this article is the accusation that Sarkeesian is cherry-picking, and then only in relation to her general propositions.

In the first three videos of the series, Sarkeesian discusses the 'Damsel in Distress' trope, which is the device in which a woman in peril needs to be rescued, usually by a man. The trope is nothing new, having been around since the earliest forms of story-telling, as Sarkeesian points out in her video, and having been written about extensively long before she addressed the issue. As you might expect, there's a Wikipedia page on the subject.

I said earlier that if your proposition is 'Black swans exist', then you are not cherry-picking if you do not mention that white swans also exist. Similarly, if Sarkeesian's propositions is 'The Damsel in Distress trope exists in video games', then she does not need to talk about all the games which do not feature than trope. All she needs to do is point out examples of it existing.

To reiterate, this is entirely separate from any discussion about the trope itself, whether about its sexism, or whether (as I have read elsewhere) it really oppresses men. This trope essentially consists of 'man rescues woman'; all that is needed as evidence for the proposition 'The Damsel in Distress trope exists in video games' is an example of 'man rescues woman' in video games. We'll talk a bit about the other stuff later.

Let's look at the three examples I gave earlier in the context of the Damsel in Distress trope.

• It is cherry-picking to say 'All video games use the Damsel in Distress trope', and only mention games which do.
• It is cherry-picking to say 'Most video games use the Damsel in Distress trope', and manipulate your evidence so that this appears to be the case.
• It is not cherry-picking to say 'Some video games use the Damsel in Distress trope', and only mention games which do.

To clarify, if Sarkeesian is only making the claim that some video games use the Damsel in Distress trope, she does not need to mention games which do not, and is not cherry-picking if she does not.

Sexism

The same point can be made of accusations of sexism in video games, as follows:

• It is cherry-picking to say 'All video games use sexist tropes', and only mention games which do.
• It is cherry-picking to say 'Most video games use sexist tropes', and manipulate your evidence so that this appears to be the case.
• It is not cherry-picking to say 'Some video games use sexist tropes', and only mention games which do.

Similarly,

• It is cherry-picking to say 'All video games are sexist', and only mention games which are sexist.
• It is cherry-picking to say 'Most video games are sexist', and manipulate your evidence so that this appears to be the case.
• It is not cherry-picking to say 'Some video games are sexist', and only mention games which are.

In other words, Sarkeesian can claim that some games use the Damsel in Distress trope, that some games use sexist tropes, and that some games are sexist, by only mentioning games which support those claims. She is not cherry-picking if she does not take account of games which do not support her claims in these cases, because those games are not relevant or significant evidence.

Final Thoughts

Note that the issue of cherry-picking, as I've presented it here, is in some ways an issue of semantics. Propositions about 'All' video games (or all swans) must be verified in relation to all video games (or swans). Propositions about 'Most' video games depend on a majority. Propositions about 'Some' depend only on some. The same is true of terms like 'common'—at which point do you decide that something is common? Does 'common' mean the same as 'all'? Does it mean 'frequent'? And what does 'frequent' mean? If something happens 25% of the time, is that 'frequent'? Most recent US Presidents have been elected on the votes of between a quarter and a third of the population: does that mean they are 'popular'?

This might seem like it is splitting hairs, and it is; but as I hope I've explained, in order to establish whether someone is cherry-picking, it is important to understand exactly what they are claiming. Not what you think they are claiming, but what they are actually claiming.

I also want to be clear that I am not saying that Sarkeesian has never been guilty of cherry-picking. That would take a detailed analysis of every claim she has ever made, which goes beyond the scope of this post. But as far as I understand her general claims to relate to some games only, she is not cherry-picking. You may wish to debate whether, for example, the Damsel in Distress trope is truly sexism, but then the onus is on you to show that it is not (especially in the light of previous research on the topic, which almost exclusively does regard it as a sexist and chauvinist trope). You may wish to debate whether Princess Peach is truly a Damsel in Distress, despite her needing to be rescued so often. But these questions are about the meaning of the trope, rather than whether Sarkeesian's pointing out its existence in certain video games is cherry-picking.

Sharing