Weirdly Cited

Pigeons learned to tell cancerous from benign breast-tissue slides. On slides they had never seen, they averaged 85% correct.

In a 2015 study, researchers trained pigeons with food rewards to sort images of breast tissue, cancer or not. A bird pecked one of two coloured buttons, one for benign and one for malignant. Over 15 days of training, the birds went from 50%, the level of guessing, to 85% correct. Then they were shown slides they had never seen. They averaged 85% correct on those, and 87% on the slides they already knew, a difference the authors call non-significant.

This is a claim about what the birds did on these slides. The cases were clear-cut ones, and the authors write that they are not proposing pigeons as diagnosticians.

Every number about the study here is from the paper, Levenson, Krupinski, Navarro and Wasserman, PLoS ONE, 2015. We read its full text, its figures and its supplementary files.

Every claim in the Short, and the sentence behind it

C5: Food for the right button

The birds were kept just below their free-feeding weight and earned food for a correct answer. They answered by pecking one of two buttons in different colours, one for benign and one for malignant.

"We provided food reinforcement to pigeon subjects, maintained just below their free-feeding weight, for correctly discriminating ... images of benign or malignant pathology specimens."

Source: Introduction. The ellipsis stands over the paper's own aside, "via two distinctively colored response buttons".

C2: 50% to 85% in 15 days

On the first day the birds were at chance. By the end of training they were at 85% correct.

"correct choice responses levels rising from 50% at the outset (i.e., at chance level) to 85% over 15 days of training"

Source: Results, Experiment 1, full-color histopathology (Fig 6A).

C3: Slides they had never seen

The real test was new slides, mixed in with the familiar ones. The birds did about as well on both.

"averaging 87% and 85% correct on familiar and novel examples, respectively, a non-significant difference"

Source: Results, novel stimulus testing (Fig 7).

C4: How many birds

The tissue experiment used 8 pigeons. Half of them saw the slides in normal colour, and the other half saw them in monochrome. The two 85% figures above are averages for the 4 birds trained on the colour slides.

"Experiment 1 involved a cohort of 8 pigeons (Columba livia); Experiments 2 and 3 each involved 4 pigeons."

"In Experiment 1, 4 pigeons were trained with normal images and the other 4 pigeons were trained with hue- and brightness-balanced monochrome images."

Source: Materials and Methods, subjects and apparatus.

C6: Straightforward cases

The tissue images came from routine cases at a university medical center, and they were clear examples of each kind.

"These images consisted of equal numbers of relatively straightforward benign and malignant examples from routine pathology cases at the University of California Davis Medical Center."

Source: Materials and Methods, standard breast histology stimuli.

C13: Easy for a pathologist

The authors say the same in their Discussion.

"although the examples shown are easily classified by a trained pathologist"

Source: Discussion, histopathology.

C1: Not documented before

The authors describe the birds as stand-in observers of medical images, and say this had not been documented before.

"can serve as promising surrogate observers of medical images, a capability not previously documented."

Source: Abstract.

C12: Tiny calcium specks

In a second experiment, 4 other pigeons looked at crops of mammograms, sorting those with clusters of microcalcifications, tiny calcium specks, from those without. They also went from 50% to over 85%.

"accuracy scores rose significantly from 50% to over 85% with 14 days of training"

Source: Results, Experiment 2 (Fig 11A).

C9: Masses on mammograms

The third experiment was the hardest: mammograms with masses, benign or malignant. Every bird in it failed on new images. The authors read that as rote memorisation of the training images.

"all of the pigeons failed to transfer their performance to novel exemplars, suggesting that their performance was based on rote memorization."

Source: Fig 12 caption, Experiment 3.

C7: The flock vote

In the monochrome group, the 4 birds' answers were pooled into one vote. The flock beat every single bird. Its score, the area under the curve, was 0.99. That is a score, not a percent correct.

"individual bird performance was surpassed by the flock score; indeed, the area under the curve for the flock was 0.99."

Source: Results, full-set monochrome training and flock sourcing (Fig 9).

C8: The Discussion's 99%

The Discussion restates the flock result as a 99% group accuracy level. It also says the birds were shown "the full set of uncompressed images", where the Results say the flock was the monochrome group. We give the 99% only as the paper's own words, with that mismatch beside it.

"accuracy level reached an amazing 99%"

Source: Discussion, histopathology.

C10: Not a role for pigeons

The paper's last paragraph names other animals put forward as diagnosticians, then says this.

"We are not (yet) proposing such a role for pigeons."

Source: Discussion, final paragraph.

C11: What the birds could be for

The authors see the birds' use in testing the quality of medical images, not in diagnosing anyone.

"Although pigeons are unlikely to be called upon to offer clinical diagnostic support"

Source: Introduction. The sentence goes on: "it does seem quite possible that their discriminative abilities may be turned to a useful purpose."

What the study doesn't say

This page reports what trained birds did with research images in a lab. It says nothing about anyone's health, and it is not advice about screening.

The pigeons did not diagnose anyone. The tissue cases were straightforward ones, and the authors are not proposing pigeons for the job.

The 85% figures are averages per bird, for 4 birds. They do not say every pigeon reached 85%.

The pigeons did not learn to read mammograms. They learned the calcium specks, and they failed on new images of masses.

The 99% is not a percent correct. The Results give the flock's area under the curve, 0.99, and the Discussion restates it as 99%.

The source

Levenson RM, Krupinski EA, Navarro VM, Wasserman EA. 2015. Pigeons (Columba livia) as Trainable Observers of Pathology and Radiology Breast Cancer Images. PLoS ONE 10(11):e0141357. doi:10.1371/journal.pone.0141357. Published November 18, 2015. https://pmc.ncbi.nlm.nih.gov/articles/PMC4651348/

The article is open access under a Creative Commons licence, CC BY 4.0:

"This is an open-access article distributed under the terms of the Creative Commons Attribution License"

Its figures may be shown with the credit line Levenson et al. 2015, PLoS ONE, CC BY 4.0. This page shows none of them.

The Short