The crucial role of protocols

October 7th, 2014 | Posted by citizenscience in article | control | crowdscience | protocol - (0 comments)

Several of the most cited articles we examined stress the importance of a functional protocol. Protocols are considered critical for establishing control over the tasks performed by citizen scientists. In fact, scientists setting up citizen science projects are typically concerned with accuracy, reliability and usability of data collected by citizens. How can amateurs collect data that are as good as those generated by professional re- searchers? According to the scientists interviewed by Cohn (2008), amateurs can collect reliable data and help advance scientific knowledge if they are properly trained to use instruments, collect and read data. Furthermore, it is important to design specific protocols that limit the tasks assigned to amateurs, test them and see whether reliable data are collected.

What are protocols, by the way? Bonney et al. (2009) described clearly what a protocol is and what it is for. They tell us that protocols specify when, where, and how data should be gathered. Used in large projects spanning multiple locations, such as, for example, the Seed Preference Test (SPT), which in 1994 attracted more than 17,000 participants of all ages and birding abilities (Trumbull et al., 2000), protocols “define a formal design or action plan for data collection” (p. 980), which allows observation made by many independent amateurs to be combined and used for analysis. These protocols should be clear, easy to use, and engaging for volunteer participants. Bonney et al. (2009) described how project designers working at the Cornell Lab of Ornithology (CLO) have tested draft protocols with both local groups, by accompanying them in the field and observing them as they collect and submit data, and with distant groups, by collecting their feedback online.

Unsurprisingly, protocols are one of the pillars supporting the engagement of citizen scientists, as emerging from our reading of the articles. Arguably, we could say that they act as ‘representatives’ of professional scientists, acting as “boundary objects”, aligning heterogeneous participants and professional scientists, such as in the SPT project. They reflect a normative view of how science should be performed and normative expectations of what the “scientific citizen” should do, once involved in a research project. Similarly to a speed bump, which is a technical artifact with an inbuilt script that prescribes drivers to proceed slowly (Latour, 1992), protocols have an inbuilt script that prescribes citizen scientists what to observe and report about. Control over observation tasks is delegated to this tool. Obviously, drivers can choose to ignore speed bumps – just fly over them and do not slow down. Similarly, citizen scientists can choose to ignore the protocols – perhaps over-reporting certain species of birds and under-reporting others. They will not be fined for their behavior as they would be by policemen, if they decided to ignore speed bumps, but their data is unlikely to pass scientists’ scrutiny.


Bonney R (Bonney, Rick); Cooper CB (Cooper, Caren B.); Dickinson J (Dickinson, Janis); Kelling S (Kelling, Steve); Phillips T (Phillips, Tina); Rosenberg KV (Rosenberg, Kenneth V.); Shirk J (Shirk, Jennifer), 2009, Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy, BIOSCIENCE 59 (11): 977-984.

Cohn, J. (2008). Citizen Science: Can Volunteers Do Real Research?

Latour, B. (1992). Where are the Missing Masses? The Sociology of a Few Mundane Artifacts.
In W. E. Bijker & J. Law (Eds.), Shaping Technology/ Building Society: Studies in
Sociotechnical Change
(pp. 225 -258). Cambridge, MA: MIT Press.

Trumbull, D., Bonney, R., Bascom, D., & Cabral, A. (2000). Thinking Scientifically during Participation in a Citizen-Science Project. Science Education, v84 n2 p265-75.

The invisible citizen scientist?

October 2nd, 2014 | Posted by citizenscience in article | review - (0 comments)

In early September this year, Caren B. Cooper, Jennifer Shirk and Benjamin Zuckerberg published an article called The Invisible Prevalence of Citizen Science in Global Research: Migratory Birds and Climate Change. This article analyzes the role of citizen science in the (most cited) articles describing the “impacts of climate change on avian migration”. The results are quite interesting:

We found that 85 of the 171 papers that we could classify were based on citizen science, constituting 5 to 20 papers per claim (Appendix S1). Citizen science heavily informed claims related to ecological patterns and consequences and was less frequently cited for claims about mechanisms (Table 1).

In other words, when it comes to avian migration and climate change, citizen scientists contribute to almost half of the body of scientific facts that we rely on for knowing about this phenomenon. Moreover, the quality of the data was examined, and the authors found no deviation among the observations performed by citizen scientists vs. the observations made by conventional means.

However, Cooper, Shirk and Zuckerberg point to a problem of visibility of the citizen scientists. It seems like the scientific community has not yet recognized the contribution of citizen scientists properly, and the authors argue that there is a “stigma” attached to involving the public:

The use of citizen science data in an active field of ecological research, such as migration phenology, is strong evidence that any stigma associated with the use of data collected by volunteers is unwarranted. Yet, the contributions of citizen science were not readily detectable in most cases. Thus, the stigma may persist unless researchers begin to draw attention to the citizen-science elements in their research papers.

As a consequence, scientific articles do not always render visible that citizens have participated in keywords, titles or abstracts. Thus, the authors suggest that the keyword “citizen science” should be used as a standardized keyword for all further studies that involve their contribution.

Cooper CB, Shirk J, Zuckerberg B (2014) The Invisible Prevalence of Citizen Science in Global Research: Migratory Birds and Climate Change. PLoS ONE 9(9): e106508. doi:10.1371/journal.pone.0106508