Orchid Project

Technologies > Citizen Science

  • The Power of the Crowd
Human Agent Collectives: The Power of the Crowd

An overview of ORCHID’s research in the crowdsourcing domain.

Outrun Cancer

Word-of-mouth, referral, or viral marketing is a highly sought-after way of advertising. We undertook a field experiment that compared incentive mechanisms for encouraging social media shares to promote participation in a crowdsourcing or citizen science project or to support a given cause. Our experiment took place on a website set up to promote a fundraising drive by a large cancer research charity. Site visitors who chose to sign up to support the cause were then asked to spread the word about the cause on Facebook, Twitter or other channels. Visitors were randomly assigned to one of four groups that differed in the way social sharing activities were incentivised. Under the control group, no extra incentive was provided. Under two others, the sharers were offered a fixed number of points that helped to take the campaign further. We compared low and high levels of such incentives for direct referrals. The final group, were offered a multi-level incentive mechanism that rewarded direct as well as indirect referrals (where referred contacts refer others). We found that providing a high level of incentives resulted in a statistically significant increase in sharing behaviour.

  • TREC crowdsourcing challenge
TREC crowdsourcing challenge

Combining machine intelligence with the skills of human crowds produces powerful systems for analysing data and meeting complexing information needs. For the TREC crowdsourcing challenge, we demonstrated a Bayesian probabilistic method for combining the responses of a crowd with features extracted using natural language processing, allowing a small number of crowdsourced labels to train a classifier. This automation reduces the cost of crowdsourcing and expands the size of datasets we can manage.  Furthermore, Bayesian modelling approaches allow us to select optimal pairs of workers and tasks, vastly increasing the efficiency and speed of learning.

  • Making the most of citizen science
Making the most of citizen science

  • New Forest Cicada Project
Smartphone-powered Crowdsourcing Hunt for the New Forest Cicada

The New Forest Cicada is an insect native to the UK in great danger of becoming extinct. Several entomologists have searched for it in the past few years, but no sighting has been confirmed for over a decade. However, the area where the cicada could be found is vast, extending to the entire New Forest and potentially other sites in Southern England. This cicada makes a high-piched call that is difficult for adults to hear as it is at the edges of our ears’ frequency range.

Visitors to the New Forest may use their smartphones to capture and classify the sound produced by the cicada, effectively mapping where the insect can or cannot be found. A smartphone “app” to classify the call has been developed and can be downloaded from the New Forest Cicada Website.

  • Predicting Human Mobility Patterns
Predicting Human Mobility Patterns

Understanding human mobility patterns is a significant research endeavour that has recently received considerable attention. Developing the science to describe and predict how people move from one place to another during their daily lives promises to address a wide range of societal challenges: from predicting the spread of infectious diseases, improving urban planning, to devising effective emergency response strategies. Individuals are also set to benefit from this area of research, as mobile devices will be able to analyse their mobility pattern and offer context-aware assistance and information. For example, a service could warn about travel disruptions before the user is likely to encounter them, or provide recommendations and mobile vouchers for local services that promise to be of high value to the user, based on their predicted future plans. More ambitiously, control systems for home heating and electric vehicle charging could be enhanced with knowledge of when the user will be home.

Associated Papers
  • Rahwan, I., Dsouza, S., Rutherford, A., Naroditskiy, V., McInerney, J., & Venanzi, M., et al. (2013). Global manhunt pushes the limits of social mobilization. IEEE Computer. Get Bibtex Citation
    @article{
       orchid84,
      title = {Global manhunt pushes the limits of social mobilization.},
      author = {Iyad Rahwan and Sohan Dsouza and Alex Rutherford and Victor Naroditskiy and James McInerney and Matteo Venanzi and Nicholas R Jennings and Manuel Cebrian},
      year = {2013},
      journal = {IEEE Computer},
    keywords = {Crowd-sourcing, Incentive Engineering} }
  • McInerney, J., Rogers, A., & Jennings, N. R. (2012). Improving location prediction services for new users with probabilistic latent semantic analysis. In 4th International Workshop on Location-Based Social Networks. Get Bibtex Citation Download as PDF
    @inproceedings{
       orchid82,
      booktitle = {4th International Workshop on Location-Based Social Networks},
      month = {September},
      title = {Improving location prediction services for new users with probabilistic latent semantic analysis},
      author = {James McInerney and Alex Rogers and Nicholas R Jennings},
      year = {2012},
      keywords = {Human-Agent Interaction, Machine Learning, Ubiquitous Computing, Applications, Flexible Autonomy},
    howpublished = {https://www.orchid.ac.uk/eprints/82/1/paper%5flbsn.pdf} }
  • McInerney, J., Stein, S., Rogers, A., & Jennings, N. R. (2012). Exploring periods of low predictability in daily life mobility. In Mobile Data Challenge by Nokia Newcastle upon Tyne, UK: Nokia. Get Bibtex Citation
    @incollection{
       orchid69,
      month = {June},
      author = {James McInerney and Sebastian Stein and Alex Rogers and Nicholas R Jennings},
      booktitle = {Mobile Data Challenge by Nokia},
      address = {Newcastle upon Tyne, UK},
      title = {Exploring periods of low predictability in daily life mobility.},
      publisher = {Nokia},
      journal = {Mobile Data Challenge by Nokia},
      year = {2012},
      keywords = {Human-Agent Interaction, Applications, Agile Teaming},
Trust-Based Learning Methods for Crowdsourced Radiation Monitoring in the Fukushima Disaster

We developed a novel trust-based method for learning from information provided from untrustworthy responders (or sensors) in disaster environments. Specifically, our method is derived from the integration of a sensor trust model with the Gaussian process and it is able to estimate jointly the observed spatial function and the individual sensor’s trustworthiness. The video shows the application of such a method to the scenario of crowdsourced radiation monitoring in the 2011 Fukushima disaster.

A web-based demo is available here

  • Tag Challenge Winner Demo
TAG Challenge

The Tag Challenge sent five individuals to roam the streets of New York, Washington DC, Bratislava, Stockholm, and London on March 31, 2012. The challenge was to locate them based only on a single photo of each individual released at 8am. ORCHID researchers joined scientists from Masdar Institute and UCSD to form the CrowdScanner team and win this seemingly impossible challenge. The lessons from the challenge appeared in Computer magazine and The Economist. This research will guide the design of future information-gathering crowdsourcing and citizen science projects.

Associated Papers
  • Rahwan, I., Dsouza, S., Rutherford, A., Naroditskiy, V., McInerney, J., & Venanzi, M., et al. (2012). Global manhunt pushes the limits of social mobilization. IEEE Computer. Get Bibtex Citation
    @article{
      orchid84,
      title = {Global manhunt pushes the limits of social mobilization.},
      author = {Iyad Rahwan and Sohan Dsouza and Alex Rutherford and Victor Naroditskiy and James McInerney and Matteo Venanzi and Nicholas R Jennings and Manuel Cebrian},
      year = {2012},
      journal = {IEEE Computer},
    keywords ={Crowd-sourcing, Incentive Engineering} }
  • Naroditskiy, V., Rahwan, I., Cebrian, M., & Jennings, N. R. (2012). Verification in referral-based crowdsourcing. PLoS ONE. Get Bibtex Citation Download as PDF
    @article{
      orchid83,
      month = {October},
      title = {Verification in referral-based crowdsourcing. },
      author = {Victor Naroditskiy and Iyad Rahwan and Manuel Cebrian and Nicholas R Jennings},
      year = {2012},
      journal = {PLoS ONE },
      keywords = {Crowd-sourcing, Incentive Engineering},
    howpublished = {https://www.orchid.ac.uk/eprints/83/1/journal.pone.0045924.pdf} }
‘Facebook for Animals’ tested on birds

We have recently developed a methodology for extracting social network structure from spatio-temporal datasets that describe timestamped occurrences of individuals. Our approach identifies temporal regions of dense agent activity and links are drawn between individuals based on their co-occurrences across these ‘gathering events’. The statistical significance of these connections is then tested against an appropriate null model. Such a framework allows us to exploit the wealth of analytical and computational tools of network analysis in settings where the underlying connectivity pattern between interacting agents (commonly termed the adjacency matrix) is not given a priori. As part of this work we have performed experiments on two large-scale datasets (with millions of data points) of great tit (Parus major) wild bird foraging records and illustrate the use of this approach by examining the temporal dynamics of pairing behaviour, a process that was previously very hard to observe. We show that established pair bonds are maintained continuously, whereas new pair bonds form at variable times before breeding, but are characterized by a rapid development of network proximity. The method proposed is general, and can be applied to any system with information about the temporal co-occurrence of interacting agents.
This work was supported by Microsoft Research.

Associated Papers
  • Psorakis, Y., Roberts, S. J., Rezek, I., & Sheldon, B. C. (2012). Inferring social network structure in ecological systems from spatio-temporal data streams. Get Bibtex Citation Download as PDF
    @inproceedings{
      orchid62,
      month = {June},
      title = {Inferring social network structure in ecological systems from spatio-temporal data streams },
      author = {Yannis Psorakis and Stephen J. Roberts and Iead Rezek and Ben C. Sheldon},
      publisher = {Royal Society Publishing},
      year = {2012},
      journal = {Journal of Royal Society Interface},
      keywords = {Machine Learning, Incentive Engineering},
      howpublished = {https://rsif.royalsocietypublishing.org/content/early/2012/06/07/rsif.2012.0223.abstract?sid=ef32f28e-f1f3-42bb-878d-f946e679cf5d},
    url = {https://www.orchid.ac.uk/eprints/62/} }
Zooniverse Labs

The Zooniverse Labs platform  has been developed to allow machine learning algorithms and predictive models developed within ORCHID to be plugged in to improve human performance in galaxy identification tasks. The firt project in the labs has been launched and concerns searching for interesting signals coming from the Kepler Field. The hope is that we will find another planet!

Associated Papers
  • Simpson, E., Roberts, S. J., Smith, A., & Lintott, C. (2011). Bayesian Combination of Multiple, Imperfect Classifiers. In NIPS 2011, Oxford. Get Bibtex Citation Download as PDF
    @inproceedings{
      orchid7,
      month = {December},
      author = {Edwin Simpson and Stephen J Roberts and Arfon Smith and Chris Lintott},
      booktitle = {NIPS 2011},
      address = {Oxford},
      title = {Bayesian Combination of Multiple, Imperfect Classifiers},
      publisher = {University of Oxford},
      pages = {1--8},
      year = {2011},
      keywords = {Machine Learning, Applications, Incentive Engineering},
      howpublished = {https://www.orchid.ac.uk/eprints/7/1/vbibcc%5fworkshop.pdf},
      url = {https://www.orchid.ac.uk/eprints/7/},
    }
  • Simpson, E., Roberts, S. J., Psorakis, I., & Lintott, C. (2012). Dynamic Bayesian Combination of Multiple Imperfect Classifiers. In Decision Making with Imperfect Decision Makers Springer. Get Bibtex Citation Download as PDF
    @incollection{
      orchid32,
      booktitle = {Decision Making with Imperfect Decision Makers},
      title = {Dynamic Bayesian Combination of Multiple Imperfect Classifiers},
      author = {Edwin Simpson and Stephen J Roberts and Ioannis Psorakis and Chris Lintott},
      publisher = {Springer},
      year = {2012},
      series = {Intelligient Systems Reference Library},
      keywords = {Machine Learning, Applications, Incentive Engineering},
    howpublished = {https://www.orchid.ac.uk/eprints/32/1/galaxyZooSN%5fsimpson%5fetal.pdf}