Globally Significant Research at the University of Melbourne
"The University of Melbourne’s SWARM Project is attempting to achieve fundamental advances in collaborative reasoning."
The University of Melbourne’s SWARM Project is attempting to achieve fundamental advances in collaborative reasoning.
With major funding from the Intelligence Advanced Research Projects Activity (IARPA), our international, multidisciplinary team is studying collaborative reasoning, in particular as it relates to improving intelligence analysis. As part of this project we are developing a new platform for online collaborative reasoning.
What is the IARPA CREATE Program?
What is IARPA?
IARPA is the United States Intelligence Advanced Research Projects Activity. IARPA invests in high-risk, high-payoff research programs to tackle some of the most difficult challenges of the agencies and disciplines in the Intelligence Community (IC).
What is the CREATE Program?
In 2017 IARPA launched the CREATE program, which stands for Crowdsourcing Evidence, Argumentation, Thinking and Evaluation. This program aims to “achieve fundamental advances in human reasoning,” by combining two approaches: crowdsourcing, and structured analytical techniques:
“The CREATE program seeks proposals to develop, and experimentally test, systems that use crowdsourcing and structured analytic techniques (STs) to improve analytic reasoning. These systems will help people better understand the evidence and assumptions that support—or conflict with—conclusions. Secondarily, they will also help users better communicate their reasoning and conclusions.”
In short, CREATE is focused on determining whether crowds of people working together in a structured way can deliver superior reasoning and superior communication of their findings (superior to people acting alone and superior to people working together using existing tools such as Google docs).
How many teams are participating in CREATE?
CREATE is supporting four teams, known as “performers,” to develop systems addressing this challenge. The four performers are engaged in a kind of friendly competition to produce the most effective system.
Eventually one of the systems, or a successor, might be deployed to teams of analysts working in the intelligence community, helping them deliver intelligence products superior in quality to those generated using existing methods.
SWARM is one of four ‘performer teams’ participating in the CREATE Program, an initiative of IARPA.
"In 2017 IARPA launched the CREATE program, which stands for Crowdsourcing Evidence, Argumentation, Thinking and Evaluation."
The SWARM Project, based the University of Melbourne, Australia was selected to be one of the four teams in the CREATE competition. Our international team consists of researchers with considerable experience in many aspects of the CREATE problem domain, including
Structured analytical techniques, particularly argument mapping
Group judgement, particularly the use of Delphi-type techniques to improve estimation
Visualisation of complex argumentation
Logic and probability, including Bayesian approaches
"Our international team consists of researchers with considerable experience
in many aspects of the CREATE problem
What is our approach?
What kind of system might help crowds of people work together to produce stronger, clearer reasoning?
The essence of SWARM’s approach is to support effective teamwork. Specifically, our system is designed to nurture what we call High Performance Reasoning Teams. This is inspired by elite teams in other domains such as professional sports, military special operations, and surgery.
One important attribute of high performance teams is that distinctive processes, i.e. ways of working together. SWARM reasoning teams are equipped with two such “modi operandi.” These can be thought of as high-level structured analytical techniques.
One is Contending Analyses. This technique involves the team exploring and refining multiple distinct analyses of any given problem, and selecting the best as representing their collective response.
The second is the “multifocal” approach. A “lens” is any kind of method, tool or concept which can be brought to bear on a problem. A classic structured analytical technique such as Key Assumptions Check is one example of a logical lens. A strong analysis will generally involve applying a number of lenses, with the choice of lens depending very much on the kind of problem. This is similar to the way a doctor would draw on a range of concepts and techniques in diagnosing a condition and planning a treatment. A strong reasoning team will be collectively proficient in applying a wide range of logical lenses to intelligence problems. The SWARM system assists teams in identifying and applying appropriate lenses.