+Panic is an interactive large-scale projection of a school of fish that senses the space around it and reacts to it.

Thousands of digital fish live in the foyer of the exhibition After Nature and interact sensitively with the visitors and their behavior.

This interactive projection symbolizes the main theme that will be treated in the exhibition: Interactions and crises of natural and social systems.

Action and Reaction

Each individual fish not only notices the other fish in its environment, but also the visitors in the room. If the visitors behave calmly and slowly approach the swarm, the fish become curious and also seek the proximity of the visitors.  As long as hectic and aggressive movements are avoided, more and more fish gather around the visitors and imitate their movements. This leads to a playful dialogue between human and digital fish.

But as soon as the activity and gestures of the human become too fast, jerky and aggressive, the swarm panics. The only way out is flight, which spreads over the other fish in a cascading and overwhelming gesture.

If the visitor calms down again, the swarm also calms down and is ready for another attempt for a dialogue.

The Algorithm

Since Craig Reynolds published his paper on the simulation of swarm behavior in 1987, countless artistic and scientific works have been developed which used these always same three rules "Cohesion, Alignment, Separation".

As the name of the work suggests, a fourth new rule was added during the process: Panic.

First flocking-test in 2D using VVVV

Next step: Porting the algorithm to HLSL. A GPU-based implementation allows thousands of autonomous agents.

After an inspiring workshop on schooling behavior with Pawel Romanczuk, we picked up the concept of Contagious Panic and integrated this into our interaction mechanics:

A fish can be panicked by visitors by making jerky and frightening movements. The fish react to this with a flight behavior, in which they move away from the visitors as quickly as possible. The special thing is that this panic is contagious. That means, a fleeing fish communicates this panic state to its peers, which in turn break out in panic. The result is cascading escape patterns that travel through the entire swarm.

A Great Kiskadee attacks a school of fish and triggers the cascade-like escape patterns. This and other videos were the inspiring basis for the further development of the fish swarm algorithm. 

These patterns are not random but, according to the scientists, have the task of confusing the attackers. *(for the complete credits please scroll to the end)

A screen capture of the realtime-simulation showing our implementation of the phenomenon.

This is screen capture of the final system where everything works together: Five different groups of fish with different behavioral parameters, curiosity of the fish to the user (green circle) and contagious panic when moving too fast (red circle).

Technical Setup

The medium of the virtual swarm is a semi-transparent 180° curtain onto which a seamless 4.5m high image is projected with the help of two ultra-wide angle projectors and a projection mapping. This rear projection can be experienced from two sides and becomes interactive thanks to five 3D cameras placed in the room. They capture every single visitor and analyze parameters such as distance to the curtain, speed and direction of movement. These values directly influence the behavior of the swarm and also generate a reactive soundscape in real time.

Video which gives an insight into the different camera analysis data. Both the visuals and the tracking were developed with VVVV.

Sound Scenography

The installation became complete by an interactive sound experience which follows the visuals and rewards mindful interaction with the swarm.

The sound created by the Berlin based studio kling klang klong operates through three sonic layers, each of them augmenting different aspects of the visuals. The movement and the emotional state of the fishes were sonified through sound design elements and an abstract composition.

One of the first sketches that demonstrates the interplay between swarm behavior and the different sound layers.

Water Ambiance

The underlying water ambience is a direct translation of the fish movements. It emphasises the interactive character of the installation while it creates a tangible underwater atmosphere.

Algorithmic Composition

The musical background is an algorithmic composition based on the real-time data of the visuals. The piece translates the forces of the swarm behaviour into musical parameters. Cohesion, Separation and Aligment are applied onto voice leading, tempo, dynamics and articulation to create an abstracted sonic embodiment of the swarming behaviour.

Emotional State

The swarm soundscape is completed with playful sound design, to emphasise the emotional state of the fish schools and encourage the visitors for interaction. If the swarm becomes curious the visitors can hear their friendly chatter, which turns into panicky bawling when they disperse.

Credits

»+Panic« is part of the inaugural exhibition »After Nature« in the Humboldt Labor at the Humboldt Forum Berlin

Media Scenography
schnellebuntebilder

Sound Scenography
kling klang klong

Exhibition Design
Inside Outside/Petra Blaisse

Media Planning
schnellebuntebilder in collaboration with hergarten / interactive environments
COOP. Projektgesellschaft mbH

Project Management Humboldt Labor
Maria Ollesch
Anna Kalvelage
Frauke Stuhl (till March 2020)

Curatorial Direction
Dr. Gorch Pieken

Copyright Photography
Humboldt-Universität zu Berlin Philipp Plum

Camera Video Documentation
Nikolaus Götz

Editing Video Documentation
Tim Rumpf

Composition Video Documentation
kling klang klong

Many thanks to Pawel Romanczuk (Science of Intelligence HU Berlin) for the inspiring workshop.

* Video provided courtesy of Pawel Romanczuk / Cluster of Excellence "Science of Intelligence," Technical University of Berlin & Humboldt Universität zu Berlin. More information and credits: Doran et al, Current Biology, 2021,  https://doi.org/10.1016/j.cub.2021.11.068