Nocturnal Fugue + EchoVision

Jiabao Li, Matt McCorkle, Botao Hu, 2024

Inspired by Thomas Nagel's "What Is it Like to Be a Bat?", "Nocturnal Fugue" immerses audiences in the intricate language of bats through AI-discovered social calls and digital habitat simulations, using spatial sound, projection mapping, and haptic transducers. "EchoVision" complements this with a mixed reality experience simulating bat echolocation.

Download EchoVision from Apple App Store

Nocturnal Fugue

With the help of AI, researchers found bats have more complex language than previously thought, with individual names, the ability to argue over food, and vocal learning. Just like human baby talk, mother bats lower their pitch when talking to babies. What would a baby bat lullaby sound like? Would a song made of bat mating calls be a love song?

Enter into Nocturnal Fugue, where you experience an immersive world of bats. Their social vocalizations transform into evocative music set to imaginative digital recreations of their natural habitats. Each digital habitat crafts a different sense of scale and time, creating a mood centric to the social vocalizations that are being heard and felt. Explore and feel the identity of bats through their isolation, sleeping, kissing, mating, feeding, grooming, and fighting vocalizations. As their sounds are transmogrified into imaginative soundscapes you are transported into the unique sonic world of bats through spatial sound, immersive projections, and haptic vibrations.

Inspired by Thomas Nagel's philosophical inquiry in "What Is it Like to Be a Bat?", this project delves into the enigma of truly comprehending the experience of another species. While we may grasp the superficial metadata of bat behavior, the true content of their communication remains elusive. Can we ever fully understand the language of bats, even with the aid of AI? We might not be able to find many words from the bat world in human’s dictionary.

EchoVision

EchoVision is an immersive art installation that allows participants to experience the world of bats using sound visualization and mixed reality technology. With a custom-designed, bat-shaped headset from the open-source HoloKit project, users can simulate echolocation, the natural navigation system bats use in the dark. They do this by using their voices and interpreting the returned echoes.

The exhibit adjusts visual feedback based on the pitch and tone of the user's voice, offering a dynamic and interactive depiction of how bats perceive their environment. This installation combines scientific learning with empathetic engagement, encouraging an ecocentric design perspective and understanding between species. "Echoing Bat" educates and inspires a deeper appreciation for the unique ways non-human creatures interact with their ecosystems.

Appeared on Xinhua News 新华社

在人工智能的助翼下，研究人员发现蝙蝠的语言比我们所知的更加复杂，它们有名字，会争论，还能进行语言学习。就像我们和婴儿说话时会用可爱的声调，蝙蝠妈妈在和蝙蝠宝宝说话时会降低声调。想象蝙蝠的摇篮曲，会是怎样的旋律？蝙蝠求偶之声，又能谱出怎样的情歌？

步入“夜曲赋格”，沉浸于蝙蝠的幻境世界。蝙蝠的语言谱出情感丰沛的乐章，与虚拟的自然栖所相辉映。

每一处栖所，皆有别具一格的规模与时光感，营造出与所闻所感的蝙蝠的语言相应的情境氛围。探索蝙蝠的独特身份，体会它们在孤独、沉睡、亲吻、交配、觅食、梳理及争斗时的各种声音。

本项目灵感源自托马斯·内格尔的哲学探究《成为一只蝙蝠会是什么样？》，深入探讨真正理解另一物种体验的奥秘。我们或许能触及蝙蝠行为的表层元数据，但它们交流的真谛，仍遥不可及。即便借助人工智能，我们真的可以完全解读蝙蝠的语言吗？我们的词典里可能找不到很多数蝙蝠用到的语汇。

Exhibitions

SXSW, XR Experience, Austin, 2025. EchoVision.

Plásmata, Athens, Greece, 2025. We’ve met before, haven’t we? EchoVision, invited by Onassis Foundation

FACT, Liverpool, UK, 2025. Nocturnal Fugue and EchoVision

Millenium Docs Against Gravity Festival, Warsaw, Poland, 2025

Currents New Media Art Festival, Santa Fe, NM, 2025

“Hope”, Ars Electronica, Linz, Austria. Sep 2024. “Of Sublime and the More-Than-Humans”

Vancouver International Film Festival, 2024

Sheffield DocFest, Alternate Reality, UK, 2024

TANK art festival, Shanghai, China, 2024

Wuzhen World Internet Science and Technology Museum, Wuzhen. New Matter of Nexus: A Digital Art Exhibition, 2024

West Bund Art Festival, Shanghai, 2024

“Host: Fusebox Program with Jiabao Li”, The Contemporary Austin, 2024

SIGGRAPH Asia, Art gallery, XR, Tokyo, Japan, 2024

f.l.i.p. X Lapin et Halot, “interactivité”, Tokyo, Japan, 2024

Ubicomp-ISWC, Design Exhibition, Melbourne, Australia. “A Handheld Mixed Reality Mask for Experiencing Bat Echolocation”. 2024

Progenitorial Hysteresis 祖细迟滞. 12 Dec 2023 - 13 Apr 2024. Duande Art Museum 巽美术馆, China

Awards

AUREA Award, Germany

Best Demo Award, Augmented Human, Abu Dhabi

Publications

Botao Amber Hu*, Jiabao Li*, Jianan Johanna Liu, Danlin Huang, Xiaobo Aaron Hu, and Yilan Elan Tao. 2025. “EchoVision.” IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2025), Saint-Malo, France. https://ieeexplore.ieee.org/document/10972681

Botao Amber Hu*, Jiabao Li*, Jianan Johanna Liu, Danlin Huang, Xiaobo Aaron Hu, and Yilan Elan Tao. 2025. “Seeing with Sound: Demonstrating "EchoVision", a Mixed Reality Simulation of Bat Echolocation” In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '25), Yokohama, Japan. ACM. https://doi.org/10.1145/3706599.3721192

Botao Amber Hu*, Jiabao Li*, Danlin Huang, Jianan Johanna Liu, Xiaobo Aaron Hu, and Yilan Elan Tao. 2024. “Becoming Bats with ‘EchoVision’: Towards Eco-Phenomenological Mixed Reality.” SIGGRAPH Asia 2024 Art Papers. Tokyo, Japan. https://dl.acm.org/doi/10.1145/3680530.3695460.

Jiabao Li, Matt McCorkle, Botao Amber Hu, “Becoming Bat.” Halfway to the Future Symposium (HttF). Santa Cruz, CA. https://doi.org/10.1145/3686169.3686212

Botao Amber Hu*, Jiabao Li*, Danlin Huang, Jianan Johanna Liu, Xiaobo Aaron Hu, and Yilan Elan Tao. “EchoVision: A Handheld Mixed Reality Mask for Experiencing Bat Echolocation.” UbiComp '24: Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing. Melbourne, Australia. https://doi.org/10.1145/3675094.3681951

Botao Amber Hu*, Jiabao Li*, Danlin Huang, Jianan Johanna Liu, Xiaobo Aaron Hu, and Yilan Elan Tao. “EchoVision” IEEE VIS Art Program 2024. https://ieeevis.org/year/2024/program/paper_a-visap-1004.html.

*The two authors contributed equally to this work. They are co-first authors.

Press

Charlie Fink, XR At SXSW 2025: Expanding The Boundaries Of Immersion, Forbes, Mar 15, 2025

Brianna Caleri, SXSW pedestrians meet live bats and experience echolocation through XR, Culture Map Austin, Mar 11, 2025

Plásmata 3: An Art Trail Where Reality Melts Into Illusion – Onassis Stegi, Artlyst, May 31, 2025

Plásmata ΙΙΙ: Τι είδαμε στο Πεδίον του Άρεως, Athens Voice, May 28, 2025

Αφροδίτη Παναγιωτάκου, Ξενάγηση στα Plāsmata: Κεφάλαιο 5, LiFO, Jun 1, 2025

Εχουμε συναντηθεί ξανά, σωστά; Τα Plásmata 3 έφτασαν στο Πεδίον του Άρεως, Huffpost, May 27, 2025

Two projects and an award for Onassis ONX at the SXSW 2025 in Texas, Onassis, Mar 2025

Martin Hieslmair, Ars Electronica Features: Beacons of Hope from Around the World, Ars Electronica, July 10, 2024

摄影机不要停：跨学科影像艺术中的人类未来, 智族 GQ Magazine, 2024年04期

李佳宝｜用艺术提出问题，用科技解决问题, Kering 跃动她影, Jul 29, 2024

XÙN访谈｜对话李佳宝：祖细迟滞, 巽美术馆, Jun 4, 2024

At Sheffield DocFest, Alternative Realities, 2024, UK

Introduction

The exhibit adjusts visual feedback based on the pitch and tone of the user's voice, offering a dynamic and interactive depiction of how bats perceive their environment. This installation combines scientific learning with empathetic engagement, encouraging an ecocentric design perspective and understanding between species. EchoVision educates and inspires a deeper appreciation for the unique ways non-human creatures interact with their ecosystems.

"What Is it Like to Be a Bat?"

In "What Is it Like to Be a Bat?" Thomas Nagel explores the unique, subjective experience of consciousness, using bats as a primary example due to their use of echolocation, a sensory perception vastly different from human experiences. He argues that understanding what it is like to be another creature is inherently limited by our inability to fully grasp their subjective experiences. This concept directly relates to EchoVision that allows participants to simulate the echolocation of bats using sound visualization and mixed reality technology. By adjusting visual feedback based on the user's voice, EchoVision bridges the gap between human and bat perception, offering a deeper, empathetic understanding of non-human sensory worlds and emphasizing the profound differences in how various species interact with their environments.

Misconception of Bats

People often harbor misconceptions about bats, viewing them as dangerous, dirty, or associated with negative superstitions. Common myths include the belief that bats are blind, that they commonly get tangled in human hair, or that they are primarily carriers of diseases (especially during the COVID-19 pandemic). These misconceptions contribute to a general fear and misunderstanding of bats, overshadowing their ecological importance and unique biological features.

Bats are ecologically vital, providing essential benefits such as pest control by consuming large quantities of insects, pollination of numerous plants, and seed dispersal that aids in forest regeneration. Their guano enriches soil fertility, contributing to nutrient cycling. These roles help maintain ecosystem balance and biodiversity.

EchoVision aims to challenge and transform these misconceptions by providing participants with an immersive, empathetic experience of the world through a bat's perspective. By simulating echolocation using sound visualization and mixed reality technology, EchoVision allows users to understand how bats navigate and perceive their environment with remarkable precision. This interactive experience highlights the sophistication and elegance of bat echolocation, fostering a greater appreciation for their role in ecosystems, such as pest control and pollination.

Through EchoVision, participants can see bats not as menacing creatures but as fascinating and integral parts of the natural world. The installation encourages scientific learning and empathetic engagement, promoting an ecocentric design perspective that values the diverse ways non-human creatures interact with their surroundings. By offering a new perspective, EchoVision helps dispel myths, reduces unfounded fears, and inspires a deeper connection to and respect for bats and their vital ecological contributions.

Echolocation

Bat echolocation is a sophisticated biological process that allows bats to navigate and hunt in the dark by emitting high-frequency sound waves and listening to the echoes that bounce back from objects in their environment. When a bat emits a sound, it travels through the air and reflects off surfaces, returning to the bat as an echo. By analyzing the time delay and changes in the frequency of the returning echoes, bats can determine the distance, size, shape, and texture of objects, as well as the speed and direction of moving prey.

In "An Immense World," Ed Yong describes how bats' echolocation abilities are finely tuned and highly specialized. He explains that bats can emit sounds at frequencies higher than the human ear can detect, often using calls that range from 20 to 200 kHz. The bat's brain processes these echoes with remarkable speed and accuracy, allowing it to create a detailed auditory map of its surroundings. This process is so efficient that bats can detect tiny insects in complete darkness and avoid obstacles with precision. Yong also highlights the diversity among bat species in their echolocation strategies, noting that different bats have evolved various adaptations to suit their ecological niches. Some bats have highly directional calls for pinpointing prey, while others use more omnidirectional calls to navigate cluttered environments like dense forests. These adaptations demonstrate the incredible versatility and effectiveness of echolocation as a sensory tool for survival.

In addition to bats, several other animals use echolocation to navigate and hunt, including dolphins, porpoises, whales (such as sperm and beluga whales), certain species of shrews, and swiftlets. One notable human who uses echolocation is Daniel Kish. Blind since infancy, Kish developed the ability to navigate his surroundings by making clicking sounds with his tongue and interpreting the echoes that bounce back from objects around him. This technique, known as human echolocation, allows him to "see" the world through sound, enabling him to walk, ride a bike, and engage in various activities independently. Kish has also founded the organization World Access for the Blind to teach echolocation to other visually impaired individuals.

LiDAR Sensor VS Echolocation

The relationship between LiDAR sensors and bat's echolocation lies in the fundamental principles of how they both detect and map their surroundings through the reflection of emitted signals.

Both LiDAR sensors and bats emit a signal that interacts with objects in the environment and then capture the reflected signal to gather information—LiDAR emits laser pulses, while bats emit ultrasonic sound waves. The time it takes for the signals to return provides data about the distance and shape of objects. Both technologies use the returned signals to create a map or image of their surroundings; LiDAR creates detailed 3D models of environments by calculating the distance between the sensor and objects, while bats interpret the time delay and intensity of the echoes to construct a mental map of their surroundings, aiding in navigation and hunting in the dark. Both methods allow for high accuracy and precision in detecting objects and their positions, with LiDAR measuring distances with great accuracy and producing high-resolution 3D models, and bats detecting tiny insects and avoiding obstacles in complete darkness.

EchoVision leverages this similarity by using sound visualization through LiDAR sensor’s reconstruction of the environment to simulate the echolocation experience, drawing a parallel between human-made technology and natural biological processes.

The Contemporary Austin - Host: Fusebox Program

We premiered EchoVision at The Contemporary Austin as part of the Fusebox Festival. The Congress Avenue Bridge in Austin, Texas, famously hosts the largest urban bat colony in the world, with roughly 1.5 million Mexican free-tailed bats residing beneath it every summer. Our performance started under the bat bridge with our collaborator, the bat conservationist Merlin Tuttle, who saved the 1.5 million bats under the Austin Bridge from extermination. About 500 people came. They listened to Merlin’s captivating stories with bats, watched the huge number of bats flying out of the bat bridge after sunset, and experienced EchoVision to be like a bat and to have a rare glimpse into the bat's perceptual universe.

Then, the adventure progressed with a walk straight down Congress Avenue to The Contemporary Austin Museum, where Jiabao Li and Matt McCorkle performed a live bat-inspired soundscape event called "Nocturnal Fugue." Nocturnal Fugue uses bat vocalizations—from social calls to mating rituals—to artistically create a mesmerizing sonic experience. This soundscape is combined with visual projections that represent the environment in which the specific bat call is being made or in response to. At the rooftop, we served Tequila. Bats are agave pollinators. They are part of the key process that turns agave flowers into tequila.

Credits

EchoVision

Directed by Jiabao Li & Botao Amber Hu

Co-producer: The Onassis Foundation

Interaction Engineer: Botao Amber Hu

Interaction Designer: Jiabao Li

Visual Effect: Aaron Hu

Industrial Design of Head-Mounted AR Mask: Jianan Liu, Danning Huang

Logistics Producer: Elan Tao

Nocturnal Fugue

Directed by Jiabao Li & Matt McCorkle

Worldbuilding & Music Composition by Matt McCorkle

Executive Producer: Jiabao Li

Bat Expert Consultant: Merlin Tuttle, Teresa Nichta

Producers: Teresa Nichta, Jessica Kidd

Guitarists: Mark Sidney Johnson, Joe Daleki

Special Thanks: Michael Ryan, Janet Tyburec, Melissa Donnelly, Dianne Odegard, Lee Mackenzie, Austin Bat Refuge, Merlin Tuttle's Bat Conservation, Onassis ONX Studio, Fusebox Festival, The Contemporary Austin

Bat Vocalization Recordings by Yosef Prat, Mor Taub, Ester Pratt, Yossi Yovel

Prat, Yosef; Taub, Mor; Pratt, Ester; Yovel, Yossi (2017). An annotated dataset of Egyptian fruit bat vocalizations across varying contexts and during vocal ontogeny. https://doi.org/10.6084/m9.figshare.c.3666502.v2