Nature AI Avatars

"Nature Avatars" is a research project at the intersection of technology, art, and the different Ways of Being that exist in our world. It attempts to develop hybrid manifestations that can serve as expression medium for beyond-human entities (from individual trees to entire watersheds). Such manifestations may play the roles of ambassadors, advocates and educators while bridging the worlds of their source entities and human communities and systems.

The avatars are manifest via digital tools while maintaining a direct connection to their source entities via real time remote sensing devices. Being powered by AI, they have the ability to interpret, impersonate and articulate the state/activity of their counterpart while interacting with a user.

One of the goals of such avatars is turning technical data and literature into non-technical insights to the general public. And this is where Generative AI (more specifically Large Language Models), play a big role. By programming these models to receive, analyze, interpret and impersonate data from the natural, we create a more natural interface for humans to engage with both the data and its underlying insights.

Another aspect of their hybrid nature is that they possess dual embodiment for sure interactions: in the digital sphere, they may exist as agents embedded in varied applications and even 3D modeled characters. At the same time, they are brought to life in the physical world through artifacts and sculptures, often made from natural materials intertwined with embedded devices running their operating code.

Rio Sagrado Avatar - Prime Example

The first Nature Avatar instance - a digital/physical representation for the Rio Sagrado (Morretes-PR, Brazil), designed for educational and political purposes.

The Rio Sagrado ("Sacred River"), runs through the largest well preserved continuum of Atlantic Rainforest in Brazil, and this whole region is currently under the threat of an international railroad project that seeks to cut through many of the water sources of this river and watershed.

The local community has been mobilizing in face of such madness, with one of the strategies being the understanding of the Rio Sagrado as a living entity. In this sense, my contribution to this process lies in the creation of multimedia mediums for the expression of the River, like a short film, and the avatar itself. The latter has been designed as a manifestation of the River's history and culture via participatory processes with the local community, leading up to the design of a sculpture by a local artisan that sought to highlight the local fauna and flora in his craft.

Embedded in the sculpture is a Raspberry Pi (model 5) that runs Speech-to-Text and Text-to-Speech models, controls LED effects, stream audio from and to user, and interact with an online LLM (GTP-4 from OpenAI, via Retune API calls, sponsored by GiveETH). This LLM is fine-tuned with over 30 scientific papers on the river's watershed, as well as community accounts on the local culture. And one can interact with that via the sculpture itself, or a dedicated website, or even Whatsapp. We have also created a metaverse experience for digital immersion in the Rio Sagrado alongside its avatar.

But the most important thing about this whole process is that it has actually catalyzed local conversations around the recognition of the rights of the Rio Sagrado in the municipality, as showcased below during a community meeting that we had with actual lawyers who wrote draft bills for the recognition of the River's rights.

Lahn River Avatar

The second Nature Avatar is a hybrid representation for the Lahn River, developed in the context of the Planetary Agency Fellowship and Residency- alongside artist Ingvild Syntropia - as part of the Panel the Planetary Thinking at JLU in Giesen, Germany.

As its predecessor, the Lahn Avatar also makes use of Large Language Models as expression mediums for the cultural and scientific knowledge around the River. However, in this case, multiple open source models were combined to give life to the avatar (Mistral, Gemma 3 families), as per provision by the residency's hosting institution (JLU, via GWDG). This required the development of a unique architecture for their optimized functioning. Furthermore, the Lahn River avatar expands the first concept implementation by to include:

• Greater user interactivity via a live camera feed embedded into the avatar sculpture:

A Raspberry Pi (model 5) was also used to run Speech-to-Text and Text-to-Speech models, control LED effects, audio and video streaming and for interaction with the custom LLM framework, hosted at a virtual server at JLU's servers.

• Real-time environmental data monitoring through dedicated hydrology sensor module:

Atlas Scientific has conveniently integrated their kit with ThingSpeak for IoT Projects, enabling easy data storage and access via API calls. By utilizing LLM Toll Calling we are able to provide the sensor data to the Lahn Avatar upon request, in real time.

With sensors installed at the Lahnfenster river observatory in Giessen, the sculpture stands open to the public as both a poetic and political gesture as shown by Giessen's mayor, Alexander Wright, asking the first question to the river avatar during its opening at Lahnfenster.

The Lanh avatar can also be reached via a dedicated website. However, it was also made present in a first-of-its-kind digital forum, designed in partnership with Consider.it, towards structured collective deliberation between people and the River (via is avatar), around the question of What are the biggest opportunities for the Lanh's future?

Finally

We are actively developing this project, by both providing maintenance/upgrades to the avatars that have been developed and considering future applications. For instance, we are currently working on integrating latest speech-to-speech models to the Lahn Avatar, so as to improve German proficiency and reduce lag response time.

With all of this work, it's our hope that the Nature Avatars concept, inspired by the underlying philosophical perspective of Symbiocracy, will help us imagine and prototype emerging futures that more deeply integrate the rights, sentience and will of our more-than-human planet. Especially as we continue to apply remote sensing and AI technologies towards the understanding of beyond-human "language", and eventually towards interspecies communication.