FathomNet Aims to "Enable Artificial Intelligence in the Ocean" with an Open Source Image Database

Researchers from the Monterey Bay Aquarium Research Institute (MBARI), the California Institute of Technology, Smithsonian Institution, the National Oceanic and Atmospheric Administration (NOAA), Ocean Discovery League, and CVision AI have released an open source image database which they hope will help with "enabling artificial intelligence in the ocean."

"A big ocean needs big data," claims MBARI principal engineer Kakani Katija of the FathomNet project. "Researchers are collecting large quantities of visual data to observe life in the ocean. How can we possibly process all this information without automation? Machine learning provides a pathway forwards, however these approaches rely on massive datasets for training. FathomNet has been built to fill this gap."

"In the past five years, machine learning has revolutionized the landscape of automated visual analysis, driven largely by massive collections of labeled data," adds Ben Woodward, co-founder and chief executive of CVision AI and FathomNet co-founder. "ImageNet and Microsoft COCO are benchmark datasets for terrestrial applications that machine-learning and computer-vision researchers flock to, but we haven't even begun to scratch the surface of machine-learning capabilities for underwater visual analysis. With FathomNet, we aim to provide a rich, interesting benchmark to engage the machine-learning community in a new domain."

In total, the opensource database offers 84,454 images representing 175,875 localizations from 81 collections covering 2,243 concepts — and is continuing to grow, with a target of 1,000 independent observations for over 200,000 animal species across diverse poses and imaging conditions for a goal of over 200 million total observations. For that, its creators say, it needs community collaboration — aided by its open-access nature.tional Geographic and NOAA, captured by the former's autonomous Deep Sea Camera System benthic lander platform and the latter's remote-operated submersible vehicle deployed from the Okeanos Explorer.

In total, the open source database offers 84,454 images representing 175,875 localizations from 81 collections covering 2,243 concepts — and is continuing to grow, with a target of 1,000 independent observations for over 200,000 animal species across diverse poses and imaging conditions for a goal of over 200 million total observations. For that, its creators say, it needs community collaboration — aided by its open-access nature.

"While FathomNet is a web-based platform built on an API where people can download labeled data to train novel algorithms," Katija explains, "we also want it to serve as a community where ocean explorers and enthusiasts from all backgrounds can contribute their knowledge and expertise and help solve challenges related to ocean visual data that are impossible without widespread engagement."

"Four years ago, we envisioned using machine learning to analyze thousands of hours of ocean video, but at the time, it wasn't possible primarily due to a lack of annotated images," claims FathomNet co-founder and Ocean Discovery League president and founder Katy Croff Bell. "FathomNet will now make that vision a reality, unlocking discoveries and enabling tools that explorers, scientists, and the public can use to accelerate the pace of ocean discovery."

A paper describing FathomNet has been published in the journal Scientific Reports under open-access terms; those looking to engage with the project can access the database and collaborate with its creators via the project website. Source code for a Python client and links to machine learning models trained on FathomNet data are available on the project GitHub repository.