Abstract

FathomNet is a worldwide picture  |The ocean is changing at an unprecedented rate, making it difficult to visually monitor marine biota at the spatiotemporal scales required for good stewardship. The amount and velocity of the necessary data gathering are outpacing our capacity to process and analyse them at a quick rate as baselines are sought by the research community.

Due to the lack of data consistency, inadequate formatting, and the need for huge, labelled datasets, recent breakthroughs in machine learning enable quick, intelligent analysis of visual data. To meet this requirement, we created FathomNet, an open-source picture database that harmonises and compiles data that has been carefully selected and categorised.

In order to facilitate future contributions from dispersed data sources, FathomNet has been seeded with existing iconic and non-iconic photos of marine creatures, underwater machinery, detritus, and other themes. We show how the usage of FathomNet data on other institutional footage may be utilised to train and deploy models, reducing the need for manual annotation and enabling autonomous monitoring of underwater ideas when combined with robotic vehicles. We can speed up the processing of visual data to attain a healthy and sustainable global ocean as FathomNet expands and incorporates more labelled data from the community.

Introduction

Traditional, resource-intensive (e.g., time, person-hours, cost) sampling methodologies are constrained in their capacity to scale in spatiotemporal resolution and engage diverse communities when monitoring an area as vast as the ocean1, which is filled with life that we have yet to describe2, and which is teeming with unknown species3. However, we are starting to witness a paradigm shift in ocean exploration and discovery as a result of the development of contemporary robotics4, low-cost observation platforms, and distributed sensing5. Distributed platforms and open data architectures are pushing the chemical and remote sensing communities to new scales of observation6, 7, as seen in the oceanographic monitoring conducted by satellite worldwide ARGO float array and remote sensing of near-surface ocean conditions.

Large-scale sampling of biological populations or processes below the ocean’s surface waters has mainly trailed behind due to a number of obstacles.

There are three popular methods—acoustics, “-omics,” and imaging—for viewing marine life and biological processes, each with advantages and disadvantages. Acoustics makes it possible to observe population- and group-scale dynamics, but individual-scale observations—particularly identifying creatures at lower taxonomic levels like species—are difficult to do. 8. Based on their DNA shed in the water column, living communities can be recognised thanks to the promising field of eDNA.

The spatial origin of the DNA, relating the measurements to population sizes, and the presence of confounding non-marine biological markers in samples are active areas of research that still need to be addressed9. eDNA studies offer broad-scale views of biological communities with only a few discrete samples. Ultimately, the verification of -omics and acoustics methods depends on visual observations. Imaging,

a non-extractive tool for observing the ocean, allows for the species-level identification of numerous creatures, clarifies community structure and spatial linkages in a range of environments, and displays fine-scale behaviour of animal groupsHowever, processing visual data is a resource-intensive process that cannot be extended without a substantial investment in creating capacity and automation as well as data with complex settings and creatures that need expert classifications.

Due to the ease of technological deployment and the availability of several remotely operated and autonomous platforms, imaging is becoming a more popular method for sampling biological populations in a range of situations.
Maging has also been utilised to operate and navigate underwater vehicles in real-time while completing challenging tasks in complicated environments14.

Additionally, photography is a powerful engagement tool for communicating knowledge about marine species and the problems confronting the ocean with larger communities. In summary, visual data is a crucial tool for gaining a deeper understanding of the ocean and disseminating that knowledge widely.

Given all the uses for maritime imaging, many technologies for managing and analysing visual data have been created. These efforts have produced a variety of effective software solutions that may be used either locally on a computer, on field trips, or widely via the Internet.

However, new techniques for automated annotation of marine visual data are badly needed due to the scarcity of specialists and the exorbitant costs of annotation and storage. The creation and use of artificial intelligence and data science techniques for ocean ecology is driven by this requirement.

The phrase “artificial intelligence” (AI) covers a wide range of methodologies, some of which have been used to the study of maritime systems. The plankton imaging community has employed statistical learning techniques like random forests to achieve automatic categorization of microscale plants and animals with accuracy levels better than 90%.

With little data and previous knowledge of marine habitats, unsupervised learning may be used, but these algorithms have limited value for automatically detecting and classifying items in marine images with enough granularity and detail to be utilised for annotation.

The performance of automated annotation and classification tasks to finer taxonomic levels has been improved by deep learning algorithms trained on visual data where all objects have been identified however, this method requires publicly available, large-scale labelled image datasets for training.

The computer vision (CV) field has had access to mage repositories for terrestrial applications for a long time. With a long-term objective to gather 500 to 1 k full-resolution photos for 80 k ideas, or 50 M images22, ImageNet was the first labelled collection based on a hierarchy or the quantity of WordNet’s classes (or “objects”). ImageNet uses photographs pulled from Flickr to achieve this size, resulting in a collection of primarily recognisable pictures (e.g., centred objects in relatively uncluttered environments).