Where Are They Now? Understanding Underwater Video Content

SmartBay Where are they now

Where Are They Now? Understanding Underwater Video Content

Vision, being paramount among our senses, enables us to understand the world around us.

Advances in machine learning, computer vision, and AI, mean that it is now possible to use cameras as sensors to better understand our environment at near-human levels of perception.

With funding from the Marine Institute’s SmartBay Ireland National Infrastructure Access Programme (NIAP) 2016, the team at SFI’s Research Centre, Insight, were granted access to the SmartBay facility, along with all its data feeds, in order to carry out scientific research for their Understanding Underwater Video Content project.

Based in Dublin City University, Professor Noel O’Connor and his team from Insight got to work using the video content that was available to them from the SmartBay Observatory in 2017. SmartBay provided Insight with the opportunity to explore the application of the latest Artificial Intelligence (AI) techniques on the underwater camera deployed in Galway Bay. This provided Insight with access to a challenging real-world data source that they had not previously encountered.

Speaking about the project at SmartBay, Professor Noel O’Connor, Insight CEO, explained that they “focused on a common computer vision pre-processing step that often needs to be solved before any use of advanced AI and analytics.”

During this time working at SmartBay, Insight began working on Background Subtraction from the underwater video camera data. Background Subtraction means separating foreground objects from the background. Using SmartBay’s underwater camera footage and data, Insight were able to separate the fish in the foreground from the surrounding water (i.e. the background). This is a pre-requisite for tasks such as fish counting or species identification.

So, where are they now?

Insight, a world leading SFI Research Centre, is now one of the largest data analytics centres in Europe. It seeks to derive value from Big Data and provides innovative technology solutions for industry and society, by enabling better decision making.

Using the video data collected by the SmartBay Observatory in Galway Bay, Insight were able to demonstrate how many state-of-the-art deep learning based background subtraction and object detection techniques were unable to cope with the vagaries of video data from the buoy.

Insight re-applied for round two of the National Infrastructure Access Programme in 2018, in which they were successful in obtaining a second round of funding to continue their research.

Emanating from the NIAP funding, Insight went on to publish an academic research paper, entitled ‘A large scale under-sea dataset for marine observation’ in Global Oceans 2020: Singapore-US Gulf Coast, pages 1-5. IEEE 2020.

Today, Insight believes the work that took place at SmartBay has continued to contribute to their ongoing research. It has broadened their understanding of application domains that could benefit from their fundamental research on better machine learning methods for vision.

Alan Berry, Research Infrastructures Manager at the Marine Institute, said, "It is a very exciting time as these emerging AI and machine learning technologies evolve. The success of Insight’s work in utilising the SmartBay Observatory and its data is very rewarding. We wish every continued success to the Insight team in bringing this important technology forward which has very significant impacts in its contribution to a greater understanding of the seas around us."

The Marine Institute facilitates access to the SmartBay Facility providing opportunity for both national and international scientists to test new sensors and devices for monitoring the marine environment.