Demystifying the Ocean Through Underwater Video Analysis @ SmartBay

DCU image

Demystifying the Ocean Through Underwater Video Analysis @ SmartBay

Researchers from the INSIGHT Centre for Data Analytics at Dublin City University are currently using many years’ worth of video footage recorded at the SmartBay subsea observatory to enable better understanding of the marine environment.

Utilizing the state-of-the-art sub-sea observatory deployed and managed by SmartBay Ireland and the Marine Institute, the project investigates how modern computer vision algorithms can assist marine scientists in better understanding the Irish marine environment. A large benchmark dataset, with ground truth, will be created during the first phase of the project. A dataset such as this, which currently does not exist anywhere, provides the fundamental resources for developing, training and evaluating deep learning models that convert raw video feed into statistical information. The dataset will also be made publicly available to researchers worldwide. This strategy (publishing large scale benchmark dataset, then allowing researchers worldwide to apply their own methods) has gained significant success in progressing the performance of many other computer vision tasks such as object detection, object recognition, video captioning, etc., where computer vision has achieved near human performance. These successes have shown the value of large benchmark datasets such as that recorded at the SmartBay Test Site.

In the second phase of this project, the focus is to investigate how these methods can be transferred, either retrained or fine-tuned, to tackle underwater observation tasks. Ultimately, an indexing and retrieval demo system will be developed for better managing the information extracted from video feeds. 


Figure 1: A sample frame of the benchmark dataset, which provides the ground truth for deep learning model development.

Dr. Dian Zhang of the INSIGHT, Centre for Data Analytics at Dublin City University commented

"The large scale visual dataset captured from the undersea camera platform at the SmartBay Test Site provides a vital component, which drives the success of this project. This unique dataset offers a great opportunity for academic researchers to develop novel deep machine learning technologies to explore the wonderful underwater world. "

With an estimated coastline length of 3,171km and Exclusive Economic Zone covering an estimated 89 million hectares, the Irish marine resource provides vital and increasing exploitation opportunities. According to the National Oceanic and Atmospheric Administration in U.S., more than eighty percent of the Ocean remains unobserved and unexplored. High resolution spatial and temporal video datasets have the potential to provide incredible insights into the Irish marine underwater environment.

John Breslin, General Manager SmartBay Ireland commented

“The ability to apply machine learning and automation to the analysis of video data can help scientists to both reduce human error and the effort required to review and analyse hours of footage. Videography is currently employed to quantify the abundance of species such as the Dublin Bay prawn and for mapping sensitive biological communities and habitats. Automation offers huge potential to improve both the efficiency and effectiveness of these analyses.” 

This project is one of seven projects awarded under the 2018/2019 National Infrastructure Access Programme funding call.  The National Infrastructure Access Programme is funded by the Marine Institute under the Marine Research Programme with the support of the Irish Government.