Faster Analysis of Data to Evaluate Bycatch Reduction Efforts in Pollock Fishery

Sexton trawl camera used to collect video footage inside pollock trawls to test the effectiveness of bycatch reduction devices in nets. Credit: NOAA Fisheries.

Scientists used a model to detect and classify fish in videos more quickly than humans. The detection model is called You Only Look Once, version 11 (or YOLOv11). It's helping scientists evaluate the effectiveness of excluders that help salmon escape from fishing nets intended to catch pollock.

YOLOv11 is an object detection deep learning model for images. Scientists at the Alaska Fisheries Science Center customized it to detect and identify both pollock and salmon in fishing nets. This allows scientists to semi-automate the video review process used to evaluate the effectiveness of bycatch reduction devices. They can also observe fish behavior to improve the performance of these devices.

Example image of the salmon excluder that is placed in the last taper section of the net with the camera placement (white box) and the approximate field of view (dashed triangle) shown. The diameter of the net is approximately 2 m at the beginning of the excluder. Credit: NOAA Fisheries.

There were record-high bycatch levels for Chinook salmon in 2005 and chum salmon in 2006. Since then, the commercial fishing industry has been working with fishery scientists and engineers to improve the efficiency of their gear and avoid and reduce salmon bycatch. The industry continues to invest in:

• New fishing net designs
• Modern technologies
• Vessel notification systems when salmon are present

These tools could help them avoid catching salmon in the first place.

Artificial Intelligence Needed to Help Monitor Footage

This new studyusing deep learning was ideal for testing the feasibility of automating the detection of salmon to evaluate the effectiveness of bycatch reduction devices. These devices within the net, known as salmon excluders, allow salmon to exit while keeping pollock in the net. To test their effectiveness, a camera with lighting is positioned with full view of the entry to the bycatch reduction device. It collects video, which scientists review to monitor these devices during fishing.

The use of video in fisheries improves:

• Understanding how processes operate
• Helps study ways to catch fish better and faster
• Using resources in a way that doesn't harm the environment, so we can keep using them in the future

In recent years, video collection has increased because of the availability of low-cost and high-quality camera systems that can be used in many different environments. However, having more footage means that scientists need to spend more time reviewing the video. Machine learning aids scientists in this effort.

Examples of fish detections from three frames for a model trained to detect both pollock and salmon (top, multi-class model) and a model trained to only detect salmon (bottom, salmon only model). Salmon (white) and pollock (grey) detections are shown with the confidence score of the detection as a percentage at the top of the boxes. False positive salmon detections are present for herring in (B, bottom-left), (C), and (F) and salmon is missed in (D) and (E). Credit: NOAA Fisheries.

"We were able to train a publicly available deep learning object detection model to identify salmon and pollock. The model compared well to human performance, with some variability. And the models save us so much time. They can process fishing tows in a matter of hours; humans would need days to weeks to review the same data," said Katherine Wilson, study lead and physical scientist at NOAA Fisheries Alaska Fisheries Science Center.

Our scientists worked with the Pacific States Marine Fisheries Commission on the project, and a contracted B&N Fisheries vessel collected the data. Wilson added, "There are improvements to be made, but this work shows promise. It was challenging for people familiar with identifying salmon to review so many hours of video and identify every salmon. Long-term automated video monitoring will be more cost effective and reduce human error."

Deep Learning Models

Advancements in computer vision, machine learning, and deep learning are helping us process data. These advancements have driven the development of artificial intelligence, machine-learning based solutions that can perform basic tasks, like object identification.

Deep learning is a subset of machine learning. It uses neural networks to capture complex patterns and relationships in data. Convolutional neural networks are a class of deep learning models that perform well with image recognition. Industries use convolutional neural networks to automate video and imagery analysis tasks, such as detecting product defects in manufacturing and diseases in medical imaging. This allows vehicles, equipment, and robots to navigate autonomously.

For research and other marine applications, convolutional neural networks have been used to classify or detect many marine fish species to assist in imagery analysis. Deep learning uses image annotation-the process of labeling and tagging images that will train computer vision models to detect or identify objects. This research used 11,572 salmon and 73,394 pollock annotations from 16,998 video frames.

We collected videos during multiple tows of fishing nets in the summer season. Salmon bycatch is mostly larger chum salmon during summer; in the winter season younger, smaller Chinook salmon are more likely to present.

Scientists evaluated model performance overall and during the following conditions:

• Krill presence
• Varying fish density
• Camera occlusions
• Low lighting

The best model detected 97 percent of fish with 82 percent prediction accuracy. Model results varied across the different conditions; there were many incorrect salmon detections when herring was present in the videos. However, there was a reduced number of misidentified salmon detections for the multi-class salmon and pollock model, compared to a model that only identifies salmon. This indicates that some of the herring errors could be minimized by including it as a class for the model to detect.

We need to do further work to use these deep learning models in the pollock fishery for salmon bycatch mitigation. For example, the models need to be tested and evaluated on videos where more Chinook salmon are present, especially smaller Chinook, to understand their detection performance for those scenarios. Also, we should explore developing models that can distinguish between chum and Chinook.

Fishing net towing behind a vessel. Credit: NOAA Fisheries.

Smarter Fishing, More Sustainable Future

"Automating video analysis has the potential to save time and money," said Wilson. "This research gives more tools to the fishing industry to reduce bycatch and be more sustainable. Developing post-processing tools to automate the evaluation of excluders could also enable bycatch reduction methods to advance more rapidly, and developing real-time tools may provide fishers with the necessary knowledge to operate even more sustainably. Our salmon and pollock detection work showed that deep learning methods are accessible and robust."