Been writing on the impact of eFADS in the tuna fishery for a new years now… I find them fascinating, and they have revolutionised the fishery, to a point where I think that if they were to disappear, a substantial amount (50%?) of the fleet would collapse… particularly as many of the “pre eFAD” skippers retire and their traditional knowledge goes with them.
I see them as treat but an opportunity at the same time… recently wrote about an paper analysing if drifting eFADs can be a good source of independent data for Stock Assessments…. So when new FADs papers come I’m keen… and particularly if they can contribute to stock assessment.
Ergo, one named “TUN-AI: Tuna biomass estimation with Machine Learning models trained on oceanography and echosounder FAD data” was in that alley.
I’m not going to pretend I understood the whole paper… as is dense in statistics and programming…. But the abstract, discussion and conclusion make sense in my little brain… so here they are. But as always: Read the original!
Abstract
The use of dFADs by tuna purse-seine fisheries is widespread across oceans, and the echo-sounder buoys attached to these dFADs provide fishermen with estimates of tuna biomass aggregated to them. This information has potential for gaining insight into tuna behaviour and abundance, but has traditionally been difficult to process and use. The current study combines FAD logbook data, oceanographic data and echo-sounder buoy data to evaluate different Machine Learning models and establish a pipeline, named TUN-AI, for processing echo-sounder buoy data and estimating tuna biomass (in metric tons, t) at various levels of complexity: binary classification, ternary classification and regression. Models were trained and tested on over 5000 sets and over 6000 deployments. Of all the models evaluated, the best performing one uses a 3-day window of echo-sounder data, oceanographic data and position/time derived features. This model is able to estimate if tuna biomass was higher than 10 t or lower than 10 t with an F1-score of 0.925. When directly estimating tuna biomass, the best model (Gradient Boosting) has an error (MAE) of 21.6 t and a relative error (SMAPE) of 29.5%, when evaluated over sets. All models tested improved when enriched with oceanographic and position-derived features, highlighting the importance of these features when using echo-sounder buoy data. Potential applications of this methodology, and future improvements, are discussed.
Discussion
The purpose of this paper is to present a new pipeline for estimating tuna biomass aggregated at dFADs, named TUN-AI. The pipeline uses echo-sounder buoy, oceanographic and FAD logbook data to train multiple Machine Learning models that solve different tasks relevant to fisheries operations. To find the most accurate methodology, we tested the performance of classification and regression methods, as well as the relative impact of including different data sources on model performance. The approach used in the current study differs from previous work in several ways. Although the methodology in Baidai et al. (2020) is similar to ours, they only tackle the classification problem, and thus they are not able to directly estimate the metric tons of tuna under the dFAD. They also have a smaller sample size in terms of sets (albeit similar) that covers only the Atlantic and Indian oceans. Finally, we have also tested several models for each task in order to find the one with the best overall performance. Model performance of the two studies are hard to compare directly, since the models have been trained on different datasets. It is worth noting that other studies that address the regression problem, like Orue et al. (2019a); Lopez et al. (2016), cannot be directly compared with this study for a number of reasons: First, their sample size is orders of magnitude smaller (21 and 138 sets, respectively). Second, they only have data from a single ocean (Atlantic and Indian, respectively). Finally, they perform a statistical model fit, while our study involves a full ML approach with train-test split and a much larger dataset. This means that TUN-AI is expected to have the reported performance on new, unseen data, while there is no guarantee that the models in Orue et al. (2019a) and Lopez et al. (2016) will generalize as well, as they use the same dataset for model fit and error evaluation. In addition, the assumptions and data-processing methods applied in other work may not be directly comparable to the process described here. For example, Orue et al. (2019a); Lopez et al. (2016) assume that tunas only occupy layers deeper than 25 m, thus omitting biomass estimates from shallower layers in their analyses. In our case, all layers were considered, as skipjack tuna are known to prefer warmer surface waters in areas where the thermocline is shallow (Andrade, 2003). In fact, later studies using the same approach as Lopez et al. (2016) did not achieve significant improvements on biomass estimates (Orue et al., 2019b). When developing tuna presence/absence and classification models, Baidai et al. (2020) also chose to consider all layers in their analyses, which used data from a different brand of echo-sounder buoys in the Atlantic and Indian oceans, but did not consider oceanographic parameters in their models.
Our analysis also evaluated the impact of oceanographic conditions and position-derived variables on model performance. Across all tasks and models, the inclusion of additional features clearly improved when compared to the model that only used echo-sounder data. This highlights the importance of enriching biomass estimates with contextual information when using data from echo-sounder buoys attached to dFADs. Although at first glance this would prove laborious, the current pipe-line draws from automated processes for extracting the oceanographic data and relating it to the other available datasets, thus the added complexity translated to only a few minutes of additional computation time on standard equipment. Given the improvement in model accuracy when including this information, and the potential applications of having accurate methods for estimating tuna biomass at dFADs, we consider that it is worthwhile to use all available information.
Previous studies have investigated the relationship between tropical tuna distribution and oceanographic conditions, both through catch data from observer logbooks and from dFAD data. For instance, in the Atlantic and Indian oceans skipjack tuna has been known to aggregate around upwelling systems and productive features where feeding habitat is favorable, and variables such as sea surface temperature or SSH have been shown to have a significant relation with tuna distribution (Druon et al., 2017; Lopez, 2017). In addition, Spanish fishers using echo-sounder buoys on dFADs consider that the oceanographic context of the dFAD, and the characteristics of each ocean influence the accuracy of biomass estimates provided by buoys (Lopez and Scott, 2014). It is worth noting that the oceanographic variables included in the current study were at surface level only (except for thermocline depth and SSHa). However, given the fact that tuna distribution within the water column is largely temperature dependent (Aoki et al., 2020; Hino et al., 2019; Tanabe et al., 2017) it is likely that models would further improve when considering variables depth-wise.
Models could be also enriched by considering dFAD soak time, which has been relevant in previous research, or presence/absence of bycatch species and other species of tuna (Orue et al., 2019b; Lopez, 2017; Forget et al., 2015). Indeed, the presence of small schooling bycatch species such as oceanic triggerfish (Canthidermis maculata), which has been found at dFADs (Forget et al., 2015), could be affecting the biomass estimates provided by the echo-sounder buoys. We see some evidence of this when examining the accuracy of our models in the binary classification task (see Table 7): the model performed worse when being tested on sets. This is likely due to cases where species other than tuna were contributing to the echo-sounder signal, so the buoy’s biomass estimates were high even though real catch of tuna at the dFAD was low. This was reflected particularly in the Atlantic Ocean, where accuracy was lowest and bycatch is higher than in other oceans (Restrepo et al., 2017). Future studies could include the bycatch information recorded in the FAD logbooks to account for this effect. Looking more closely into the binary classification model, we can see that the worst performance comes from trying to distinguish sets that catch less than 10 t from the rest. This makes sense, since the fact that a fishing vessel decided to set on a specific FAD is probably already a good indicator of the echo-sounder measurements showing a strong signal, which probably correspond to other fish species.
These are clearly the hardest observations to distinguish. In the current study, species composition of the catch data was not considered. As the echo-sounder buoys used in this study calculate biomass estimates based on the target strength of skipjack tuna, it is likely that the presence of other tuna species such as bigeye, which has a lower target strength (Boyra et al., 2018, 2019) and thus stronger acoustic response, would impact biomass estimates from the echo-sounder buoys, contributing to errors within the models used to estimate aggregation size.
Most traditional echo-sounder buoys do not currently differentiate between species when giving biomass estimates, though recent buoy models, such as the ISD+buoys included in the study, provide a daily estimate of species composition together with biomass estimates. Although previous studies have highlighted the importance of considering species composition when estimating biomass (Moreno et al., 2019; Santiago et al., 2016), the information from these buoys has not yet been applied, and should be considered in future studies. Likewise, this study only used echo-sounder information from one buoy manufacturer, although vessels may use buoys from up to four different brands.
The echo-sounders and buoys from each manufacturer vary on a number of levels: beam angle, sampling method, echo-sounder frequency, etc.; so the same machine learning models used here cannot be applied directly. Nonetheless, future studies could explore the application of similar ML models to the biomass estimates provided by other buoy brands in order to establish manufacturer-specific echo-sounder signal processing pipelines. In the case of classification models, the confusion matrices in Fig. 4 showed that most cases where the model misclassified the tuna aggregation size were when biomass estimates were medium (10 t ≤ y < 30 t) or high (y ≥ _30 t). On the other hand, when examining the regression models we found that estimated tuna biomass tended to be lower than observed tuna biomass as the latter increased (Fig. 6). This could be due to various factors: firstly, catches over 100 t were relatively rare (in our data, 315 events, 8.1%) and thus the model did not have sufficient examples to properly learn from them; secondly, buoys are only able to estimate the biomass of tuna within the echo-sounder beam, and in tuna aggregations over 100 t it is unlikely that the entire school is under the buoy at the same time. Furthermore, it is possible that with large schools of tuna the echo-sounder signal saturates, and the biomass estimates provided by the echo-sounder buoy become an underestimation. To resolve this issue, it could be interesting to apply specialist models which could be adjusted according to when aggregations are predicted to be small or large. It is also worth noting that fishermen do not choose on which buoys they set at random, but based on the biomass estimation provided to them, and thus could be biased towards buoys with higher biomass estimations. This could be a further reason why our ML models underestimated the observed tuna biomass when its values were above 30 t in the case of the ternary classification, or 100 t in the case of the regression tasks. Future studies exploring the reasons behind fishermen’s decisions to visit a buoy could provide further insight into this point. This tendency to underestimate should also be taken into account when using information derived from echo-sounder buoys for stock assessments (Santiago et al., 2016), although consistent underestimation should have no effect on patterns present in the temporal series.
The pelagic and migratory nature of tuna make it a challenging species to study using traditional methods, as only a small part of this species’ habitat can be observed in real time at any given moment. However, dFADs equipped with high-tech echo-sounder buoys, as those used in the current study, can be used as floating open-ocean sampling stations, gathering constant and current information from various sensors.
As shown here, and as remarked by other authors (Orue et al., 2019a), although the information provided by the echo-sounder alone is valuable, it still requires extensive cleaning and filtering prior to use. These initial protocols can avoid errors due to measurements taken by the buoys on board or on land, but the ML techniques used here go a step further in processing the echo-sounder signal to correctly estimate tuna biomass beneath any given echo-sounder buoy. This way, the echo-sounder signals can provide insight into the whereabouts and behaviour of tuna around dFADs, at a fraction of the cost of what scientific expeditions with the same scope could achieve. As highlighted by previous authors, this type of data could be used for fishery-independent abundance indices, improving knowledge on species distribution or better understanding the factors driving aggregation and disaggregation processes of tuna at dFADs (Santiago et al., 2016; Lopez et al., 2016; Moreno et al., 2019).
The current study represents an important step in this direction, being the first to successfully evaluate the performance of numerous ML models, following correct ML methodology using large amounts of data to both train and test each model. This has allowed Tun-AI to tackle the most tasks of various complexities, including directly estimating the amount of tuna aggregated to the dFAD, achieving high degrees of accuracy. As evidenced here, when the massive data provided by echo-sounder buoys attached to dFADs is further enriched with remote-sensing data on conditions across oceans, and trained with reliable ground-truthed data, ML proves a powerful tool for extracting otherwise hidden patterns in these datasets, potentially furthering knowledge on pelagic species.