LPAS: Live Plankton Analysis System

Following discussions with farmers trialling the system, it is believed the retrofit LPAS will have a major impact on better and quicker identification of HABs, which would reduce labour and provide early warning of the presence HABs of interest.

BACKGROUND

Several types of plankton can seriously affect the gill health of fish on marine farms, negatively impacting performance and potentially causing mortality. Also, feeding when these harmful plankton species are present only exacerbates the issue as fish are drawn up to the surface, where there’s more plankton and they have an increased oxygen demand while there, so drawing even more plankton into their gills. The type of damage depends on the species and density of plankton; some may produce toxic compounds while others can cause physical damage or create hypoxic conditions.

Traditionally, once a day, farmers manually test water samples using a microscope, looking for certain species. If a harmful type is detected at a high enough concentration, the farmers will act to protect the fish by stopping feeding, and/or activating a bubble curtain or aeration systems.

This sampling method has a number of weaknesses as it is performed only once or twice a day, with samples taken from a single point. Also, it is not consistent, as different people may make different interpretations. It is also highly time-consuming.

Existing flow-cytometers are accurate; however they are prohibitively expensive and complex.  Processing time for information is very slow, as data cannot be accessed live but is post-processed.  While data can be collected over longer periods, the cost of this system mean a single system will usually be left at a strategic point and then post-processed by environmental teams; a laborious process that does not provide warning at the right time to implement mitigation measures to protect the stocks.

The LPAS system uses microscope camera technology, which will be combined in future developments with an automatic water sampling system. Once the sample has been taken, the camera takes an image of it and transfers it to the onboard computer. The AI deep learning-based image analysis software will then process images live onboard the device and provide a digital output of what it has measured. LPAS could be used as a stand-alone sensor providing a reliable, low-cost alert system for deploying on multiple locations around each loch or farm, creating a monitoring and warning network.

AIMS

The aim of the project was to build and test the feasibility of a complete prototype sensor that can automatically sample, identify and count harmful phytoplankton species using video analysis. The system would include a programmed water-sampler, a microscopic camera system, and a computer system with software that could carry out automatic recognition and counting of the different plankton species.

Experimental study

Not all objectives have been achieved, but additional tasks were taken on. The extensive effort required to train the AI on the original number of species of interest was quickly realised as being unachievable within the scope of the project. Instead, the team focused on demonstrating that autonomous identification to a usable accuracy was possible using a smaller number of HAB species, proving the principle and enabling a viable product on target to be trialled.

The first piece of work was to develop a fully operational water sampling system. The components of this system were the resident pumps, the water lines to the microscope camera, and the integration of the water lines with the sample chamber in the microscope camera.

In the original project plan, the requirements of a sampling system were to pump continuously 24/7 and analyse the samples for any HAB species present. This would then ensure that continuous sampling could be achieved and therefore early warnings could be presented to the farmers at any time. For a successful design, the pump modules were pivotal in their capability of conveying water to the surface from multiple depths, while retaining the integrity of any microscopic organisms in the sample.

A number of different pumps were tested and two designs were identified as giving the required outcomes. One was a peristaltic pump and the other a hydraulic cylinder pump. The latter was preferred for its ability to pump water in a more controlled way. There will be a need to develop the sampling system further in the future for the ability to undertake repeated sampling from different water depths.

Most relevant to the integration of the water lines to the microscope camera were biofouling, air bubbles, and low phytoplankton numbers in the flow cell volume, as well as certain plankton species preferring to stick to the inside of the flow-through sample chamber. While these challenges have so far prevented the development a fully automated, low-maintenance sampling system, the insights gained during this phase of the development have been invaluable.

The microscope camera was configured successfully to enable the collection of images at the right quality for the machine vision AI to train on. Achieving this involved the testing of different microscopes with camera hardware, configurations and magnifications.

The illumination was done using a multicolour LED, which could be controlled in such way that the water sample was lit up with different wavelengths of light. This configuration enabled images to be taken in all combinations of wavelength and illumination. Each series of images was recorded so that the machine vision algorithm could then begin the analysis on a computer system.

The requirements of the imaging process included the ability to capture the morphological features and dimensions of each phytoplankton cell. Automating this process is challenging, and several issues were found with the magnification and sharpness of the image. To remedy this, the initial microscope and camera set-up were exchanged for a standard, off-the-shelf, brightfield microscope in combination with a high-resolution 20Mp camera.

Access to pure cultures of known harmful and non-harmful algal species was essential for acquiring the best training phytoplankton images. To train a machine vision system for just one species requires hundreds of images in different life stages, orientations and quality. This activity proved highly time-consuming, so the scope of the training work was restricted to two species, aiming to prove the approach was feasible and to develop a learning process for future image library development.

The University of Aberdeen team provided samples of different species that could be imaged, resulting in a successful first AI recognition of two species and therefore proving the feasibility of the project.

In parallel with the development of the microscope and water sampler, a flow cytometry technology method to identify and identify HAB species and capture images for algorithm training purposes was developed. Flow cytometry is a well-established and powerful method that captures the dimensions and morphology of microscopic cells in a stream of fluid. It is especially relevant for wild samples, where the target species are typically clumped together in colonies alongside other types of microorganisms and debris.

Although a very powerful laboratory method, the cytometry approach did not deliver the same kind of images as taken by the microscope camera. These images were not effective for training the AI in that they were too dissimilar. This issue, and the prohibitive cost, made the team decide to continue with microscope identification only. Farm staff is already familiar with microscopes and their optics, and processing units for machine vision and AI are dropping in price, with modules economically viable for use in commercial products aimed at the aquaculture market. This means that it will be possible to integrate a trained HAB identification AI onto already-existing procedures and equipment.

The team has accumulated a significant collection of important HAB species. Post-project, the HAB species currently in culture at Aberdeen are Dinophysis sp, Acuminta sp, Ceratium sp., Scripsiella trochoidea, Chaetoceros sp., Pseudonitzschia sp, Rhizosolenia setigera and Alexandium sp.; possibly one of the largest HAB culture collections currently held in Scotland.

The prototype system deployed at various farms relies on a manual water sampling and while this is not optimal, it is acceptable, since the benefit of the AI system will still save labour and bring a significant consistency to the identification and counting results. It is, however, OTAQ’s ultimate goal to deliver a fully automated, low-maintenance system that is robust and reliable.

The purpose of this prototype system was to build up the training image library for the machine vision AI. Additionally, the project has already created awareness of the possibilities of an upcoming AI system to help the analysis and counting of HAB species.

The partnership between the University of Aberdeen and OTAQ continues in a KTP project that aims to deliver a first-of-its-kind commercial product and on-going developments towards a more automated system. The first release of the HAB identification AI took place summer 2023, offering rapid and autonomous identification of several species.

Outcomes

The project showed the difficulties in achieving clear and meaningful images of plankton and showed the need for real-life samples from aquaculture sites as being the most representative of actual conditions.

The time it would take to obtain representative species from farms and get cultures set up was underestimated. However, the project demonstrated what was feasible to do and supported the continued investment in the ongoing follow-on KTP project, which will not only deliver the first-to-market AI retrofit system but also bring on a new team member from the KTP to build OTAQ’s capabilities in this new AI offering.

OTAQ has, since the project concluded, provided and deployed a number of test devices at fish farms around the world, and these are continuously taking images that will further train the AI to eventually identify all relevant HAB species with high certainty.

There is presently no other commercial offering of machine vision-based identification of HABs. During the LPAS project, OTAQ was able to undertake some preliminary testing of the sampling unit and the image capture features, however it was not until the conclusion of the project that the first devices were deployed at several farms.

Following discussions with farmers trialling the system, it is believed the retrofit LPAS will have a major impact on better and quicker identification of HABs, which would reduce labour and provide early warning of the presence HABs of interest.

A much deeper understanding of HABs species and lifecycles was gained. The difficulty in obtaining samples that can be used to train the AI was originally underestimated. It seems that laboratory-based samples do not form a good basis for AI training. Additionally, the requirements for the sampling system are now better understood, which will eventually help with future developments in that direction.

During and after the project, a number of trade shows were visited including Aquaculture UK 2022 in Aviemore and, through SAIC support, Aqua Nor 2023 in Trondheim, Norway, as part of the Scottish Pavilion. The project and outcomes were also presented at the SAIC Sustainable Aquaculture Summit in May 2023 in Glasgow.

SAIC is also supporting OTAQ and the wider project team with dissemination in the form of PR and animation collateral.

The project and ongoing work have attracted a large amount of interest from commercial and academic partners. Supported by a GCRF grant, the team visited South Africa where Abalone farmers are also open to trailing the LPAS system to detect and identify HABs that are relevant to their region.

ADDITIONAL INFORMATION

Prototype HAB alarms ‘will be on farms by January’ (fishfarmingexpert.com)

Automated identification of harmful algae blooms - Fish Farmer Magazine

Using AI to tackle harmful algal blooms | The Fish Site

Best Farm management software for aquaculture 2023 | NeuroSYS

Scottish aquaculture industry warns about the growing risk of harmful algal blooms (sustainableaquaculture.com)