A semi-supervised generative adversarial network for amodal instance segmentation of piglets in farrowing pens

Authors：Huang, Endai; He, Zheng; Mao, Axiu; Ceballos, Maria Camila; Parsons, Thomas D.; Liu, Kai^*

Source：COMPUTERS AND ELECTRONICS IN AGRICULTURE, 2023, 209: 107839.

DOI：10.1016/j.compag.2023.107839

• Precision livestock farming
• Deep learning
• Computer vision
• Animal monitoring
• Farrowing crate
• De-occlusion

Occlusions, such as farrowing pens in piggeries, hinder computer vision applications for automated animal monitoring. Amodal instance segmentation (AIS), aiming to predict a complete mask of an occluded target, is a promising solution. However, AIS usually requires amodal datasets, which are challenging to create and limit the application of AIS. To solve this problem, we proposed a novel semi-supervised generative adversarial network (GAN) for AIS, denoted "the AISGAN". Our AISGAN only requires a regular modal dataset and generate amodal samples by random occlusions, making the AIS method more applicable. A corresponding segmentation loss was added to overcome mode collapse of GAN. The results showed that the AISGAN achieved a mean Intersection of Union (mIoU) of 0.823 and outperformed the mIoUs of Mask RCNN, Raw, and Convex Hull (0.801, 0.780, and 0.778, respectively). As a semi-supervised method, the mIoU of our AISGAN was further enhanced (by 0.6%) when we fine-tuned it with unlabeled new data, showing its extensibility to new unseen scenarios. The visual-ization demonstrates that the AISGAN can produce realistic masks of piglets, including details of their noses and legs, even under heavily occluded conditions. With the AISGAN, we achieved an occlusion-resistant spatial distribution analysis of the piglets in farrowing pens. Thus, the AISGAN is a promising tool to manage occlusion problems for automated animal monitoring in complex housing environments.