Enhancing Visual Coding Through Collaborative Perception

作者：An, Lingling; Yan, Zhen; Wang, Weizheng^*; Liu, Jian K.; Yu, Keping

来源：IEEE Transactions on Cognitive and Developmental Systems, 2023, 15(4): 1744-1753.

DOI：10.1109/TCDS.2022.3203422

• Feature compilation
• multimodal stimulus
• nonlinearity
• visual coding

A central challenge facing the nature human-computer interaction involves understanding how neural circuits process visual perceptual information to improve the user's operation ability under complex tasks. Visual coding models aim to explore the biological characteristics of retinal ganglion cells to provide quantitative predictions of responses to a range of visual stimuli. The existing visual coding models lack adaptability in natural and complex scenes. Therefore this article proposes an enhanced visual coding model through collaborative perception. Our model first extracts the multimodal spatiotemporal features of the input video to simulate the retinal response characteristics adaptively. Second, it uses the basis function to compile the input stimulus into a multimodal stimulus matrix. Afterward, the upstream and downstream filters reform the stimulus matrix to generate the spike sequence. Experiments show that the proposed model reproduces the physiological characteristics of ganglion cells in the biological retina, leading to the high accuracy, good adaptability, and biological interpretability in comparison with its rivals.

• 单位
  西安电子科技大学