A Fast Identification Method for <i>Fritillaria</i> Varieties Based on the Fusion of Electronic Tongue and Electronic Eye Information

MA Jingyu; SUN Tao; WANG Yanrong; LI Xin; ZENG Wanqing; WANG Zhiqiang

doi:10.13386/j.issn1002-0306.2024020161

Volume 45 Issue 18

Sep. 2024

Turn off MathJax

Article Contents

Abstract

References

Supplements

Science and Technology of Food Industry > 2024 > 45(18): 9-18. > DOI: 10.13386/j.issn1002-0306.2024020161

MA Jingyu, SUN Tao, WANG Yanrong, et al. A Fast Identification Method for Fritillaria Varieties Based on the Fusion of Electronic Tongue and Electronic Eye Information[J]. Science and Technology of Food Industry, 2024, 45(18): 9−18. (in Chinese with English abstract). doi: 10.13386/j.issn1002-0306.2024020161.

Citation:

PDF (3712 KB)

A Fast Identification Method for Fritillaria Varieties Based on the Fusion of Electronic Tongue and Electronic Eye Information

School of Computer Science and Technology, Shandong University of Science and Technology, Zibo 255049, China

More Information

Received Date: February 20, 2024
Available Online: July 21, 2024

Graphical Abstract

Abstract

Abstract

Fritillaria is a widely used traditional Chinese medicine, with a complex source and a wide variety of medicinal materials. Different varieties have similar external characteristics, making it difficult to distinguish using traditional methods. To achieve rapid and objective identification of Fritillaria species, this study proposed a method for rapid identification based on electronic tongue and electronic eye combined with a deep learning model. Electronic tongue and electronic eye were utilized to collect gustatory fingerprint and visual image information from different categories of Fritillaria, respectively. An enhanced Transformer encoder based on causal attention mechanism was employed to extract time-series features from the ET signals and augment the ability to extract local features. Meanwhile, an improved ShuffleNetV2 network based on coordinate attention mechanism was used to extract morphological features of EE image and suppress background noise. Subsequently, a feature weighted fusion module was presented to adaptively integrate the feature information extracted from both the electronic tongue and electronic eye, and achieve classification and recognition of the fused features. The experimental results indicated that the proposed information fusion method had better classification performance compared to separate usage of electronic tongue and electronic eye, with a testing accuracy of 98.4%. This study provides a novel approach for rapidly identifying Fritillaria varieties, which offers research insights into the classification and traceability analysis of other Chinese medicinal materials.
- electronic tongue,
- electronic eye,
- ShuffleNetV2,
- Transformer,
- attention mechanism,
- fritillaria

FullText(HTML)

References (31)

References

[1]	BORJIGIN G, WEI F S, JIANG S, et al. Extraction, purification, structural characterization and biological activity of polysaccharides from Fritillaria:A review[J]. International Journal of Biological Macromolecules,2023,242:124817. doi: 10.1016/j.ijbiomac.2023.124817
[2]	张富丽, 刘炜, 毛建霏, 等. 基于核苷类物质建立川贝母药材真伪鉴别模型研究[J]. 中国中药杂志,2021,46(13):3337−3348. [ZHANG F L, LIU WEI, MAO J F, et al. Analysis of flavor components of 6 kinds of Yanshan chestnut by GC-MS combined with electronic nose/electronic tongue[J]. China Journal of Chinese Materia Medica,2021,46(13):3337−3348.] ZHANG F L, LIU WEI, MAO J F, et al. Analysis of flavor components of 6 kinds of Yanshan chestnut by GC-MS combined with electronic nose/electronic tongue[J]. China Journal of Chinese Materia Medica, 2021, 46（13）: 3337−3348.
[3]	AN Y L, LI Y, WEI W L, et al. Species discrimination of multiple botanical origins of Fritillaria species based on infrared spectroscopy, thin layer chromatography-image analysis and untargeted metabolomics[J]. Phytomedicine,2023,123:155228.
[4]	刘燕德, 徐振, 胡军, 等. 基于太赫兹光谱技术的贝母品种鉴别方法研究[J]. 光谱学与光谱分析,2021,41(11):3357−3362. [LIU Y D, XU Zhen, HU Jun, et al. Research on variety identification of fritillaria based on terahertz spectroscopy[J]. Spectroscopy and Spectral Analysis,2021,41(11):3357−3362.] LIU Y D, XU Zhen, HU Jun, et al. Research on variety identification of fritillaria based on terahertz spectroscopy[J]. Spectroscopy and Spectral Analysis, 2021, 41（11）: 3357−3362.
[5]	翟映红, 于国强, 杨永茂, 等. 从川贝母中检出平贝母的理化鉴别方法[J]. 华西药学杂志,2017,32(6):633−635. [ZHAI Yinghong, YU Guoqiang, YANG Yongmao, et al. Physical and chemical identification on distinguishing Fritillariae ussuriensis Bulbus from Fritillariae cirrhosa Bulbus[J]. West China JOurnal of Pharmaceutical Sciences,2017,32(6):633−635.] ZHAI Yinghong, YU Guoqiang, YANG Yongmao, et al. Physical and chemical identification on distinguishing Fritillariae ussuriensis Bulbus from Fritillariae cirrhosa Bulbus[J]. West China JOurnal of Pharmaceutical Sciences, 2017, 32（6）: 633−635.
[6]	王恬, 谈娜娜, 陆佳, 等. 川贝母生物碱的化学组成分析与IL-6结合活性成分的虚拟筛选[J]. 宝鸡文理学院学报(自然科学版),2023,43(4):30−34. [WANG Tian, TAN Nana, LU Jia, et al. Chemical composition analysis of alkaloids form Fritillaria cirrhosa D. Don and virtual screening of IL-6 binding active ingredients[J]. Journal of Baoji University of Arts and Sciences (Natural Science),2023,43(4):30−34.] WANG Tian, TAN Nana, LU Jia, et al. Chemical composition analysis of alkaloids form Fritillaria cirrhosa D. Don and virtual screening of IL-6 binding active ingredients[J]. Journal of Baoji University of Arts and Sciences （Natural Science）, 2023, 43（4）: 30−34.
[7]	杨银爱, 郜海燕, 牛犇, 等. 基于电子舌对不同品种鲜莲子滋味成分的分析[J]. 食品工业科技, 2023, 44(24):319-326. YANG Yin'ai, GAO Haiyan, NIU Ben, et al. Analysis of taste components in different varieties of fresh lotus seeds based on electronic tongue[J]. Science and Technology of Food Industry, 2023, 44(24):319−326.
[8]	张晓, 吴宏伟, 于现阔, 等. 基于电子眼技术的穿心莲质量评价[J]. 中国实验方剂学杂志,2019,25(1):189−195. [ZHANG Xiao, WU Hongwei, YU Xiankuo, et al. Quality evaluation of andrographis herba based on electronic-eye technique[J]. Chinese Journal of Experimental Traditional Medical Formulae,2019,25(1):189−195.] ZHANG Xiao, WU Hongwei, YU Xiankuo, et al. Quality evaluation of andrographis herba based on electronic-eye technique[J]. Chinese Journal of Experimental Traditional Medical Formulae, 2019, 25（1）: 189−195.
[9]	段金芳, 肖洋, 刘影, 等. 一测多评法与电子眼和电子舌技术相结合优化山茱萸蒸制时间[J]. 中草药,2017,48(6):1108−1116. [DUAN Jinfang, XIAO Yang, LIU Ying, et al. Optimization of steaming time of Cornus officinalis by QAMS combined with electronic-eye and electronic-tongue techniques[J]. Chinese Traditional and Herbal Drugs,2017,48(6):1108−1116.] DUAN Jinfang, XIAO Yang, LIU Ying, et al. Optimization of steaming time of Cornus officinalis by QAMS combined with electronic-eye and electronic-tongue techniques[J]. Chinese Traditional and Herbal Drugs, 2017, 48（6）: 1108−1116.
[10]	陈佳瑜, 袁海波, 沈帅, 等. 基于智能感官多源信息融合技术的滇红工夫茶汤综合感官品质评价[J]. 食品科学,2022,43(16):294−301. [CHEN Jiayu, YUAN Haibo, SHEN Shuai, et al. Comprehensive sensory quality evaluation of dianhong congou tea infusions using intelligent sensory multi-source information fusion technology[J]. Food Science,2022,43(16):294−301.] doi: 10.7506/spkx1002-6630-20211028-321 CHEN Jiayu, YUAN Haibo, SHEN Shuai, et al. Comprehensive sensory quality evaluation of dianhong congou tea infusions using intelligent sensory multi-source information fusion technology[J]. Food Science, 2022, 43（16）: 294−301. doi: 10.7506/spkx1002-6630-20211028-321
[11]	WANG S C, ZHANG Q, LIU C Z, et al. Synergetic application of an E-tongue, E-nose and E-eye combined with CNN models and an attention mechanism to detect the origin of black pepper[J]. Sensors and Actuators A:Physical,2023,357:114417. doi: 10.1016/j.sna.2023.114417
[12]	LU L, HU Z Q, HU X Q, et al. Quantitative approach of multidimensional interactive sensing for rice quality using electronic tongue sensor array based on information entropy[J]. Sensors and Actuators B:Chemical,2021,329:129254. doi: 10.1016/j.snb.2020.129254
[13]	王首程, 于雪莹, 高继勇, 等. 基于电子舌和电子鼻结合DenseNet-ELM的陈醋年限检测[J]. 食品与机械,2022,38(4):72−80,133. [WANG Shoucheng, YU Xueying, GAO Jiyong, et al. Age detection of mature vinegar based on electronic tongue and electronic nose combined with DenseNet-ELM[J]. Food & Machinery,2022,38(4):72−80,133.] WANG Shoucheng, YU Xueying, GAO Jiyong, et al. Age detection of mature vinegar based on electronic tongue and electronic nose combined with DenseNet-ELM[J]. Food & Machinery, 2022, 38（4）: 72−80,133.
[14]	SUN Y T, MA H Z. Research progress on oil-immersed transformer mechanical condition identification based on vibration signals[J]. Renewable and Sustainable Energy Reviews,2024,196:114327. doi: 10.1016/j.rser.2024.114327
[15]	ZHANG S Y, LIAN C, XU B R, et al. A token selection-based multi-scale dual-branch CNN-transformer network for 12-lead ECG signal classification[J]. Knowledge-Based Systems,2023,280:111006. doi: 10.1016/j.knosys.2023.111006
[16]	GONG A J, GUO L B, YU Y, et al. Spectrum-image dual-modality fusion empowered accurate and efficient classification system for traditional Chinese medicine[J]. Information Fusion,2024,101:101981. doi: 10.1016/j.inffus.2023.101981
[17]	王健庆, 戴恺, 李子柔. 基于深度学习的中药饮片图像识别研究 [J]. 时珍国医国药, 2020, 31(12):2930-2933. WANG Jianqing, DAI Kai, LI Zirong. Research on image recognition of traditional chinese medicine decoction pieces based on deep learning[J]. Lishizhen Medicine and Materia Medica Research, 2020, 31(12):2930−2993.
[18]	周丽媛, 赵启军, 高定国. 基于注意力引导深度纹理特征学习的复杂背景藏药材切片图像识别[J]. 世界科学技术-中医药现代化,2022,24(12):4825−4832. [ZHOU Liyuan, ZHAO Qijun, GAO Dingguo. Recognition of Tibetan medicinal slices in images with complex background via attention-guided deep texture feature learning[J]. World Science and Technology—Modernization of Traditional Chinese Medicine and Materia Medica,2022,24(12):4825−4832.] ZHOU Liyuan, ZHAO Qijun, GAO Dingguo. Recognition of Tibetan medicinal slices in images with complex background via attention-guided deep texture feature learning[J]. World Science and Technology—Modernization of Traditional Chinese Medicine and Materia Medica, 2022, 24（12）: 4825−4832.
[19]	CHEN Y, HUANG Y Q, ZHANG Z Z, et al. Plant image recognition with deep learning:A review[J]. Computers and Electronics in Agriculture,2023,212:108072. doi: 10.1016/j.compag.2023.108072
[20]	SHANG Y Y, XU X S, JIAO Y T, et al. Using lightweight deep learning algorithm for real-time detection of apple flowers in natural environments[J]. Computers and Electronics in Agriculture,2023,207:107765. doi: 10.1016/j.compag.2023.107765
[21]	彭红星, 何慧君, 高宗梅, 等. 基于改进ShuffleNetV2模型的荔枝病虫害识别方法[J]. 农业机械学报,2022,53(12):290−300. [PENG Hongxing, HE Huijun, GAO Zongmei, et al. Litchi diseases and insect pests identification method based on improved ShuffleNetV2[J]. Transactions of the Chinese Society for Agricultural Machinery,2022,53(12):290−300.] doi: 10.6041/j.issn.1000-1298.2022.12.028 PENG Hongxing, HE Huijun, GAO Zongmei, et al. Litchi diseases and insect pests identification method based on improved ShuffleNetV2[J]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53（12）: 290−300. doi: 10.6041/j.issn.1000-1298.2022.12.028
[22]	SUI Y, WANG X, WU J, et al. Causal attention for interpretable and generalizable graph classification[Z]. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Washington DC, USA; Association for Computing Machinery. 2022:1696–705.10. 1145/3534678.3539366.
[23]	WEN G, LI S, LIU F, et al. YOLOv5s-CA: A modified YOLOv5s network with coordinate attention for underwater target detection [J]. Sensors, 2023, 23(7): 3367.
[24]	杨正伟, 张鑫, 李庆盛, 等. 基于电子舌及一维深度CNN-ELM模型的普洱茶贮藏年限快速检测[J]. 食品与机械,2020,36(8):45−52. [YANG Zhengwei, ZHANG Xin, LI Qingsheng, et al. A fast detection Pu-erh tea storage based on the voltammetric electronic tongue and one-dimension CNN-ELM[J]. Food & Machinery,2020,36(8):45−52.] YANG Zhengwei, ZHANG Xin, LI Qingsheng, et al. A fast detection Pu-erh tea storage based on the voltammetric electronic tongue and one-dimension CNN-ELM[J]. Food & Machinery, 2020, 36（8）: 45−52.
[25]	YANG Z W, GAO J Y, WANG S C, et al. Synergetic application of E-tongue and E-eye based on deep learning to discrimination of Pu-erh tea storage time[J]. Computers and Electronics in Agriculture,2021,187:106297. doi: 10.1016/j.compag.2021.106297
[26]	金鑫宁, 刘铭, 桑恒亮, 等. 基于电子舌和电子眼结合改进MobileNetv3的黄芪快速溯源检测[J]. 食品与机械,2023,39(6):37−47. [JIN Xinning, LIU Ming, SANG Hengliang, et al. Fast traceability detection of Astragalus membranaceus based on the combination of electronic tongue and electronic eye to improve MobileNetv3[J]. Food & Machinery,2023,39(6):37−47.] JIN Xinning, LIU Ming, SANG Hengliang, et al. Fast traceability detection of Astragalus membranaceus based on the combination of electronic tongue and electronic eye to improve MobileNetv3[J]. Food & Machinery, 2023, 39（6）: 37−47.
[27]	李练兵, 高国强, 吴伟强, 等. 考虑特征重组与改进Transformer的风电功率短期日前预测方法[J]. 电网技术,2024,48(4):1466−1480. [LI Lianbing, GAO Guoqiang, WU Weiqiang, et al. A short-term day-ahead wind power prediction method considering feature recombination and improved transformer[J]. Power System Technology,2024,48(4):1466−1480.] LI Lianbing, GAO Guoqiang, WU Weiqiang, et al. A short-term day-ahead wind power prediction method considering feature recombination and improved transformer[J]. Power System Technology, 2024, 48（4）: 1466−1480.
[28]	JIN X, JIANG J X, LI Y, et al. Improved ShuffleNetV2 for action recognition in BPPV treatment[J]. Biomedical Signal Processing and Control,2024,88:105601. doi: 10.1016/j.bspc.2023.105601
[29]	游小荣, 李淑芳. 融合迁移学习和集成学习的服装风格图像分类方法[J/OL]. 现代纺织技术:1−8[2024-06-18]. https://doi.org/10.19398/j.att.202401007. [YOU Xiaorong, LI Shufang. Fashion style image classification method integrating transfer learning and ensemble learning: 1−8[2024-06-18]. https://doi.org/10.19398/j.att.202401007.] YOU Xiaorong, LI Shufang. Fashion style image classification method integrating transfer learning and ensemble learning: 1−8[2024-06-18]. https://doi.org/10.19398/j.att.202401007.
[30]	武晓春, 郜文祥. 基于tSNE多特征融合的JTC轨旁设备故障检测[J]. 铁道科学与工程学报,2024,21(3):1244−1255. [WU Xiaochun, GAO Wenxiang. Fault detection of JTC trackside equipment based on tSNE multi-feature fusion[J]. Journal of Railway Science and Engineering,2024,21(3):1244−1255.] WU Xiaochun, GAO Wenxiang. Fault detection of JTC trackside equipment based on tSNE multi-feature fusion[J]. Journal of Railway Science and Engineering, 2024, 21（3）: 1244−1255.
[31]	崔佳旭, 杨博. 贝叶斯优化方法和应用综述[J]. 软件学报,2018,29(10):3068−3090. [CUI J X, YANG B. Survey on Bayesian optimization methodology and applications[J]. Journal of Software,2018,29(10):3068−3090.] CUI J X, YANG B. Survey on Bayesian optimization methodology and applications[J]. Journal of Software, 2018, 29（10）: 3068−3090.