A Fast Identification Method for <i>Fritillaria</i> Varieties Based on the Fusion of Electronic Tongue and Electronic Eye Information

MA Jingyu; SUN Tao; WANG Yanrong; LI Xin; ZENG Wanqing; WANG Zhiqiang

doi:10.13386/j.issn1002-0306.2024020161

Volume 45 Issue 18

Sep. 2024

Turn off MathJax

Article Contents

Abstract

References

Supplements

Science and Technology of Food Industry > 2024 > 45(18): 9-18. > DOI: 10.13386/j.issn1002-0306.2024020161

MA Jingyu, SUN Tao, WANG Yanrong, et al. A Fast Identification Method for Fritillaria Varieties Based on the Fusion of Electronic Tongue and Electronic Eye Information[J]. Science and Technology of Food Industry, 2024, 45(18): 9−18. (in Chinese with English abstract). doi: 10.13386/j.issn1002-0306.2024020161.

Citation:

PDF (3712 KB)

A Fast Identification Method for Fritillaria Varieties Based on the Fusion of Electronic Tongue and Electronic Eye Information

School of Computer Science and Technology, Shandong University of Science and Technology, Zibo 255049, China

More Information

Received Date: February 20, 2024
Available Online: July 21, 2024

Graphical Abstract

Abstract

Abstract

Fritillaria is a widely used traditional Chinese medicine, with a complex source and a wide variety of medicinal materials. Different varieties have similar external characteristics, making it difficult to distinguish using traditional methods. To achieve rapid and objective identification of Fritillaria species, this study proposed a method for rapid identification based on electronic tongue and electronic eye combined with a deep learning model. Electronic tongue and electronic eye were utilized to collect gustatory fingerprint and visual image information from different categories of Fritillaria, respectively. An enhanced Transformer encoder based on causal attention mechanism was employed to extract time-series features from the ET signals and augment the ability to extract local features. Meanwhile, an improved ShuffleNetV2 network based on coordinate attention mechanism was used to extract morphological features of EE image and suppress background noise. Subsequently, a feature weighted fusion module was presented to adaptively integrate the feature information extracted from both the electronic tongue and electronic eye, and achieve classification and recognition of the fused features. The experimental results indicated that the proposed information fusion method had better classification performance compared to separate usage of electronic tongue and electronic eye, with a testing accuracy of 98.4%. This study provides a novel approach for rapidly identifying Fritillaria varieties, which offers research insights into the classification and traceability analysis of other Chinese medicinal materials.
- electronic tongue,
- electronic eye,
- ShuffleNetV2,
- Transformer,
- attention mechanism,
- fritillaria

FullText(HTML)

References (31)

References

[1]	BORJIGIN G, WEI F S, JIANG S, et al. Extraction, purification, structural characterization and biological activity of polysaccharides from Fritillaria:A review[J]. International Journal of Biological Macromolecules,2023,242:124817. doi: 10.1016/j.ijbiomac.2023.124817
[2]	张富丽, 刘炜, 毛建霏, 等. 基于核苷类物质建立川贝母药材真伪鉴别模型研究[J]. 中国中药杂志,2021,46(13):3337−3348. [ZHANG F L, LIU WEI, MAO J F, et al. Analysis of flavor components of 6 kinds of Yanshan chestnut by GC-MS combined with electronic nose/electronic tongue[J]. China Journal of Chinese Materia Medica,2021,46(13):3337−3348.] ZHANG F L, LIU WEI, MAO J F, et al. Analysis of flavor components of 6 kinds of Yanshan chestnut by GC-MS combined with electronic nose/electronic tongue[J]. China Journal of Chinese Materia Medica, 2021, 46（13）: 3337−3348.
[3]	AN Y L, LI Y, WEI W L, et al. Species discrimination of multiple botanical origins of Fritillaria species based on infrared spectroscopy, thin layer chromatography-image analysis and untargeted metabolomics[J]. Phytomedicine,2023,123:155228.
[4]	刘燕德, 徐振, 胡军, 等. 基于太赫兹光谱技术的贝母品种鉴别方法研究[J]. 光谱学与光谱分析,2021,41(11):3357−3362. [LIU Y D, XU Zhen, HU Jun, et al. Research on variety identification of fritillaria based on terahertz spectroscopy[J]. Spectroscopy and Spectral Analysis,2021,41(11):3357−3362.] LIU Y D, XU Zhen, HU Jun, et al. Research on variety identification of fritillaria based on terahertz spectroscopy[J]. Spectroscopy and Spectral Analysis, 2021, 41（11）: 3357−3362.
[5]	翟映红, 于国强, 杨永茂, 等. 从川贝母中检出平贝母的理化鉴别方法[J]. 华西药学杂志,2017,32(6):633−635. [ZHAI Yinghong, YU Guoqiang, YANG Yongmao, et al. Physical and chemical identification on distinguishing Fritillariae ussuriensis Bulbus from Fritillariae cirrhosa Bulbus[J]. West China JOurnal of Pharmaceutical Sciences,2017,32(6):633−635.] ZHAI Yinghong, YU Guoqiang, YANG Yongmao, et al. Physical and chemical identification on distinguishing Fritillariae ussuriensis Bulbus from Fritillariae cirrhosa Bulbus[J]. West China JOurnal of Pharmaceutical Sciences, 2017, 32（6）: 633−635.
[6]	王恬, 谈娜娜, 陆佳, 等. 川贝母生物碱的化学组成分析与IL-6结合活性成分的虚拟筛选[J]. 宝鸡文理学院学报(自然科学版),2023,43(4):30−34. [WANG Tian, TAN Nana, LU Jia, et al. Chemical composition analysis of alkaloids form Fritillaria cirrhosa D. Don and virtual screening of IL-6 binding active ingredients[J]. Journal of Baoji University of Arts and Sciences (Natural Science),2023,43(4):30−34.] WANG Tian, TAN Nana, LU Jia, et al. Chemical composition analysis of alkaloids form Fritillaria cirrhosa D. Don and virtual screening of IL-6 binding active ingredients[J]. Journal of Baoji University of Arts and Sciences （Natural Science）, 2023, 43（4）: 30−34.
[7]	杨银爱, 郜海燕, 牛犇, 等. 基于电子舌对不同品种鲜莲子滋味成分的分析[J]. 食品工业科技, 2023, 44(24):319-326. YANG Yin'ai, GAO Haiyan, NIU Ben, et al. Analysis of taste components in different varieties of fresh lotus seeds based on electronic tongue[J]. Science and Technology of Food Industry, 2023, 44(24):319−326.
[8]	张晓, 吴宏伟, 于现阔, 等. 基于电子眼技术的穿心莲质量评价[J]. 中国实验方剂学杂志,2019,25(1):189−195. [ZHANG Xiao, WU Hongwei, YU Xiankuo, et al. Quality evaluation of andrographis herba based on electronic-eye technique[J]. Chinese Journal of Experimental Traditional Medical Formulae,2019,25(1):189−195.] ZHANG Xiao, WU Hongwei, YU Xiankuo, et al. Quality evaluation of andrographis herba based on electronic-eye technique[J]. Chinese Journal of Experimental Traditional Medical Formulae, 2019, 25（1）: 189−195.
[9]	段金芳, 肖洋, 刘影, 等. 一测多评法与电子眼和电子舌技术相结合优化山茱萸蒸制时间[J]. 中草药,2017,48(6):1108−1116. [DUAN Jinfang, XIAO Yang, LIU Ying, et al. Optimization of steaming time of Cornus officinalis by QAMS combined with electronic-eye and electronic-tongue techniques[J]. Chinese Traditional and Herbal Drugs,2017,48(6):1108−1116.] DUAN Jinfang, XIAO Yang, LIU Ying, et al. Optimization of steaming time of Cornus officinalis by QAMS combined with electronic-eye and electronic-tongue techniques[J]. Chinese Traditional and Herbal Drugs, 2017, 48（6）: 1108−1116.
[10]	陈佳瑜, 袁海波, 沈帅, 等. 基于智能感官多源信息融合技术的滇红工夫茶汤综合感官品质评价[J]. 食品科学,2022,43(16):294−301. [CHEN Jiayu, YUAN Haibo, SHEN Shuai, et al. Comprehensive sensory quality evaluation of dianhong congou tea infusions using intelligent sensory multi-source information fusion technology[J]. Food Science,2022,43(16):294−301.] doi: 10.7506/spkx1002-6630-20211028-321 CHEN Jiayu, YUAN Haibo, SHEN Shuai, et al. Comprehensive sensory quality evaluation of dianhong congou tea infusions using intelligent sensory multi-source information fusion technology[J]. Food Science, 2022, 43（16）: 294−301. doi: 10.7506/spkx1002-6630-20211028-321
[11]	WANG S C, ZHANG Q, LIU C Z, et al. Synergetic application of an E-tongue, E-nose and E-eye combined with CNN models and an attention mechanism to detect the origin of black pepper[J]. Sensors and Actuators A:Physical,2023,357:114417. doi: 10.1016/j.sna.2023.114417
[12]	LU L, HU Z Q, HU X Q, et al. Quantitative approach of multidimensional interactive sensing for rice quality using electronic tongue sensor array based on information entropy[J]. Sensors and Actuators B:Chemical,2021,329:129254. doi: 10.1016/j.snb.2020.129254
[13]	王首程, 于雪莹, 高继勇, 等. 基于电子舌和电子鼻结合DenseNet-ELM的陈醋年限检测[J]. 食品与机械,2022,38(4):72−80,133. [WANG Shoucheng, YU Xueying, GAO Jiyong, et al. Age detection of mature vinegar based on electronic tongue and electronic nose combined with DenseNet-ELM[J]. Food & Machinery,2022,38(4):72−80,133.] WANG Shoucheng, YU Xueying, GAO Jiyong, et al. Age detection of mature vinegar based on electronic tongue and electronic nose combined with DenseNet-ELM[J]. Food & Machinery, 2022, 38（4）: 72−80,133.
[14]	SUN Y T, MA H Z. Research progress on oil-immersed transformer mechanical condition identification based on vibration signals[J]. Renewable and Sustainable Energy Reviews,2024,196:114327. doi: 10.1016/j.rser.2024.114327
[15]	ZHANG S Y, LIAN C, XU B R, et al. A token selection-based multi-scale dual-branch CNN-transformer network for 12-lead ECG signal classification[J]. Knowledge-Based Systems,2023,280:111006. doi: 10.1016/j.knosys.2023.111006
[16]	GONG A J, GUO L B, YU Y, et al. Spectrum-image dual-modality fusion empowered accurate and efficient classification system for traditional Chinese medicine[J]. Information Fusion,2024,101:101981. doi: 10.1016/j.inffus.2023.101981
[17]	王健庆, 戴恺, 李子柔. 基于深度学习的中药饮片图像识别研究 [J]. 时珍国医国药, 2020, 31(12):2930-2933. WANG Jianqing, DAI Kai, LI Zirong. Research on image recognition of traditional chinese medicine decoction pieces based on deep learning[J]. Lishizhen Medicine and Materia Medica Research, 2020, 31(12):2930−2993.
[18]	周丽媛, 赵启军, 高定国. 基于注意力引导深度纹理特征学习的复杂背景藏药材切片图像识别[J]. 世界科学技术-中医药现代化,2022,24(12):4825−4832. [ZHOU Liyuan, ZHAO Qijun, GAO Dingguo. Recognition of Tibetan medicinal slices in images with complex background via attention-guided deep texture feature learning[J]. World Science and Technology—Modernization of Traditional Chinese Medicine and Materia Medica,2022,24(12):4825−4832.] ZHOU Liyuan, ZHAO Qijun, GAO Dingguo. Recognition of Tibetan medicinal slices in images with complex background via attention-guided deep texture feature learning[J]. World Science and Technology—Modernization of Traditional Chinese Medicine and Materia Medica, 2022, 24（12）: 4825−4832.
[19]	CHEN Y, HUANG Y Q, ZHANG Z Z, et al. Plant image recognition with deep learning:A review[J]. Computers and Electronics in Agriculture,2023,212:108072. doi: 10.1016/j.compag.2023.108072
[20]	SHANG Y Y, XU X S, JIAO Y T, et al. Using lightweight deep learning algorithm for real-time detection of apple flowers in natural environments[J]. Computers and Electronics in Agriculture,2023,207:107765. doi: 10.1016/j.compag.2023.107765
[21]	彭红星, 何慧君, 高宗梅, 等. 基于改进ShuffleNetV2模型的荔枝病虫害识别方法[J]. 农业机械学报,2022,53(12):290−300. [PENG Hongxing, HE Huijun, GAO Zongmei, et al. Litchi diseases and insect pests identification method based on improved ShuffleNetV2[J]. Transactions of the Chinese Society for Agricultural Machinery,2022,53(12):290−300.] doi: 10.6041/j.issn.1000-1298.2022.12.028 PENG Hongxing, HE Huijun, GAO Zongmei, et al. Litchi diseases and insect pests identification method based on improved ShuffleNetV2[J]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53（12）: 290−300. doi: 10.6041/j.issn.1000-1298.2022.12.028
[22]	SUI Y, WANG X, WU J, et al. Causal attention for interpretable and generalizable graph classification[Z]. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Washington DC, USA; Association for Computing Machinery. 2022:1696–705.10. 1145/3534678.3539366.
[23]	WEN G, LI S, LIU F, et al. YOLOv5s-CA: A modified YOLOv5s network with coordinate attention for underwater target detection [J]. Sensors, 2023, 23(7): 3367.
[24]	杨正伟, 张鑫, 李庆盛, 等. 基于电子舌及一维深度CNN-ELM模型的普洱茶贮藏年限快速检测[J]. 食品与机械,2020,36(8):45−52. [YANG Zhengwei, ZHANG Xin, LI Qingsheng, et al. A fast detection Pu-erh tea storage based on the voltammetric electronic tongue and one-dimension CNN-ELM[J]. Food & Machinery,2020,36(8):45−52.] YANG Zhengwei, ZHANG Xin, LI Qingsheng, et al. A fast detection Pu-erh tea storage based on the voltammetric electronic tongue and one-dimension CNN-ELM[J]. Food & Machinery, 2020, 36（8）: 45−52.
[25]	YANG Z W, GAO J Y, WANG S C, et al. Synergetic application of E-tongue and E-eye based on deep learning to discrimination of Pu-erh tea storage time[J]. Computers and Electronics in Agriculture,2021,187:106297. doi: 10.1016/j.compag.2021.106297
[26]	金鑫宁, 刘铭, 桑恒亮, 等. 基于电子舌和电子眼结合改进MobileNetv3的黄芪快速溯源检测[J]. 食品与机械,2023,39(6):37−47. [JIN Xinning, LIU Ming, SANG Hengliang, et al. Fast traceability detection of Astragalus membranaceus based on the combination of electronic tongue and electronic eye to improve MobileNetv3[J]. Food & Machinery,2023,39(6):37−47.] JIN Xinning, LIU Ming, SANG Hengliang, et al. Fast traceability detection of Astragalus membranaceus based on the combination of electronic tongue and electronic eye to improve MobileNetv3[J]. Food & Machinery, 2023, 39（6）: 37−47.
[27]	李练兵, 高国强, 吴伟强, 等. 考虑特征重组与改进Transformer的风电功率短期日前预测方法[J]. 电网技术,2024,48(4):1466−1480. [LI Lianbing, GAO Guoqiang, WU Weiqiang, et al. A short-term day-ahead wind power prediction method considering feature recombination and improved transformer[J]. Power System Technology,2024,48(4):1466−1480.] LI Lianbing, GAO Guoqiang, WU Weiqiang, et al. A short-term day-ahead wind power prediction method considering feature recombination and improved transformer[J]. Power System Technology, 2024, 48（4）: 1466−1480.
[28]	JIN X, JIANG J X, LI Y, et al. Improved ShuffleNetV2 for action recognition in BPPV treatment[J]. Biomedical Signal Processing and Control,2024,88:105601. doi: 10.1016/j.bspc.2023.105601
[29]	游小荣, 李淑芳. 融合迁移学习和集成学习的服装风格图像分类方法[J/OL]. 现代纺织技术:1−8[2024-06-18]. https://doi.org/10.19398/j.att.202401007. [YOU Xiaorong, LI Shufang. Fashion style image classification method integrating transfer learning and ensemble learning: 1−8[2024-06-18]. https://doi.org/10.19398/j.att.202401007.] YOU Xiaorong, LI Shufang. Fashion style image classification method integrating transfer learning and ensemble learning: 1−8[2024-06-18]. https://doi.org/10.19398/j.att.202401007.
[30]	武晓春, 郜文祥. 基于tSNE多特征融合的JTC轨旁设备故障检测[J]. 铁道科学与工程学报,2024,21(3):1244−1255. [WU Xiaochun, GAO Wenxiang. Fault detection of JTC trackside equipment based on tSNE multi-feature fusion[J]. Journal of Railway Science and Engineering,2024,21(3):1244−1255.] WU Xiaochun, GAO Wenxiang. Fault detection of JTC trackside equipment based on tSNE multi-feature fusion[J]. Journal of Railway Science and Engineering, 2024, 21（3）: 1244−1255.
[31]	崔佳旭, 杨博. 贝叶斯优化方法和应用综述[J]. 软件学报,2018,29(10):3068−3090. [CUI J X, YANG B. Survey on Bayesian optimization methodology and applications[J]. Journal of Software,2018,29(10):3068−3090.] CUI J X, YANG B. Survey on Bayesian optimization methodology and applications[J]. Journal of Software, 2018, 29（10）: 3068−3090.

[1]	HUANG Fan, ZHANG Ting, LIU Xiao, WANG Xiaoping, TANG Xiaobo, WANG Yun, LI Chunhua, XIA Chen. Analysis of Characteristic Aroma Components of Sichuan Dark Tea[J]. Science and Technology of Food Industry, 2023, 44(12): 328-336. DOI: 10.13386/j.issn1002-0306.2022070344
[2]	YANG Ting, SHEN Shiyan, WANG Zhineng, YANG Liu, SHANG Shixiong, CUI Jie, YING Xiongmei. Analysis and Comparison of Nutritional Components and Aroma Components of Brown Sugar with Different Processing Methods[J]. Science and Technology of Food Industry, 2021, 42(19): 43-55. DOI: 10.13386/j.issn1002-0306.2020120282
[3]	YANG Ying-di, LI Min, PENG Bang-zhu. Research on the Metabolism of Aroma Components in Apple Cider[J]. Science and Technology of Food Industry, 2018, 39(19): 314-320. DOI: 10.13386/j.issn1002-0306.2018.19.055
[4]	WANG Li-xia, LIU Ying, GAO Han, HE Jun-ping. Analysis of aroma components in red raspberry before and after fermentation[J]. Science and Technology of Food Industry, 2018, 39(6): 217-222. DOI: 10.13386/j.issn1002-0306.2018.06.039
[5]	LI Yan-jie, HUANG Jia-jia, DONG Fang, SU Xin-guo, DU Bing. Effects of electron beam irradiation on sterilization and aroma components of three spices[J]. Science and Technology of Food Industry, 2017, (16): 19-23. DOI: 10.13386/j.issn1002-0306.2017.16.005
[6]	WANG Bao-chun, XIE Min-hua, WANG Xue-xi, WU Xiao-hua, CHEN Bai, ZHANG Xin, NIU Ji-jun. Effects of 1-MCP treatment on fruit quality and aroma components of Huaniu apples during cold storage[J]. Science and Technology of Food Industry, 2017, (07): 331-339. DOI: 10.13386/j.issn1002-0306.2017.07.056
[7]	HAN Zong-yuan, LI Xiao-jing, YANG Yu-hong, WANG Nan, LIU Yun-nan, WANG Jia-qing, XIAO Zhi-gang. Study on the quality and aroma components in raspberry juice[J]. Science and Technology of Food Industry, 2016, (15): 270-275. DOI: 10.13386/j.issn1002-0306.2016.15.044
[8]	YANG Jun, YIN Jie, LIU Qin, ZHU Li-yun, WU Jun-qing, ZHANG Chun-miao, JIN Hao, ZHANG Yong- jun, GAO Yong-sheng, SONG Lin-zhen. Preparation,nutrition and aroma components analysis of bayberry extract[J]. Science and Technology of Food Industry, 2016, (13): 290-294. DOI: 10.13386/j.issn1002-0306.2016.13.051
[9]	LI A-min, WANG Xiao-rong, JIANG He-ti. Research of the change of the aroma components in the kiwi spirit during aging[J]. Science and Technology of Food Industry, 2015, (09): 281-286. DOI: 10.13386/j.issn1002-0306.2015.09.053
[10]	MENG Ai-li, PANG Xiao-li, WEN Shun-wei, WEI Qiu-yi, SI Hui-qing. Analysis of aroma characteristics and aroma components from mengding yellow tea[J]. Science and Technology of Food Industry, 2014, (18): 106-112. DOI: 10.13386/j.issn1002-0306.2014.18.014

Supplements (1)

Supplements
Other Related Supplements

Cited By

Cited by

Periodical cited type(11)

1.	毕海心，段珺婕，周宇轩，赵前程，李智博. 抑菌性纳米颗粒的制备方法及在食品保鲜领域的应用进展. 食品安全质量检测学报. 2024(05): 12-22 .
2.	石彬，冉曜琦，李咏富，龙明秀，何扬波，罗其琪，李伟. 百香果与柠檬对贵州红酸汤发酵品质的影响. 食品安全质量检测学报. 2024(05): 237-245 .
3.	傅雯雯，曾小芸，李静雯，周崇冰，黄凌，谭飔. 胭脂萝卜花色苷活性智能指示膜的制备及其特性分析. 食品工业. 2024(03): 45-49 .
4.	江东阳，李楠楠，黄海源，周泽广，卢彦越，钟磊. 淀粉/羧甲基纤维素/甘油可降解复合膜的制备及性能研究. 林产化学与工业. 2024(02): 94-102 .
5.	蔡秋爽，李鸿梅，何嘉欣，张立世. 改性玉米苞叶纤维素/纳米滑石粉/豌豆淀粉复合膜的制备及性能表征. 食品工业科技. 2023(04): 287-295 . 本站查看
6.	蔡文琪，李泽坤，翟佳乐，陈宇，唐旭，李晓钰，杨立娜. 大豆种皮纤维素薄膜的制备及性能研究. 包装与食品机械. 2023(01): 1-7+13 .
7.	刘忠，徐艳，朱荣耀，赵志强. 纳米银的制备及其应用研究进展. 天津科技大学学报. 2023(02): 75-80 .
8.	黄舒琦，陈熠，李慧玲，范丽萍，王楠. KGM/PEC/TiO_2复合膜的制备及其性能研究. 食品工业. 2023(06): 31-38 .
9.	房迅，王嘉伟，吴迎花，屈靖朝，李锦辉，郭少波，刘智峰. 银纳米颗粒的制备、抑菌机制及应用的研究进展. 化工技术与开发. 2023(08): 33-37 .
10.	张朝涛，王春慧，李雪，石洪玮，刘贵巧，高光晔，毛雪飞. 酶提取-单颗粒电感耦合等离子质谱法分析樱桃番茄纳米银颗粒及其吸收规律研究. 中国无机分析化学. 2023(12): 1271-1281 .
11.	葛瑾，杨钧翔，李金金，刘玲，贾晓昱，张鹏，李江阔. 果蔬绿色防腐包装材料及其应用效果研究进展. 食品科技. 2022(08): 21-27 .