[1]张冬,梁平,顾进广.基于双融合图注意力网络多模态知识图谱链路预测[J].计算机技术与发展,2024,34(07):123-130.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0089]
　ZHANG Dong,LIANG Ping,GU Jin-guang.Multi-modal Knowledge Graph Link Prediction Based on Dual Fusion and Graph Attention Networks[J].,2024,34(07):123-130.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0089]
点击复制

基于双融合图注意力网络多模态知识图谱链路预测

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
34
期数:
2024年07期
页码:
123-130
栏目:
人工智能
出版日期:
2024-07-10

文章信息/Info

Title:
Multi-modal Knowledge Graph Link Prediction Based on Dual Fusion and Graph Attention Networks
文章编号:
1673-629X(2024)07-0123-08
作者:
张冬1; 2; 梁平1; 2; 顾进广1; 2
1. 武汉科技大学 计算机科学与技术学院,湖北 武汉 430065; 2. 智能信息处理与实时工业系统湖北省重点实验室,湖北 武汉 430065
Author(s):
ZHANG Dong1; 2; LIANG Ping1; 2; GU Jin-guang1; 2
1. School of Computer Science and Technology,Wuhan University of Science and Technology,Wuhan 430065,China; 2. Hubei Province Key Laboratory of Intelligent Information Processing and Real -time Industrial System,Wuhan 430065,China
关键词:
多模态; 知识图谱; 链路预测; 模态融合; 图注意力网络
Keywords:
multi-modal; knowledge graph; link prediction; model fusion; graph attention network
分类号:
TP182
DOI:
10.20165/j.cnki.ISSN1673-629X.2024.0089
摘要:
知识图谱链路预测是一种根据知识图谱已存在的事实去预测缺失事实的任务,旨在解决知识图谱不完整性问题。但是现有的知识图谱链路预测有一定的缺陷,传统方法只使用单一的数据模态,没有充分利用不同数据模态的丰富信息,并且在图神经网络中孤立地看待实体和关系,没有考虑到不同邻域实体关系权重的不同。 为了解决上述缺陷,提出了基于双融合图注意力网络的多模态知识图谱链路预测模型。 首先,使用了图像、文本和属性 3 种模态,同时为了保证数据模态特征的一致性和互补性,设计了一个基于早期融合和晚期融合结合的双融合机制对多模态信息进行融合;然后,为了加强知识图中实体关系的融合以及邻域关系,同时考虑了实体以及关系的多样性,融合了实体表示和关系表示,并通过图注意力网络进行聚合以加强实体的特征表示。 通过在 4 个公开的数据集 FB15K-237、WN18RR、DB15K 以及 YAGO15K 进行模拟实验,结果表明,提出的多模态知识图谱链路预测方法具有较好的性能。
Abstract:
Knowledge graph link prediction aims to predict missing facts within the knowledge graph,addressing the issue of knowledge graph incompleteness. However,existing knowledge graph link prediction methods exhibit certain limitations. Traditional methods are re-stricted to utilizing a single data modality,thereby missing the opportunity to fully leverage the wealth of information provided by diverse data modalities. Moreover, in the context of graph neural networks, entities and relationships are frequently treated as independent elements,often overlooking the varying significance of entity relationships within distinct neighborhoods. To address these problems,a multi-modal knowledge graph link prediction model based on dual-fusion graph attention network is proposed. Firstly,three modalities of image,text and attribute were incorporated. To ensure consistency and synergy among data modal features,a dual-fusion mechanism that combines early and late fusion strategies was devised for multi - modal data amalgamation. To strengthen the fusion of entity relationships and neighborhood relationships in the knowledge graph,consideration is given to the diversity of entities and relationships.The fusion of entity and relationship representations is followed by aggregation through the graph attention network,thereby enhancing the feature representation of the entities. By conducting simulation experiments on four public datasets,specifically FB15K-237,WN18RR,DB15K,and YAGO15K,the results demonstrate the strong performance of the proposed multi-modal knowledge graph link prediction method.

相似文献/References:

[1]李彩云[],张著洪[]. 求解单目标区间数规划的改进型免疫优化算法[J].计算机技术与发展,2015,25(09):102.
　LI Cai-yun[],ZHANG Zhu-hong[]. Improved Immune Optimization Algorithm Solving Single-objective Interval Number Programming[J].,2015,25(07):102.
[2]孙艳,田丽梅. 基于多维尺度分析的舆情研究主题词知识图谱[J].计算机技术与发展,2016,26(04):187.
　SUN Yan,TIAN Li-mei. Mapping Knowledge Domain on Subject Headings of Public Sentiment Research Based on Multi-dimensional Scaling[J].,2016,26(07):187.
[3]刘申凯,周霁婷,朱永华,等.融合知识图谱和 ESA 方法的网络新词识别[J].计算机技术与发展,2019,29(03):12.[doi:10.3969/ j. issn.1673-629X.2019.03.003]
　LIU Shen-kai,ZHOU Ji-ting,ZHU Yong-hua,et al.Network New Word Recognition Based on Fusion of Knowledge Graph and ESA[J].,2019,29(07):12.[doi:10.3969/ j. issn.1673-629X.2019.03.003]
[4]戈其平,钟艳如.基于数学教学的知识图谱构建[J].计算机技术与发展,2019,29(03):187.[doi:10.3969/ j. issn.1673-629X.2019.03.039]
　GE Qi-ping,ZHONG Yan-ru.Construction of Knowledge Atlas Based on Mathematics Teaching[J].,2019,29(07):187.[doi:10.3969/ j. issn.1673-629X.2019.03.039]
[5]魏 瑾,李伟华,潘 炜.基于知识图谱的智能决策支持技术及应用研究[J].计算机技术与发展,2020,30(01):1.[doi:10. 3969 / j. issn. 1673-629X. 2020. 01. 001]
　WEI Jin,LI Wei-hua,PAN Wei.Research on Intelligent Decision Support Technology and Application Based on Knowledge Graph[J].,2020,30(07):1.[doi:10. 3969 / j. issn. 1673-629X. 2020. 01. 001]
[6]项 威,王 邦.中文事件抽取研究综述[J].计算机技术与发展,2020,30(02):1.[doi:10. 3969 / j. issn. 1673-629X. 2020. 02. 001]
　XIANG Wei,WANG Bang.Survey of Chinese Event Extraction Research[J].,2020,30(07):1.[doi:10. 3969 / j. issn. 1673-629X. 2020. 02. 001]
[7]刘家祝,郭 强,吴碧伟,等.基于子图相交的社交账号与知识图谱实体对齐[J].计算机技术与发展,2020,30(05):10.[doi:10. 3969 / j. issn. 1673-629X. 2020. 05. 003]
　LIU Jia-zhu,GUO Qiang,WU Bi-wei,et al.Subgraph Intersection Based Alignment between Social Media Account and Knowledge Graph Entity[J].,2020,30(07):10.[doi:10. 3969 / j. issn. 1673-629X. 2020. 05. 003]
[8]陆菁宇,张绍阳,黄文旎.学科发展状态的知识图谱构建[J].计算机技术与发展,2020,30(06):145.[doi:10. 3969 / j. issn. 1673-629X. 2020. 06. 028]
　LU Jing-yu,ZHANG Shao-yang,HUANG Wen-ni.Analysis of Development Status of Discipline Based on Knowledge Graph[J].,2020,30(07):145.[doi:10. 3969 / j. issn. 1673-629X. 2020. 06. 028]
[9]黄东晋,秦 汉,郭 昊.基于 BERT-CNN 的电影原声智能问答系统[J].计算机技术与发展,2020,30(11):158.[doi:10. 3969 / j. issn. 1673-629X. 2020. 11. 029]
　HUANG Dong-jin,QIN Han,GUO Hao.Movie Soundtrack Intelligent Question and Answer System Based on BERT-CNN[J].,2020,30(07):158.[doi:10. 3969 / j. issn. 1673-629X. 2020. 11. 029]
[10]任佳妮,杨 阳.全球医疗机器人技术领域创新态势分析[J].计算机技术与发展,2021,31(04):158.[doi:10. 3969 / j. issn. 1673-629X. 2021. 04. 027]
　REN Jia-ni,YANG Yang.Analysis of Innovation Situation in Field of Global MedicalRobot Technology[J].,2021,31(07):158.[doi:10. 3969 / j. issn. 1673-629X. 2021. 04. 027]
[11]曹茂俊,林世友,肖阳,等.面向测井领域的多模态知识图谱构建[J].计算机技术与发展,2024,34(09):195.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0132]
　CAO Mao-jun,LIN Shi-you,XIAO Yang,et al.Construction of Multi-modal Knowledge Graph for Logging Field[J].,2024,34(07):195.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0132]

更新日期/Last Update: 2024-07-10