Representation Learning from Text and Structured Data

Rui Zhang

doi:10.21996/39jm-9m60

Representation Learning from Text and Structured Data

Rui Zhang

Publikation: Afhandling › Ph.d.-afhandling

Abstract

Tekst og netværk er to almindelige former for data. De kan altid bruges samme til at beskrive forskellige applikationer, såsom kommentarsystemer, sociale netværk, og akademiske netværk. Dataanalyse bliver vigtig nu. Det er et afgørende spørgsmål
at præsentere tekst- og netværksdata på en effektiv måde. Mange repræsentation læringsmodeller er blevet foreslået til dette problem. Men de fleste metoder har brug for dataetiketter, komplekse systemer og/eller højdimensionelle vektorer for
at opnå de gode repræsentationer, og dette er ofte udfordrende for beregning og lagring af både upstream og downstream applikationer. Derfor adresserer denne afhandling ovenstående udfordringer og yder bidrag til repræsentationslæring på tekst- og tekstbaserede netværksdata.

Til tekstrepræsentation læring foreslås en multi-label-læringsmodel baseret på semantisk etiketlæring for at kategorisere tekstbaserede publikationer med hierarkisk
kategoristruktur. Denne model lærer først repræsentationer af publikationer og kategorier. Derefter genkender og videregiver modellen den matchende information hierarkisk. Endelig opnår denne model bedre forudsigelser i hierarkisk kategori af publikationer.

Til tekstbaserede netværksdata foreslås først en metapath-baseret repræsentationsmodel. Denne model kan lære lavdimensionelle repræsentationer for målknuder fra deres tekstattributter og topologiske strukturer ved hjælp af en kaskadestyret selvovervåget læring mekanisme. For at overvinde begrænsningen af metapath og reducere de ekstra omkostninger, foreslår vi også en selvovervåget metapath-fri algoritme med relationsbaseret nabo-graf kontrastlæring. Denne model kan producere globale repræsentationer ved at lære alle knudepunkter og links. Repræsentationerne kan bruges til mange downstream-opgaver. Modellen udkonkurrerer de nyeste metoder.

Samlet set giver afhandlingen en omfattende gennemgang af eksisterende repræsentation læringsmetoder og foreslår nye metoder baseret på dyb læring for at producere meget mere effektive og effektive repræsentationer til tekst og netværk. Bidragene er empirisk valideret på adskillige datasæt og opgaver i den virkelige verden.

Originalsprog	Engelsk
Bevilgende institution	Syddansk Universitet
Vejledere/rådgivere	Zimek, Arthur, Hovedvejleder Schneider-Kamp, Peter, Bivejleder
Dato for forsvar	20. maj 2022
Udgivelsessted	Odense
Udgiver	Syddansk Universitet. Det Naturvidenskabelige Fakultet
DOI	https://doi.org/10.21996/39jm-9m60
Status	Udgivet - 25. maj 2022

Note vedr. afhandling

Afhandlingen kan læses på SDUs bibliotek.

Dokumenter og links

10.21996/39jm-9m60

2 Konferencebidrag i proceedings

Improving semantic similarity of words by retrofitting word vectors in sense level
Zhang, R., Schneider-Kamp, P. & Zimek, A., 2020, Proceedings of the 12th International Conference on Agents and Artificial Intelligence. Rocha, A., Steels, L. & van den Herik, J. (red.). SCITEPRESS Digital Library, s. 108-119
Publikation: Kapitel i bog/rapport/konference-proceeding › Konferencebidrag i proceedings › Forskning › peer review
Matching research publications to the united nations' sustainable development goals by multi-label-learning with hierarchical categories
Zhang, R., Vignes, M., Steiner, U. & Zimek, A., okt. 2020, Proceedings - 2020 IEEE 7th International Conference on Data Science and Advanced Analytics, DSAA 2020. Webb, G., Zhang, Z., Tseng, V. S., Williams, G., Vlachos, M. & Cao, L. (red.). IEEE, s. 516-525 9260025
Publikation: Kapitel i bog/rapport/konference-proceeding › Konferencebidrag i proceedings › Forskning › peer review

Citationsformater

@misc{5169eb110dcd47a0bc65e539b6b675b5,

title = "Representation Learning from Text and Structured Data",

abstract = "Text and networks, as two common forms of data, always appear cooperatively in describing diverse applications in the real world, such as review systems, social networks, and citation networks. As the demand of data analytics continues to grow, how to effectively and efficiently represent text and network data has become a critical research issue. To resolve this problem, various machine learning models have been proposed for text and network representation learning, but most of them mainly rely on tons of manually labeled training samples, complex systems, and/or high-dimensional vectors to improve the accuracy and precision of representations, which often bring new challenges to computation and storage costs in both upstream and downstream applications. Thus, this thesis addresses the above challengesand makes contributions to representation learning on the text and text-attributed network data.For text representation learning, a label-semantic augmented multi-label-learning model is proposed to categorize text-based publications with hierarchical category structure, which creatively learns representations of publications and categories, recognizes and passes their matching information hierarchically, and as a result, achieves better hierarchical-category predictions.For text-attributed network representation learning, a meta-path-based embedding method is first developed, which is able to learn low-dimensional representations for target-typed nodes from their text attributes and topological structures by a cascaded self-supervised mechanism. Moreover, in order to overcome the limitation of preset meta-paths and reduce the extra learning cost, we also propose a selfsupervised meta-path-free algorithm with relation-based neighbor-graph contrast learning, which could produce global node representations by encoding all-typednodes and relations. These representations can be used for a variety of downstream tasks and outperform state-of-the-art baselines.Overall, this thesis provides a comprehensive review of existing representation learning methods and proposes several novel approaches based on deep learning to produce much more effective and efficient representations for text and networks.The contributions are empirically validated on several real-world datasets and tasks.",

author = "Rui Zhang",

year = "2022",

month = may,

day = "25",

doi = "10.21996/39jm-9m60",

language = "English",

publisher = "Syddansk Universitet. Det Naturvidenskabelige Fakultet",

address = "Denmark",

school = "SDU",

}

TY - GEN

T1 - Representation Learning from Text and Structured Data

AU - Zhang, Rui

PY - 2022/5/25

Y1 - 2022/5/25

N2 - Text and networks, as two common forms of data, always appear cooperatively in describing diverse applications in the real world, such as review systems, social networks, and citation networks. As the demand of data analytics continues to grow, how to effectively and efficiently represent text and network data has become a critical research issue. To resolve this problem, various machine learning models have been proposed for text and network representation learning, but most of them mainly rely on tons of manually labeled training samples, complex systems, and/or high-dimensional vectors to improve the accuracy and precision of representations, which often bring new challenges to computation and storage costs in both upstream and downstream applications. Thus, this thesis addresses the above challengesand makes contributions to representation learning on the text and text-attributed network data.For text representation learning, a label-semantic augmented multi-label-learning model is proposed to categorize text-based publications with hierarchical category structure, which creatively learns representations of publications and categories, recognizes and passes their matching information hierarchically, and as a result, achieves better hierarchical-category predictions.For text-attributed network representation learning, a meta-path-based embedding method is first developed, which is able to learn low-dimensional representations for target-typed nodes from their text attributes and topological structures by a cascaded self-supervised mechanism. Moreover, in order to overcome the limitation of preset meta-paths and reduce the extra learning cost, we also propose a selfsupervised meta-path-free algorithm with relation-based neighbor-graph contrast learning, which could produce global node representations by encoding all-typednodes and relations. These representations can be used for a variety of downstream tasks and outperform state-of-the-art baselines.Overall, this thesis provides a comprehensive review of existing representation learning methods and proposes several novel approaches based on deep learning to produce much more effective and efficient representations for text and networks.The contributions are empirically validated on several real-world datasets and tasks.

AB - Text and networks, as two common forms of data, always appear cooperatively in describing diverse applications in the real world, such as review systems, social networks, and citation networks. As the demand of data analytics continues to grow, how to effectively and efficiently represent text and network data has become a critical research issue. To resolve this problem, various machine learning models have been proposed for text and network representation learning, but most of them mainly rely on tons of manually labeled training samples, complex systems, and/or high-dimensional vectors to improve the accuracy and precision of representations, which often bring new challenges to computation and storage costs in both upstream and downstream applications. Thus, this thesis addresses the above challengesand makes contributions to representation learning on the text and text-attributed network data.For text representation learning, a label-semantic augmented multi-label-learning model is proposed to categorize text-based publications with hierarchical category structure, which creatively learns representations of publications and categories, recognizes and passes their matching information hierarchically, and as a result, achieves better hierarchical-category predictions.For text-attributed network representation learning, a meta-path-based embedding method is first developed, which is able to learn low-dimensional representations for target-typed nodes from their text attributes and topological structures by a cascaded self-supervised mechanism. Moreover, in order to overcome the limitation of preset meta-paths and reduce the extra learning cost, we also propose a selfsupervised meta-path-free algorithm with relation-based neighbor-graph contrast learning, which could produce global node representations by encoding all-typednodes and relations. These representations can be used for a variety of downstream tasks and outperform state-of-the-art baselines.Overall, this thesis provides a comprehensive review of existing representation learning methods and proposes several novel approaches based on deep learning to produce much more effective and efficient representations for text and networks.The contributions are empirically validated on several real-world datasets and tasks.

U2 - 10.21996/39jm-9m60

DO - 10.21996/39jm-9m60

M3 - Ph.D. thesis

PB - Syddansk Universitet. Det Naturvidenskabelige Fakultet

CY - Odense

ER -

Representation Learning from Text and Structured Data

Abstract

Note vedr. afhandling

Dokumenter og links

Fingeraftryk

Relaterede publikationer

Improving semantic similarity of words by retrofitting word vectors in sense level

Matching research publications to the united nations' sustainable development goals by multi-label-learning with hierarchical categories

Citationsformater