Research on the construction of Tai Chi teaching resource library in colleges and universities driven by multimodal knowledge graph and somatosensory interaction

Abstract

Abstract: Purpose of the study: In 2020, "Tai Chi" was included in the UNESCO Intangible Cultural

Heritage List, and the country's efforts to protect traditional culture are continuously strengthening. Data

from the Ministry of Education show that by 2025, more than half of the country's colleges and universities

will have opened Tai Chi courses, but most of them are based on the 24-style simplified Tai Chi, lacking

the principles of attack and defense and the theory of Chinese medicine meridians, which has led to most

students becoming tired of learning. The "Action Plan for Education Informatization 2.0" proposes to

"develop an intelligent education ecology." In the context of digital inheritance of intangible cultural

heritage, Tai Chi teaching is still in the dual dilemma of low efficiency of the traditional apprenticeship

system, scattered teaching resources, and faults in the deconstruction of cultural connotations. The

application of intelligent technology is mostly concentrated on single action recognition and lacks

knowledge system construction. This paper aims to explore the application of multimodal knowledge

graphs and somatosensory interaction technology in the construction of Tai Chi teaching resource libraries

in colleges and universities. The goal is to achieve the systematization, intelligence, and personalization of

Tai Chi teaching resources, thereby improving the quality and efficiency of Tai Chi teaching. The research

aims to achieve the following objectives: first, to construct a multimodal knowledge graph covering Tai

Chi theoretical knowledge, technical movements, and cultural connotations; second, to enable students to

interact with Tai Chi teaching resources through somatosensory interaction technology, thereby enhancing

their learning interest and participation; and third, to utilize intelligent algorithms to provide personalized

learning path resource recommendations based on students' learning situations and needs, thereby

promoting the personalized development of Tai Chi teaching.

Research Methods: Using CNKI, weconducted advanced searches for the keywords "multimodal knowledge graph," "body interaction," and

"Tai Chi teaching" over a five-year period, from June 2020 to June 2025. We selected papers from core

journals, including those from SCI , EI , and Peking University Core. We identified 145 relevant academic

journal articles through advanced searches, and ultimately, 86 valid references were identified after

screening. Using literature review, logical analysis, and case study methods, we systematically reviewed

existing research findings both domestically and internationally, providing theoretical support and

methodological reference for this study. Using the expert interview method, we invited 6 experts, scholars

and front-line teachers with rich teaching experience in the field of Tai Chi to conduct in-depth interviews

on the needs, content, and technical implementation of the construction of Tai Chi teaching resource library.

Research results: Multimodal knowledge graph is an intelligent knowledge organization tool that

integrates multiple information forms such as text, images, videos, and motion capture data. It takes the

ontology model as its core and integrates the theoretical knowledge, technical movements, and cultural

connotations of Tai Chi through entity, attribute, and relationship triples to achieve semantic association

and unified storage of multimodal data, providing a dynamic and expandable knowledge base for

intelligent teaching. As the third generation of revolutionary technology for human-computer interaction,

somatosensory interaction technology uses sensor equipment to capture biomechanical information such as

user body movements, gestures, and postures in real time. After computer analysis, it conducts two-way

information transmission and conversion with the established multimodal Tai Chi knowledge database,

self-checks movement deviations, and realizes dynamic error correction through visualization or voice

guidance. Based on the innovative integration of the two, a university Tai Chi teaching resource library is

constructed to integrate scattered resources and improve student learning efficiency. (1) Construct a

multimodal knowledge graph. Through academic databases such as China National Knowledge

Infrastructure (CNKI), we collected literature on Tai Chi’s theories, technical movements, and cultural

connotations. We obtained professional knowledge and experience through expert interviews and

integrated various information forms such as text and images to form a comprehensive Tai Chi knowledge

base. Based on the characteristics and knowledge structure of Tai Chi, we designed a Tai Chi ontology

model, integrating abstract theories, scattered technical movements, and cultural resources into a

computable and scalable digital framework. We also used semantic association technology to associate

databases of different modalities and establish a unified multimodal Tai Chi database. (2) Integration of the

somatosensory interaction system. In order to meet the experimental requirements and control costs, we

deployed an Orbbec Astra Pro camera to collect the coordinates of students’ skeletal points at 30 fps,

capture RGB images and depth maps, and use a six-axis IMU sensor to capture the rotation angle and

speed of the wrist and ankle and send sensor data. We used the Kabsch algorithm to align the captured

skeletal points and other data with the database in the Tai Chi ontology model, and perform two-way

information transmission and conversion, real-time action error correction, and improve learning efficiency.

(3) Application and optimization of the teaching resource library. Utilize intelligent algorithms to analyze

students' learning situations and needs, and provide personalized learning path resource recommendations;

combine somatosensory interaction technology to design interactive teaching sessions to enhance student

participation and learning interest; regularly collect student feedback and expert opinions, update and

optimize the teaching resource library, and continuously introduce new technological achievements and

teaching concepts to maintain the advancement and practicality of the database.

Research Conclusions: Inthis study of constructing a university Tai Chi teaching resource library driven by multimodal knowledge

graphs and somatosensory interaction, the fusion of multimodal knowledge graphs and somatosensory

interaction technology was applied to the integration and optimization of Tai Chi teaching resources,

achieving the systematization, intelligence, and personalization of Tai Chi teaching resources. By

constructing a multimodal knowledge graph covering Tai Chi theoretical knowledge, technical movements,

and cultural connotations, and integrating a somatosensory interaction system, it can not only enhance

students' interest and participation in learning Tai Chi, but also effectively improve the quality and

efficiency of Tai Chi teaching. This innovative integration provides new paths and methods for Tai Chi

teaching in universities, is practical and feasible, and also provides a useful reference for the digital

inheritance of other traditional sports.