Alibaba Cloud Big Data AI Platform

Dec 11, 2023 · Artificial Intelligence

How Hyperbolic Space and Contrastive Learning Boost Domain-Specific Language Models

This article introduces the KANGAROO model, which injects hierarchical semantic information via hyperbolic space and leverages contrastive learning on dense subgraph structures to overcome global sparsity in vertical‑domain knowledge‑enhanced pre‑trained language models, and evaluates its performance on finance and medical tasks.