0 views collected around this technical thread.
This talk explores how 360's advertising recommendation system leverages language models—from BERT to large‑scale LLMs—to improve user interest modeling, feature extraction, and conversion‑rate prediction, detailing practical challenges, engineering solutions, experimental results, and future research directions.
The diagram contrasts a Base LLM, which merely predicts the next word from training data and can continue stories or answer simple facts but may generate unsafe text, with an Instruction‑Tuned LLM that is fine‑tuned via RLHF to understand and follow commands, delivering more accurate, useful, and safe responses.
Researchers discovered that prompting ChatGPT with repeated words can cause the model to leak private data such as phone numbers and email addresses, highlighting a serious repeat‑prompt vulnerability that reveals substantial personally identifiable information from its training corpus.
This talk presents how large language models are leveraged for predicting protein, antibody, and RNA structures, covering background, model stability, generative approaches, antibody-specific models, RNA modeling, and protein‑RNA interaction prediction, along with experimental results and future research directions.
This article provides a comprehensive overview of natural language processing and language‑model research, tracing the historical development from early rule‑based and statistical N‑gram models through neural network approaches such as RNN, LSTM, ELMo, and Transformer, and detailing the architectures, strengths, and limitations of the GPT series up to GPT‑4, while also discussing evaluation metrics, practical applications, and future challenges.
This article surveys the development of natural language processing from early rule‑based systems through statistical n‑gram models, neural language models, RNNs, LSTMs, ELMo, Transformers and BERT, and then details the architecture, training methods, advantages and limitations of the GPT‑1, GPT‑2, GPT‑3, ChatGPT and GPT‑4 models, concluding with a discussion of future challenges and references.
This article reviews a study that compares the stage‑wise language acquisition of infants with the learning trajectory of GPT‑2, using linguistic probes and statistical tests to determine whether deep language models follow sequential or parallel learning patterns similar to children.
This article introduces prompt engineering for ChatGPT, explains key design principles, and demonstrates a series of practical applications such as text classification, summarization, role‑playing, terminal emulation, output formatting, temperature control, iterative fine‑tuning, and reverse‑engineering of prompts.
This article explains how ChatGPT generates text by predicting the next token using large language models, describes the role of probability, temperature, and attention mechanisms in transformers, and discusses neural network training, embeddings, semantic spaces, and the broader implications for artificial intelligence research.
Airbnb replaced its XLM‑RoBERTa ranking with an MT5 encoder‑decoder for content recommendation, built a real‑time generative assistant for reply suggestions and intent detection, and deployed a T5‑based paraphrase chatbot, showing that large‑scale pre‑trained transformers improve relevance, agent efficiency, and user satisfaction.
The article explains that ChatGPT produces responses by converting words into high‑dimensional vectors, feeding them through neural networks, and selecting tokens based on probability distributions, while also contrasting GPT with BERT and describing a related training event.
The article examines the public testing of the new Bing chatbot, contrasting its internet‑enabled, citation‑rich responses and occasional erratic, immature behavior with ChatGPT’s more stable output, while exploring user‑reported failures, speculative technical reasons, and the ethical implications of deploying advanced language models.
This article provides a comprehensive overview of the GPT series and explains how InstructGPT and ChatGPT are built by combining supervised fine‑tuning, reward modeling, and Proximal Policy Optimization, detailing their datasets, training pipeline, performance advantages, limitations, and future research directions.
This article explains why ChatGPT performs so well by tracing the evolution of natural‑language processing from rule‑based grammars through statistical n‑gram models to neural architectures like RNNs, LSTMs, attention mechanisms, Transformers, and the massive data and training methods that power modern large language models.
This article explains the underlying mechanisms of ChatGPT, including its GPT‑3 foundation, the role of supervised fine‑tuning, human‑feedback reinforcement learning (RLHF), PPO optimization, consistency issues, evaluation metrics, and the limitations of these training strategies, with references to key research papers.
This article surveys the most influential Transformer‑based research papers—from the original Attention Is All You Need work to recent models such as Autoformer and FEDformer—covering breakthroughs in natural language processing, computer vision, speech recognition, and long‑term series forecasting, and provides download links for each.
This article traces OpenAI’s progression from GPT‑1 through GPT‑3, Codex, InstructGPT, and ChatGPT, highlighting how increasing model scale, prompt‑based task integration, and human‑feedback alignment have driven the evolution toward more capable, generalizable language intelligence aimed at achieving artificial general intelligence.
This article provides a comprehensive overview of natural language processing, covering its definition, historical development from one‑hot encoding to modern models such as word2vec, ELMo, GPT, and BERT, and discusses the advantages, limitations, and key concepts of each technique.
This article explores how AI and natural language processing technologies are applied to music lyric creation, covering background challenges, rhyme retrieval methods, advanced language models such as SongNet, decoding strategies, style transfer, and a multi‑level generation platform that aims to streamline professional songwriting.
This article introduces the importance of pre‑training in natural language processing, reviews classic pre‑training models such as Skip‑thoughts, BERT, GPT‑2 and T5, presents the modular UER‑py framework and its Chinese resources, compares it with Huggingface Transformers, and outlines practical deployment steps in industry.