Tagged articles
Llama
19 articles · Page 1 of 1
Meta's Muse personal AI agent topped the US App Store and boosted stock 27%, but analysts debate whether its IM-style interface, virtual-machine architecture, and aggressive real-world integration can sustain momentum amid privacy flaws and 'false need' skepticism.
AI agentsLlamaMeta
0 likes · 7 min read

Meta’s debut of a ‘super‑intelligent’ large model, led by Scale AI founder Alexandr Wang, signals a strategic shift toward open‑source AI development and intensifies the competition for top talent, reshaping the industry’s roadmap toward AGI.
AGILlamaMeta
0 likes · 5 min read

This article breaks down the core principles, architectures, and formulas of Llama, ChatGLM, and T5, compares their strengths and weaknesses, and provides a zero‑data, no‑VPN beginner project that trains and evaluates all three models on sentiment analysis and summarization, visualizes the results, and offers practical deployment tips.
ChatGLMLlamaNLP
0 likes · 24 min read

This article explains Meta's LLaMA model series, detailing LLaMA 2's architectural innovations—RMSNorm, SwiGLU, rotary positional encoding, and grouped-query attention—and LLaMA 3's post-training pipeline including supervised fine-tuning, reward modeling, rejection sampling, and direct preference optimization.
DPOGrouped-Query AttentionLLM Architecture
0 likes · 10 min read

The article examines how Meta’s CEO Mark Zuckerberg’s aggressive talent‑buying and commercial focus have sidelined Turing‑award winner Yang Li‑kun, detailing the restructuring of Meta’s AI labs, the clash over research directions, and the broader dilemma of balancing academic innovation with business imperatives in the AI industry.
AI industryAI researchJEPA
0 likes · 14 min read

This article provides a comprehensive technical analysis of Meta's Llama series, tracing the evolution from Llama‑1 through Llama‑2 to Llama‑3, detailing model architectures, training data pipelines, optimization methods, benchmark results, and the broader impact on the open‑source AI community.
AI researchLlamalarge language models
0 likes · 25 min read

This tutorial walks through LLaMA‑Factory fine‑tuning by explaining how to choose learning rate, epochs, batch size, cutoff length, LoRA rank, and validation split, and shows how to estimate and reduce GPU memory usage with techniques like gradient accumulation, liger_kernel, and DeepSpeed.
AIDeepSpeedLlama
0 likes · 25 min read

This tutorial explains how large language models can use function calling to access real‑time data, walks through setting up a Flask endpoint, demonstrates integration with Mistral Small, Llama 3.2‑1B, and Qwen models, and provides complete Python code examples for end‑to‑end execution.
APIFunction CallingLLM
0 likes · 10 min read

This comprehensive technical report details the development, architecture, training methodology, extensive benchmark evaluations, safety measures, and inference optimizations of Meta's open‑source Llama 3 large language model series, covering models up to 405 billion parameters and supporting multilingual, multimodal, and tool‑use capabilities.
AILlamalarge language model
0 likes · 115 min read

This step‑by‑step guide shows how to use Hugging Face's Trainer API to pre‑train an ultra‑small LLaMA‑3 model (under 20 M parameters) on the TinyStories dataset, covering model configuration, tokenizer setup, data preprocessing, collators, training arguments, and inference results.
Hugging FaceLanguage ModelLlama
0 likes · 27 min read

Meta's Llama series has progressed from the 7‑65B Llama‑1 in early 2023 to the 8B and 70B Llama‑3 in 2024, scaling token counts from 1 T to over 15 T, adopting decoder‑only Transformers with RMSNorm, SwiGLU, RoPE and GQA, and adding supervised fine‑tuning, RLHF and DPO, resulting in state‑of‑the‑art benchmark performance and a vibrant open‑source ecosystem.
AILlamalarge language models
0 likes · 25 min read

The article reviews LLaMA’s Transformer and RoPE architecture, explains why its context windows (4K‑128K tokens) are limited, and evaluates industry‑proven extension techniques—including linear, NTK‑aware, and YaRN interpolation plus LongLoRA sparse attention—while addressing memory and quadratic‑cost challenges and presenting a KubeAI workflow for fine‑tuning and deployment.
AILlamaLongLoRA
0 likes · 17 min read


AIWalkerMar 4, 2024 · Artificial Intelligence
VisionLLaMA extends the LLaMA transformer architecture to 2‑D visual data with naive and pyramid designs, introduces a 2‑D rotary position encoding (AS2DRoPE), and achieves faster convergence and superior performance over state‑of‑the‑art ViT models on image generation, classification, segmentation, and detection tasks.
AS2DRoPEComputer VisionLlama
0 likes · 6 min read

This article reviews the LLaMA large‑language‑model series, covering its background, architectural innovations such as Add&Norm, SwiGLU, and RoPE, a known reversal‑curse bug, and provides step‑by‑step MindSpore Transformers code for model configuration, inference, and pipeline usage while previewing the upcoming LLaMA‑2 session.
AILlamaMindSpore
0 likes · 6 min read

This article investigates how mixing data from mathematical reasoning, code generation, and general instruction-following tasks influences supervised fine‑tuning of LLaMA models, revealing distinct scaling curves, resource‑dependent performance conflicts, and a two‑stage DMT strategy that mitigates catastrophic forgetting while boosting overall capability.
DMT StrategyData ScalingLlama
0 likes · 14 min read

This guide shows how to download, convert, 4‑bit quantize, and run Meta’s 7‑billion‑parameter LLaMA model on a single 16‑inch Apple laptop using Python, torch, and the llama.cpp repository, demonstrating that the quantized model fits in memory and generates responses quickly, with optional scaling to larger models.
7B modelAILlama
0 likes · 5 min read

Megatron-LLaMA is an open‑source high‑performance training framework for LLaMA models, offering tensor, pipeline, and sequence parallelism, an overlapped optimizer, and near‑linear scalability, achieving up to 176% speedup on 32 GPUs and robust performance even with limited network bandwidth.
DeepSpeedGPU optimizationLlama
0 likes · 10 min read

This article analyzes upgrade paths of major LLMs—ChatGLM, LLaMA, Baichuan—detailing performance, context length, and architectural changes, then examines essential capabilities, data cleaning, tokenizer and attention design, and offers practical guidance for balanced scaling and efficient model construction.
BaichuanChatGLMLLM Architecture
0 likes · 32 min read

Before training large language models like LLaMA, MetaAI applies a multi‑stage CCNet pipeline that crawls web data, stores it in WET format, deduplicates paragraphs, detects and filters languages using fastText, and further refines content by similarity to Wikipedia and citation‑based linear models.
CCNetLlamadata preprocessing
0 likes · 7 min read
