Author

Big Data Technology Tribe

Focused on computer science and cutting‑edge tech, we distill complex knowledge into clear, actionable insights. We track tech evolution, share industry trends and deep analysis, helping you keep learning, boost your technical edge, and ride the digital wave forward.

Articles

Likes

102

Views

Comments

Latest from Big Data Technology Tribe

41 recent articles

Big Data Technology Tribe

Feb 25, 2026 · Databases

How Lance Implements MVCC Transactions with Optimistic Concurrency and Automatic Conflict Resolution

Lance uses Multi-Version Concurrency Control to provide ACID guarantees, creating immutable table versions on each commit and employing atomic storage primitives, rebase logic, and retry mechanisms to handle concurrent writes, conflict detection, and resolution across multiple transaction types.

Concurrency ControlDatabase InternalsLance

0 likes · 16 min read

How Lance Implements MVCC Transactions with Optimistic Concurrency and Automatic Conflict Resolution

Big Data Technology Tribe

Jan 31, 2026 · Backend Development

Debugging Lance‑Spark & Lance‑Ray on HDFS: Build Wheels and Fix Common Errors

This guide walks through building custom pylance and lance‑namespace wheels to enable HDFS support, resolves common ModuleNotFoundError, Hive dependency, and native library issues, clarifies correct table_id usage, and provides a complete Python script that reads and modifies a Lance dataset with Ray.

Pythonhdfslance-ray

0 likes · 9 min read

Debugging Lance‑Spark & Lance‑Ray on HDFS: Build Wheels and Fix Common Errors

Big Data Technology Tribe

Jan 20, 2026 · Big Data

Extending Spark SQL with LanceSparkSessionExtensions: A Complete Guide

This article explains how to inject the LanceSpark plugin into Spark, covering the core LanceSparkSessionExtensions class, various ways to register extensions, the custom parser and planner strategy implementations, and the underlying Spark mechanisms such as injectParser, injectPlannerStrategy, and PredicateHelper.

DataSourceV2LanceSparkPlannerStrategy

0 likes · 14 min read

Extending Spark SQL with LanceSparkSessionExtensions: A Complete Guide

Big Data Technology Tribe

Dec 19, 2025 · Big Data

Why Did Our HDFS Standby NameNode Crash? A Deep Dive into Block Recovery Bugs

A recent HDFS outage caused the Standby and Observer NameNodes to crash after heavy client load triggered block recovery failures, exposing a bug in commitBlockSynchronization that leads to mismatched block IDs and edit‑log inconsistencies, which can be fixed by applying HDFS‑17861.

BlockRecoveryCrashHadoop

0 likes · 15 min read

Why Did Our HDFS Standby NameNode Crash? A Deep Dive into Block Recovery Bugs

Big Data Technology Tribe

Nov 23, 2025 · Artificial Intelligence

How Ray Data Accelerates AI Workloads with Streaming Execution

Ray Data is a scalable library built on Ray that offers high‑performance, streaming‑execution APIs for AI workloads, enabling efficient batch inference, data preprocessing, and training data ingestion across CPU and GPU resources, while supporting diverse data formats and seamless integration with popular frameworks.

AI data processingPythonRay Data

0 likes · 11 min read

How Ray Data Accelerates AI Workloads with Streaming Execution

Big Data Technology Tribe

Nov 21, 2025 · Fundamentals

Mastering Ray: Core Concepts of Tasks, Actors, and Objects for Distributed Computing

This guide explains Ray's fundamental building blocks—including Tasks, Actors, remote Objects, Placement Groups, and environment dependencies—showing how to define, schedule, and retrieve distributed workloads with code examples and command‑line utilities.

ActorsDistributed ComputingObject Store

0 likes · 8 min read

Mastering Ray: Core Concepts of Tasks, Actors, and Objects for Distributed Computing

Big Data Technology Tribe

Nov 18, 2025 · Cloud Native

Master Dockerfile: Core Commands, Multi‑Stage Builds, and Handy Tricks

This guide explains Dockerfile fundamentals, covering essential instructions like FROM, RUN, COPY, WORKDIR, ENV, EXPOSE, CMD, and ENTRYPOINT, shows a complete example, demonstrates multi‑stage builds to shrink images, and shares practical tips for reliable builds.

DevOpsDockerfileimage-optimization

0 likes · 6 min read

Master Dockerfile: Core Commands, Multi‑Stage Builds, and Handy Tricks

Big Data Technology Tribe

Oct 18, 2025 · Databases

How Adaptive Structural Encoding Boosts Random Access in Columnar Storage

This article examines how adaptive structural encoding in columnar formats like Lance dramatically improves random‑access performance on NVMe storage, compares it with Apache Parquet and Arrow, and discusses the trade‑offs between scan speed, memory usage, and compression.

Columnar StorageLanceNVMe

0 likes · 17 min read

How Adaptive Structural Encoding Boosts Random Access in Columnar Storage

Big Data Technology Tribe

Sep 20, 2025 · Fundamentals

Master Rust Conditional Compilation: From #[cfg] to cfg! with Real-World Examples

This guide thoroughly explains Rust's conditional compilation, covering the #[cfg] attribute, the cfg! macro, custom feature flags, dependency selection, testing, and practical GUI demos, providing code snippets and best practices for writing portable, maintainable cross‑platform Rust applications.

CFGCross‑platformFeatures

0 likes · 11 min read

Master Rust Conditional Compilation: From #[cfg] to cfg! with Real-World Examples

Big Data Technology Tribe

Sep 18, 2025 · Fundamentals

Understanding Rust Crates, Cargo.toml, and Common Cargo Commands

This guide introduces Rust’s core building blocks—crates as compilation units, the Cargo.toml configuration file, and essential Cargo commands—illustrated with code examples and a sample project layout to help newcomers quickly grasp Rust’s package ecosystem.

CratesPackage ManagementRust

0 likes · 5 min read

Understanding Rust Crates, Cargo.toml, and Common Cargo Commands