This article explains the origins and market growth of data lakes, compares them with traditional data warehouses, showcases major implementations like Amazon Galaxy and Club Factory, and provides practical guidance on choosing open‑source or commercial cloud solutions to construct a data lake efficiently while minimizing risk.
An analysis of Indeed's 2014‑2019 US job data reveals that SQL, Java, and Python dominate employer demand, while AWS shows explosive growth, highlighting shifting trends in programming languages, cloud services, and data‑science‑related roles across the tech job market.
An analysis of recent data from Canalys, Synergy Research Group and other sources reveals that the global cloud computing market grew 37.6% to $107.1 billion in 2019, projects continued rapid expansion to $2840 billion by 2024, and compares the performance and strategies of AWS, Azure, Google Cloud and Alibaba Cloud.
This article details how a single engineer created Listen Notes, a podcast search engine and API, by leveraging standard web technologies, AWS infrastructure, Django backend, React frontend, Ansible-driven DevOps, and a suite of monitoring and productivity tools, proving that a solo effort can launch a robust internet product.
AWS CTO Werner Vogels explained at re:Invent how the Nitro system offloads networking, storage and management to custom hardware, delivering near‑bare‑metal performance, stronger isolation, and a modular architecture that also powers containers, serverless workloads, and next‑generation EC2 instances.
This roundup highlights major cloud‑native developments, including AWS re:Invent 2019 announcements such as SageMaker Operators for Kubernetes and EKS on Fargate, the 2020 KubeCon Europe schedule, Rancher 2.3.3 release, upstream Kubernetes enhancements, and a curated list of essential open‑source projects and reading recommendations for practitioners.
An analysis of Indeed's 2014‑2019 job‑posting data reveals that SQL, Java, and Python lead the demand rankings, while AWS, Azure, Docker, and machine‑learning skills show explosive growth, and traditional technologies like Oracle and .NET are rapidly declining.
The article explains Coinbase’s unique security and infrastructure requirements for blockchain nodes, describes the challenges of blue‑green deployments, and details the Snapchain system built on AWS that enables fast, reliable snapshot‑based node provisioning, upgrades, and high‑availability scaling.
This guide explains CloudEndure’s features and walks through a step‑by‑step example of configuring AWS cross‑region disaster recovery, covering initial project setup, data replication, test and recovery mode switching, and the failback process, while highlighting networking and security considerations.
This guide walks through setting up a highly available Redis 3.0 master‑slave cluster with Sentinel and a floating VIP on AWS EC2, covering environment preparation, configuration files, scripts, service startup, and failover testing.
The article details NetEase's experience designing and implementing dynamic server scaling for overseas games on AWS, comparing Auto Scaling and GameLift, describing a custom scaling platform, the challenges faced, lessons learned, and future directions for cloud‑based game operations.
The article examines AWS's Open Distro for Elasticsearch, critiques its impact on open‑source licensing, discusses aggressive cloud‑provider practices, and explores how the SSPL controversy highlights the need for new standards to protect open‑source innovation.
Meltwater details how it processes millions of daily media posts using a custom‑tuned Elasticsearch 1.7.6 cluster of over 400 nodes on AWS, covering data volume, query complexity, node configuration, indexing strategy, performance optimizations, and lessons learned for large‑scale search deployments.
This presentation details NetEase Games' journey of moving its mobile titles overseas onto AWS, covering the challenges of performance, elasticity, security and global connectivity, the systematic cloud evaluation framework they built, and the architectural and operational practices they adopted to achieve reliable, cost‑effective worldwide game services.
This article provides a step‑by‑step tutorial for migrating MySQL databases—whether hosted on Amazon RDS or self‑managed servers—to Amazon Aurora, covering four scenarios from simple snapshot migration with downtime to complex minimal‑downtime approaches using read replicas, Percona XtraBackup, and AWS DMS.
The article argues that AWS’s relationship to Kubernetes mirrors Windows’s historic dominance over Linux, examining vendor lock‑in, the evolution of cloud services, the economics of scaling, and why enterprises should consider open‑source alternatives and distribution‑based Kubernetes solutions to avoid being trapped.
Reflecting on a decade of AWS, this article shares ten hard‑earned lessons—from building evolvable systems and anticipating failures to prioritizing security, automation, and open platforms—that guide the design, operation, and scaling of cloud services for today’s enterprises.
The article describes Ctrip's motivation, design choices, and implementation details for deploying a self‑managed Kubernetes cluster on AWS, covering network architecture with VPC and ENI, custom image distribution, logging and monitoring, data synchronization, and operational challenges such as ELB loopback and kubelet configuration.
MongoDB changed its open‑source license to the SSPL to curb cloud providers, prompting AWS to launch its own compatible DocumentDB fork, while industry leaders debate the move’s impact on open‑source freedom, developer choice, and MongoDB’s market performance.
Meltwater’s media‑monitoring platform runs a custom Elasticsearch 1.7.6 cluster of over 400 nodes on AWS, handling 200 TB of primary data and 3 million daily documents while serving thousands of complex queries per minute, achieved through careful shard design, master‑node configuration, extensive performance tuning, and automated provisioning.
The article examines AWS Outposts as a hybrid‑cloud solution, arguing that while it brings public‑cloud services to on‑premises data centers, it cannot match the ultra‑high SLA of pure public cloud, and discusses the broader implications of private‑cloud versus public‑cloud dominance in the IT industry.
Amazon is rapidly shifting its enterprise workloads from Oracle's proprietary databases to its own AWS services, planning to retire Oracle software by early 2020, a move that highlights the intensifying competition between the two tech giants and the broader shift toward cloud-native infrastructure.
This article chronicles Netflix’s evolution from a DVD‑rental startup to a cloud‑native streaming giant, highlighting its partnership with AWS, the development of chaos‑engineering tools like Chaos Monkey and the Simian Army, and the open‑source technologies that underpin its resilient, scalable architecture.
The article traces Netflix’s evolution from DVD rentals to a cloud‑native streaming giant, explains how it leverages AWS for massive scale, and details its chaos‑engineering tools—Chaos Monkey, Simian Army, and related monkeys—that continuously test and improve system resilience.
This article explains how AWS Fargate eliminates the need for traditional container orchestration by providing a serverless, pay‑as‑you‑go platform that integrates with Amazon VPC, Auto Scaling, ELB, IAM and compliance standards, reshaping the way developers build and run cloud‑native applications.
This step‑by‑step guide shows how to configure Route53 DNS, prepare a VM with required tools, create an S3 state store, provision a Kubernetes cluster on AWS using kops, validate it, expose a sample service, and clean up the resources.
This article explains why distributed caching is essential for large‑scale internet applications, outlines the business and technical benefits of cloud services, and details EVCache’s cloud‑native design, cross‑region replication, high‑availability mechanisms, and real‑world use cases such as Netflix’s recommendation system.
The author recounts a 16‑hour drive to Las Vegas for AWS re:Invent, highlighting AI‑focused sessions such as Alexa, Lex, Polly, serverless Lambda, the MXNet vs TensorFlow competition, and emerging video‑processing research, while noting strengths, limitations, and future growth prospects.
The author reflects on attending AWS re:Invent 2017, highlighting Alexa's AI capabilities, the ecosystem for skill development, the technical challenges of building voice assistants, and the need for tight integration of product, algorithm, and engineering to advance intelligent assistants.
The article introduces AWS S3 SELECT for in‑object SQL filtering, highlights Aurora's new Multi‑Master and Serverless capabilities, describes DynamoDB's on‑demand backup and Global Tables, and provides a brief overview of the managed graph database Neptune, emphasizing performance and cost benefits.
The AWS re:Invent 2017 conference in Las Vegas introduced a range of compute innovations—including Amazon EKS, AWS Fargate, new EC2 instance families, Spot Instance hibernation, Time Sync Service, and a Serverless Application Repository—while showcasing sessions from Netflix and other industry leaders.
This article examines the concepts, strengths, and limitations of Google App Engine and Amazon AWS as Platform‑as‑a‑Service offerings, focusing on their application scenarios, development support, operations management, and associated services, and draws conclusions for game server development.
The article describes the challenges of resource isolation and permission management in a microservice environment, explains AWS IAM concepts and policies, and details the design, development, and expected benefits of the internally built OpenIAM service for unified authentication and authorization across services.
The article explains Netflix's end‑to‑end build and deployment pipeline, detailing how Nebula, Jenkins, Spinnaker, and AWS are combined with a culture of cloud‑native microservices to deliver code from local compilation to immutable AMI‑based releases across regions in about sixteen minutes.
The article analyzes 2017 Q2 earnings, Stack Overflow traffic, language preferences, industry and geographic trends to compare the popularity and growth of Amazon Web Services and Microsoft Azure, highlighting AWS's faster growth and Azure's strong presence in certain sectors and regions.
This article explores whether Facebook could be hosted on AWS by estimating Facebook's server count, comparing hardware performance, evaluating software compatibility, and calculating the massive cost implications, ultimately concluding that while technically possible, it would be far more expensive than maintaining its own data centers.
This article details how REA Group implements a comprehensive DevOps workflow for its micro‑service architecture on AWS, covering the 12‑Factor principles, toolchain components, automated deployment with rea‑shipper, monitoring, alerting, and the collaborative processes that enable continuous delivery at scale.
Amazon Aurora separates compute and storage using a service‑centric design, employs a quorum‑based replication across three availability zones, slices data into 10 GB protection groups, and leverages S3‑backed redo logs to deliver MySQL‑compatible performance with rapid recovery and strong consistency.
This step‑by‑step guide shows how to register for Amazon AWS, launch a free‑tier EC2 instance with Linux, bind an Elastic IP, set up a MySQL RDS database, and deploy a WordPress site, providing a practical hands‑on learning path for Linux operations and cloud computing.
The article examines public cloud’s cost‑effective benefits and its reliability challenges, citing major outages from AWS, Azure and Alibaba Cloud, and advises both providers and users to adopt multi‑cloud, backup, and disaster‑recovery strategies for safer cloud adoption.
The article shares a comprehensive recap of AWS re:Invent 2016, highlighting over a thousand new features across compute, storage, networking, security and tools, discussing serverless concepts, the Modern Data Architecture framework, and practical lessons learned by the English‑Fluent‑Talk engineering team.
This step‑by‑step guide shows how to set up an AWS Tokyo VM, install Git and Ansible, clone microservice lifecycle and bookstore repositories, configure Vagrant, build Docker images, run tests with docker‑compose, and push the resulting images to a private registry.
This article explains how to set up Dockerized hosts using Docker‑Machine on Windows, covers driver options, environment configuration, and demonstrates building, publishing, and running a .NET Core application in containers on both local and cloud platforms.
This article details Medium’s evolution from its initial AWS‑based backend stack to a sophisticated, service‑oriented architecture that leverages Node.js, Go, DynamoDB, Redis, Aurora, Neo4j, Spark, and CI/CD pipelines to support millions of users.
500px handles massive image uploads and billions of daily views by leveraging a Go‑based microservice architecture on AWS, using Nginx, HAProxy, RabbitMQ, and VIPS to achieve up to 1000 requests per second with sub‑180 ms response times.
Airbnb’s engineering VP Mike Curtis explains how the company leverages Amazon Web Services, a Hadoop‑based big‑data platform, and custom tools like Aerosolve, Airflow, and Airpal to power its global marketplace, enabling rapid scaling, dynamic pricing, and personalized search through extensive cloud infrastructure and machine‑learning pipelines.
This guide walks you through creating an AWS EC2 instance, installing and configuring Squid as a proxy, restarting the service, setting up local proxy settings, and securing the connection with an SSH tunnel for encrypted traffic.
The article reviews the evolution of the public cloud market, highlights Amazon AWS as the dominant player, discusses notable provider outages such as Nirvanix and AWS incidents, and offers guidance for enterprises on multi‑cloud strategies, data redundancy, and provider responsibilities.
The article analyzes why stability, not raw performance, should be the primary criterion when choosing a cloud provider, describes a 7‑day benchmark across AWS, Azure, Alibaba Cloud, Tencent Cloud and UCloud, and presents detailed results for CPU, memory, disk, database and storage stability.
Using Apache JMeter, this study objectively benchmarks hot object storage performance (RT and TPS) and pricing across five major cloud vendors—AWS, Azure, Alibaba Cloud, Qiniu, and Huawei—evaluating download/upload speeds for various file sizes and calculating a performance‑to‑price ratio to guide optimal provider selection.
This article presents a systematic performance and cost‑effectiveness comparison of six major cloud object‑storage providers—AWS, Azure, Qiniu, Huawei Cloud, Alibaba Cloud, and Kingsoft Cloud—using JMeter‑driven upload/download tests across various file sizes and analyzing price data to rank their overall value.
This article explains how Netflix leverages Amazon S3 and Elastic MapReduce to build a virtually unlimited, dynamically scalable Hadoop data warehouse in the cloud, and introduces Genie—a Hadoop platform‑as‑a‑service that abstracts job submission, resource management, and cluster orchestration.
Airbnb leverages AWS, Hadoop, Presto, Airflow, and custom machine‑learning tools to power its global marketplace, optimizing search, pricing, and data pipelines while achieving significant cost savings and operational efficiency.
This article shares two real‑world operations‑automation case studies—cloud‑based scaling on AWS with Chef and Jenkins, and a comprehensive automated deployment workflow for an advertising company's C++ RTB system, Java platform, and data clusters—highlighting processes, tools, and key lessons.
The article describes how a mobile analytics service processes billions of events daily using a Lambda‑style architecture that combines Kafka, Storm, Amazon EMR, and S3 to achieve scalable, fault‑tolerant batch and real‑time computation, while ensuring reliable event ingestion and graceful degradation.
This article describes how a legacy .NET contract management system was transformed into a set of independent microservices using a strategic decoupling approach, explains microservice fundamentals and advantages, and introduces the Ruby‑based Stencil framework with code generation, CI templates, and one‑click AWS deployment to accelerate development and operations.
