Tag

s3

0 views collected around this technical thread.

Linux Ops Smart Journey
Linux Ops Smart Journey
Jan 19, 2025 · Cloud Native

How to Integrate GitLab with External MinIO and CertManager for Secure Cloud‑Native Storage

This guide walks you through configuring GitLab to use an external MinIO object store and CertManager for TLS certificates, covering credential setup, bucket creation, Helm chart adjustments, secret management, and verification steps to achieve a secure, efficient cloud‑native deployment.

GitLabMinIOcertmanager
0 likes · 9 min read
How to Integrate GitLab with External MinIO and CertManager for Secure Cloud‑Native Storage
Test Development Learning Exchange
Test Development Learning Exchange
Dec 4, 2024 · Cloud Computing

Using AWS S3 for Data Storage and Processing with Python

This tutorial guides readers through creating an AWS account, configuring the AWS CLI, uploading and downloading CSV files to S3, and using Python with pandas to clean, deduplicate, and aggregate the data before re‑uploading the results, illustrating end‑to‑end cloud data processing.

AWSCLICloud Computing
0 likes · 6 min read
Using AWS S3 for Data Storage and Processing with Python
Xiaolei Talks DB
Xiaolei Talks DB
Nov 14, 2024 · Cloud Native

How to Back Up OceanBase 4.2.5 on Kubernetes to Baidu Cloud BOS (S3)

This guide walks you through upgrading the ob‑operator, creating S3 credentials, configuring an OBTenantBackupPolicy YAML, applying the backup, and verifying the backup status to achieve a full data backup of an OceanBase 4.2.5 tenant to Baidu Cloud BOS using Kubernetes.

BOSKubernetesOceanBase
0 likes · 8 min read
How to Back Up OceanBase 4.2.5 on Kubernetes to Baidu Cloud BOS (S3)
Cloud Native Technology Community
Cloud Native Technology Community
May 30, 2023 · Cloud Native

Improving Third‑Party Service Performance in Kubernetes with Nginx Caching

This article explains how to boost performance and reliability of third‑party service calls in a Kubernetes cluster by configuring Nginx as a caching gateway, covering cache settings, deployment, persistent shared cache via S3, and the associated limitations.

CachingDevOpsKubernetes
0 likes · 10 min read
Improving Third‑Party Service Performance in Kubernetes with Nginx Caching
DataFunSummit
DataFunSummit
Jan 1, 2023 · Big Data

Shopee Data Infra Presentation: Storage Status, Acceleration, Serviceization, and Future Plans

The Shopee Data Infra talk details the current storage architecture, Presto‑based acceleration with Alluxio caching, service‑oriented storage solutions using Alluxio Fuse and S3 APIs, and outlines future enhancements for Spark/Hive integration and CSI/Fuse optimizations, providing a comprehensive view of large‑scale big data storage engineering.

AlluxioCache ManagerData Infrastructure
0 likes · 16 min read
Shopee Data Infra Presentation: Storage Status, Acceleration, Serviceization, and Future Plans
DevOps Cloud Academy
DevOps Cloud Academy
Mar 2, 2022 · Operations

Promoter: Rendering AlertManager Graphs for DingTalk Notifications Using Go

The article introduces Promoter, a Go‑based webhook that fetches Prometheus metrics, renders alert graphs with gonum/plot, stores the images in S3‑compatible object storage, and embeds them in DingTalk notifications, providing deployment instructions, template customization, and core implementation details.

AlertmanagerGoPrometheus
0 likes · 10 min read
Promoter: Rendering AlertManager Graphs for DingTalk Notifications Using Go
Shopee Tech Team
Shopee Tech Team
Oct 8, 2021 · Databases

Cold‑Hot Storage Architecture and Practices in Shopee ClickHouse

Shopee ClickHouse implements a cold‑hot storage architecture by mounting JuiceFS‑backed S3 as a remote volume, using table‑level storage policies and TTL‑driven background moves to shift data from SSD to cold storage, achieving cost‑effective scalability, improved disk utilization, and validated performance despite operational challenges such as Redis memory pressure and small‑file handling.

ClickHouseCold-Hot StorageData Tiering
0 likes · 22 min read
Cold‑Hot Storage Architecture and Practices in Shopee ClickHouse
Architects Research Society
Architects Research Society
Dec 24, 2020 · Cloud Computing

Enforcing Squid Access Policies for Amazon S3 and Yum in an AWS VPC

This tutorial demonstrates how to configure an open‑source Squid proxy in an AWS VPC to restrict Internet access, allow only approved Amazon S3 buckets and Yum repositories, route traffic through specific gateways, and achieve high availability using Auto Scaling and Route 53.

AWSHigh AvailabilityProxy
0 likes · 20 min read
Enforcing Squid Access Policies for Amazon S3 and Yum in an AWS VPC
NetEase Game Operations Platform
NetEase Game Operations Platform
Dec 28, 2019 · Operations

Operational Pitfalls of Using S3 Website Hosting and CDN Query‑String Caching

The article analyses practical drawbacks of S3 static‑website hosting, the complexities of CDN query‑string caching, and protocol incompatibilities such as HEAD + Range and If‑Match handling, offering concrete lessons and mitigation strategies for cloud operations teams.

CDNCloudFrontOperations
0 likes · 10 min read
Operational Pitfalls of Using S3 Website Hosting and CDN Query‑String Caching
NetEase Game Operations Platform
NetEase Game Operations Platform
Aug 10, 2019 · Operations

Resolving Chinese Filename Encoding Issues on S3 Website with Nginx Proxy

The article explains how Chinese filenames cause encoding errors when uploading to Amazon S3, why S3 forces UTF‑8 for website access, and presents a practical Nginx‑based proxy solution that transparently converts GBK‑encoded URIs to UTF‑8 to ensure reliable file retrieval.

GBKNginxOperations
0 likes · 9 min read
Resolving Chinese Filename Encoding Issues on S3 Website with Nginx Proxy
HomeTech
HomeTech
May 23, 2019 · Frontend Development

Building a Cross‑Platform S3 Upload GUI with Electron and Vue

This article explains how to create a cross‑platform desktop client for uploading local resources to S3‑backed CDN using Electron, Vue, and lowdb, covering UI design, project structure, main‑renderer process communication, persistent storage, command‑line integration, packaging, and update handling.

Cross‑PlatformDesktop ApplicationElectron
0 likes · 16 min read
Building a Cross‑Platform S3 Upload GUI with Electron and Vue
HomeTech
HomeTech
Feb 20, 2019 · Cloud Computing

Storage Concepts, Front‑end JS Integration, and Migration Practices Using Ceph and S3

This article explains block, object, and file storage concepts based on Ceph, describes how front‑end JavaScript resources are migrated to an S3‑compatible object store, outlines the supporting Lua/Nginx combo tool, caching strategies, and step‑by‑step e‑commerce migration procedures.

CephNginxbackend
0 likes · 16 min read
Storage Concepts, Front‑end JS Integration, and Migration Practices Using Ceph and S3
Architecture Digest
Architecture Digest
Mar 11, 2018 · Backend Development

Handling 1 Million Requests per Minute with Go: A Scalable Backend Architecture

The article describes how a Go‑based backend, using a two‑layer job/worker pattern with buffered channels and configurable worker pools, can reliably ingest millions of POST requests per minute, serialize payloads to Amazon S3, and dramatically reduce server count through Elastic Beanstalk auto‑scaling.

GoJob Queuebackend
0 likes · 12 min read
Handling 1 Million Requests per Minute with Go: A Scalable Backend Architecture