Avoid These 6 Log Management Anti‑Patterns to Keep Your Observability Reliable

Background

Observing system status and troubleshooting rely heavily on logs, a long‑standing observability method. A scientific local log‑management strategy should retain complete history, minimize performance overhead, and facilitate collection and analysis, yet many real‑world anti‑patterns break these goals for collectors like LoongCollector, Filebeat, Fluentbit, Vector, and OpenTelemetry Collector.

Anti‑Patterns

Copy‑truncate rotation creates a new file then truncates the original, which is non‑atomic. This can cause log loss, duplicate collection due to inode changes, and a time window where writes are missed.

Using NAS/OSS as log storage introduces eventual‑consistency metadata mismatches, empty reads, data‑delay, and possible loss because directory listings are slow and inotify is unsupported.

Multi‑process writes to the same file lead to interleaved entries, incomplete collection, and file‑lock contention.

Creating file holes to free space alters the file signature, causing collectors to treat the file as new, potentially duplicating data or losing history, and can fragment the filesystem.

Frequent overwriting of the whole log file results in metadata/content inconsistencies, data loss, and loss of historical logs.

Editing logs with Vim replaces the file, changing its inode and header, which may trigger duplicate collection or loss.

Recommendations

Prefer create mode for rotation: create a new file and rename the old one to preserve continuity.

Store logs on local disks or EBS for consistent performance and reliability.

Use append‑only writes combined with standard logrotate tools.

When freeing space, use fallocate instead of truncate or dd.

For read‑only access, use tools like less or grep.

If unavoidable, implement robust de‑duplication and exception handling on the consumer side.