Essential Practices for Developing, Deploying, and Operating Microservices

Development

Source Code Control

Each service should have its own repository to keep checkout size small, simplify logs, and enable fine‑grained access control; services are not deployed together, so their code should not be co‑located.

Standardizing source‑code control further simplifies team work and makes CI/CD easier.

Development Environment

Developers need a fast local environment; to provide a consistent environment across operating systems, package the development environment as a virtual machine.

Because microservice architectures involve many services, developers cannot realistically work on all services on a single machine. Instead, combine locally run services with isolated cloud environments, allowing rapid iteration locally while testing against cloud‑hosted services. Isolation is critical; shared environments cause chaos from unintended changes.

Continuous Integration

Integrate code into the main branch as soon as possible. Updates to the main branch trigger CI systems to build automatically, which then runs automated tests to verify build quality.

Automated tests run on developers' machines, while CI can run more complex, time‑consuming tests in parallel across a cluster, speeding up work.

If all tests pass, CI publishes the build artifact to the automated deployment system.

Benefits include faster code integration, early conflict detection, more frequent full test runs to catch bugs quickly, and greater confidence in small, incremental changes.

Continuous Delivery

The goal is to release small changes quickly without large batch deployments, keeping the system running.

Achieving this requires a robust CI pipeline and automated deployment workflow.

Feature flags ensure users only receive fully ready functionality, enabling incremental deployment.

Risks of Shared Libraries

The biggest challenge is not controlling when updates to a shared library are deployed by consuming services, which can cause days‑long delays.

Publish a deprecation schedule and coordinate with service teams to apply updates promptly, ensuring backward compatibility.

Shared libraries are suitable for cross‑cutting concerns like connectivity, transport, logging, and monitoring, but business logic should not reside there.

Service Templates

Beyond core business logic, services handle tasks such as registration, monitoring, client load balancing, rate limiting, and circuit breaking. Teams should use templates to bootstrap these common tasks and integrate properly with the platform.

Templates accelerate work but are not mandatory; teams may choose to build custom solutions for required behaviors.

Providing templates for each supported technology stack yields benefits: reduced duplicate tooling, easier cross‑team code reviews, and smoother onboarding.

Service Replaceability

As the number of services grows, architectural limits become apparent. Maintaining simple, focused services makes replacement easier.

Changing databases or languages is feasible if APIs and event streams are well maintained, allowing full implementation swaps without affecting other services.

Deployment

Deployment Packages

Standardized deployment packages are a key component of automated pipelines.

Packages must be deployable anywhere, keep external configuration and secrets out of the package, and ensure isolation to prevent resource contention.

System images satisfy these requirements; each service gets its own image with version control, enabling resource limits, monitoring, and isolation equivalent to running a single service per host.

When using system images, updates must be performed by replacing the entire image rather than in‑place changes, ensuring identical environments across dev, staging, and prod and avoiding configuration drift.

Automated Deployment

Deploy any service version to any environment via a unified trigger, enabling fully automated, simple deployments and frequent small changes.

The goal is zero‑downtime updates. Rolling restarts or parallel running of old and new versions behind a load balancer allow graceful upgrades and quick rollbacks.

Feature Flags

Feature flags let code enable or disable functionality at runtime, decoupling code deployment from feature release.

Teams can query and manage flags via a shared library.

Flags enable staged rollouts to subsets of users, early problem detection, and rapid rollback by disabling the flag.

Flags should be short‑lived; once a feature is fully released, the flag should be removed to reduce support and testing complexity.

Encapsulate flags at entry points rather than scattering them throughout code.

Configuration Management

Deployable packages must not contain environment‑specific options or secrets; independent solutions are required.

Common approaches include environment variables, filesystem volumes, and shared key/value stores.

Be cautious with environment variables as they can leak; consider clearing them after reading.

Operations

Centralized Logging

Each service instance generates logs, but logs should not be stored on the instance because new deployments replace instances.

Use a platform‑provided centralized logging system with a standard log format, enabling search across services, instances, or the entire platform.

Logging code can reside in shared libraries or service templates.

Centralized Monitoring

Centralized monitoring is essential for quickly understanding the impact and root cause of failures.

Run multiple instances behind a load balancer; monitoring must aggregate metrics across instances and allow drill‑down to individual components.

Key metrics include infrastructure (CPU, memory, disk, network), request volume and latency, error counts, downstream request metrics, and service‑specific metrics.

Automatic capture of these metrics can be provided by service templates or shared libraries.

Distributed Tracing

Distributed tracing links request IDs across services, visualizing request flow and identifying performance bottlenecks.

Autoscaling

Stateless services scale by adding instances behind a load balancer; monitoring provides the needed metrics.

Many platforms expose a declarative interface to set desired instance counts, enabling programmatic autoscaling and fault‑tolerant recovery.

External Services

Services often depend on external systems such as databases, caches, message queues, or email services, which may be third‑party or internally hosted.

These external systems should be managed and automated similarly to internal services, with careful isolation to avoid resource contention.

Personnel

Full Lifecycle Ownership

Service teams must own, operate, and evolve their services from creation through retirement, not just at launch.

This ownership helps teams understand operational pain points and improve stability.

Autonomous Full‑Stack Teams

Each team member becomes the owner of multiple small services, possessing all skills needed for development, deployment, and operation.

Teams must be able to handle member turnover without leaving “orphaned” services.

Consistent platform practices reduce learning curves when services change hands.

Team size should be large enough to be self‑sufficient but small enough to avoid excessive coordination overhead; the “two‑pizza team” model is a common guideline.

References

Netflix: migration, deployment, build

Gilt: scaling, essence, implementation

SoundCloud: microservices, monolith decomposition, stack standardization

Google: monolith to microservices, deep experience

Amazon

eBay

Yelp

Spotify

REA Group

Otto

AutoScout24

Martin Fowler

12‑factor apps

SOA without ESB

Building microservices

Microservice architecture