Run a mock round

Avoid microservices when:

• The domain is still being explored. Wrong boundaries are far more expensive to move than within a monolith.
• The team is small. Each service multiplies deployment, monitoring, on-call and security effort; a handful of engineers cannot operate dozens of services well.
• You lack operational maturity: automated CI/CD, centralized logging, metrics, tracing and infrastructure as code.
• The workload is low scale and the business needs fast iteration, where a monolith ships faster.
• Strong transactional consistency is central and distribution would only add complexity.
• There is no clear independent scaling or ownership driver.

A modular monolith with clear internal boundaries gives most of the maintainability benefit at a fraction of the cost, and lets you extract services later along proven seams. The rule of thumb is to earn distribution: adopt it only when a concrete, measurable problem makes it necessary.

In a dynamic environment, service instances start, stop and move, so callers cannot rely on fixed hostnames. Service discovery maintains a registry of healthy instances and lets clients resolve a logical service name to a live address.

Two patterns:

• Client-side discovery: the caller queries the registry (Consul, Eureka) and load-balances itself. Fewer hops but discovery logic in each client.
• Server-side discovery: the caller hits a load balancer or virtual IP, and the platform (Kubernetes Services, cloud load balancers) routes to a healthy instance. Simpler clients, one extra hop.

# Kubernetes resolves this name to ready pods
http://order-service.default.svc.cluster.local

Health checks are essential so unhealthy instances are removed quickly. In Kubernetes, DNS plus a Service is built-in discovery; for VMs you typically need a registry. Combine discovery with retries, timeouts and circuit breakers so a stale address does not cascade into an outage.

CAP states that during a network partition a distributed system must choose between consistency (every read sees the latest write) and availability (every request gets a non-error response). Since partitions are unavoidable, the real choice is CP or AP.

• CP systems refuse writes or reads rather than risk divergence; examples include systems using consensus such as etcd or ZooKeeper, and relational databases with strong quorum.
• AP systems keep serving and reconcile later; examples include Cassandra and DynamoDB, which favour availability with eventual consistency.

Partition occurs -> choose: reject (CP) or serve stale (AP)

PACELC extends this: even without a partition, systems trade latency against consistency. In microservices, apply the choice per operation: money movements may demand consistency, while a product view counter can be eventually consistent. Design compensating flows, idempotency and conflict resolution where you accept AP, and avoid assuming a globally consistent clock or snapshot.

Synchronous calls (HTTP/REST, gRPC) are simple, easy to reason about and return results immediately, which suits queries and request-response workflows. The downside is temporal coupling: if the callee is slow or down, the caller waits or fails, and failures can cascade.

Asynchronous messaging decouples services through a broker. The producer publishes an event and moves on, consumers process at their own pace, and the system absorbs traffic spikes and partial outages. The cost is eventual consistency, harder debugging, duplicate delivery and the need for idempotent consumers, plus a broker to operate.

Order HTTP -> Payment service            (sync, needs answer now)
Order -> order.created event -> Email    (async, fire and forget)

A common design uses sync for the user-facing read path and async for side effects such as emails, analytics, inventory updates and notifications. Prefer async when the caller does not need an immediate answer and when decoupling or burst handling matters.

Two-phase commit is rarely used in microservices because it needs a coordinating transaction manager, holds locks across services, blocks on coordinator failure and hurts availability. Instead, prefer an eventual-consistency design:

• Saga: model the workflow as local transactions with compensations (orchestrated or choreographed). Each step commits independently and failures trigger undo actions.
• Outbox pattern: write the business row and an event row in one local transaction, then publish the event asynchronously. This avoids the dual-write problem where a crash after the DB commit but before the publish loses the event.
• Idempotent consumers: deduplicate by message id or use upserts so retries and duplicate deliveries are harmless.
• Reconciliation: background jobs compare states and repair drift, and reads may use a read model projected from events.

BEGIN;
INSERT INTO orders ...;
INSERT INTO outbox (event_type, payload) VALUES ('order.created', :json);
COMMIT;

Choose compensations that are reversible or harmless, and make every step idempotent, because distributed atomicity is replaced by retries and repair.

An API gateway is the single entry point for clients, sitting in front of backend services. It typically handles:

• Request routing to the correct service.
• Authentication and authorization, terminating TLS and validating tokens.
• Rate limiting, quotas and basic request validation.
• Response aggregation or transformation, sometimes exposing a Backend for Frontend tailored to a client.
• Observability: access logs, metrics, tracing headers, correlation ids.

routes:
  - path: /orders/**
    uri: lb://order-service
    filters: [TokenRelay, RateLimit]

Benefits are a smaller client surface and consistent cross-cutting concerns. Risks are a single point of failure, a potential bottleneck and a temptation to accumulate business logic. Keep it thin, make it highly available, and avoid coupling it to service internals. Service meshes move some of these concerns to sidecars, but the gateway usually remains the public edge.

A saga replaces a single ACID transaction across services with a sequence of local transactions, each publishing an event or invoking the next step. If a step fails, previously completed steps are undone by compensating transactions, which are semantic inverses such as refunding a payment or releasing a reservation.

Two coordination styles:

• Orchestration: a central saga orchestrator tells each service what to do and handles failures. Easier to understand and monitor, but adds a coordinator.
• Choreography: each service reacts to events and emits the next. Loosely coupled, but the overall flow is harder to see and can turn into a tangle.

CreateOrder -> ReserveStock -> ChargePayment -> Ship
     cancel <-  release      <- refund        <- fail

Compensations are not true rollbacks: an email already sent cannot be unsent, so design steps to be reversible or idempotent. Combine with the outbox pattern and idempotent consumers to avoid lost or duplicate events.

Owning its data is what makes a service independently deployable. With a shared database, a schema change in one service can break another, teams coordinate releases, and the database becomes a hidden coupling point. Database-per-service enforces encapsulation: other services can only reach the data through a published API or events.

Benefits: independent schema evolution, freedom to choose the right store (relational, document, search, time-series), and failure isolation. Costs: no cross-service joins or foreign keys, so you compose data in the application or maintain read models, and consistency becomes eventual.

order-service  -> orders_db
user-service   -> users_db

To keep read models fresh, services publish domain events and consumers project the data they need, using the outbox pattern for reliability. Avoid the trap of one database server with logical separation but shared credentials and direct cross-schema queries: that is still a distributed monolith.

With many services and network hops, you cannot debug from a single log file. Observability rests on three pillars plus correlation:

• Structured logs: JSON with consistent fields and a correlation or trace id on every line.
• Metrics: counters, gauges and histograms for request rate, error rate and latency, plus resource usage. Track percentiles, not just averages.
• Distributed tracing: propagate a trace context (for example W3C traceparent) so one request can be followed across services; OpenTelemetry is the common standard.
• Health and readiness endpoints so orchestrators route traffic only to healthy instances.

traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

Add centralized dashboards and meaningful alerts on symptoms such as error rate and saturation rather than every host metric. A service mesh can emit much of this automatically. The goal is to answer "what happened to this request and why" without SSHing into production hosts.