Load Balancer & Service Registry

• Support scalability by routing incoming traffic to a stateless service instance

• Upstream client is unaware of the load balancer

• Most promote availability and therefore resiliency using health probes and checks

• Health probes can work with custom health check endpoints

• Either provide or support service registry systems

• On-premise physical or software options, and managed options in the cloud

• Round Robin: Simplest type of routing, can result in uneven traffic

• Least Connections: Routes based on number of client connections to server, Useful for chat or streaming applications

• Least Response Time: Routes based on how quickly servers respond

• IP Hash: Routes client to server based on IP; Useful for stateful sessions

• Record and provide microservice instance locations

• Microservices register on startup and deregister on shutdown

• Used by traffic routing mechanisms like load balancers

• Develop your own or DNS on cloud PAAS solutions and container orchestration engines

API Gateway & BFF APIs

• Central entry point, exposing microservices using one interface

• Can handle security: Authentication and authorization

• Can also provide call rate limiting, monitoring, caching, and logging

• Can provide load balancing and service registry functionality

• Central entry point, exposing microservices using one interface for a specific client

• Can handle security: Authentication and authorization

• You would use in addition to load balancers and service registry

• Custom logic to consolidate and format data from downstream microservices

Synchronous Communication

• Client to service, and service to service and service to external

• Your upstream software makes a request to a downstream software

• Upstream software waits for a response from the downstream software

• Huge compatibility across hardware and software thanks to its use on the Internet

• If you use the Internet, you are already using HTTP

• Both parties must be available, and then you wait for a response

• Performance subject to network quality

• Client must know the location of the service (host/port)

• Reducing temporal coupling and avoiding distributed transactions

API Style for microservice

• Consumer and developer-friendly by providing a predictable style

• REST principles are in line with microservices

• Decouples client and service

• Functionality to honor contracts (versioned endpoints for backwards compatibility)

• Encourage stateless services

• Ideal contract and data format for caching strategies

• RPC, GRPC, SOAP, GraphQL and many more

Resiliency Pattern

1. Timeout

2. Circuit breaker: after a number of trials, break the connection and return error

3. Retry

4. Cache



 

• Configure short timeouts on calling code to avoid waiting forever for a response

• Calling code stops calling downstream services if too many failures occur

• Calling of downstream service resumes after a specified threshold

• Ideal for transient faults: Momentary loss of connectivity or service

• Limit the number of retries and use with the circuit breaker pattern

• Separate cache database serving microservice instances

• Serve matching data from a cache that hasn't expired instead of making a sync call

OpenAPI and API Catalogs

• Document your microservice contracts (endpoints)

• Document using machine readable format (JSON)

• Use document to generate interactive websites

• Create an API catalogue

• Use machine readable documentation with many other tools

• Development teams working on microservices concurrently

• Business teams needing to see the data

• Scoping and designing new parts of your architecture

Asynchronous Communication

• Decouples client and service and reduces sync com and temporal coupling

• Sender also can be called client and publisher

• Many queues with many listeners to send and receive data

• Data sent referred to has as a message (command) or event

• Many queueing patterns

• Message brokers provide resiliency in your architecture

• Event-driven architecture and eventual-consistency

• Both concepts are helpful for microservices architecture

Transaction Manager for logical distributed transactions

• We can avoid physical distributed transactions in terms of waiting

• Avoided using asynchronous communication and messaging systems

• We need to manage the state of a logical data transaction (saga pattern)

• Centralize the transaction state: Somewhere, we can see that state of each part

• Have failure compensation: Functionality to undo a partially complete transaction

• Transaction managers and the saga pattern also allow you to cancel a transaction

Eventual Consistency and Event-driven Architecture

• A microservice has a local copy of data belonging to another service

• Local copy avoids having to make a sync call to retrieve the data from the source

• Uses publisher and subscriber model: Where client and service completely decoupled

• Microservice that owns the data publishes data changes as events (messages)

• Subscribers interested in the data subscribe to the events (optionally store them locally)

• This duplicated data across our microservices is eventual consistent

• Many eventual consistency patterns that balance availability and consistency

• Complete decoupling of publisher and subscriber: Publish data changes as events

• Create new subscribers, i.e., new use cases for your data without changing publishers

Deployment using virtual machines

• Waste of hardware resources (CPU, Memory, and storage)

• Multiple services on one machine violates the autonomous principle

• Also violates the resiliency principle (scalability and availability)

• Better resource utilization (CPU, Memory, and storage)

• Templating virtual machine images and snapshot backups

• Each service instance on a small VM promotes design principles

• Support for AC (Infrastructure as code)

• Special VM operating systems to manage VM's

• Multiple services still venerable to host failures

• Each VM still has an unnecessarily large footprint and are slow to startup

Deployment using containers

• Like VMs, isolate services from each other

• Single service per container (single deployable unit containing all dependencies)

• Much more resource-efficient and startup faster than VMs

• Most popular way of deploying microservices

• Multiple container images containing microservices architecture run using one file

• Recreate full microservices architecture environments (Spin up production on dev)

• Managing multiple containers on multiple servers: Load balancing and discovery

• Run containers: Ensure numbers and availability across these servers

• Containers without an orchestration engine are not ideal!

• Well supported on cloud platforms and on-premise

Deployment using cloud

• Only if you already have the budget, teams, and location/equipment

• Cloud provides managed and serverless services

• Infrastructure as a service: When you still need to manage the OS/Runtime

• You still must manage and update the OS and runtime

• Managing applications/services and data only

• No control over the runtime/OS running the application

• Cloud platform handles the running, concurrency, and resources

• Cloud platform may limit concurrency, duration of running, and tech choices

Security

• Use HTTPS everywhere!

• Network security: private and public network

• Call rate-limiting at API Gateway or BFF API level

• Use a reputable Identity provider/service
• OAuth2 and OpenID Connect standards
• Authentication: Identify the user (Identity token)
• Authorization: Can the client app access the microservice? (access token)
• Authorization: What can the user do? (access token)
• Pass these tokens between clients and services: all services validate tokens

• Standards: OAuth2, OpenID connect, and Token exchange

• Identity providers: Azure AD, Okta, Pingldentity, IdentityServer, and many more!

• Secure message broker using HTTPS, authentication, and message validation

Central Logging

• Reputable product that supports scalability and availability

• Centralized portal for querying, filtering, and managing log data

• Managing data: Archiving and backing up log data

• Security: secure connections (HTTPS), authentication, and authorization

• Examples: Elastic/Kibana, Splunk, and Graphite

• Structured logging: Support for custom fields, logging levels, and correlation IDs

• Lightweight log data format like JSON

• Data compliance requlations

• Communicate data policy

Central Monitoring

• Real-time metrics across servers

• Visualize and aggregate data

• Vast alerting options: SMS, email, visual and custom calls!

• Examples: Nagios, PRTG, and New Relic

• Archiving and backup of data

• Custom or off-shelf library for consistent stats across architecture

• Preconfigured machine, VM, or container templates