A highly available, cluster orchestration system can be employed to manage Microservice deployments across heterogeneous environments. Cluster orchestrator will own the responsibility of ensuring the availability of Microservices with minimal human intervention. The system will consist of two components:

• Cluster Orchestration Server(s): Centralized management server
• Cluster Orchestration Agents: A thin native software deployed on every instance (virtual machine) in a cluster dedicated for hosting the Microservices.

Cluster orchestrator deploys an agent on every hosting instance (virtual machine) in a cluster which is the used to control and communicate with the host operating system and the services deployed on the machine. Once a connection between the agent and the server is established, cluster orchestrator handles the following responsibilities associated with managing the Microservices:

• Availability
• Data replication
• Data partitioning
• Load balancing
• Rolling updates and rollbacks
• Resource placement (high density deployment)
• Failover and recovery
• Health monitoring and healing

The following diagram illustrates a cluster orchestrator in action:

Microsoft Azure Service Fabric has inbuilt orchestration capabilities capable of handling hyper scale Microservice deployments across heterogeneous environments. More about Service Fabric cluster orchestration capabilities can be found here: https://azure.microsoft.com/en-us/documentation/articles/service-fabric-architecture/.