User:Cloudsoftcorp/draft article intelligent cloud application mobility

Summary:
Intelligent Application Mobility is a design principle for software systems that is intended to help transactional business applications achieve:


 * near-instant scalability across machines, data centres and clouds
 * proximity of processing to demand and/or data
 * disaster avoidance
 * compliance with business policies and jurisdictional constraints

Contents:

 * Core Concepts and objectives
 * Problems caused by traditional approaches to scaling and distributing transactional business applications
 * The 6 key characteristics of Intelligent Application Mobility

Core Concepts:
Intelligent Application Mobility enables a business application to be activated as a collection of very fine-grained processing components, called segments, which can move independently and near-instantly to any point(s) in a data processing infrastructure while continuing to operate without interruption. Each segment is managed individually and automatically by user-defined policies, which are location-aware and determine the best location for each segment in the infrastructure, and will move segments across machines, geographies, and clouds to achieve the optimum deployment. As circumstances change – such as operating conditions, user locations, resource availability – Intelligent Application Mobility continually adjusts application deployment in real-time to maintain the best balance of performance, reliability, location and costs, consistent with the user’s policies.


 * a segment is a unit of processing with private state which communicates via messages. Segments are independent and individually managed, thereby enabling greater flexibility for solving scaling and distribution problems
 * segment mobility enables segments to move independently across machines, geographies and clouds, so that the overall application tier can adapt in real time to variations in workloads, changing provenance of demand, location of data, business constraints, and resource availability
 * non-stop mobility guarantees there is no interruption to service when segments move; any in-flight requests are guaranteed to arrive once only and in the correct sequence – even when segments are moving
 * policies are user-defined and automate the management of individual segments to enable continuous real-time optimization, plus the automatic enforcement of regulations and business compliance

A pre-requisite for implementing Intelligent Application Mobility is the use of a Layer 7 “overlay network” to facilitate the transparent re-distribution of segments across machines, geographies and clouds without interrupting service availability. This overlay network must completely abstract the location of segments so that a client-requester never has to know where an application segment is located – instead the client-requester makes a regular method call and the overlay-network transparently routes the requests to the required segment. When a segment moves to a new location the overlay network must re-wire itself so that requests (including those already in-flight) are re-routed to the new location and arrive exactly once in the correct order.

Objectives:
The above capabilities are intended to help achieve: Conceptually, applications typically consist of many thousands of segments (as a rule of thumb: one segment per stateful object). As Intelligent Application Mobility enables segments to dynamically reorganise themselves across machines as needed, this means a single application can dynamically scale-out (and scale-back) as required by several orders of magnitude without any duplication of segments across machines. This last point is particularly important for transactional business applications for the reasons outlined below (“Problems caused by traditional approaches to scaling and distributing transactional business applications”).
 * Elastic scalability:

Enterprise applications often service demand in multiple locations. As segments are fine-grained and independently mobile, specific segments can each move closer to where the demand for that segment is located, thereby reducing round-trip latencies. And segments can re-locate as the locus of demand changes (e.g. “follow-the-sun” processing).
 * Affinity of processing and demand:

Segments encapsulate just the state that is critical to their core function, and hence are both lightweight and fast to move around – even across wide-area networks. Consequently it will often be more efficient and lower latency to move the segment to where data is located rather than ship data across the network to the segment for processing.
 * Affinity of processing and data:

The real-time monitoring data needed to inform policy-based mobility, combined with the policy framework itself, can help enforce compliance. For example, Intelligent Application Mobility needs to be location aware, which in turn can be combined with policies to help ensure that certain data is only processed in certain jurisdictions where required by regulation.
 * Automatic compliance:

Through moving just the segments that make up an application, Intelligent Application Mobility can re-locate an entire application significantly faster than alternatives that also move the application’s complete environment (such as implementing mobility at the level of a virtual machine). Consequently it needs less notice of an impending catastrophic event in order to avoid a disaster by re-locating an application. Through its dynamic overlay network, Intelligent Application Mobility can also create new paths to bypass network congestion; and can create redundant paths to protect against network failure.
 * Disaster avoidance:

Problems caused by traditional approaches to scaling and distributing transactional business applications
Applications that execute business transactions are difficult to scale and distribute as they must maintain consistency and integrity when changes are made to data, which imposes technical constraints. Maintaining these constraints is rarely a problem when data contention is low, but this changes as business applications scale-out to handle significant volumes, and particularly where applications run across geographies.

Once an application needs to span multiple machines (e.g. because a single machine doesn’t have enough processing capacity or because processing needs to take place in more than one location) then holding state in the processing tier creates problems. This is because a the integrity and consistency of an application’s in process state now needs to be maintained across multiple machines. Traditional approaches to solving these issues are detailed below, together with the problems they themselves introduce.

Stateless Processing Tier:
An obvious way to avoid issues associated with holding state in the processing tier is to remove state from the processing tier – hence the advent of stateless programming. The advantage of this approach is that it doesn’t matter which instance of a server handles any given request, therefore you can have as many instances of the server as you want with a load-balancer distributing the workload across those instances, which in turn means the number of servers used can be scaled out and scaled back very easily.

However the stateless programming model is unnatural in that the usual techniques for storing in process state – including local variables – are not available. It is also inefficient whenever a process needs to persist data between requests. Some design patterns introduce external databases or data caching fabrics to fill this gap, but this is efficient only in a limited number of cases: the need for frequent synchronisation can introduce a large amount of complexity and delay. For transactional applications, and highly-contended data in particular, this overhead can preclude the use of the stateless model altogether.

Static Partitioning:
One technique for maintaining state within a processing tier that spans multiple machines while avoiding any duplication of state is to statically partition the application across those machines. For example, a typical partitioning scheme might dedicate the first server machine to items in the range A E, the second to the range F-J, the third to K-N, and so on. However, while this technique ensures that each stateful process exists only once, it now means that each request has to be routed to a specific server machine – the one that hosts the unique stateful process required to service the request. Hence a stateful routing mechanism is needed – typically a message broker. In addition partitioning in static; any change to the partitioning scheme is disruptive; and you also have to ensure that each partition has enough processing resource to handle its maximum workload as there is no way for a partition to change on-demand. This “just in case” provisioning of resources is inevitably wasteful most of the time, and may be inadequate at other times.

Cloning + Data Replication:
A technique for maintaining state in a processing tier that spans multiple machines while avoiding the problems of static partitioning is to clone the entire application – including its state – as many times as needed to meet the current workload. Conceptually this approach re-creates the flexibility and location transparency of stateless processing – it doesn’t matter which instance of a server handles any given request, which again means you can simply vary the number of clones in line with varying workloads, and dynamically load-balance across the number of clones.

However, maintaining an instance of the same state in each clone means that any update made by one clone must be propagated to all others. This multiplies the number of updates n times, where n is the number of clones; plus when an update takes place, then n-1 clones will be out of date until the update is propagated – which is incompatible with high data contention.

The 6 key characteristics of Intelligent Application Mobility
Intelligent Application Mobility enables state to be maintained in the processing tier without any need to either partition or duplicate that state. This is achieved through the following 6 characteristics:

1. Much finer-grained scalability and distribution are required

Decomposing an application into services, so that individual services can be distributed across multiple machines, is a well-established pattern for scaling applications. Effectively the service is the unit-of-scalability: the greater the number of services, the greater the number of machines a transactional application can scale across before partitioning or cloning is required.

Inevitably the number of services into which an application can be decomposed is limited, so the scalability of this approach is equally limited. However scalability will increase by orders of magnitude if the ‘unit-of-scalability’ is an application segment rather than an application service> This is because a single service contains a potentially unlimited number of segments. For example, a securities trading application typically consists of several services, one of them being the ‘order book’ service (where bids and asks for securities are recorded and matched). The order book service may contain tens of thousands of individual ‘order-books’ – one for each security being traded. In this example the order-book is the segment. Hence scaling at the segment (order-book) level makes the order-book service dramatically more scalable – instead of being limited to a single machine, the order-book service can now scale anywhere up to the point where an entire machine is allocated to each individual order-book. This very fine-grained unit-of-scalability is also, by definition, the unit-of-distribution – which enables much more efficient and effective distribution of transactional applications as discussed below. The importance of operating at a very fine-grained level is also detailed in Microsoft’s paper on their Orleans framework (http://research.microsoft.com/apps/pubs/default.aspx?id=141999).

2. Each segment must be mobile

The potential offered by much finer-grained scalability can only be fully realised if segments are mobile i.e. they can migrate dynamically between multiple machines, geographies and clouds. Without mobility, the way you initially deploy the segments is the way they stay deployed – which effectively provides nothing more than an alternative (albeit enhanced) mechanism for statically partitioning a service.

Very fine-grained mobility allows individual segments to move separately or collectively, wherever is best, and whenever required. To scale out a service, individual segments are simply moved to one or more additional server machines – which may be in the same datacentre, or spread across geographies and clouds. There’s no static partitioning, and no cloning or replication. Effectively the application can now dynamically repartition itself to achieve just-in-time (rather than just-in-case) provisioning.

3. Mobility must not interrupt or degrade the service

A limitation of conventional mobility technologies is that they move entire virtual machines or LPARS, which consumes a lot of resource, takes a considerable amount of time, and services are noticeably paused as they are moved. Consequently moving applications is necessarily an infrequent event – even within the same datacentre, and more so across the wide-area-network. By comparison, mobility at the segment level is lightweight and very fine-grained, which in turn offers the possibility of continual, very precise optimisation of resource-usage. In other words, individual segments can be distributed and redistributed dynamically between available resources in order to continually achieve the optimal allocation of workload to resources. However this continual resource optimisation is only feasible if frequent reconfiguration is possible – which requires the ability to move transactional segments around with no interruption or degradation to service. If mobility incurs any performance penalty it will be curtailed, so it’s essential to move segments while they are still running and especially without pausing them.

4. Mobility must be near-instantaneous

Avoiding performance penalties is only feasible if mobility is near-instantaneous. Whereas the process of moving a coarse-grained entity such as a virtual machine may take many minutes from initiation to completion, moving even large numbers of segments typically takes just milliseconds as relatively little state, and no code, needs to be moved. Couple this with the ability to move segments while they are still running with no interruption or degradation to service, and intelligent application mobility now positions your application to respond near-instantly to any changes in the operating environment, and to precisely match resource usage to rapidly fluctuating workloads.

5. Full mobility must be available across the wide area network

There are several scenarios where mobility over the wide area network is valuable. For example...


 * dynamically move processing to the location(s) where the greatest demand is being generated
 * re-locate specific segments closer to specific users in order to improve latency
 * move segments to where the data is located, when that data is too big to move over the network
 * “follow-the-sun” processing
 * use the most cost-effective computing resource, perhaps in datacentres that are under-utilised, or in clouds that are offering the best spot-prices
 * high availability and disaster avoidance

The problems caused by implementing mobility at a coarse-grained level (e.g. moving virtual machines) are greatly exacerbated by the bandwidth and latency constraints a wide-area network. By comparison, very fine-grained mobility at the segment level is significantly faster and lighter-weight. This means bandwidth is rarely an obstacle, and latency constraints can be a positive reason to implement wide area mobility – latency can be reduced by moving a segment closer to its user.

6. Automatic control and governance

The capabilities described above create highly mobile application segments that can dynamically and continuously reconfigure themselves – including across geographies – in response to changing workloads, resource availability, user demand, performance criteria, and costs. With so many factors in play, it is only feasible to implement Intelligent Application Mobility with automatic management of these capabilities, which requires:


 * the continuous, real-time monitoring of workloads, capacities, resource availability, user demand, locations, costs etc; and
 * matching these monitoring data against business policies in order to determine when, where and how to respond to changes in any of the above.

Using a policy-based framework to automate the management of this type of dynamic application is not just essential for driving down the cost of management and driving up the elasticity benefits of cloud computing. It’s also essential for ensuring that good governance is enforced. For example, in many industries there are regulations governing where data processing and storage must, and must not, take place. A policy framework that is geo-location aware can ensure compliance with such regulations – even as data and processes dynamically move around a cloud.