Tenant Aware Load Balancer is the upcoming Load Balancer from WSO2. Here in this article i will briefly describe it's architecture and how it works.
Introduction
LoadBalancer
Load balancing is a key concern in a Platform-as-a-Service (PaaS) or a middleware platform deployed on the Cloud.
The primary functionality of a load balancer is to distribute the load of the incoming traffic amongst a set of back end worker nodes. This set of worker nodes can be either statically configured or can be dynamically discovered. In WSO2 carbon based products we use cluster messages which based on axis2 clustering to identify the node join to cluster or leave cluster.
Multi-tenancy - Tenant-aware Load Balancing
If a Cloud deployment has to be able to scale to thousands, hundreds of thousands or millions of tenants, we need tenant-partitioning. This means, for a single Cloud service, there will be multiple clusters, and each of these Service clusters will handle a subset of the tenants in the system. Creating dynamic tenant clusters & tenant partitioning strategies are some of the ongoing research areas. In such a tenant-partitioned deployment, the load balancers themselves need to be tenant-aware, in order to be able to route the requests to the proper tenant clusters. This means the load balancer has to be tenant-aware as well as Service-aware, since it is the Service clusters that are partitioned according to the tenants. WSO2 Load Balancer is now capable of routing messages in both tenant aware and service aware way.
Load balancing is a key concern in a Platform-as-a-Service (PaaS) or a middleware platform deployed on the Cloud.
The primary functionality of a load balancer is to distribute the load of the incoming traffic amongst a set of back end worker nodes. This set of worker nodes can be either statically configured or can be dynamically discovered. In WSO2 carbon based products we use cluster messages which based on axis2 clustering to identify the node join to cluster or leave cluster.
Multi-tenancy - Tenant-aware Load Balancing
If a Cloud deployment has to be able to scale to thousands, hundreds of thousands or millions of tenants, we need tenant-partitioning. This means, for a single Cloud service, there will be multiple clusters, and each of these Service clusters will handle a subset of the tenants in the system. Creating dynamic tenant clusters & tenant partitioning strategies are some of the ongoing research areas. In such a tenant-partitioned deployment, the load balancers themselves need to be tenant-aware, in order to be able to route the requests to the proper tenant clusters. This means the load balancer has to be tenant-aware as well as Service-aware, since it is the Service clusters that are partitioned according to the tenants. WSO2 Load Balancer is now capable of routing messages in both tenant aware and service aware way.
How WSO2 Tenant Aware Load Balancer Works
Load Balancer's task is to distribute the load of the incoming traffic amongst a set of back end worker nodes.
We will see how this works now.
when the request comes in to Load Balancer we have to identify host header and determine cluster domain
Eg: Host header = https://appserver.cloud-test.wso2.com/t/ttt.ttt/carbon/admin/login.jsp
Service domain is appserver domain
Then determine tenant Id from the url. Here tenant domain is ttt.ttt so we can determine tenant domain.
These parameters can extract from host header.
Now we have both service domain and tenant domain. so with both of them we can direct this request to correct cluster as we defined in loadbalancer.conf. There we can configure services with their cluster domains and tenant ranges. When load balancer loads it picks those ranges and cluster domains.
Load Balancer's task is to distribute the load of the incoming traffic amongst a set of back end worker nodes.
We will see how this works now.
when the request comes in to Load Balancer we have to identify host header and determine cluster domain
Eg: Host header = https://appserver.cloud-test.wso2.com/t/ttt.ttt/carbon/admin/login.jsp
Service domain is appserver domain
Then determine tenant Id from the url. Here tenant domain is ttt.ttt so we can determine tenant domain.
These parameters can extract from host header.
Now we have both service domain and tenant domain. so with both of them we can direct this request to correct cluster as we defined in loadbalancer.conf. There we can configure services with their cluster domains and tenant ranges. When load balancer loads it picks those ranges and cluster domains.
See the following sample configuration . You can find those configuration under the “repository/conf” folder. There you will find loadbalancer.conf file.
appserver {
# multiple hosts should be separated by a comma.
hosts appserver.cloud-test.wso2.com;
domains {
wso2.as1.domain {
tenant_range 1-100;
}
wso2.as.domain {
tenant_range *;
}
}
}
Let’s examine this configuration.
Above configuration is for Application service with two service clusters. Let’s examine it line by line.
First line says appserver is the main node of this configurations.
In next line you will find hosts entries. There we have to add host address that maps to application server service. If you need to add multiple service domains add it with separated by commas.
Next you will find domains there you will find two domains with pre defined tenant ranges.
wso2.as1.domain named cluster is responsible for load tenants with tenant id 1 to 100
wso2.as.domain named cluster is responsible for load tenants with all tenants except the tenants belong to above range.
Then it can forward request to correct cluster based on service domain and tenant domain.
How Load Balance End Point works
For load balancing we are using synapse end point as a end point to determine how message should forward. So here we have defined external end point which is named as Tenant Aware Load Balance End point. It has logic to forward messages to correct back end node. We can define loadbalance end point in synapse main sequence.
In order to do this you have to change main sequence with correct load balance end point. If you need to implement your own custom load balancing implementation you can write your own end point and configure load balancer to use it.
As an example we can load balance based on any other parameter which is available in request header (something similar to tenant domain). It's simple and straightforward.
For load balancing we are using synapse end point as a end point to determine how message should forward. So here we have defined external end point which is named as Tenant Aware Load Balance End point. It has logic to forward messages to correct back end node. We can define loadbalance end point in synapse main sequence.
In order to do this you have to change main sequence with correct load balance end point. If you need to implement your own custom load balancing implementation you can write your own end point and configure load balancer to use it.
As an example we can load balance based on any other parameter which is available in request header (something similar to tenant domain). It's simple and straightforward.
See the following diagram to get general idea on how it works.
Advantages and other features of WSO2 Tenant Aware Load Balancer
- Tenants are loaded in demand to the pre defined cluster based on the configurations in loadbalancer.conf.
- Unload unused tenants after paticular predefined time.
- Tenant Partitioning
- Tenant may have multiple service clusters
- Tenants will define the partitions.
- When a new instance added, notify the LB instances.
- Auto scaling - starting and terminating of instances based on the load. This supports any type of infrastructure as a service. What we have to do is write simple adapter to auto-scaler service based on the service vendor's API. It’s straight forward and simple. External components Autoscaler service can add, assign nodes into LB. So load Balancer is independent from the infrastructure that we are using.
- Group communication between clusters using cluster messages.
- Notifying all the load balancers when a new node is added to cluster.
- Data partitioning(tenants data) - we can use this design to data partitioning based on the Geography or any other factors. As an example if we decide tenant 1 to 100 should loaded and store their data on United state we can deploy new application cluster at united states and inform it to loadbalancer. So we have to add configuration to loadbalancer.conf file saying tenant 1 to 100 are loaded at particular cluster(named as wso2.as.usa.domain). Then all requests come to the tenant 1 to 100 will forward to that cluster. so all data related to those tenants (Application files, databases) will stored at united state servers.
- Private jet mode - Also we can assign some particular application cluster to specific tenant. Its like private jet. If you are willing to pay for entire cluster we can assign entire cluster for your tenant. This is again we can do using configurations at loadbalancer.conf file. So all request comes to that tenant will forward to that particular cluster.
- Always running an additional cluster(Default cluster) to quickly load new tenant. This cluster is responsible for all newly created tenants in the system and super admin tenant. We can define this cluster in load balancer configuration file with * mark as upper tenant ID.
After next release of WSO2 Load Balancer you will be able to use all above mentioned functionalities. Please visit http://wso2.org/projects/load-balancer
