Load Balancer

server decommissioning process template

Load Balancer

Web Traffic- Management

Indexes

• Server Scalability.
• Load-Balancing.
• Session Replications and Failover.

Problem Statement

Johna wants to build the web application which can be access by around 500 concurrent users. But his backend server is only able to serve the request for approx. 100 concurrent users only. What Johna needs to take actions to fulfill his purpose.

 Answer: Server Scalability.

Scalability : Scalability is the capability of a system, network, or process to handle a growing amount of work, or its potential to be enlarged to accommodate that growth.

A system whose performance improves after adding resources, proportionally to the capacity added, is said to be a scalable system.

Methods of adding more resources for a particular application fall into two broad categories:-

To scale horizontally:- means to add more nodes to (or remove nodes from) a system, such as adding a new computer to a distributed software application

To scale vertically:-means to add resources to (or remove resources from) a single node in a system, typically involving the addition of CPUs or memory to a single computer

Horizontally Scaling

Server Load Balancing

Load balancing is the distribution of a workload across many nodes.

A Load Balancer

allows users to intelligently distribute traffic to a single IP across any number of servers using a number of different protocols.

It increases the reliability of your web application.

If one of your server nodes fails, the traffic is programmatically distributed to your other nodes without any interruption of service.

Methods of Load Balancing

Round Robin: With the round robin method of load balancing, the load balancer will send traffic to each node in succession.

Weighted Round Robin:This builds on the simple Round Robin load balancing method. In the weighted version each server in the pool is given a static numerical weighting. Servers with higher ratings get more requests sent to them.

Pre-requisites for load balancing

Apache2 ( /etc/apache2 )

Libapache2-mod-jk (/etc/libapache2-mod-jk)

Jdk

Tomcat

Recap

Scalability is related to server’s ability to handle efficiently many concurrent requests simultaneously.

High Availability is the set of technologies aimed at providing some guarantees that the application’s service will be available for the clients for the longest possible time. It is also known as web application up-time, and it is usually expected to be 100%.

Load Balancing is a technology aimed at distributing request load among a collection of servers

Load Balancer is the server that performs load balancing duties by distributing requests among servers on the cluster.

Horizontal Scalling VS Vertical Scaling

Sticky Session

A method used with Application Load Balancing, to achieve server-affinity.

A router or load balancer with sticky-session support can assign a single server to a particular user, based on their HTTP session or IP address.

Tomcat adds the name of the Tomcat instance to the end of its session id cookie, separated with a dot (.) from the session id.

If the Apache web server finds a dot in the value of the stickyness cookie, it only uses the part behind the dot to search for the route

In order to let Tomcat know about its instance name, you need to set the attribute jvmRoute inside the Tomcat configuration file conf/server.xml

The name of the session cookie used by Tomcat (and more generally by Java web applications based on servlets) is JSESSIONID (upper case) but can be configured to something else.

Session Replication

How to setup Session Replication in tomcat

Before going to session replication we need to understand 2 important concepts. Multicast Session Manager in Tomcat   Multicast is To transmit a single message to a select group of recipients. here multicast used by tomcat cluster to identify the instances those part of cluster. There are two types of Cluster, Static Cluster and Dynamic Cluster. In static cluster there is no need multicast, because each tomcat we statically defined/configured the other instances. But dynamic Cluster we are not defined anything. so each tomcat in that cluster some how to identify the other tomcat instances. so here multicast concepts is used. each and every tomcat first joining to single multicast group. and send the heartbeat signals in periodic interval. so other tomcat instances received these signal and add the member to the cluster.

Session Manager in Tomcat

Session Manager is used to create and manage the session behalf the application. In Servlet Specification request.getSession(); line is mention that container (tomcat) is responsible for create the session. here tomcat use the Session Manager for this purpose.

there is 4 types of Session Manager

• Standard Manager
• Persistent Manager
• Delta Manager
• Backup Manager

Standard Manager

• Its is the default manager used by tomcat. Even though we are not mention in our web application tomcat use this manager for managing our session.
• If u want to customize the this standard manager then add <Manager> tag in context.xml file.

<Manager className=“org.apache.catalina.session.StandardManager” />

here org.apache.catalina.session.StandardManager is fully qualified class name of the Standard Manager.

Persistent Manger

This mnager is to sote the session information into persistent place after some interval. here two types of store is available.

• File Store
• JDBC Store
• File Store helps to store all session information in separate files in underlying file system (local HDD or shared file-system like NFS,..) 
• JDBC Store helps to store the session information to relational database.

so using the Persistent Manager we can achieve the tomcat cluster. but its not swapped out in real time. its pushes the information after certain interval. so if anything badly happen(crash) before that interval then in-memory session data is gone.

Delta Manger

It’s replicate the session to all other instances. so this manager usually used clustered environment but not good for large cluster.

Backup Manager

this manager usually used clustered environment. Its like delta manger. but it will  replicate to exactly one other instance(backup instance). Its acted like one instance is Primary  and another instance as backup  

Delta Manager

Steps to make Session Replication in Tomcat Clustering

Add <Cluster> Entries in conf/server.xml file for all instances. This very important part for tomcat clustering. We need to Add <Cluster> tag in conf/server.xml file in all tomcat instances.

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>

we can add this <Cluster> tag in either inside the<Engine> tag or <Host> tag.

here SimpleTcpCluster is Tomcat Cluster implementation

Enable the Web Application as distributable. Add the tag</distributable> in web.xml of your application.

<Manager className="org.apache.catalina.ha.session.DeltaManager"/>

here Manager tag define the delta manager. Delta manager means replicate to all instances.

<Channel className="org.apache.catalina.tribes.group.GroupChannel">

Tomcat Clustering use the Apache Tribes communication framework.  This group commnication framework is responsible for dynamic membership (using multicast) , send and receive the session delta information using uni-cast (normal TCP connection).

<Membership className="org.apache.catalina.tribes.membership.McastService“ address="228.0.0.4" port="45564“ frequency="500“ dropTime="3000"/>

This is Membership definition. here address is multicast address. we can pick any address from Class D address range (224.0.0.0 to 239.255.255.255)and any port number.

Each and every tomcat send the heart beat signal to multicast address in periodic (frequency) interval. all other tomcat whose joined the multicast address they can receive these signals and add the membership to the cluster. if heat beat signal is not revive some particular interval (dropTime) from any one of the tomcat, then we need to consider that tomcat is failed.

Note:-All tomcat instances which is part of the clustering, should have same multicast address and port number.

<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
here sender use the PooledParallelSender have pooled connections to use the send the session information concurrently. so its speedup the session replication process.

<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
here we define which port Receiver can bind and used for receiving the session replicate information. here two properties are important. address and port. here address is ur system IP address and port is any unused port. here address="auto" its automatically pick the system IP address.

Backup Manager

In delta manager each tomcat instance need to replicate the session information to all other tomcat instances. Its take more time and replication if our cluster size is increased. so there is alternative manager is there Its Backup Manager.

Backup Manager is replicate the copy of session data to exactly one other tomcat instances. This big difference between both managers. here which tomcat creates that is primary copyof the session. and another tomcat whose hold the replicate session is backup copy. If any one of the tomcat is down. back up tomcat serve the session. Its achieve the fail over capability.

The setup process of backup manager is same as Delta manager. except we need to mention the Manager as BacupManager (org.apache.catalina.ha.session.DeltaManager)  inside <Cluster> element.

Suppose we have 3 tomcat instances and i configured into backup manager.

now user try access the page. User request comes to load balancer, and load balancer redirect the request to suppose tomcat1. Now tomcat one create the session, now tomcat1 is responsible to replicate exactly one copy to any one of the tomcat. so tomcat1 picks any tomcat which is part of the cluster (multicast). here tomcat1 picks tomcat3 as a backup. so tomcat3 hold the backup copy of the session.

we are run the load balancer in sticky session mode. so all further request from that particular user  is redirect to tomcat1 only. all modification in tomcat1 is replicate to tomcat3.

now tomcat1 is crashed/shutdown for some reason

now same user try to access the page. this time load balancer try to redirect to tomcat1. but tomcat1 is down. so load-balancer pick one tomcat from the remaining tomcats. here interestingly 2 case are there.

Case 1:

Suppose Load balancer pick the tomcat3 then tomcat3 receive the request and tomcat3 itself hold the backup copy of the session. so tomcat3 make that session as primary copy and tomcat3 pick any one tomcat as backup copy. so here remaining only one tomcat is there. so tomcat3 replicate the session to tomcat2. so now tomcat3 hold primary copy and tomcat2 hold the backup copy. now tomcat3 give the response to user. all further request is handled by tomcat3 (sticky session).

Case 2:

Suppose Load balancer pick the tomcat2 then tomcat2 receive the request and tomcat2 don't have the session. so tomcat2 session manager (Backup Manager) ask to all other tomcat manager "hi anybody hold the session for this user (based on session id [cookie])". Actually tomcat3 have the backup session. so tomcat3 inform to tomcat2. and replicate the session to tomcat2. now  tomcat2 make that session as primary copy and tomcat3 whose already have copy of session as remains as a backup copy of that session. so now tomcat2 hold primary copy and tomcat3 hold the backup copy. now tomcat2 give the response to user. all further request is handled by tomcat2 (sticky session).

so in either case our session is replicate and maintained by backup manager. Its good for large cluster.