Hi this is my third part of the Tomcat Clustering Series. In this post we are going to discuss the how to setup session replication in tomcat clustering environment. Session replication makes High availability and full fail-over capability to our clustering environment.[Check the video below for better understanding]
In my previous post we discussed about setup simple load balancer and how to make session affinity concepts.
How to setup Session Replication in tomcat
before going to session replication we need to understand 2 important concepts
- Multicast
- Session Manager in Tomcat
Multicast
is To transmit a single message to a select group of recipients.
here multicast used by tomcat cluster to identify the instances those
part of cluster.
There is 2 types
of cluster
- Static Tomcat Cluster
- Dynamic Tomcat Cluster
so here multicast concepts is used. each and every tomcat
first joining to single multicast group.
and send the heartbeat signals in periodic interval. so other tomcat
instances received these signal and add the member to the cluster.
Session
Manager in Tomcat
Session Manager is used to create and manage the session behalf
the application. In Servlet Specification request.getSession(); line is mention
that container (tomcat) is responsible for create the session. here
tomcat use the Session Manager for this purpose.
there is 4 types of
Session Manager
- Standard Manager
- Persistent Manager
- Delta Manager
- Backup Manager
Standard Manager
Its is the default manager used by tomcat. Even though we are not
mention in our web application tomcat use this manager for managing our
session. If u want to customize the this standard manager then add
<Manager> tag in context.xml file.
<Manager
className=“org.apache.catalina.session.StandardManager” />
here org.apache.catalina.session.StandardManager
is fully qualified class name of the Standard Manager.
Persistent Manger
This mnager is to sote the session information into persistent place
after some interval. here two types of store is available.
- File Store
- JDBC Store
JDBC Store helps to
store the session information to relational database.
so using the Persistent Manager we can achieve the tomcat cluster. but
its not swapped out in real time. its pushes the information after
certain interval. so if anything badly happen(crash) before that
interval then in-memory session data is gone.
Delta Manger
In this post we are going to use this manager. Its replicate the session
to all other instances. so this
manager usually used clustered environment. but not good for large
cluster.
Backup Manager
this manager usually used clustered environment. Its like delta manger.
but it will replicate to exactly one other instance(backup instance).
Its acted like one instance is Primary and another instance as backup
Steps to make Session Replication in Tomcat Clustering
here i will continue from exactly where i left in last session affinity post. so check that post and make sure jumRoute all are set properly. so steps are
- Enable Multicast routing
- Add <Cluster> Entries in conf/server.xml file for all instances.
- Enable the Web Application as distributable
1. Enable Multicast routing
In Linux Environment most of the system kernel is capable to
process the multicast address. but we need to add route entry in kernel
routing table.
sudo route add
-net 224.0.0.0 netmask 240.0.0.0 dev eth0
here eth0 is my Ethernet interface. so change according to your interface
In multicast address is belong to Class D address Range (224.0.0.0 to
239.255.255.255). so we inform to kernel if any one access these
address then it goes through eth0 interface.
2. Add <Cluster> Entries in conf/server.xml file for all instances.
This very important part for tomcat clustering. We need to Add
<Cluster> tag in conf/server.xml file in all tomcat instances.
<Cluster
className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
we can add this <Cluster> tag in either inside the<Engine>
tag or <Host> tag.
here SimpleTcpCluster is Tomcat Cluster implementation
This tag is looks like simple but its has many inner tags. if
we omitted then its takes the default values. if we want do any
cutomization (like change multicat address, receving address port) we
need to use complete <Cluster> tag
this is complete <Cluster>
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel
className="org.apache.catalina.tribes.group.GroupChannel">
<Membership
className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564"
frequency="500"
dropTime="3000"/>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
Check my sample conf/server.xml file (for reference)
here most of the code are boiler plate code. just copy and paste. if we need we can customize. for example we can change the multicat address and port number.
<Manager className="org.apache.catalina.ha.session.DeltaManager"/>
here Manager tag define the delta manager. Delta manager means replicate to all instances.
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
Tomcat Clustering use the Apache Tribes communication framework. This group commnication framework is responsible for dynamic membership (using multicast) , send and receive the session delta information using uni-cast (normal TCP connection).
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564"
frequency="500"
dropTime="3000"/>
This is Membership definition. here address is multicast address. we can pick any address from Class D address range (224.0.0.0 to 239.255.255.255)and any port number.
Each and every tomcat send the heart beat signal to multicast address in periodic (frequency) interval. all other tomcat whose joined the multicast address they can receive these signals and add the membership to the cluster. if heat beat signal is not revive some particular interval (dropTime) from any one of the tomcat, then we need to consider that tomcat is failed.
Note:-
All tomcat instances which is part of the clustering, should have same multicast address and port number.
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
here sender use the PooledParallelSender have pooled connections to use the send the session information concurrently. so its speedup the session replication process.
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
here we define which port Receiver can bind and used for receiving the session replicate information. here two properties are important. address and port. here address is ur system IP address and port is any unused port. here address="auto" its automatically pick the system IP address.
we have some interceptor
TcpFailureDetector -Its ensure that instance are dead. In some case multicast messages are delayed, all tomcat instances are think about that tomcat is dead. but this interceptor to make tcp unicast to failed tomcat and ensure that instances is actually failed or not
another important listener is JvmRouteSessionIDBinderListener, we talk about later
3. Enable the Web Application as distributable
Note:
All session in our web application must me serializable.
Do these steps to all tomcat instances and start the tomcat and httpd
server. check my configuration in my github repo or get as ZIP
now client make the request and first tomcat process and create the session, then looks like this
now all tomcat instance have exact copy of the session. so if tomcat 1 crashed or shutdown, then any other tomcat still can process the request [see the video below]
We used session affinity like previous post. based on that cookie id contain the tomcat name (worker name). so when first tomcat1 return the session id end with tomcat1. but when tomcat 1 is failed and tomcat 2 take the responsible for all further request. but session id still contain the tomcat1. so its makes the load balancer difficult. because tomcat1 is down. and load balancer pick any other tomcat. but actually tomcat2 takes the responsible. so we need to reflect these changes in session id.
JvmRouteSessionIDBinderListener take care to change the client session id to tomcat2 when failure is occurred so load balancer redirect to tomcat2 without confusing.
Check the git hub for all configuration files and tomcat clustering setup is available. or u can download as ZIP
