Here is info on how I setup clustering on User Application 4.0.1 using TCP and Unicast instead of UDP and Multicast. Novell Documentation on UA 4.0.1 covers how to setup clustering using UDP and Multicast communcation. I have setup UA for several clients, none of which would allow Multicast on their networks. Ergo, I have never setup a UA installation using UDP/Multicast. I have always had to reconfigure for TCP/Unicast. Novell does not say much about how to set this up.

Start with TID #3448409

This only covers one piece of the puzzle. Read here for info on JBoss clustering in general. User App 4.0.1 is based on JBoss 5.1GA

Seems even the JBoss Community has noticed a dearth of info on doing TCP Clustering. From what I can tell, JBoss itself has two caches that need to be clustered. JBoss Experts can chime in here anytime to correct my thinking. The JBoss community link provides some detail on how to configure both. One issue with the JBoss docs, Novell has modified the standard JBoss configuration quite a bit. So much so, it does not totally match what is in the JBoss Wiki article. From what I found in the UA XML files, the second cache, referred to as JBoss Messaging, or JBM, is commented out completely. Is JBM needed for UA Clustering? Novell? (or is it NetIQ now?) Once I get an answer from Novell/NetIQ, I will post it here. For now, assume it stays as installed, no changes.

Well, no answer forthcoming from Novell/NetIQ on if JBM clustering is required. So, I proceeded without it and it appears to be working.

Ok, we need to edit a few files on the JBoss server. The first is the startup script, which can be found in /opt/novell/idm on linux.(assuming you pick the default installation location. If you did not, well, you know where you installed it, don't you?). If you are using Windows, make the necessary adjustments. For Linux, it is start_jboss.sh. Here we need to add a few extra startup options. Some are covered in the Novell/NetIQ TID, some are not.

Here is what I had for the default;

#!/bin/sh

JAVA_HOME=/opt/novell/idm/jre
export JAVA_HOME

# The heap size values here have been optimized for your system.
# In order to use these settings, you need to uncomment the setting
# of JAVA_OPTS.
#
# If you specified it, we have also added a setting for your cluster,
# "workflow engine id". 
# 

# Make sure that our JRE is picked up first.
PATH=/opt/novell/idm/jre/bin:$PATH
export PATH

JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -server -Xms1024m -Xmx1024m -XX:MaxPermSize=512m "
export JAVA_OPTS

exec /opt/novell/idm/jboss/bin/run.sh -Djboss.service.binding.set=ports-01 -c IDMProv -b 0.0.0.0 -Dcom.novell.afw.wf.engine-id=Engine1 

This is what I made it look like;
#!/bin/sh

JAVA_HOME=/opt/novell/idm/jre
export JAVA_HOME

# The heap size values here have been optimized for your system.
# In order to use these settings, you need to uncomment the setting
# of JAVA_OPTS.
#
# If you specified it, we have also added a setting for your cluster,
# "workflow engine id". 
# 

# Make sure that our JRE is picked up first.
PATH=/opt/novell/idm/jre/bin:$PATH
export PATH

JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -server -Xms1024m -Xmx1024m -XX:MaxPermSize=512m "
export JAVA_OPTS

exec /opt/novell/idm/jboss/bin/run.sh -Djboss.service.binding.set=ports-01 -c IDMProv -b 0.0.0.0 -Dcom.novell.afw.wf.engine-id=Engine1 -Djboss.partition.name=DEVIDM -Djboss.default.jgroups.stack=tcp -Djgroups.tcpping.initial_hosts=10.27.37.23[7600],10.27.37.24[7600] -Dnovell.jgroups.tcp.tcpping.initial_hosts=10.27.37.23[7815],10.27.37.24[7815]

The key are two JGroups variables that define the members of the cluster, one for JBoss (-Djgroups.tcpping.initial_hosts), one for Novell (-Dnovell.jgroups.tcp.tcpping.inital_hosts). Pay attention to the port numbers in brackets. Then there is the item that says use tcp (-Djboss.default.jgroups.stack=tcp) for communications. If you are running this cluster on the same network/VLAN/whatever as another User App Cluster, give yours a name (-Djboss.partition.name=DEVIDM in this case). This will keep them isolated so nodes do not try and join the wrong cluster. The tcp protocol should prevent this, but just to be sure...

Ok, now that we have set all our environment info in the startup script, we need to switch the tcp stack from using MPING (Multicast based, we do not want it) to TCPPING. That is in a file buried in the JBoss file structure. For default installs this is here;

/opt/novell/idm/jboss/server/IDMProv/deploy/cluster/jgroups-channelfacory.sar/META-INF/jgroups-channelfactory-stacks.xml (have fun typing that one)

Edit this file. Look for this string;

stack name="tcp" (including quotes)

under that item, comment out the MPING entry and uncomment the TCPPING entry.

            <!-- Alternative 1: multicast-based automatic discovery.  
            <MPING timeout="3000"
                   num_initial_members="3"
                   mcast_addr="${jboss.partition.udpGroup:230.11.11.11}"
                   mcast_port="${jgroups.tcp.mping_mcast_port:45700}"
                   ip_ttl="${jgroups.udp.ip_ttl:2}"/> -->            
            <!-- Alternative 2: non multicast-based replacement for MPING. Requires a static configuration
                 of *all* possible cluster members.   -->
            <TCPPING timeout="3000"
                     initial_hosts="${jgroups.tcpping.initial_hosts:localhost[7600],localhost[7601]}"
                     port_range="1"
                     num_initial_members="2"/>

I changed the num_initial_members value to the number of nodes in the cluster. The docs say this represents the minimum number of cluster nodes that need to be active. Set yours as you see fit.

Perform both these edits on all cluster nodes. This info is in the link to the JBoss Wiki page above. There is one more setting to be done, and that is in the Novell/NetIQ TID. That is changing the cluster string in the Cache settings page in User App.

Open User App and login as an Admin User. Go to the Administraton tab, then the Caching Configuration menu. Under Cluster Configuration, set things according to the TID (above). The cluster properties is shown below for reference.


Current Cluster Properties:TCP(start_port=7815;loopback=true):TCPPING(initial_hosts=${novell.jgroups.tcp.tcpping.initial_hosts};port_range=10;timeout=3000;num_initial_members=3):MERGE2(max_interval=10000;min_interval=5000):FD(shun=true;timeout=2500;max_tries=5):VERIFY_SUSPECT(timeout=1500):pbcast.NAKACK(gc_lag=100;retransmit_timeout=3000):pbcast.STABLE(desired_avg_gossip=20000):pbcast.GMS(print_local_addr=true;join_timeout=5000;shun=true;view_bundling=true)

Once you have all these settings done, restart JBoss (/etc/init.d/jboss_init). Use a separate start/stop command.

TIP: if you want the command line saved in a local file, export JBOSS_CONSOLE=<path to file>. Then you can tailf the file and watch the console, see the cluster build (or not build as may be).

More Information#

There might be more information for this subject on one of the following:

Add new attachment

Only authorized users are allowed to upload new attachments.
« This page (revision-16) was last changed on 11-Nov-2011 08:56 by jim