Labels
The following settings can be used for a general cluster configuration. Do note that you want to be EXTREMELY careful in setting high values here so as to not cause a non-detection event where the cluster does not failover as expected: Likewise, turning up the sensitivity may cause false failover events to happen when not needed:
- If any nodes are currently paused, unpause them all using the failover cluster MMC
- Set the following cluster properties on the passive node(s) from the command line:
cluster /prop SameSubnetDelay=1000
cluster /prop SameSubnetThreshold=10
cluster res ClusterName /priv HostRecordTTL=300
cluster res AnyOtherClusterName /priv HostRecordTTL=300 - Also set the following cluster properties for the individual node via the command line:
cluster /priv HeartBeatLostInterfaceTicks=10:DWORD
cluster /priv HeartBeatLostNodeTicks=10:DWORD - Stop and start the cluster service on each node to confirm the settings have taken effect
Do note that the general settings and how to set these are found at the following link below:
What has changed for Windows 2008 and the heartbeat network?
Speacial Note: Dynamic DNS and Windows 2008 Failover Clusters
If you are seeing random issues here where DDNS is failing even after the changes are made above you can try going the alternate route and performing the following changes:
- Change the cluster DNS registration TTL's to to reflect a (1) day expiration:
cluster res ClusterName /priv HostRecordTTL=86400
cluster res AnyOtherClusterName /priv HostRecordTTL=86400 - Uncheck the "Register this connection's addresses in DNS" in the advanced DNS properties tab on all NIC's for all nodes of a cluster
- Choose the "Disable NetBIOS over TCP/IP" option in the advanced WINS properties tab on all NIC's for all nodes of a cluster
NOTE - This change is basically making the DNS registration of the cluster's network name valid for the prescribed time frame. If the IP Address is changed manually on purpose, you may need to restart the cluster and/or clear out the DNS cache to ensure this gets updated to the latest IP Address. Also, proper testing should be performed to ensure this option is valid in any particular customer's environment.
Fixing 'too many open files' error
Once in a while we stumble upon 'too many open files' error in log files especially for java application servers. What this means is that you have hit a limit on number of open file descriptors (fd)/file handles assigned for the process or user the application server is running as.
Let's use tomcat for our example.
So to get the number of fd's used by tomcat we run
$ pgrep tomcat
12345
$ sudo ls /proc/12345/fd | wc -l
2040
Don't use `lsof | wc -l` to get the number of fd's. `lsof` lists much more than the number of fd's in use. If you look at the FD column you'll see memory mapped files, current working directory, root directoy, the text segment of the running process etc listed which don't utilize file descriptors.
To see the fd limit run `ulimit` as the application user
$ sudo -u tomcat sh -c 'ulimit -n'
2048
One can also check for soft limit(-S) and hard limit(-H)
$ sudo -u tomcat sh -c 'ulimit -Hn'
2048
To see the system wide fd limit
$ cat /proc/sys/fs/file-max
148922
Or
$ sysctl fs.file-max
fs.file-max = 148922
To find the current system wide fd usage
$ cat /proc/sys/fs/file-nr
1184 0 148922
Or
$ sysctl fs.file-nr
fs.file-nr = 1184 0 148922
The first field shows the number of allocated fd's, second field gives the number of allocated but unused fd's and the last field shows the maximum number of fd's. So in the above example 1184 fd's are allocated and all are in use.
So once you hit a limit the next step, obviously, is to increase it a bit. So let's do that.
For example to change the limit for tomcat user append the below lines to /etc/security/limits.conf
tomcat soft nofile 1024
tomcat hard nofile 3072
We can also put
ulimit -n 3072
in the shell script (startup wrapper script or rc script) that invokes the application server to change the limit for that process as all processes spawned from the shell will inherit the limit.
To increase the system wide limit we have to change the kernel runtime parameter fs.file-max and save it in /etc/sysctl.conf
# echo 'fs.file-max = 200000' >> /etc/sysctl.conf
# sysctl -p
# sysctl fs.file-max
fs.file-max = 200000
Steps to create a new user:
- Login into the target server that has access to modify Active Directory
- Shared AD Environment => Login into the appropriate admin station
- Dedicated AD Environment => Login into the customers domain controller
- Shared AD Environment => Login into the appropriate admin station
- Click on the start button and navigate to Programs | Administrative Tools and click on Active Directory Users and Computers
- In the left-hand pane of the console tree, right-click the folder in which you want to add a user account. Navigate to New and then click User
- In First name, type the user's first name
- In Last name, type the user's last name
- In Full name, type in the first initial of the first name and the full last name
- In User login name, retype the full name used above
- Click on Next
- Type in a suitable password using a password generator where possible
- Do not check the User must change password at next login check box
- Click on Next and then Finish
- Right-click the newly created user, navigate to and click on Properties
- Click on the Members of tab
- Click on Add
- Add any security groups that are needed by typing in the security group into the space provided and clicking on Check Names
- Click OK and OK to exit out
** *Note - The user creation process does not allow UNIX attributes to be edited unless you are in the Domain Admins security group. If this is the case you MUST use the command line script on the admin stations in IAD03 OR open a ticket with Operations Engineering. Otherwise, if the account will be a service account or if no UID is required, then you are done. If this is for a dedicated domain, please use the MMC as mentioned above in creating a new user.
RHEL5 rhcs cluster nodes will try to fence each other if they lose multicast cluster communications between nodes.
In these types of problems the culprit is almost always a failure or configuration issue with multicast at the network switch.
Environment
Red Hat Cluster Suite
Diagnosing the problem
1. On all cluster nodes enable recognition of icmp broadcasts:
echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
2. On all cluster nodes run "cman_tool status" and note:
A. Number of cluster nodes.
B. The cluster multicast address:
[root@node5 ~]# cman_tool status
Version: 6.1.0
Config Version: 8
Cluster Name: rhel_cluster
Cluster Id: 31412
Cluster Member: Yes
Cluster Generation: 180
Membership state: Cluster-Member
Nodes: 2
Expected votes: 1
Total votes: 2
Quorum: 1
Active subsystems: 9
Flags: 2node Dirty
Ports Bound: 0 11 177
Node name: node5
Node ID: 5
Multicast addresses: 239.192.122.47
Node addresses: 9.47.65.165
3. Do a 2-count multicast ping:
ping -c 2 -I bond0 -b -L 239.192.122.47
4. If there are no problems then you should see multicast responses from all other cluster nodes:
PING 239.192.122.47 (239.192.122.47) from 9.47.65.165 bond0: 56(84) bytes of data.
64 bytes from 9.47.65.166: icmp_seq=1 ttl=64 time=1.23 ms
64 bytes from 9.47.65.166: icmp_seq=2 ttl=64 time=2.28 ms
-- 239.192.122.47 ping statistics --
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 1.230/1.755/2.281/0.527 ms
5. If you don't get the proper responses, then:
A. Make sure that you previously ran the following on ALL the cluster nodes:
echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
B. If you confirmed that Step 5A was done, then the either a node or network
switch is misconfigured or there are hardware problems such as bad
network cables.
6. On all cluster nodes disable recognition of icmp broadcasts:
echo 1 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
Alternative Method
'omping' can be started on both nodes, taking the the ip of the remote node and the own ip as parameters.
Reference : How do I test multicast?
Ensure you run the same command on both nodes at the same time as mentioned in the above reference URL to properly check multicasting between the nodes.
A tale of two servers
The Apache httpd web server and the Apache Tomcat server are two very popular products in use for delivering web content.
About the Apache httpd server
The Apache httpd web server grew out of the NCSA web server. At its core, httpd implements the HTTP protocol specification. It was primarily designed to serve static files to browsers. Over the years, many features have been added and the core web server can be extended through the use of modules. Some popular modules include:
- mod_rewrite : this module provides a powerful mechanism for the administrator to rewrite URL requests that come in or get sent out of httpd
- mod_cache : this module provides a top-level module framework for caching web content
- mod_ssl : this module adds strong encryption to httpd using the OpenSSL toolkit
- mod_dbd : this module adds a database connectivity layer to httpd
- mod_mime : this module allows the administrator to control actions within the server based on MIME types and file extensions, it allows for the defining of handlers and filters for output
The Apache httpd web server ships with over 70 modules and there is a repository of 3rd party modules to extend it even further (currently with 490 modules.)
About the Apache Tomcat server
The Apache Tomcat server is a project that started at Sun Microsystems as the reference implementation of the Java Servlet and Java Server Pages Specifications. The Tomcat code base was donated by Sun to the Apache Software Foundation in 1999. As the Java Servlet and JSP specifications develop and mature, Tomcat's versatility grows and expands. As implied, the Tomcat server is run in a Java virtual machine and requires, at least, a JRE (Java Runtime Environment) to run although it is more commonly run using a JDK (Java Development Kit.) Tomcat should run anywhere a JDK is available (Linux, Windows, UNIX, etc.).
Tomcat also implements the HTTP protocol and can be extended to execute non-Java scripts, such as Python, PHP and Ruby.
For the remainder of the article, the Apache httpd web server will be referred to as "Apache" and the Apache Tomcat server will be referred to as "Tomcat."
The relationship between Apache and Tomcat
Both Apache and Tomcat implement the HTTP protocol for serving files to browsers. Both can be extended to serve dynamic content using many popular scripting languages. So what are the differences? Why would you choose one over the other? To answer that, let's look at the strengths.
- There is no module to allow Apache to directly execute Java code. Since Tomcat is a Java application executing in a Java virtual machine, Java becomes the native language of the web applications it serves.
- It can be argued that the integration of scripting languages like PHP and Python is better with Apache than with Tomcat but this isn't easily proven or refuted.
- Apache generally has a smaller memory footprint than Tomcat and serves static content faster using fewer resources.
- Virtual hosting, aliasing and URL rewriting are, generally, easier to configure in Apache.
Because of these points, people usually use Apache for supporting scripting languages and serving static content and use Tomcat for Java applications. Where they merge together is in the use of an Apache module, mod_jk for the 1.3.x and 2.0.x versions of Apache and mod_proxy_ajp for the 2.2.x version of Apache. The module allows Apache to proxy requests between the browser and a Tomcat server seamlessly through the use of a binary protocol, with persistent connections, to streamline communications, making it the preferred mechanism for connecting the two servers.
Another way to connect Apache and Tomcat is to use the mod_proxy_http module in Apache 2.2.x. This is a generic HTTP proxy and does not provide the benefits of the mod_jk/mod_proxy_ajp modules.
The combination of Apache and Tomcat provides for a powerful and scalable web server architecture
CNF objects are objects that occur in an Active Directory forest where there is a "collision" of a particular name where two different domain controllers have the same object. To ensure all objects are "unique" one of the objects gets appended with CNF followed by a GUID. In general, one of these objects should be deleted preferably the one with the older date or CNF in its name. But, either object can be valid. For more information, you an refer to the link below:
Back in the old days when you wanted to update a vDisk, you needed to make a complete copy of the current vDisk, then import it back into the vDisk Store, set it to Private Mode and link it to a Master device to apply your changes. After making all the changes, you needed to put the vDisk back to Standard Image mode and copy the complete vDisk to the rest of the Provisioning Services servers.
Now with Citrix Provisioning Services vDisk versioning you can add versions to your current vDisk. This method of updating the vDisk creates a delta file next to the vDisk itself. This delta file is usually considerable smaller than the complete vDisk file and you don’t need to change the vDisk mode to apply changes to it so that it can continue to be used.
My previous blog was about how to create a Citrix XenApp 6.5 vDisk. In this blog I will show you how to update this vDisk with Provisioning Services vDisk versioning and what steps you should always go through for Citrix XenApp 6.5.
Base
The first version, when creating the vDisk, is called Base (version 0). At file level, the Base consists of a .vhd file (the vDisk itself), a .pvp file (the vDisk properties file) and the .lok file (when the vDisk is in use).
Creating a vDisk version
The first step is to create a new version to the current vDisk, this will be version 1. To do that, open the Citrix Provisioning Services console, browse to the vDisk, right click on it and select Versions.
Click New
Version 1 is now created. Notice that the default access type is Maintenance. Click Done.
At file level you see that there is a *.1.avhd and a *.1.pvp file created. These are the “delta files” for Version 1, all new changes are saved in these files.
Prepare the XenApp Master server for the vDisk update
Just like when creating the vDisk, I use a dedicated XenApp Master server for updating the vDisk. This Master server has no other active user sessions and have no policies applied to it. But any server in the farm can be used to apply the updates as long as there are no active user sessions on it.
Within the Provisioning Services console open the properties of the “Master” server and change theType to Maintenance. Click OK.
Apply changes to the vDisk (version)
Make sure that the correct vDisk is linked to the XenApp “Master” server and start it.
Now that the vDisk version and the Master server are both in maintenance mode, a menu will appear at startup. Option 1 will start the server from vDisk version 1 in read/write mode. Option 2 will start the server from the base (read-only mode).
Press 1 and hit Enter
After the Master server is started, you can logon and apply the updates.
NOTE: Always reapply the steps that are needed to give certain software a unique ID or GUID as described in Step 6 AND always run the “Prepare this server for imaging and provisioning” wizard as described in Step 7 of this blog (link will be open in new window) before closing the vDisk / shutting down the master server.
After shutting down the master server, you see at file level that the *.1.avhd file has increased in size.
Test the new vDisk version (optional)
You have the ability to test the new vDisk version on different XenApp servers before promoting it to production. When promoting a vDisk version to test means also that you put it back to read-only mode, you can start multiple servers from a vDisk version in Test mode.
It is recommended to test the vDisk version on a server another then where you applied the updates. To promote the vDisk version to Test, do the following;
Open the Versions dialog of the vDisk and click on Promote. Select Test and click Ok.
Click Done
Open the properties of the Target Device you want to test de new vDisk version on and change the Type to Test. Click OK.
Now when starting the Target Device in Test mode, on startup there will be a menu appear. Option 1 will start the server from vDisk version 1 in read-only mode. Option 2 will start the server from the base.
Promote the vDisk version to production
When al tests are successful you can promote the vDisk to production (when not you can revert it back to maintenance and apply new changes). All servers assigned to this vDisk will boot the next time from the latest vDisk version with production state (until then they remain using the base version of the vDisk).
To promote the vDisk version to production, open the Versions dialog of the vDisk.
Select version 1 and click Promote.
Select Immediate or schedule the promotion for a later time. Click OK.
Click Done
If you have more than one Citrix Provisioning Services servers in your environment, don’t forget to copy the vDisk version to the other PVS servers. Check the vDisk replication status to see if all PVS servers are up-to-date.
vDisk version Merging
Each vDisk version depends on the underlying versions. For example, version 2 needs version 1 and the base to work well.
You can merge vDisk versions so that they are no longer dependent on underlying versions.
With Merging vDisk versions, you have two options;
– Merged Updates – All updates from last base
– Merged Base – Last base + all updates from that base
In this blog I will show you both options, starting with “Merged Updates”. For this examples I already created a second version so I can show you the difference better.
Open the vDisk Versions dialog and click Merge. Select Merged Updates and click OK
There will now be created a third version which contains both updates from version 1 and 2. This process takes a few minutes.
After a few minutes version 3 is ready.
When promoting version 3 to production, version 1 and 2 become obsolete and can be deleted safely. (Indicated by the red X)
To merge the base with all the updates, select the latest version and click Merge
Select Merged Base – Last base + all updates from that base and click OK
There will now be created a fourth version which contains the base and all updates. After a few minutes version 4 is ready.
When promoting version 4 to production, the base and version 3 become obsolete and can be deleted safely.
After deleting the base and version 3, only version 4 is left. This will be the new base. On file level you have now one big file again.
Share the post "How to update a XenApp 6.5 vDisk with Provisioning Services Versioning"
run the command below
# timedatect1 Set-ntp Yes



