This scenario generally can occur when upgrading an existing ESX/ESXi host to a newer version of ESX/ESXi server on VMware vSphere 4. VMware and ESXi server handles the networks differently within its OS which long story short treats the fact that if the management network and VMkernel networks are on teh same VLAN that it complains whereas in prior versions it ignored this condition. To rid yourself of this error you need to perform two things:

1. Set every ESX/ESXi server to the same exact Virtual Network Name. If you have VMware Distributed Switches this is an easy endeavor. Else, you have to manually set each host or run a script to do so.
2. On the VMware vSphere console, right-click the cluster and in the VMware HA settings go into the Advanced settings and add the following:
   
   Option: das.allowNetwork0
   Value: Service Console
   
   NOTE: The Value for your installation will be whatever you set the management console name is for your environment.
   
   
3. Sage your settings and right-click the host in questions and choose to reconfigure HA and the error should go away!

In order to upgrade an existing ESX/ESXi server to ESXi 4.1 server you need to either need to be at the local console or have remote access over an appropriate lights-out management card. For our purposes, we'll assume the latter is the case. To properly upgrade the system, you need to fulfill the prerequisite steps as well as perform the actual migration steps both as detailed below:

Prerequisites

1. Download the appropriate ISO image from VMware's website at http://www.vmware.com
2. Sign up for an account or have existing credentials to gain access to the downloads area
3. Lights-Out Management card access with DVD/CDROM capabilities OR
4. Lights-Out Management card access with a localized DVD/CDROM with the ESXi ISO burned and made bootable
5. Configure the BIOS on the server to boot the DVD/CDROM prior to the local disk

Install/Upgrade Procedures

1. Log into iLO/iDRAC of the server you are upgrading
   • Login into the remote system OR
   • In the case of a new install reboot the server
   • Ensure you have either a virtual ISO or physical CDROM in the server to boot from
   • The server should boot off the ISO/CDROM in which will load the boot image where you should get the welcome screen below:
     
     
   • Hit enter to continue on welcome
2. You should see the End User License Agreement screen as pictured below:
   • Accept the EULA by pressing F11
     
     
3. On the Select a Disk screen choose the local disk AND NOT the REMOTE disk
   • If there are no SAN disk currently hooked up to the system you will not see them in the view; only local disk
   • Use the arrow keys to move up and down through the selections if you need to
   • Once you have the target disk selected hit enter
     
     
   • Confirm by pressing enter on the selection you selected previously
     
     
4. The VMware installer will now ask you to confirm that you want to install with your chosen selections
   • To confirm and start the install press f11 on the Confirm Install window
     
     
5. The install will then format and install the necessary binaries and look like the screen below
   
   
6. On the Install Complete screen press enter to complete the installation and reboot
   
   
   
   

Post Installation Configuration

1. Once the server restarts and loads the ESXi software you will be at the welcome screen
   • To configure ESXi so you can get to it remotely press F2
     
     
   • This will bring up the login prompt
   • The default login for a new ESXi build is username: root and password: (blank)
   • Press enter to login whereby you will be dumped into the configuration screen
     
     
2. Once logged into the system there are a few configurations you should configure and are denoted below for each section
   
   
   • Configure Password
     1. Select the Change Password option and press enter
     2. Enter in your password and confirm your choice on the next line
     3. Once complete press enter to confirm and set the new password
        
        
   • Configure Management Network
     1. Hit enter to choose the Configure Management Network option
        
        
        • The Network Adapters option
          1. NOTE - You do not need to configure by default and is beyond the scope of this document. However, this will allow you to choose which interface is used for the management network should you have more than one NIC in the server.
             
             
        • IP Configuration option
          1. Choose the IP Configuration option
          2. In the IP Cnfiguration window enter in the following information:
             • IP Address
             • Subnet Mask
             • Default Gateway
          3. Once complete press enter to save the configuration
             
             
        • DNS Configuration option
          1. Choose the DNS Configuration option
          2. In the DNS Configuration window configure the following information:
             • Set the DNS Server information
             • Set the DNS hostname for the ESXi server
          3. Once complete press enter to save the configuration
             
             
        • Custom DNS Suffixes option
          1. Choose the Custom DNS Suffixes option
          2. In the Custom DNS Suffixes window configure the following information
             • Set the DNS Suffixes separates by a space for each entry
          3. Once complete press enter to save the configuration
             
             
     2. Once all network configuration is doe press ESX from the Configure Management Network screen and the system will prompt you to apply network settings and restart the network
        • Press Y to apply the settings and restart the network
     3. Test Management Network option
        • Choose the Test Management Network option
        • In the Test Management Network windows review the IP's and hostname to test and press enter to run the self-test
        • If all checks out OK the the system is correctly talking on the network and doing DNS resolution
        • Once complete press enter to return to the previous screen
          
          
   • Press ESC to logout of the configuration

There have been a couple instances where customers have complained about poor performance or high Memory utilization, after looking at the vSphere console we discovered Memory limits or CAPs limiting a servers Memory.

Here are the steps you can take to view if your Client's server has a memory limit
1. Login to vSphere in any data center including Cloud
(note Cloud is not exempt from being affected)
2. Right click on a VM Host to find, "Edit Settings..."

3. Navigate to the resource tab and select, "Memory"
Here you can see if there is a memory limit


If you need a limit removed check with the VM team

In order to perform enable SNMP from ESXi you need to ensure you load the VMware CLI tools onto the vSphere/vCenter box so that you can execute the commands listed below. Please note you can download these bunaries from VMware directly at the following URL:
http://www.vmware.com
You will need to register if you haven't used their site before to get at the download section.

---

In order to enable SNMP you will need the IP Address of the SNMP source you intent to allow traffic to. ESXi will only allow a single target (as tested at the time of this articles creation time) to be used for the SNMP configuration. To configure SNMP simply enter in the following commands:
If you have more than one server to perform enter in the following commands instead substituting the content in the parenthesis with the last octet of the servers you want to configure:
For more information on this subject, please reference the following link:
http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1008065&sliceId=1&docTypeID=DT_KB_1_1&dialogID=152643734&stateId=0%200%20152641980

By default there is no SSH capabilities enabled by ESXi server as the console is locked down by default. However, VMware conveniently allowed a limited backdoor that can be enabled via the Remote CLI or on the console of the box if you have access. You basically once you have logged into the service console enable the following options:

• Enable Local Tech Support
• Enable Remote Tech Support (SSH)
• Modify Tech Support Timeout

For more information, please reference the following link below:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1017910

1.) Create a local user on the server. In our example we will use user: mand1.
2.) Make the user a member of the local administrators group on the server.
3.) Login in with this user and customize the profile. (for example the start menu).
4.) Logoff and login again with an administrator account.
5.) Create a share on your file server. For our example we will use \\SRV-RDSDC-01\TSmandatory.
6.) For share permissions choose - Everyone - Full Control. For NTFS permissions choose - Authenticated Users - Read
7.) Turn off Caching on this share
8.) Copy the complete template folder from the C:\Users directory (C:\users\mand1 in our example) to the new TSmandatory share
9.) Rename the template folder from mand1 to TSmandatory.V2
Please be aware that you have to add the .V2 in the name of your folder, because it's the new profile type in Windows Server 2008 and 2008 R2.
10.) Delete the Local and LocalLow folders from the AppData folder
The next step is to add the right permissions on the mandatory profile
12.) Open REGEDIT and highlight the HKEY_USERS subfolder. Choose File -> Load Hive and load the NTUSER.DAT file from the profile. Give the key any name (eg: TSmandatory)
13.) Right-click on the TS Mandatory profile and choose permissions
14.) Delete the template user and add the Authenticated Users (Full Control)
15.) Unload the NTUSER.DAT from your registry. (File -> Unload Hive)
16.) Rename the NTUSER.DAT to NTUSER.MAN in the profile.
17.) You can configure the profile path under individual users on the Remote Desktop Services Profile tab. Simply enter \\SRV-RDSDC-01\TSmandatory\TSmandatory without the .V2.
Alternatively you can configure a GPO to specify the location of the Mandatory profile.

DNS configuration

1. For DNS this assumes that an Active Directory forest is installed and configured to support this installation. As such you will need to ensure that the DNS suffix is referenced accordingly for internal names versus what may be used externally.If you are stuck into a DNS split brain configuration, you MUST ensure the records match accordingly for the access levels you are providing for so that:
   • Internal DNS names resolve correctly for whatever is being configured
   • External DNS names referenced for the RD Web Server are accessible for external clients and are valid to the external facing IP Address used for the connection
2. Create round robin records that are A record matches to the RDS servers in the backend cluster
   • RD01P => tsfarm.companydnsname.com => 10.0.0.101
   • RD02P => tsfarm.companydnsname.com => 10.0.0.102
   • Additional Nodes => tsfarm.companydnsname.com => Next IP Address in range preferably sequential
3. Creation of individual A records for each server used in the setup (By default, DDNS should get this if enabled and the client is set to register on login)

NOTE - For this document we assume that we are doing split brain configurations as this leads to a more common configuration given most customers won't think about this beforehand and will use the same domain suffix for their AD installation.

SSL Certificate Configuration

1. For an RD Web Server configuration to work externally, all records that are being referenced for the initial Web Address connection must be valid
2. For the RD Gateway connection configured for the same which MUST be accessible externally the certificate used to validate the gateway connection MUST be valid if you are specifying a specific RD Gateway server to use for the RD connection; You can choose the "Automatically detect RD Gateway server settings" but should that not work you need to have a valid SSL certificate that the client can confirm else you will get an SSL error and won't be able to connect.
3. OPTIONAL: You can sign the RDP sessions in the RDS configuration but in order to prevent any references to download an untrusted root certificate you should procure server certificates from an external CA provider. Else, you can create steps to have the end user install the certificate local to their system to bypass that error or to check the box to ignore the same for subsequent sessions.

Active Directory Security Group Creation

1. Creation of Remote Desktop CAP Access group => Grants access for AD users and groups to the RDS farm; This SHOULD NOT have any computer objects in here
   • Remote Desktop CAP Access group added to the Remote Desktop Users local group on all WA and RD servers
   • This should include any user that is not defined as an administrator here for access to the RDP sessions on the system
2. Creation of Remote Desktop RAP Access group => Grants access for AD computers to the RDS farm; This SHOULD NOT have any users or groups in here
   • Remote Desktop RAP Access group added to the TS Web Computers local group on the local group for each WA server containing the TS Web Computers local group
   • This should include any computer that is configured as a session host and is allowed to access the RDP session of the server (namely any system acting as a gateway serve or RD Session Host backend)

Note: All AD security groups added here will be used for the RDS Gateway configuration in granting access to the front-end WA and back-end RD systems

Server Configuration

1. Install the following roles on all WA Server:
   • Remote Desktop Web Access
   • Remote Desktop Gateway
   • Remote Desktop Connection Broker
   • Remote Desktop Session Host
2. Install the following roles on all RDS servers:
   • - Remote Desktop Session Host
   • - Remote Desktop Connection Broker
3. Install the following role on the main AD server
   • - Remote Desktop Terminal Licensing => Install to directory other than C:\Windows
   • - Ensure the environment which should be in the same subnet to avoid firewall ports being blocked to it

WA Server Configuration (Detailed):

1. For the Authentication Method choose "Do not require Network Level Authentication"
2. For the Licensing Mode configure the option for "Per User"
3. For the User Groups add the following groups for the "Select User Groups Allowed to Access To This RD Session Host Server": Administrators (Default), Remote Desktop CAP Access group
4. For the Configure Client Experience accept the defaults
5. Install Self Signed certificate
6. For CAP and RAP Authorization policies configure the following:
   • RD Gateway User Groups => Administrators (Default), Remote Desktop CAP Access group
   • RD CAP Policy (TS_CAP_01) => Choose Password checkbox
   • RD RAP Policy (TS_RAP_01) => Choose "Allow users to connect to computers in the following group:" and select the Remote Desktop RAP Access group
7. Network Policy and Access Services => Accept the defaults
8. Web Server (IIS) => Accept the defaults

Tools that show installed once complete:

1. RemoteApp Manager
   • RD Session Host Server Tab
     • Server Name: gateway.companydnsname.com
     • RDP Port: 3389 (default)
     • Check the "Show a demote desktop connection to this RD Session Host server in RD Web Access"
     • Do not allow users to start unlisted programs on initial connection
     • Select the option "Do not allow users to start unlisted programs on initial connection
   • RD Gateway
     • Select "Automatically detect RD Gateway server settings"
     • OPTIONAL: You can configure a specific name here for load balancing but it REQUIRES a valid certificate on the name to resolve else the connection will not work; Default to the option above if this does not work as expected
   • Digital Signature
     • OPTIONAL: To help bypass the multiple security warnings displayed when connecting into these hosts, procure a valid SSL certificate from an externally trusted provider to avoid having to load a certificate per end user machine; Do not select this option if you plan on using a self-signed certificate
   • Custom RDP Settings
     • #*** Uncheck all checkboxes and choose the settings you want namely "Clipboard"
       #Remote Desktop Connection
   • Configure this by right-clicking the Remote Desktop Connection Manager icon and choosing properties
   • Connection Settings
     • Display Name: Enterprise Remote Access
     • Connection ID: gateway.companydnsname.com
   • RD Web Access
     • Server Name: Enter in server name of RD Web Access server namely "wa01p.companydnsname.com"
   • On the sub object called RemoteApp Sources configure the following:
     • RemoteApp Sources: enter in the machine that has the RD Web Access server role namely "wa01p.companydnsname.com"
2. RD Gateway Manager
   • Right-Click the WA server object and choose "Properties"
     • Server Farm
       • Add the RD gateway server farm member: gateway.companydnsname.com
   • Select the Connection Authorization Policies folder to create a CAP Access policy:
     • General Tab
       • Name the policy "TS_CAP_01"
       • Check the box to enable the policy
     • Requirements
       • Check the "Password" box; Uncheck the "Smart Card" box
       • User Group Membership
         • Add to the User group membership list: Remote Desktop CAP Access, BUILTIN\Administrators
         • Remote from the User group membership list: Domain Users
       • Client Computer Group Membership
         • Add to the list: Remote Desktop RAP Access
         • Remove from the list: Domain Users
       • Timeouts (Optional)
         • Enable Idle (120 minute) and Session (480 minute) timeouts, if appropriate
   • Select the Resource Authorization Policies folder to create a RAP Access policy:
     • General Tab
       • Name the policy "TS_RAP_01"
       • Check the box to enable the policy
     • User Groups
       • Add to the list: Remote Desktop RAP Access
       • Remove from the list: Domain Users
     • Network Resource
       • Add to the Select an Active Directory Domain Services network resource group: Remote Desktop RAP Access
3. RD Session Host Configuration => Configure licensing settings for RCAL's and the RD Connection Broker Settings
   • Licensing Tab:
     • Select "Per User"
     • Assign license server to appropriate resource by clicking on the Add button and selecting the correct server source
   • RD Connection Broker:
     • Click on Change Settings and Dedicated farm redirection
       • Choose Farm member and fill in the following:
         • RD Connection Broker server name: wa01p.companydnsname.com
         • Farm name: tsfarm.companydnsname.com
       • Once the above is configured click OK to go back to the previous screen and configure the following:
         • Select IP Address to be used for reconnection: Check the default IP Address
4. Remote Deskop Services Configuration => No configuration needed here as this just shows the active sessions on the local host
5. Web Server (IIS)
   • Go to the default.aspx file under the \RDWeb\pages\ folder and modify the setting "ShowDesktops" to "false"
6. In the Local Users and Groups section for:
   • Session Broker Computers add the following: Remote Desktop RAP Access group
   • TS Web Access Computers add the following: Remote Desktop RAP Access group
   • Remote Desktop Users add the following: Remote Desktop CAP Access group

==========================

RD Server Configuration (Detailed):

1. #* For the Authentication Method choose "Do not require Network Level Authentication"
   • For the Licensing Mode configure the option for "Per User"
   • For user groups to be able to access this RD Session Host server enter in: Administrators (Default), Remote Desktop CAP Access group
   • For Client Experience accept the defaults
   • Web Server (IIS) => Accept the defaults

Tools that show installed once complete:

1. RemoteApp Manager => Leave as is as this is not configured on the RD servers
2. Remote Desktop Connection Manager => Leave as is as this is not configured on the RD servers
3. RD Session Host Configuration => Configure licensing settings for RCALS and the RD Connection Broker Settings
   • Licensing Tab:
     • Select "Per User"
     • Assign license server to appropriate resource by clicking on the Add button and selecting the correct server source
   • RD Connection Broker:
     • Click on Change Settings and Farm Member
       • Choose Farm member and fill in the following:
         • RD Connection Broker server name: gateway.companydnsname.com
         • Farm name: tsfarm.companydnsname.com
       • Once the above is configured click OK to go back to the previous screen and configure the following:
         • Participate in Connection Broker Load-Balancing: Checked
         • Relative weight of this server in the farm: 100
         • Select IP Address to be used for reconnection: Check the default IP Address
4. Remote Desktop Services Manager => No configuration needed here as this just shows the active sessions on the local host
5. In the Local Users and Groups section for:
   • Session Broker Computers add the following: Remote Desktop RAP Access group
   • TS Web Access Computers add the following: Remote Desktop RAP Access group
   • Remote Desktop Users add the following: Remote Desktop CAP Access group

The following excerpt describes how to perform a SSL certificate conversion if you have a given format and need to move it over to another format. For example if you want to go from IIS to Apache or from P7B to .pem. The original link appears below if you wanted more information on the subject (as well as to give credit to the site owner).
Link to SSL Shopper Website:
https://www.sslshopper.com/ssl-converter.html

---

PEM Format

The PEM format is the most common format that Certificate Authorities issue certificates in. PEM certificates usually have extentions such as .pem, .crt, .cer, and .key. They are Base64 encoded ASCII files and contain "----BEGIN CERTIFICATE---" and "---END CERTIFICATE----" statements. Server certificates, intermediate certificates, and private keys can all be put into the PEM format.
Apache and other similar servers use PEM format certificates. Several PEM certificates, and even the private key, can be included in one file, one below the other, but most platforms, such as Apache, expect the certificates and private key to be in separate files.

DER Format

The DER format is simply a binary form of a certificate instead of the ASCII PEM format. It sometimes has a file extension of .der but it often has a file extension of .cer so the only way to tell the difference between a DER .cer file and a PEM .cer file is to open it in a text editor and look for the BEGIN/END statements. All types of certificates and private keys can be encoded in DER format. DER is typically used with Java platforms. The SSL Converter can only convert certificates to DER format. If you need to convert a private key to DER, please use the OpenSSL commands on this page.

PKCS#7/P7B Format

The PKCS#7 or P7B format is usually stored in Base64 ASCII format and has a file extention of .p7b or .p7c. P7B certificates contain "----BEGIN PKCS7---" and "---END PKCS7----" statements. A P7B file only contains certificates and chain certificates, not the private key. Several platforms support P7B files including Microsoft Windows and Java Tomcat.

PKCS#12/PFX Format

The PKCS#12 or PFX format is a binary format for storing the server certificate, any intermediate certificates, and the private key in one encryptable file. PFX files usually have extensions such as .pfx and .p12. PFX files are typically used on Windows machines to import and export certificates and private keys.
When converting a PFX file to PEM format, OpenSSL will put all the certificates and the private key into a single file. You will need to open the file in a text editor and copy each certificate and private key (including the BEGIN/END statments) to its own individual text file and save them as certificate.cer, CACert.cer, and privateKey.key respectively.

OpenSSL Commands to Convert SSL Certificates on Your Machine

It is highly recommended that you convert to and from .pfx files on your own machine using OpenSSL so you can keep the private key there. Use the following OpenSSL commands to convert SSL certificate to different formats on your own machine:

OpenSSL Convert PEM

OpenSSL Convert DER

OpenSSL Convert P7B

OpenSSL Convert PFX

If you need to convert a Java Keystore file to a different format, it usually easier to create a new private key and certificates but it is possible to convert a Java Keystore to PEM format.

SSL and Redirects

In OOD, you have two options for SSL: install the SSL Cert into the web server and configure appropriately or off-load the SSL encryption to the load balancers. This page presents methods to accomplish both.

SSL in Web Server

In this scenario, the SSL Cert is installed on the web server and the web server has full SSL capabilities (server-side SSL Certs for authentication, changing cipher suites, etc.)

SSL on Load Balancers

In this scenario, the SSL Cert has been installed on the load balancers. The load balancers will handle all SSL encryption/decryption and the web server will only see unencrypted traffic, usually on port 80.
One particular problem this poses is forcing redirects to HTTPS sites. Since the load balancers handle all HTTPS traffic, the web server will never see traffic on port 443 so you need to "force" the issue.

Configuration #1

Configure the load balancer to send HTTP traffic to one port and HTTPS traffic to another (after decrypting it, of course.) Apache will need to listen on both ports. Once configured, you can redirect your traffic appropriately. Here's an example configuration using Apache as the web server:

• The load balancer is configured to forward HTTP traffic to port 80 on our server and HTTPS traffic to port 81.
• Add two 'Listen' directives
  <code>
  Listen 80
  Listen 81
  </code>
• Set up NamedVirtualHosts:
  <code>
  NamedVirtualHost 10.10.10.10:80
  NamedVirtualHost 10.10.10.10:81
  </code>
• Set up a VirtualHost for each "site", configured appropriately:
  <code>
  <VirtualHost 10.10.10.10:80>

1. This is the HTTP "site"
   ServerName www.nothing.cop
2. Redirect all requests to the HTTPS site
   RedirectMatch .* https://www.nothing.cop
   </VirtualHost>
   <VirtualHost 10.10.10.10:81>
3. This is the HTTPS "site"
   ServerName www.nothing.cop
   DocumentRoot /var/www/html
   Options Indexes
   </VirtualHost>
   </code>

• Open this URL :https://www.secure128.com/signin.aspx

• Under Returning Customers, fill out Email and Password from above doc.

• In My Portal section, under Functions, click Order History.

• Under section Order Console Filters, check box called Domain, type your Domain name, click Search.

• Under Console Details section, click red button Details.

• Scroll down to middle of page for section Certificate Signing Request.

• Copy the entire contents of the CSR box and paste into CSR decoder tool: http://www.secure128.com/resource/csr-decoder.aspx.

• Click Validate your CSR.

• Review the output under Your Certificate Signing Request Information and send to customer to confirm.

These instructions were developed using IIS 6.0 on Windows 2003 Server SP1 and OpenSSL 0.9.5a.

1. Export your SSL Certificate from IIS:
   • Open 'Internet Information Services' from 'Administrative Tools' on your Start menu.
   • Right-click on your website and select 'Properties'.
   • Choose the 'Directory Security' tab and click on 'View Certificate'.
   • Choose the 'Details' tab, and click 'Copy To File'.
     • Click 'Next' and choose 'Yes, export the private key'.
     • Make sure 'Personal Information Exchange - PKCS#12 (.PFX) is selected.
     • Clear all checkboxes and click 'Next'.
     • Enter a password of at least 4 characters and click 'Next'. (Remember this - You will need it later!)
     • Enter the path to save the file to, and click 'Next'.
     • Click 'Finish' then 'OK'.
     • Close all dialog boxes, and 'Internet Information Services'.
2. Convert your PFX certificate to PEM
   • Download OpenSSL to the same folder you saved your certificate to. You can find it at:https://www.zoneedit.com/doc/partner/perl-utils/openssl-win32-binaries/openssl.exe
   • Start a Command Prompt and go to the folder with OpenSSL and your certificate.
   • Type 'openssl pkcs12 -in filename.pfx -out filename.pem -nodes' (without quotes).
   • Enter your password from step 1.d.iv at the prompt and press return.
   • You should see the message 'MAC verified OK'.
   • Exit the command prompt.
3. Create the .cer and .key files
   • Create two blank documents with the extensions .cer and .key.
   • Start 'WordPad' and open the .pem file you created in step 2.
   • Open the blank .cer and .key files in "Notepad".
   • Locate the section of the file that begins with '----BEGIN RSA PRIVATE KEY----' in 'WordPad'.
   • Copy that line and everything under it up to and including '----END RSA PRIVATE KEY----'
   • Paste that text into your .key file and save it.
   • Locate the section of the file that begins with '----BEGIN CERTIFICATE----' in 'WordPad'.
   • Copy that line and everything under it up to and including '----END CERTIFICATE----'.
   • Paste that text into your .cer file and save it.
   • Close 'WordPad' and both instances of 'Notepad'.

The only files you need to keep are the .key and .cer files that you have created. Rename the .cer file to .crt. The password you will be asked for when using the SSL Utility will be the same one you entered in step 1.d.iv.

Save the CA signed certificate to a text file with a .cer extension. For example: signed-certificate.cer. Include the Begin Certificate and End Certificate lines when you save the file. For example:
---BEGIN CERTIFICATE---
MIIFPDCCBCSgAwIBAgIQdc/+38qo3f3bZwozWEiUVjANBgkqhkiG9w0BAQUFADCB
vDELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
DTE0MDcwNzAwMDAwMFoXDTE1MDcwODIzNTk1OVowejAYELMAkGA1UEBhMCVVMxEzAR
---END CERTIFICATE---

1. Double-click on the new file that you created and a Certificate dialog box opens
2. Click on the Certification Path tab.
3. Look at the tree-like structure representing the full certificate chain. The top of the chain is referred to as the root Certificate Authority (CA). The bottom of the chain represents your server's certificate. If your server is not listed one-level below the root CA, then your certificate was issued by an intermediary CA. However, if your server is listed one-level below the root CA, then the certificate was issued by the root CA. For example, the following screen capture shows a certificate chain where an root CA, VeriSign, issued a certificate for support.vertafore.com

You must export the certificate from the chain; first let’s start with the Root Certificate.

1. Select the top of the chain, double-click the server's certificate (i.e. server.cer) file and a Certificate dialog box opens.

a.Click Certification Path tab.
b.Highlight an entry of the certificate chain.

c.Click View Certificate.
d.In the Certificate dialog window, click the Details tab.
e.Click Copy to File...
f.In the Certificate Export Wizard that appears, click Next.
g.Select Base-64 encoded X.509 (.CER), and click Next.
h.Type in a unique name for the certificate you are exporting and click Next.
For example, "VS-root-CA".
i.Click Finish.
j.Click OK in the dialog box that displays the following message:  The export was successful.
k.Repeat the preceding sub steps for each intermediate certificate in the chain. Note that there is no need to repeat these steps for the bottom entry of the chain because the server’s certificate already exists.
Intermediate Certificate
  2.  Select the next entry in the chain, double-click the server's certificate (i.e. server.cer) file and a Certificate dialog box opens.
     a)   Click Certification Path tab.
     b)   Highlight an entry of the certificate chain.

 c)  Click View Certificate.
d)  In the Certificate dialog window, click the Details tab.
e)  Click Copy to File...
f)   In the Certificate Export Wizard that appears, click Next.
g)  Select Base-64 encoded X.509 (.CER), and click Next.
h)  Type in a unique name for the certificate you are exporting and click Next. For example, " VS-intermediary-CA".
i)   Click Finish.
j)   Click OK in the dialog box that displays the following message: The export was successful.
When you are done, you will have a certificate file (.cer) for each entry of the chain. In our example, there are three certificate files:

Certificate type	Name	Certificate file name
Root	VeriSign	VS-root-CA.cer
Intermediary	VeriSign Class 3 International Server CA – G3	VS-intermediary-CA.cer
Server	support.vertafore.com	Support_vertafore.cer

What is DKIM?

DomainKeys Identified Mail (DKIM) is a method for associating a domain name to an email message, thereby allowing a person, role, or organization to claim some responsibility for the message. The association is set up by means of a digital signature which can be validated by recipients. Responsibility is claimed by a signer by adding a DKIM-Signature: field to the message's header. The verifier recovers the signer's public key using the DNS, and then verifies that the signature matches the actual message's content.

Enabling DKIM in Sendmail

• Generate a private key

A "default.private" key file will be generated. It will be moved to a specific location later.

•   Generate a public key for this private key

A file with filename "default.public" will be generated with content like
---BEGIN PUBLIC KEY---
...
---END PUBLIC KEY---
It will be used to create a DNS TXT record. See next step.

• Create a DNS record of type TXT

Modify DNS records and add a record of type TXT:
Note that the prefix "---BEGIN PUBLIC KEY--" and suffix "--END PUBLIC KEY---" should not be put in the TXT record value. This DNS record will be retrieved by mail receivers who want to verify emails with DKIM signatures. The record name "default._domainkey" tells verifier that the "selector" of this signature is "default", therefore if you are changing selector name to something else, make sure you change all of them consistently.

• Install dkim-milter in Linux

Run the following as root to install the dkim-milter pacakge.

• Enable dkim-milter to run on start-up

Make sure dkim-milter service will run on start-up by running this command:

• Move private key to appropriate location

As root, copy the private key to the location specified by the "keylist" (refer to next step) and make sure it is readable by dkim-milter:
Make sure the filename of private key file matches the "selector" name specified in the DNS record.

• Add an entry to the keylist for dkim-milter to read

Add the following line to /etc/mail/dkim-milter/keys/keylist. Replace <domain.com> with your domain name.

• Configure dkim-milter

Open configuration file /etc/mail/dkim-milter/dkim-filter.conf and use the following configuration:
/etc/mail/dkim-milter/internalhosts is needed if you are signing e-mails on a gateway mail server
/etc/mail/dkim-milter/peerlist can have list of hosts whose connections should be accepted without processing by dkim filter.

• Add dkim to sendmail.mc dd to sendmail.mc

  INPUT_MAIL_FILTER(`dkim-filter', `S=inet:8891@localhost')

  You might need to adjust socket definitions in /etc/init.d/dkim-milter as well.

• Restart dkim milter and sendmail

Now check emails logs to see if dkim inserts are happening.

When submitting mail by using sendmail as a mail submission program, sendmail copies all messages to "/var/spool/clientmqueue" first. Later, another sendmail process, the sendmail mail transfer agent (MTA) copies the messages from /var/spool/clientmqueue to /var/spool/mqueue and sends them to their destination.
/var/spool/clientmqueue is thus the holding area used by the MSP (Mail Submission Protocol) sendmail instance before it injects the messages into the main MTA (Mail Transport Agent) sendmail instance. Sendmail will save the message in /var/spool/clientmqueue for safe keeping before trying to connect to the MTA to get the message delivered.
When files accumulate in /var/spool/clientmqueue, this is probably due to sendmail localhost MTA not running, and thus the mails don't get send.
Basic mailq commands:

• Following command show the mail submission queue ( note that this is not the amount mail in queue but waiting to be processed by queue runner)

•  Following command shows total mails in queue.

Following command shows total lost emails (to understand what lost emails are, please continue to read this article)

Note : One may alternatively use "sendmail -bp"  command that will also display mails in queue.

Parts of message queue

Files stored in the queue directory (/var/spool/mqueue) are identified by the first two letters.
df - data file -  Indicates data files or content of queued message.
qf - queue control file - Indicates headers of queued message
Qf  - bogus qf file - Indicates headers of permanently queued message or lost message.
Another two not very important (from a user perspective) parts of message are
tf - temporary qf rewrite image -  When processing a queued message, it is often necessary for sendmail to modify the contents of the qf file. This usually occurs if delivery has failed or if delivery for only a part of the recipient list succeeded. In either event, at least the message priority needs to be incremented.To prevent damage to the original qf file, sendmail makes changes to a temporary copy of that file. The temporary copy has the same queue identifier as the original, but its name begins with a t.
xf - transcript file - While calling the necessary delivery agents, sendmailsaves all the error messages it receives in a temporary file. The name of that temporary file begins with the letters xf. After all delivery agents have been called, sendmail returns any collected error messages to the sender and deletes the temporary xf file.

Understanding qf in detail

The qf file contains the message header. In addition to the header, the qf file also contains all the information necessary to:

•  It contains the sender's address and a list of recipient addresses.

S<root@10-163-120-13.cloud.opsource.net>

•  It contains a priority that determines the current message's position in a queue run of many messages.

P7140439

•  It contains the date that the message was originally queued. That date is used to time out a message.

 Fri, 21 Jan 2011 00:35:01 -0500

•  It contains the reason that the message is in the queue and possibly the error that caused it to be queued.

MDeferred: Connection timed out with alt4.gmail-smtp-in.l.google.com.

What are lost emails

Sendmail renames a qf file to Qf if there is a problem that it cannot address it by merely deferring or rejecting it, its often called lost emails. An example, would be like we have a email address no-reply@domain.com which has sent a email to a valid user but its not able to reach the receiving server, so it keeps deferring it until 5 days when it permanently rejected and sendmail tries to notify sender no-reply@domain.com that it cannot deliver but it simply fails to notify as  no-reply@domain.com isn't a valid address. Since it cannot notify, it just gives error:
savemail panics occur when sendmail is unable to deliver a bounced message and keep it as Qf file in mqueue directory.
Most of these Qf fails can be deleted, but at times there can be genuine email that could have failed to deliver. Though sendmail would not rename it back from Qf to qf to deliver, with human intervention, one can rename the failed Qf message to qf and retry it to see if it delivers.
There are also some df files that may not have any associated qf or Qf file which can be deleted as well (very few or negligible).

Flushing Sendmail queue or running queuerunner

Following command will try to forcefully run the mailq:

When sending emails through sendmail - whether it's a standalone server, or a relay server, the system sometimes wont expose itself to the outside world as being from a valid FQDN. Actually, the default for sendmail is to expose itself based upon best guess, which typically is by hostname. Because hostnames are essential to our environment (the ood.ops name) - I strongly suggest against changing the systems hostname to fix this problem. Because of this, sendmail servers will expose themselves to the outside world as <hostname>.ood.ops. I've seen twice now where this has also led to email servers as being identified as spam sources because the 'helo' isn't a valid FQDN. Please follow these steps to test this.

1. send yourself an email from the server itself. Make sure the 'mailx' rpm is installed (for hybrid customers, this is not installed by default on cloud servers)
rpm -qa | grep mailx
sudo yum install mailx

# mail <username>@Latheef.net
Subject: <type in whatever here>
<this is the data field, type whatever you please>
<enter a '.' (period or dot) then <enter> to end the email>
CC: <hit enter - no need to CC anyone>

When you receive the email in gmail, you need to view all of the headers. To get the headers you will need to:

1. open the email
2. in the drop down next to the 'reply' box, select that and 'show original'

Once you see the headers, you want to look at the last 'Received:' line before it hits the google mailservers. Keep in mind, your 'Source' is the bottom most 'Received' line, and your destination is the top most 'Received:' line. So you would read the headers from the bottom up, and not top down. An example would be:
Received-SPF: softfail (google.com: domain of transitioning gwright@jobvite.com does not designate 65.200.35.206 as permitted sender) client-ip=65.200.35.206;
Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning gwright@jobvite.com does not designate 65.200.35.206 as permitted sender) smtp.mail=gwright@jobvite.com
Received: from localhost.localdomain ([65.200.35.206]) (using TLSv1) by na3sys009amx188.postini.com ([74.125.148.10]) with SMTP;
    Thu, 08 Sep 2011 21:55:02 GMT
Received: from localhost.localdomain (localhost [127.0.0.1])
    by localhost.localdomain (8.14.4/8.14.4) with ESMTP id p88Lt04i001960
    for <gwright@Latheef.net>; Thu, 8 Sep 2011 14:55:00 -0700
Received: (from gwright@localhost)
    by localhost.localdomain (8.14.4/8.14.4/Submit) id p88Lt0PZ001959
    for gwright@Latheef.net; Thu, 8 Sep 2011 14:55:00 -0700

The important line here is:

Received: from localhost.localdomain (65.200.35.206)

As you can see from this, currently my desktop VM is 'helo'ing to the world, or exposing itself as 'localhost.localdomain'. This doesn't match my current hostname on my VM, but that doesn't matter. I assume it's getting localhost.localdomain from my /etc/hosts file. Without modifying the /etc/hosts file, or my servers hostname, there is a way to fix this in sendmail. In sendmail, you just need to open the sendmail.mc file and add the following macro line:
define(`confDOMAIN_NAME', `relay.host.com')

You will then need to rebuild the config and restart sendmail:
sudo make -C /etc/mail
sudo /sbin/service sendmail restart

You will then need to follow the above procedures to send yourself another test message and once again inspect your headers. You should see successful results like the following:
Received-SPF: softfail (google.com: domain of transitioning gwright@jobvite.com does not designate 65.200.35.206 as permitted sender) client-ip=65.200.35.206;
Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning gwright@jobvite.com does not designate 65.200.35.206 as permitted sender) smtp.mail=gwright@jobvite.com
Received: from relay.host.com ([65.200.35.206]) (using TLSv1) by na3sys009amx241.postini.com ([74.125.148.10]) with SMTP;
    Thu, 08 Sep 2011 21:57:38 GMT
Received: from localhost.localdomain (localhost [127.0.0.1])
    by relay.host.com (8.14.4/8.14.4) with ESMTP id p88LvaRv002029
    for <gwright@Latheef.net>; Thu, 8 Sep 2011 14:57:36 -0700
Received: (from gwright@localhost)
    by localhost.localdomain (8.14.4/8.14.4/Submit) id p88LvaBJ002028
    for gwright@Latheef.net; Thu, 8 Sep 2011 14:57:36 -0700

From the above, the important line is again, the one right before the google/gmail 'Received:' line.

Received: from relay.host.com (65.200.35.206)

Congrats. Now you've successfully told sendmail "I know you don't know what my public hostname is, so please use this provided one instead of guessing something that's incorrect for my purpose."

When sending emails through sendmail - whether it's a standalone server, or a relay server, the system sometimes wont expose itself to the outside world as being from a valid FQDN. Actually, the default for sendmail is to expose itself based upon best guess, which typically is by hostname. Because hostnames are essential to our environment (the ood.ops name) - I strongly suggest against changing the systems hostname to fix this problem. Because of this, sendmail servers will expose themselves to the outside world as <hostname>.ood.ops. I've seen twice now where this has also led to email servers as being identified as spam sources because the 'helo' isn't a valid FQDN. Please follow these steps to test this.

1. send yourself an email from the server itself. Make sure the 'mailx' rpm is installed (for hybrid customers, this is not installed by default on cloud servers)
rpm -qa | grep mailx
sudo yum install mailx

# mail <username>@Latheef.net
Subject: <type in whatever here>
<this is the data field, type whatever you please>
<enter a '.' (period or dot) then <enter> to end the email>
CC: <hit enter - no need to CC anyone>

When you receive the email in gmail, you need to view all of the headers. To get the headers you will need to:

1. open the email
2. in the drop down next to the 'reply' box, select that and 'show original'

Once you see the headers, you want to look at the last 'Received:' line before it hits the google mailservers. Keep in mind, your 'Source' is the bottom most 'Received' line, and your destination is the top most 'Received:' line. So you would read the headers from the bottom up, and not top down. An example would be:
Received-SPF: softfail (google.com: domain of transitioning gwright@jobvite.com does not designate 65.200.35.206 as permitted sender) client-ip=65.200.35.206;
Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning gwright@jobvite.com does not designate 65.200.35.206 as permitted sender) smtp.mail=gwright@jobvite.com
Received: from localhost.localdomain ([65.200.35.206]) (using TLSv1) by na3sys009amx188.postini.com ([74.125.148.10]) with SMTP;
    Thu, 08 Sep 2011 21:55:02 GMT
Received: from localhost.localdomain (localhost [127.0.0.1])
    by localhost.localdomain (8.14.4/8.14.4) with ESMTP id p88Lt04i001960
    for <gwright@Latheef.net>; Thu, 8 Sep 2011 14:55:00 -0700
Received: (from gwright@localhost)
    by localhost.localdomain (8.14.4/8.14.4/Submit) id p88Lt0PZ001959
    for gwright@Latheef.net; Thu, 8 Sep 2011 14:55:00 -0700

The important line here is:

Received: from localhost.localdomain (65.200.35.206)

As you can see from this, currently my desktop VM is 'helo'ing to the world, or exposing itself as 'localhost.localdomain'. This doesn't match my current hostname on my VM, but that doesn't matter. I assume it's getting localhost.localdomain from my /etc/hosts file. Without modifying the /etc/hosts file, or my servers hostname, there is a way to fix this in sendmail. In sendmail, you just need to open the sendmail.mc file and add the following macro line:
define(`confDOMAIN_NAME', `relay.host.com')

You will then need to rebuild the config and restart sendmail:
sudo make -C /etc/mail
sudo /sbin/service sendmail restart

You will then need to follow the above procedures to send yourself another test message and once again inspect your headers. You should see successful results like the following:
Received-SPF: softfail (google.com: domain of transitioning gwright@jobvite.com does not designate 65.200.35.206 as permitted sender) client-ip=65.200.35.206;
Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning gwright@jobvite.com does not designate 65.200.35.206 as permitted sender) smtp.mail=gwright@jobvite.com
Received: from relay.host.com ([65.200.35.206]) (using TLSv1) by na3sys009amx241.postini.com ([74.125.148.10]) with SMTP;
    Thu, 08 Sep 2011 21:57:38 GMT
Received: from localhost.localdomain (localhost [127.0.0.1])
    by relay.host.com (8.14.4/8.14.4) with ESMTP id p88LvaRv002029
    for <gwright@Latheef.net>; Thu, 8 Sep 2011 14:57:36 -0700
Received: (from gwright@localhost)
    by localhost.localdomain (8.14.4/8.14.4/Submit) id p88LvaBJ002028
    for gwright@Latheef.net; Thu, 8 Sep 2011 14:57:36 -0700

From the above, the important line is again, the one right before the google/gmail 'Received:' line.

Received: from relay.host.com (65.200.35.206)

Congrats. Now you've successfully told sendmail "I know you don't know what my public hostname is, so please use this provided one instead of guessing something that's incorrect for my purpose."

Installing and running sendmail

Sendmail distribution consists of 3 rpm packages:
sendmail - Truly necessary to send and receive email
sendmail-cf -- Includes configuration macros and other  files that can help reconfigure sendmail installation
sendmail-doc -- contains documentation file

•  Installing using yum

yum install sendmail
yum install sendmail-cf

• Installing using rpm

rpm --Uhv sendmail*
Starting sendmail  # /etc/init.d/sendmail start

Basic Sendmail Configuration

Most of sendmail's configuration parameters are defined in the /etc/mail/sendmail.mc file, which is then used by the m4 macros to create the /etc/mail/sendmail.cf file.
Configuration of the sendmail.mc file is much simpler than configuration of sendmail.cf, but still you may find little intimidating. Remember, few of opsource customers might directly edit sendmail.cf, so editing sendmail.mc and rebuilding the config may overwrite their settings. It is always advisable to take backup of both sendmail.mc and sendmail.cf depending upon what you edit.
Below we will cover some basic lines in sendmail.mc that are considered important or edited mostly by linux admins:

• The first few lines of sendmail.mc do some housekeeping. The divert line remove extra output when configuration file is generated. OSTYPE must be set to linux to get the proper location of files needed.

Lines that begin with dnl (delete to new line) are comment lines.

• Below lines define logging level. 

Though you see them commented above, 9 is the default logging level by default. You may find uncommented "O LogLevel=9" in sendmail.cf instead. Logs are written to /var/log/maillog by default as defined in /etc/syslog.conf

•  Below is the line for smarthost.  Smart hosts are usually used when all other methods of delivery have failed. It would be perfectly reasonable to have the hosts attempt to deliver mail directly first (local server), and if that fails then to send it to the smart host.

NOTE:  Valid entries for SMART_HOST is a FQDN like 'relay.customer.com' or IPs. Because the sendmail Docs don't make this clear, if you are trying to add an IP as a SMART_HOST it must be enclosed in '[]' (square brackets). So if you are trying to setup a relay for customer and don't want to create an FQDN to a private internal IP with an /etc/hosts entry, then an IP is valid if and only if you configure it like - define(`SMART_HOST', `[10.xxx.xxx.xxx]')

• Below 3 lines set locations for programs that distribute incoming mail (procmail, by default), mail aliases file, mail statistics file.

• Below 2 lines are are related to how mail queue should be handled. By default, sendmail will send a warning email after four hours that email has been deferred. After 5 days, it sends a bounce back message that mail delivery to intended recipient permanently failed. Remember, sendmail retries every hour to keep on sending a failed message every hour until 5 days after which the message is permanently discarded.

•  FEATURE macro is used to set sendmail special features. virtusertable options sets the location of virtusertable database.

• DAEMON_OPTIONS is commented below. Therefore incoming mail from internet is allowed as well.

• If you want your mail to appear to come from user@opsource.net and not user@thishost.ood.ops or user@localhost, then you have to consider setting up masquerading (of course, there is an option in email clients to change the @ address). masquerade modifies the domain name of all traffic originating from and passing through your mail server except from root user. Below lines are self-explanatory.

Wondering why root user can't be masqueraded, well you have to remove/comment following line and rebuild the sendmail configuration if you do not wish root to be exposed.

• Procmail is mailer by default, below configuration defines cyrus-imap to be used as imap server.

This completes a basic overview of sendmail.mc file
If you would need to edit sendmail.mc and rebuild the sendmail.cf, use m4 macro to regenerate the new configuration:

Defining Outgoing Mail Access

Every time an email message intended for outgoing mail is received by your sendmail server, the server needs to decide if it will accept or reject relaying of that message.
Example entry for /etc/mail/access
Basically, there are 4 actions that you have the server take on a match:
RELAY - It will simply send the message on to the mail server requested in the mail message.
REJECT - Message is rejected and sender is informed that it was rejected.
DISCARD - Message is silently discarded and sender is not informed.
ERROR - One can add some informative text here, customize and let the sender know why it was not relayed.
After any changes to /etc/mail/access, it must be rebuilded (or consider restarting sendmail)

Rebuilding /etc/mail/access

Note: You can also use  /etc/mail/relay-domains to set up relay. Use FQDN  only in this file. One disadvantage of this file is that controls mail based on the source domain only, and source domains can be spoofed by spam e-mail servers. The /etc/mail/access file has more capabilities, such as restricting relaying by IP address or network range and is more commonly used. If you delete /etc/mail/relay-domains, then relay access is fully determined by the /etc/mail/access file.

Configuring Virtual Users

 /etc/mail/virtusertable

The /etc/mail/virtusertable file contains a set of simple instructions on how to treat received email. The first column lists the target email address and the second column lists the local user's mail box, a remote email address, or a mailing list entry in the /etc/aliases file to which the email should be forwarded.
If there is no match in the virtusertable file, sendmail checks for the full email address in the /etc/aliases file.
Example entry for /etc/mail/virtusertable
After adding the required entries in virtusertable (use any text editor such as vi to edit /etc/mail/virtusertable), it must be rebuilded (or restart sendmail):
Rebuilding virtuserable:

/etc/aliases file

A more flexible method of handling mail delivery (sytem wide rather than being specific to one particular user) involves the /etc/aliases file. One can think of the /etc/aliases file as a mailing list file. The first column has the mailing list name (sometimes called a virtual mailbox), and the second column has the members of the mailing list separated by commas.
In the above example, there are lot of administrative users (bin, daemon, adm, lp etc.) instead of having separate mailboxes for each, they are redirected to root's mail box.  You can also use this file to create a mini mailing list, the above example has an entry for family which is forwarded to 3 addresses. Mails can also sent to be file as well, in the above example there is an entry for trouble-ticket which  stores incoming tickets in file /var/spool/trouble/incoming.
If you edit this file, use the following command to rebuild aliases (or restart sendmail)