XenApp 6.5 HRP02 install hangs

Citrix recently released the eagerly anticipated HRP02 for XenApp 6.5, which contains around 87 hotfixes bundled into this one rollup.  You can download it here.

While installing this patch onto several servers in a farm, most of them worked fine however on one server the patch installation hung part way through (at around 90%) with no apparent errors or issues recorded in the event log.

HRP02_install_hang

 

Restarting the server and retrying the install with AV disabled, Citrix services stopped, third party services stopped had no effect.  Next step was to enable logging by installing the patch using the following command line:

C:\Temp> msiexec /p XA650W2K8R2X64R02.msp /L*V c:\temp\hrp02.log

Examining the resulting log file revealed the issue – this particular server had the XenApp install run from a UNC network share that no longer existed, and could not locate the original install media:

HRP02_logA quick registry search revealed this original UNC path of the install media was set in multiple places in the registry, for each of the individual components that make up a XenApp install (eg base XenApp install, Flash extensions, Delivery Services console etc all have individual installers).  After a bit of find and replace work, and a restart of the server, the patch install completed successfully!

Winning with StoreFront!

OK, so winning might be stretching it a little bit, but I’ve been spending a lot of time on StoreFront lately and I’ve gathered a few tips and figured out a few things to help your StoreFront deployment go a little smoother.  On top of this these tips will (hopefully) make your StoreFront deployment work a little faster, but I still can’t get it to operate as quick as Web Interface.

What I won’t cover is how to set it up, load balance it, or front it with Access Gateway/NetScaler as there are plenty of guides already out there for that.  Some examples are here, here, here, and more – just google it.

Access Gateway builds

You need to use a .e, or “Enhancement” build to work properly with StoreFront.  As at time of writing, the recommended version is 10.0.73.5002.e which you can download from Citrix here (login required).  In the future you will be able to use mainstream builds, however even with the initial 10.1 release of NetScaler this is not yet the case (AGEE feature is still in tech preview mode).

Certificates

Ensure you have the right certificate on your StoreFront server(s) before you start installing as StoreFront will hard code it’s URL based on the certificate installed and bound to IIS.  Note that even if you have an Access Gateway with a “real” trusted certificate your devices will still need to trust the internal certificate on the StoreFront servers when they are inside your network.  iPads have an option to ignore trust issues with the certificate, but Android and Windows devices do not.

.Net Config Files

Get familiar with the Microsoft .Net config file structure as you will end up editing many of them.  A lot of options, even some that were previously in the GUI with Web Interface, are now controlled by options in the .config files and must be edited by hand which unfortunately makes them more prone to human error.

Aside from configuration options though, an important setting particularly if your StoreFront servers do not have direct internet access is the the following line:

<generatePublisherEvidence enabled="false"/>

As per CTX117273 make the change to the .Net config files in both C:\Windows\Microsoft.NET\Framework and C:\Windows\Microsoft.NET\Framework64.  Then search for every .config file under C:\Program Files\Citrix\Receiver StoreFront and C:\inetpub\wwwroot and check each one.  If it has the following lines

<runtime>
 <!-- Set enabled=false, if this machine does not have external access to the internet -->
 <generatePublisherEvidence enabled="true" />
 </runtime>

then go ahead and change the “true” to “false”. This will disable the checking of the certificate used to sign the code, and part of this process does CRL lookups on the fly. If you server does not have internet access the whole process slows down waiting for the lookups to time out. Even if the server does have direct internet access, disabling all these checks seems to have a measurable improvement in the responsiveness of your StoreFront site.

Note that after you enable certain features, eg you enable the pass-through authentication method, a new folder will be created under StoreFront program files folder with an additional .config file so best to check for these regularly during the setup process and a final check when you are done configuring and installing all features.
In addition to these, you can disable the CRL check in the Internet control panel as per below.CRL Check

Miscellaneous settings

In the WINS tab under Advanced Settings on your network adapter, choose the “Disable NetBIOS over TCP/IP” option.

Change the Startup mode of the “Citrix Credential Wallet” service to “Automatic (Delayed Start)”.

Change PowerShell execution policy to “bypass” using the below command.  Similar to the CRL check, it disables checking for digitally signed scripts and associated CRL lookups.

Set-ExecutionPolicy -scope LocalMachine bypass

Set-ExecutionPolicy

Use DebugView.  It’s a handy tool that will tell you why something is failing when you get no feedback from the console or event viewer.  The logfiles it generates can be pretty large, but it’s worth combing through to see what’s going on.  You can download it from the Microsoft SysInternals page here.

Conclusion

I think that’s about it!  Not all these tips will have an effect in every environment, and please understand there is a security risk that comes with disabling CRL checks.  However I’ve had much greater success with StoreFront with making all the changes here so now they are part of my standard build procedure for new installs.

If you’ve found something that works for you and I haven’t mentioned it here, please leave a comment and I’ll update the post.

UPDATE for StoreFront 2.0

Fortunately for those installing StoreFront 2.0, Citrix have made life a little easier and in every .config file I have looked at so far (generated by StoreFront), the generatePublisherEvidence setting is already set to “false”.

Everything else I’ve written here still applies, including the generatePublisherEvidence setting in Aspnet.config files under the various C:\Windows\Microsoft.NET folders.  Note also a new KB article from Citrix on speeding up the first time login for StoreFront, CTX137400.

StoreFront WTF?

Citrix’s Web Interface replacement product, StoreFront, has certainly given me many clenched teeth moments recently (blog post coming on this soon).  However this WTF momemnt today made me chuckle…

Storefront_WTF

Technically it’s probably a Google Chrome issue as none of the other browsers I tried offered to translate the page, I guess I just found it amusing given some of the other trials I’ve had with it recently :)

XenServer 6.0.2 hotfix and driver disk install summary

Confused about the multitude of XenServer 6.0.2 hotfixes and associated drivers currently available? The public hotfixes and drivers are listed here and as of April 2013 totals 88!  It’s not as daunting as it looks though – there are currently 21 hotfixes, the remainder are the drivers associated with various versions of hotfixes.  And of the 21 hotfixes a number of the older ones are superceded and included with newer hotfixes, but there are some dependencies so they need to be installed in the correct order.

Firstly, download the required hotfixes.  As mentioned we don’t need all 21 – so download the following 5 hotfixes (that’s 16 less reboots to do!)

Hotfix 6 CTX134130
Hotfix 10 CTX135225
Hotfix 19 (includes hotfixes 1 and 2) CTX137134
Hotfix 20 (includes hotfixes 4, 8, 14, 16, 18) CTX136478
Hotfix 21 (includes hotfixes 1, 3, 5, 7, 11, 13) CTX136479

Hotfix 21 links to an article (CTX136621) listing the various upgraded drivers to go with it.  To work out the drivers you need, fire up a XenServer command line via the physical console, XenCenter or SSH.  Enter the “lsmod” command to return all running modules and drivers, or combine it with the egrep command to narrow the search down.  Eg the driver page shows the modules that require updating as per below graphic (this is not the complete list, cropped for brevity):

XS602E012-Drivers

So to search for these drivers, issue the command as follows:

[root@XenHost ~]# lsmod | egrep 'bnx2x|bnx2|tg3|cxgb3|cxgb4'

Any lines that get returned indicate the drivers that are installed and need updating.  If no lines get returned, you have no drivers that need updating :)

Now we are prepared, install the hotfixes in the following order:

1 – Hotfix 6
2 – Hotfix 10
3 – Hotfix 20
4 – Hotfix 19
5 – Hotfix 21
6 – Drivers (if required)

Time and enthusiasm permitting, I’ll attempt to keep this post updated as future hotfixes get released – happy patching!

Citrix Profile Manager and cookies

If you’ve implemented Citrix Profile Manager you will no doubt be aware of the policy options to manage cookies and the index.dat file (see here)

http://support.citrix.com/proddocs/topic/user-profile-manager-sou/upm-manage-cookies.html

What isn’t mentioned anywhere (that I’ve been able to find, anyway) is that if you are using Windows 7 / 2008 R2 and redirect the APPDATA folder using Windows group policy settings, then the Citrix policy settings for cookie processing are not required and do not take effect.  This is because the cookies folder resides under the APPDATA folder (%userprofile%\AppData\Roaming\Microsoft\Windows\Cookies) therefore if you redirect it, cookies will never be stored on the local machine for the Citrix profile manager to process.

For Windows XP / Server 2003, the cookie processing settings are still required as the Cookies folder is %userprofile%\Cookies.

IBM BladeCenter CEE Switch connecting to Cisco switch

While installing an IBM BladeCenter H that had a Brocade Converged Enhanced Ethernet Switch module (IBM part# 61Y1909, Brocade model 8470) I found pretty much zero documentation on how to connect it to an external Cisco switch.  So if you are trying to figure this out yourself, hopefully I can save you the trouble.

On the Cisco switch, I had 2 x 10Gb ports connected to the Brocade.  The config on those ports is as follows:

interface TenGigabitEthernet1/5
 description -- Connected to Brocade CEE SW01 --
 switchport trunk native vlan 111
 switchport mode trunk
 channel-protocol lacp
 channel-group 2 mode active
end
interface TenGigabitEthernet1/6
 description -- Connected to Brocade CEE SW01 --
 switchport trunk native vlan 111
 switchport mode trunk
 channel-protocol lacp
 channel-group 2 mode active
end

I wanted to use LACP to bond these two ports together, partly for throughput (which shouldn’t really be an issue 10Gb networking) and partly for redundancy – as you can see the ports are members of a channel group.  The port channel interface config is:

interface Port-channel2
 description -- Connected to Brocade CEE SW01 --
 switchport
 switchport trunk native vlan 111
 switchport mode trunk
end

Note the native vlan statement – I specifically wanted VLAN1 traffic passed to the Brocade, but the Brocade expects all VLANs to be tagged.  The work-around for this was to create a dummy VLAN (VLAN111) and make this the native vlan.  This then ensures VLAN1 will be tagged :)

Now for the Brocade side of the configuration.  First, the internal 10Gb ports will need to be configured as follows:

interface InTengigabitEthernet 0/1
 fcoeport
 switchport
 switchport mode trunk
 switchport trunk allowed vlan all
 no shutdown

The two external ports are configured as follows:

interface ExTengigabitEthernet 0/15
 channel-group 2 mode active type standard
 no shutdown
 lacp timeout long

interface ExTengigabitEthernet 0/16
 channel-group 2 mode active type standard
 no shutdown
 lacp timeout long

Note you will need to issue a “no switchport” command before you can configure the channel group.  Now for the port channel interface itself:

interface Port-channel 2
 switchport
 switchport mode trunk
 switchport trunk allowed vlan all
 no shutdown

While there may be a better way to do it, this is the configuration worked for me and with the lack of documentation on Cisco interoperability I wasn’t going to burn any more time on it.  I know you can set the Cisco switch to not have a default VLAN, like the Brocade, but this would have meant reworking a lot of the Cisco side of the network which I didn’t want to do.

 

PVS and the dreaded 0x0000007B error

A client was trying to setup a lab of HP T610 thin clients to stream Windows 7 from Citrix Provisioning Services 6.1.  The T610 they used to create the image worked flawlessly, they captured the image, and could then stream it back fine in private or standard mode.  The issue appeared when they went to stream it to more than one T610 – while the original machine worked fine, on every other T610 they tried it would blue-screen on startup with the dreaded 0x0000007B error.

“It’s probably something really simple” I said – famous last words – and set about checking for differences between the T610s in the BIOS settings, firmware versions and so on, deleting and recreating machines in the PVS console, recreating the AD accounts and more.  I was already familiar with the “ghosted network card” issue (CTX133188) but this was not the issue.  A couple of hours later and no “simple” fixes in sight – I eventually stumbled across this thread and on page 4 one of the posters references MS hotfix KB2550978.  I booted the VHD in Hyper-V, removed the PVS Target Device software, installed this hotfix and reinstalled the PVS Target device – problem solved!

The issue is the T610 clients support a feature called “PCI Express Serial Number Capability” so from PVS’s point of view, the network card in every individual T610 is seen as a new NIC and needs the driver installed before it can boot successfully.  But as it’s trying to boot across the network, this causes the 0x0000007B error.  Not many clients I have come across stream to physical desktops, so this probably won’t impact too many people, but I hope this info helps someone out!