Tag Archives: Ssl

What You Need to Know About the Heartbleed Bug

You may have heard that a new critical vulnerability has been identified that has affected many Internet Web servers – specifically those that use certain versions of “Open SSL” as a means of encrypting user sessions. We have inspected all VirtualQube.com Web sites, and verified that none of our sites have this vulnerability. However, it is possible that other Web sites you use on a regular basis are, or were, vulnerable. You can find a list of the top 1000 Web sites and their status (“vulnerable” / “not vulnerable”) as of roughly 12:00 UTC yesterday at https://github.com/musalbas/heartbleed-masstest/blob/master/top1000.txt. It is possible that many of the sites listed as “vulnerable” at the time have since patched their servers. However, if you have accounts on any of these sites – and the “vulnerable” list includes some high-profile sites such as yahoo.com, flickr.com, okcupid.com, slate.com, and eventbrite.com – you should immediately change your passwords.

There is also a useful tool available at http://filippo.io/Heartbleed/ that will allow you to check out a Web site if you are unsure whether or not it is vulnerable.

For the more technical in the crowd who are wondering how this vulnerability affects Web security, it allows an attacker to extract data from the memory of a Web server in up to 64K chunks. That may not sound like much, but if enough 64K chunks are extracted, useful information can be reconstructed, including username/password combinations, and even the private encryption key of the server itself. http://www.mysqlperformanceblog.com/2014/04/08/openssl-heartbleed-cve-2014-0160/ contains a list of the specific versions of OpenSSL that are vulnerable to this exploit.

First Look at Citrix Access Gateway 5.0

At the recent Synergy Berlin conference, Citrix announced Access Gateway 5.0. We have confirmed that, as of now, 5.0 is available for download from the Citrix download site - both as an update for the CAG 2010 hardware appliance, and in Access Gateway VPX (virtual appliance) format. (Note: you will need a “mycitrix” account to download the software.)

One of the things I really like about 5.0 is that it now supports running two 2010 appliances in an active/passive HA configuration with automatic failover. This was a serious shortcoming of the original CAG appliance.

In earlier versions, if you were using the Access Gateway as a general-purpose SSL VPN, you could configure HA of a sort within the Access Gateway client plug-in, by defining primary and secondary Access Gateways for the client to connect to. However, if you were simply running the Access Gateway in “CSG replacement” mode to connect to a XenApp farm without requiring your users to first establish an SSL/VPN connection, you had no ability to provide automatic failover unless you had some kind of network load balancing device in front of multiple Access Gateway appliances. That meant, of course, that to avoid having the load balancing device become a single point of failure, you had to have some kind of HA functionality there as well. By the time you were done, the price tag had climbed to a level that just didn’t make sense for some smaller deployments.

NOTE: This specifically applies to the 2010 appliance. The CAG Enterprise models, because they are built on the NetScaler hardware platform, have always supported operation as HA pairs with automatic failover. Of course, a CAG MPX 5500 also carries a $9,000 list price, compared to $3,500 for a CAG 2010.

Now, with the release of 5.0, you can purchase two 2010 appliances (which will cost you less than a single MPX 5500), and run them as an active/passive HA pair. Thank you very much, Citrix CAG team!

Here are a couple of videos from Citrix TV. The first deals with how to upgrade an existing CAG 2010 to the 5.0 software using a USB flash drive, and then set up the basic system parameters:

The second video shows how to configure a pair of appliances for active/passive failover:

You can access several other “how-to” videos by going to http://www.citrix.com/tv, and searching on “Access Gateway 5.0.”

Compelled Certificate Creation Attacks

Last October, we published a three-part series on SSL certificates: what they are, how they work, and how they’re used to secure transactions over the Web. You’ll find the series listed in our “Security” category. For most of us, this process has worked pretty well for a long time. But I recently ran across a paper by Christopher Soghoian and Sid Stamm that points out a vulnerability that, frankly, hadn’t really occurred to me before.

NOTE: I’ve chosen to place a copy of this paper on our own Web site, because I believe that the material is important enough that I wanted to ensure that it would be available even if the link I used to find it should no longer be valid. I believe that this is permissible under the Creative Commons Attribution license cited by the authors.

As we discussed in the previous series, the security of the public key infrastructure (“PKI”) that we’ve come to rely on ultimately depends on the trustworthiness of the Certificate Authorities (“CAs”) that grant the certificates. In general, a public CA (e.g., VeriSign) assumes some responsibility for verifying the identity of the person or organization requesting an SSL certificate. The level of verification performed depends on the type of certificate purchased. A small business purchasing a certificate that will be used to secure their Outlook Web Access site can get one pretty cheaply, and typically the issuer will only require that the requester be able to reply to an email message sent to the domain in question. On the other hand, Bank of America will go through a much more detailed process to get an “Extended Validation” certificate for one of their on-line banking servers (as well they should).

But if a bad guy could somehow obtain, from a trusted CA, a certificate for a Bank of America server, and then trick a user into visiting their fake BofA Web server, there would be no easy way for the user to know that something bad was going on - because the browser would indicate that a valid SSL session had been established.

Of course, any CA that knowingly issued such a certificate would risk irreparable harm to its reputation, punitive lawsuits, and potentially have its trusted status revoked by the major Web browser manufacturers. But, as Soghoian and Stamm point out in their paper, there are no technical restrictions that would prohibit a CA from doing so. So the integrity of the entire PKI and the security of millions of users’ communications ultimately depends on hundreds of CAs around the world choosing to do the right thing.

Now, I’m not particularly worried about VeriSign or GoDaddy, because I’m pretty sure they’re not going to cooperate in something like this without a court order (more on that later). But I didn’t realize that Microsoft, Apple, and Mozilla (Firefox) all include a number of national government CAs in their default “trusted root certification authorities” databases. For example, Microsoft’s program includes the governments of France, Korea, Latvia, Serbia, Tunisia, Turkey, and Uruguay, just to name a few. I’m sure that these government CAs are included for all the best reasons. But I’m not sure that I’m particularly comfortable with the idea of having my browser, by default, trust the government of Turkey with the blanket power to issue SSL certificates for any Web site. Correction - I’m sure that I’m not comfortable with that!

Why? Because the possibility is very real that some government, somewhere, might compel a CA to issue a false certificate that can then be used to perform a “man-in-the-middle” attack for surveillance purposes. In fact, as Soghoian and Stamm point out, there is evidence that this has already been done. (If you want the details on that, read their paper.)

As a result, they are working on a Firefox add-on that is currently known as “CertLock.” Certlock will keep track of the country of origin of the root CA of each Web site you visit, and if, on a return visit, it detects that the certificate being presented chains up to a root CA in a different country, even though your browser may trust that CA, it will warn you. For example, if your banking site uses certificates issued by VeriSign, which is a US-based CA, CertLock will store that information the first time you go to your banking site. If, on some future visit to that banking site, the Web server you hit presents a certificate that - although it appears to be valid - is chained to a root certificate issued by Etisalat in the United Arab Emirates, you’ll get a warning, and a chance to abort the connection.

Is this a perfect solution? No. Admittedly there are some scenarios that won’t be caught - but those are arguably not that significant anyway, with the possible exception of #4 below. To use a few of the examples cited by Soghoian & Stamm:

  1. Assume that the US government compels VeriSign to issue a certificate for use by a law enforcement agency wishing to intercept communications between a suspect located in the US and his/her US-based bank, which uses VeriSign certificates on all its Web servers. CertLock won’t detect that, because the CA issuing the fake certificate is the same CA that issued the legitimate certificates.

    However, if the government can get a court order compelling VeriSign’s cooperation, it could just as easily - and probably more easily - get a court order directly compelling the bank to disclose the suspect’s account information. So there’s little point in the exercise.

    The same holds true if the bank’s legitimate certificates were issued by, say, GoDaddy instead of VeriSign. They’re both US-based CAs, so CertLock won’t detect the attack - but, by the same reasoning, it’s still a moot point.

  2. Assume that a resident of China is accessing his/her online account with a Chinese bank that obtained its legitimate SSL certificates from VeriSign. Assume further that the Chinese government is interested in intercepting the suspect’s online transactions, and compels the China Internet Network Information Center (“CNNIC” - a domestic Chinese CA) to issue a false certificate for the operation.

    In this scenario, CertLock would detect the attack - although, again, it’s an improbable scenario because the Chinese government could just as easily compel the Chinese bank to provide the suspect’s account information.

  3. Assume that a US executive is on a business trip to China, and is attempting to access his/her gmail account from a hotel Internet connection. Once again, the Chinese government could compel CNNIC to issue a false certificate to employ a man-in-the-middle attack, since they have no leverage to compel the assistance of VeriSign, which issued the legitimate SSL certificates. This attack would be detected by CertLock.
  4. Assume that a Chinese executive is on a business trip in the US, and attempts to access his/her Chinese bank account from a hotel Internet connection. If the Chinese bank was using legitimate VeriSign SSL certificates, and if the US government obtained a false certificate from VeriSign, there would be no way for CertLock to detect the attack.
  5. Since American CAs dominate the certificate market, and are used by many foreign organizations, that last scenario is far from hypothetical, and would seem to give the US an edge in potential intelligence-gathering.

    So the bottom line is that the approach taken by CertLock is not perfect. But it’s a step in the right direction, and I’ll be downloading it as soon as I can get my hands on it. In the meantime, particularly if you’re interested in security issues or if your job includes security-related responsibilities, I’d heartily recommend that you download and read the entire paper. Although it’s a bit complex, it’s only 19 pages long, so it shouldn’t take you more than two cups of coffee to get through it.

Citrix Continues to Virtualize Appliances

Five or six years ago, when Citrix first announced the Citrix Access Gateway appliance, I remember scratching my head and thinking, “Wait a minute, Citrix is in the software business. Why in the world do they want to start selling hardware, with all of the warranty, repair, and support issues that come along with being a hardware manufacturer?” The answer, of course, was that in order to build out the complete Application Delivery solution they envisioned, they needed components that, at the time, couldn’t be delivered using software alone.

But the world turns, and time moves on, and today Citrix has a world-class virtualization platform that runs on off-the-shelf server hardware that is itself mind-bogglingly powerful compared to what was available five or six years ago. So it makes all the sense in the world for Citrix to turn all of those hardware devices into virtual appliances as quickly as they can.

Yesterday, they formally announced virtualized versions of both the Access Gateway and the Branch Repeater. We’ll get to the virtual Branch Repeater in another post, because we’ll have our hands full in this one just covering the things you need to know about the Access Gateway VPX.

First, you need to know that the Access Gateway VPX is fundamentally a virtualized version of the 2010 CAG Appliance - with some exceptions that we’ll get into in a moment. You can download it and use XenCenter to import it directly into your XenServer environment. The cost is only $995 (compared to $3,500 for the 2010 hardware appliance), with an ongoing Subscription Advantage cost of $129/year. Here’s where it gets cool:

  • It was difficult to come up with a good solution for redundancy and automatic failover with the 2010 appliance. Unless you wanted to put a load-balancer in front of it (and if you’re going to do that, you may as well just buy a NetScaler in the first place), redundancy depended on putting primary and secondary appliance URLs or IP addresses into the CAG client. And that did you no good at all if you were trying to run it in “CSG-replacement mode” just to provide secure Web access to a XenApp farm. But the VPX virtual appliance fully supports Live Motion, XenServer HA, and NIC bonding. So the VPX doesn’t have to be redundant, because the underlying XenServer infrastructure can provide the resilience you need.
  • If you were using a 2010 appliance, and wanted to use “SmartAccess,” you had to stand up a separate “Advanced Access Control” Web server in your protected network. Obviously, that added to the cost and complexity of the solution. The VPX appliance, on the other hand, supports SmartAccess policies directly.

    Edit July 27, 2010: Not sure now where I originally picked up this information, but it is incorrect. An Advanced Access Control Web server is still required to enable SmartAccess policies with the Access Gateway VPX.

NOTE: SmartAccess, in case you’re not familiar with the term, allows you to control, at a very granular level, what applications and information a user can access, and what they can do with that information, based on the access scenario. The same user, presenting the same authentication credentials, might get a totally different level of access depending on whether s/he is connecting from inside the corporate network, from outside the network using a company-owned laptop, from home using a personal PC, or from a hotel business center using a totally untrusted device. For more information on how SmartAccess works and why it’s cool, check out this video from Citrix TV:


  • The VPX appliance fully supports the latest generation of the Citrix Receiver, and works with Dazzle and the Merchandising Server.
  • You no longer need to buy VPN client licenses to run it in “CSG replacement” mode. This is a biggie. Citrix made it clear some time ago that they would not be putting any more development time and effort into enhancing the software “Citrix Secure Gateway.” But the CSG just wouldn’t die, for one simple reason: it’s free. If you own XenApp or XenDesktop licenses with current Subscription Advantage, you’ve got the rights to use the CSG software, and your only cost is a server to run it on…and that’s pretty low in today’s virtual world. Yes, it could be argued that the CAG appliance was somewhat more secure, since it ran on a hardened Linux-derived kernel. But it cost $3,500 plus roughly $100 per concurrent user. Hmmm… CSG, free, CAG appliance, several thousand dollars. That was an easy decision for a lot of users.

    Co-incident with the release of the VPX appliance, Citrix is announcing that they’re eliminating the Access Gateway Standard User Licenses. They will no longer be sold as of June 30. Instead, when you buy an Access Gateway (physical or virtual), you get a “platform license” that entitles you to use it to secure access to a XenApp or XenDesktop farm (i.e., what’s generally referred to as “CSG Replacement Mode”) at no additional charge. So now the equation is: CSG, free, but I’ve got to put it on a server, and if it’s a Windows Server, the OS is going to cost me $700 - $800 or so. CAG VPX, $995, but I import it directly into my XenServer infrastructure and don’t have to pay for anything else unless I want the advanced access functionality. Suddenly the value proposition looks a lot more attractive.

  • Speaking of the advanced access functionality, Citrix has made some licensing changes there as well. The Access Gateway Universal licensing model has been reduced from three tiers to two, and the prices have been lowered. So now, if you didn’t purchase the XenApp or XenDesktop Platinum Editions (which include Access Gateway Universal licenses), you can purchase the Access Gateway Universal licenses separately for $100/concurrent user in quantities up to 2,500, and $50/concurrent user for 2,500+ users.

What’s the down side? Well, I’m not sure there is one. The VPX appliance isn’t going to work well as a general-purpose SSL/VPN for thousands of concurrent users, but then neither did the 2010 hardware appliance. So if that’s what you need, or if you need the high-end enterprise features like Global Server Load Balancing to enable transparent failover to a Disaster Recovery site, then we need to have a conversation about NetScalers. But for basic CSG-like functionality, or a SmartAccess deployment for a few hundred concurrent users, the virtual appliance looks pretty darned attractive to me.

For more information on the Access Gateway VPX, including a demo of just how easy it is to import it into your XenServer environment and get it running, check out the following video from Citrix TV:

SSL and Certificates - Part 3 of 3

Part 1 and Part 2 of this series covered the basic cryptographic concepts behind SSL certificates, and looked at how an SSL certificate is constructed and how it is validated. This installment will discuss what different kinds of certificates exist, some things to watch out for, and two big takeaways that will save you time, money, and aggravation.

Traditionally, an SSL server certificate, such as the Wells Fargo Bank certificate that we discussed in Part 2, were issued for the Fully Qualified Domain Name (“FQDN”) of the server the certificate is intended to secure. The certificate we discussed was specifically issued to “www.wellsfargo.com.” This is called the “Common Name” (abbreviated as “CN”), and is specified in the “Subject” field of the certificate:

The Common Name Field


That means that if there were other DNS entries, such as “remote.wellsfargo.com” or “email.wellsfargo.com” that happened to resolve to the IP address of the same physical server, and you pointed your browser at one of those, you would get a certificate error – because the certificate was issued to “www.wellsfargo.com,” not to one of those other entries, and the browser won’t be happy unless the host name in the address bar exactly matches the Common Name listed in the Subject field.

In recent years, a couple of new kinds of certificates have been introduced. One is the Multiple Domain, or “UCC” (Unified Communications Certificate) certificate. [Note: Yes, I realize that saying “UCC certificate” is inherently redundant – like saying “PIN number.” If, by chance, my former English professor is reading this, I apologize. But I’m going to do it anyway.] A UCC certificate contains an extra field called the “Subject Alternative Names” field, which can be used to list multiple subdomains that the certificate can be used to secure. For example, a UCC certificate could be used to secure “remote.mooselogic.com,” “email.mooselogic.com,” “extranet.mooselogic.com,” and so forth, provided that all of those subdomains are explicitly listed in the Subject Alternative Names field. That means that you must specify what subdomains you want listed when you purchase the certificate, and if you want to add or delete one, the certificate must be regenerated by the issuer (which will generally cost you more money).

In addition to the Subject Alternative Names field, a UCC certificate still has a “Common Name” listed in the “Subject” field. However, according to the X.509 certificate standard, if the Subject Alternative Names field is present, the client browser is supposed to ignore the contents of the Common Name field (although not all of them do). Therefore, if the common name is “www.mooselogic.com,” but that common name is not repeated as one of the Subject Alternative Names, a browser that strictly adhered to the standard would end up with a certificate error if it tried to connect to “www.mooselogic.com.” This interaction between Common Name and Subject Alternative Names has some implications for mobile devices that we’ll come back to in a bit.

The other new kind of certificate is the “Wildcard” certificate. A Wildcard certificate could be issued for, say, “*.mooselogic.com,” and used to secure any and all first level subdomains. (E.g., email.mooselogic.com is a first level subdomain; email.seattle.mooselogic.com is not a first level subdomain, and could not be secured with a Wildcard certificate.) A Wildcard certificate does not contain a Subject Alternative Names field – instead, the Wildcard (“*.mooselogic.com”) is actually listed as the Common Name in the Subject field.

If you are running a browser that was released anytime since 2003, it should support Subject Alternative Names, and probably Wildcard certificates as well. In that case, your browser will be happy if one of the following three conditions is true:

  1. The host name in the address bar of the browser exactly matches the Common Name of the certificate. (Unless the cert is a UCC cert, in which case the browser is supposed to ignore the Common Name.)
  2. The Common Name is a Wildcard, and the host name in the address bar matches the Wildcard.
  3. The cert is a UCC cert, and the host name in the address bar exactly matches one of the names listed in the Subject Alternative Names field.

However, if you are using a browser on a mobile device of some kind, it’s a different story. Windows Mobile 6.x devices support both Subject Alternative Names and Wildcards. Windows Mobile 5 devices support Subject Alternative Names, but do not support Wildcards. If you’re not running a Windows Mobile 5 or 6 device, you’re going to have to check with the vendor of your mobile device. Some support both Subject Alternative Names and Wildcards, some only support one of them, some support neither.

So what? Well, if you’re trying to use a mobile device to synchronize e-mail with an Exchange Server, it’s usually done by pointing the device at the same URL that you’re using for Outlook Web Access (“OWA”). If you’re using a Wildcard certificate to secure your OWA site, and your mobile device doesn’t support Wildcards, you’re out of luck – it’s simply not going to work. However, if you’re using a UCC certificate to secure your OWA site, and the URL of the OWA site is also the Common Name of that UCC certificate, your mobile device will be happy even if it doesn’t support UCC certificates…because it will simply look in the Common Name field and find a match.

So here’s big takeaway #1: If you’re going to use a UCC certificate to secure multiple URLs, and one of those URLs happens to be the URL you’re going to use to synch email to mobile devices, make sure that URL is the Common Name of the certificate in addition to being listed as one of the Subject Alternative Names.

Another common “gotcha” involving SSL and mobile devices involves intermediate certificates. Remember that “chain of trust” discussion from the second post in this series? It is increasingly common to find that the certificate you have purchased to secure your Web site is not chained directly to the CA’s trusted root. Instead, there is at least one intermediate certificate in the chain between the trusted root and the certificate you purchased. This isn’t a problem for “big Windows,” because the browser is smart enough to sense that the certificate the server is presenting is not chained directly to the trusted root that it knows about, and to request the intermediate certificate(s) so it can validate the complete chain of trust. Mobile devices, including Windows Mobile devices, are not that smart.

Mobile devices depend on the server to present the entire certificate chain, including any intermediate certificates, at the time of connection. And the server won’t do that unless all of the intermediate certificates are present in that server’s own local computer certificate store. Installing the certificates into IIS for use in securing the OWA Web site does not automatically put them in the local computer certificate store – you must explicitly import them.

But why purchase a commercial certificate at all? Can’t you be your own Certificate Authority if you’re running a Windows Active Directory Domain? Yes, you can…if you don’t care about supporting connections from any PCs other than ones that have been joined to the domain, and you don’t care about supporting mobile devices. For example, when you set up a Windows Small Business Server, the wizard that configures that server for OWA automatically secures it with a self-issued certificate. That’s not a problem for any PC or laptop that has been joined to your SBS domain, because the very act of joining a computer to a domain inserts the domain’s own self-issued root certificate into the computer’s trusted root certificates store. But if you then try to connect from your home PC, or your mother-in-law’s PC, or any other PC that isn’t a member of your domain, you get a certificate error. At least with a PC, you have the opportunity to override the error and connect to the Web site anyway…but a mobile device will simply fail to connect, while typically giving you very little information about what the problem is.

You may or may not be able to manually import a certificate into the trusted root certificate store of your mobile device. Some mobile operators give their subscribers that level of “management access” to their mobile devices and some don’t. Some mobile operators provide special certificate installation utilities for their smart phones, some don’t. Sometimes there are workarounds, sometimes there aren’t. To our knowledge, there is no definitive list available of which mobile devices have their certificate stores locked down and which don’t. So the question is: How much is your time worth? The first time you (or we, on your behalf) spend a half day trying to make a mobile device work with an SSL certificate that wasn’t built into the phone, you will have spent more money – not to mention the time and aggravation – than it would have cost to go to a public CA and purchase a certificate that’s already supported.

So big takeaway #2 is: If you’re going to synch e-mail to mobile devices, do yourself a favor and decide in advance what mobile devices you’re going to support, then buy an SSL certificate from a public CA whose trusted root is already supported by those mobile devices. You’ll save money in the long run, and probably keep your blood pressure lower as well.

For more information on certificates and mobile devices, including a list of the trusted root certificates that ship with Windows Mobile 5 and Windows Mobile 6 devices, download the Moose Logic Technical Bulletin entitled Recommended Best Practices for Exchange Synchronization with Mobile Devices.