Archive for the ‘Problem Solution’ Category

Slow Performance with Vyatta on VMware ESXi

Wednesday, July 13th, 2011

I use a pair of Vyatta VM’s to route between VLAN’s at home.  One is running on ESXi, the other on VMware Server 2 (yeah, I really want to move that to ESXi, haven’t had time to move all of the services I’m running on the host onto VM’s).  They are redundant, I use VRRP on the user and server facing VLAN’s and OSPF on the VLAN facing my Cisco 1720 that’s connected to the Internet.  The VM running on ESXi is prioritized in OSPF and VRRP to make lifer simpler and troubleshooting easier.

Drawing2

I recently noticed how slow the performance was when accessing FTP and HTTP for one particular host.  From outside, a FTP session to this particular host would average maybe 10 KB/s.  One moderate size image hosted via HTTP on that host basically would not load because the connection was going so slow (1 KB / s in this case).  Off and on, I noticed slow performance connecting to other hosts.  For a while, I focused my troubleshooting on the host exhibiting most of the performance issues.  Oddly enough though, when I failed everything over to the other Vyatta VM running on VMware server, the performance was much better.  I was able to receive 40 – 50 KB / s with that FTP session.  Connecting the host directly to the VLAN between my Cisco router and the Vyatta VM’s also resulted in much improved performance.  Nothing obvious jumped out from tcpdump’s or logs on the Vyatta VM’s.

I started looking at what inconsistencies there were between the two VM’s, and between the interfaces on each VM.  These VM’s originally were made from Vyatta’s OVF template.  I must have added one additional NIC to each of them.  Two of the NIC’s on the ESXi Vyatta instances were VMXNET, and the third was E1000.  Eventually, after troubleshooting this issue for quite a while, I found this particular topic on the Vyatta forums.  When I changed the two VMXNET interfaces to VMXNET3, all of the performance issues on the ESXi Vyatta instance went away.  (The Vyatta instance running on VMware server still has the NIC’s on VMXNET and doesn’t appear to suffer performance-wise from it).  From other posts in that topic, it sounds like this may be related to a issue with the open-vm-tools included in past versions of Vyatta, that hadn’t been fixed in the OVF yet.

Another post on the Vyatta forum linked to a VMware KB article.  The KB article describes performance issues on Linux guests that forward traffic if “Large Receive Offload” is enabled.  I’m not sure if this is related to the issues I was having or not.  Currently, LRO is enabled on my ESXi Vyatta instance, and performance still seems OK.  So, I’m guessing that must be a separate issue from the one I was having.

CRL Expiration

Friday, June 17th, 2011

I have my own root certificate authority that I use to sign my personal SSL certificates (for my Exchange server, other internal web servers, etc.)  I’ve noticed that I’ve been getting warnings in Google Chrome, that the browser is unable to check the CRL (Certificate Revocation List).  To my knowledge, the CRL is basically just a list of certificates that have been revoked, that is signed by the CA, and stored in some accessible location (like on a web server).  These errors were confusing however, because I do have a CRL published where the SSL certs say it is, and the browser is able to fetch the CRL file stored there.  Being the perfectionist that I am, I had to solve this problem.

After looking into file extensions, MIME types, making the location accessible from the Internet, I was looking through my OpenSSL configuration and came across this line:

default_crl_days= 365                   # how long before next CRL

I had never generated a new CRL since when I tested it initially when set up all of my internal PKI stuff.  (I have maybe 15 SSL certs, all of which are on systems I control, so I’ve never had a reason to revoke one for real).  So, it was a couple years old at this point.  I regenerated the CRL, published it at the right location, and cleared the cache in Chrome.  After doing that, the CRL warning went away.

Not sure how I would have learned this had I not come across that line in the config.  You learn something new every day. :)

Exchange 2007/2010 Forwarding Rules Not Working

Sunday, November 14th, 2010

Recently, I noticed that the forwarding rule I set up on my Exchange 2010 account at an organization I help provide IT support wasn’t working.  This was on a new Exchange install.  I wasn’t that worried about it, nobody contacts me at that address anyway, but it annoyed me a little.  Earlier this week I finally looked into it.

Apparently forwarding / redirects to external domain names is disabled by default in Exchange 2007 and 2010.  I never really realized this before, I’ve used redirect rules before in Exchange, but not on a new install (that I manage) before.  It seems like the Outlook client (or OWA) should be able to check for this, and give the user an error message when they try to create a forwarding rule that would be blocked by this setting.

To change this setting, open the Exchange Management Console, and drill down to the Organization Configuration -> Hub Transport.  Under the Remote Domains tab, open the Default domain. 

image

Then, on the format tab, check the “Allow automatic forward” box.

image

Alternatively, from the Exchange Management Shell (PowerShell rocks!), this will do the trick:

set-remotedomain -identity Default -AutoForwardEnabled $true

Copyright 2009 Simpliciti Solutions