Notes from the Ubiquiti World Conference in Chicago

March 24th, 2012

Hey, I went up to the Ubiquiti World Conference yesterday in Chicago.  Thought I’d share a few quick points from my notes.

IMG-20120323-00054

AirFiber

  • Mentioned they were shooting for one new product line per quarter – seems pretty aggressive
  • point to point wireless backhaul.  Not wifi based.
  • 700 Mbps in each direction up to about 1.5 miles.  250 Mbps out to ~10 miles (max range)
  • license free, 24 Ghz.  Might support licensed 21 / 25 Ghz later on.
  • they brought the AirFiber team in from Motorola Canopy.
  • simultaneous transmission – separate antenna for TX / RX.  Allows for better RX gain without hitting FCC limits for output.
  • $3000 for two units / a single link.

IMG-20120323-00055

Unifi

  • UAP Pro should be available in April finally.  Will have GbE, multiple ethernet ports, 802.3af, 5 Ghz support.
  • Outdoor 5 Ghz
  • Availiability issues – UAP-LR has not shipped for a few months.  Should be better soon.
  • Multi site management coming soon.  Separate databases, single place to manage all of it through.

AirOS Roadmap

  • 5.5 – multiple VLAN support, large MTU (2024 bytes)
  • 5.6 – Ubiquiti SNMP MIB
  • Multiple SSID’s might be added in a later release

AirSync

  • prevents your own AP’s on a single tower from transmitting while another is receiving.  Only prevents collisions from your own AP’s though.

ToughSwitch

  • 8 port web managed (AirOS) PoE switch
  • VLAN’s, bonding, RSTP, etc.
  • Ping monitoring, auto power cycle AP on port.
  • No fiber planned
  • no pricing announced

IMG-20120323-00056

Titanium Line

  • GbE, 802.3af, metal case.
  • Rocket Titanium, Bullet Titanium
  • Variable sector width product: can be 60 / 90 / 120 degrees, can be changed by user
  • A few months out

IMG-20120323-00059

AirControl 2

  • centralized management for AirMax devices
  • auto topology mapping, can pull GPS coords from devices.
  • proprietary management protocol, no SNMP :(
  • rule based grouping / altering / action / confic changes

AirVision

  • right now, AirCam, AirCam Dome, AirCam mini
  • same type of camera in each.  720p
  • Dome has built in mic, micro SD card
  • Mini is table top form factor, has a webcam like look to it
  • AirCam pro later this year.  Metal body, optical zoon, heating.  1080p.  IR LED
  • New AirVision software features – mobile browser support, video export, email notifications with pictures, automatic disk management
  • Working on pushing intelligence to cameras themselves to offload from AirVision server – motion detection, etc.
  • Eventually will have PTZ cameras.
  • No wireless cameras planned.

IMG-20120323-00057

Availability:

  • UAP Pro – April, price will be “disruptive”
  • AirFiber – Q2
  • Titanium products – May
  • ToughSwitch – May

Misc:

  • EdgeOS router still in development, a ways off.  More info in future conferences.
  • No plans for WiMax.  Some licensed band products possible in future.

Annoyances in IPv6

March 6th, 2012

I wouldn’t consider myself an anti-IPv6 luddite. I have IPv6 deployed in my home network (with maybe 15 different subnets, 30 or so VM’s, multiple sites, etc.a decent approximation of an SMB network) with HE.net tunnels. I’m frustrated about all the real world issues associated with IPv6 though.

IPv4 is obviously running out of addresses, and IPv6 is the only reasonable solution today for that. But beyond the issue of address space exhaustion, I’m not sure that IPv6 is going to solve many more problems than it causes.

I don’t think we can really blame these issues on the good folks at the IETF who developed IPv6. IPv6 was designed quite a long time ago, and the IPv4 landscape has changed a lot in the last 15 years. It’s frustrating though that we’re trading one set of problems (associated with IPv4) to another set of problems that IPv6 will bring.

I’d like to see more universal support for IPv6.from network vendors, software vendors, and ISP’s. Unfortunately, the organization I work at has a /8, and don’t appear to be eager to adopt IPv6. What are your thoughts?

Small Sites

Connecting a small site to multiple ISP’s via IPv6 is going to be more difficult than with IPv4 / NAT. I think this is a fairly well documented (see Ivan Pepelnjak’s post).

But in my view, the even more significant issue is that the internal addresses small sites use will be tied to an ISP.

Currently with IPv4, most smaller orgs will use RFC 1918 space internally and NAT those addresses to their public IP range. With v6, organizations that aren’t large enough to obtain their own provider independent IPv6 allocation will use their IPv6 address ranges given to them by their ISP’s – both internally and externally.

This makes changing ISP’s an extremely painful experience. With IPv4, changing your public address space would involve some relatively simple NAT changes, and probably some DNS / firewall changes on the outside. With IPv6, all of the internal devices will have to be re-IP’ed. For end-user devices using DHCPv6 or stateless autoconfiguration, that’s really not a huge deal. But for all of the routers, servers, and other infrastructure devices with static addresses.that’s not going to be a fun undertaking.

Fundamentally, I think it makes sense to decouple internal and external addresses. Overloaded NAT as is typically deployed with IPv4 is not necessarily the best way to handle this. One-to-one NAT would be a bit better. IPv6 prefix translation as RFC 6296 describes makes more sense. I’m not sure that any existing IPv6 hardware actually supports what RFC 6296 describes however. The fanatical religious opposition to any implementation of address translation in IPv6 is disheartening. I think the lack of common support for address translation in IPv6 will cause a lot more harm than good.

Link Local Addresses

Maybe this is just a personal thing. But I find link local addresses to be fairly annoying. I get that there are scenarios where multiple devices are in the same broadcast domain that need to communicate with eachother without a router sending RA’s or a DHCPv6 server. But I’m not sure I see a purpose of using link local addresses when a routable address is already assigned on an interface.

Take for instance this output from a “show ipv6 route” in Vyatta:

O>* 2001:470:c147:50::/64 [110/2] via fe80::80ab:92ff:fe6e:82ee, br1, 21:52:50
O>* 2001:470:c147:60::/64 [110/2] via fe80::80ab:92ff:fe6e:82ee, br1, 21:53:00
O>* 2001:470:c147:70::/64 [110/2] via fe80::80ab:92ff:fe6e:82ee, br1, 21:53:00
O>* 2001:470:c147:80::/64 [110/10] via fe80::80ab:92ff:fe6e:82ee, br1, 21:53:00

To me, that link local address is fairly useless information. Here’s another example, from Wireshark:

clip_image002

Now these two issues are really more related to the applications themselves than the protocol. But I see link local addresses as causing more confusion and headaches than they will solve.

Colons

I’ll admit, this is somewhat petty. But honestly, colons were probably one of the worst possible characters to use as a delimiter in IPv6. It conflicts with the delimiter for port numbers that often follow IP addresses. This often requires square brackets around the IPv6 address. As if IPv6 addresses weren’t frightening looking enough, a URL that previously looked like http://10.20.30.40:8080 will now look like http://[2001:470:c147:50::10]:84. (Granted, DNS should be used in most of these situations.)

It also requires multiple key presses on most keyboards and is more difficult to type on mobile devices.

Maybe forward “/” slash would have been worse as far as confusion and conflicts go. Other than that though, I can’t think of too many symbols that would be more of a pain.

Handling of Dual Stack Environments

Five years from now, I think it’s fairly clear that a great deal of environments will be dual stack. Sure, there will be a few places that have managed to become IPv6 only or greenfield deployments that are IPv6 only by necessity. And I’m sure there will be some lethargic organizations that will hold on to IPv4 until the end of time. But, the way it looks now, most places will be dual stack. With hosts using an IPv4 address and an IPv6 address. With routers using separate protocols and forwarding tables for IPv4 and IPv6. And with firewalls using separate access lists for IPv4 and IPv6.

In my mind, it would make sense if this were a little more unified.

Why shouldn’t I be able to apply an access list to an interface that includes both IPv4 and IPv6 elements? I think this would really come in handy with firewalls that support address groups. I apply policy Webservers_Policy to an interface that allows traffic to port 80 on everything in the Webservers_Group address group. And when I happen to add an IPv6 web server, I just add an IPv6 address to that group.  Obviously, the v4 / v6 ACL’s would just appear to be unified from a management perspective.  But surely it couldn’t be too difficult a vendor to do this.

For routing, IS-IS might make this feasible with single-topology support. See this example.

I don’t think this would make sense in every environment. But I think in a lot of cases it would make it a lot easier to add IPv6 on to an existing IPv4 environment. And that’s going to be the most common scenario in the near future.

Vendor Support

This is really more a vendor problem than a problem with the protocol itself. But it feels like a number of vendors view anything beyond basic IPv6 support as an addon or advanced feature, even if the IPv4 equivalent is included standard.

I think this situation is changing as IPv6 support becomes more of a necessity. Still though, some devices support just enough IPv6 to be able to check the “I support IPv6″ checkbox, but not enough to use in real world networks.

For instance, the image I have on a Cisco 1720 supports IPv6, and OSPFv2, but not OSPFv3. As another example, Vyatta supports things like VRRP and firewall address groups for IPv4, but not IPv6.

If I’m using OSPF for dynamic routing with IPv4, chances are I’m going to want to use OSPFv3 for IPv6.

First Thoughts on Ubiquiti Unifi

December 3rd, 2011

I purchased a Ubiquiti Unifi AP earlier this week and have been pretty impressed.  The Unifi AP’s are super super cheap ($70 – 80), and the controller software is free.  The UAP Mini’s, which have a shorter range / slower speed cost about $60, though it’s hard to find anywhere online that has them available.  ( Amusingly, I can only just barely buy a Cisco PoE injector for this amount )  These are enterprise class AP’s – they support 802.1q, multiple SSID’s, WPA enterprise, centralized management, etc.  All of the current models are 2.4 ghz only, supposedly dual-band models will be released in the future.

They also support a captive portal, which is super easy to configure with either a static password, generated one-time-use vouchers, etc.  This is the only part that actually depends on the controller service.  So, if the controller is down (or inaccessible), everything will continue to work normally except guest access.  This is pretty nice, as it let’s you have a single controller for multiple sites, without having to worry too much about what happens if the sites are disconnected.  I have screenshots of the guest policy and voucher interfaces below.

guest-controlvouchers

I do have a few small gripes / annoyances.  Hopefully some of these will be addressed as the controller software becomes more mature.  It looks like a long list – but I don’t think any of these would be deal-breakers in most small or medium sized environments.

  1. Limited to 4 SSID’s / VLAN’s on each AP.
    This probably isn’t an issue in most environments, but it could be in some.  As it is though, these AP’s are probably cheap enough that you could deploy additional AP’s to handle the additional VLAN’s and still end up spending way less than you would with Cisco, et al.
  2. Configuration changes take AP’s offline for short period
    When I make a change on one of the SSID’s / VLAN’s, the AP goes into a provisioning mode for a minute or so.  When this is occurring my wireless clients are disconnected.  Not sure how this is handled in a multi-AP environment.  My hope is that the provisioning would be staggered, so clients would be able to connect to another AP while the AP they were originally associated with is rep-provisioned.  If not, any configuration changes become events requiring downtime, which would be disappointing.
  3. No SNMP Support
    SNMP is currently not supported in the controller software, or on the devices themselves.  This isn’t a huge deal for me (It’s easier to monitor the bandwidth usage via the switchports), but I can imagine it would be super-nice to have in many environments.  The controller software has some monitoring built in to it – the controller can send email alerts if a AP goes offline and can show usage graphs (see screenshot below). 
    usage
  4. On Windows, controller software doesn’t use a service by default
    When installed on Windows, the controller software doesn’t use a Windows service out of the box.  There is an option in the software to install a service ( see this post on the Ubiquiti forum ).  I’m not sure why this isn’t done by default though.  I can’t imagine any instance where you would not want software like this to be a service.
  5. Only a single RADIUS Server can be Configured
    Only one RADIUS server configured, no failover.  Cisco’s Aironet devices support configuring multiple redundant RADIUS servers.  It would be nice to see this feature added.  You could potentially use some kind of clustering or load balancing to still have redundancy though if you wanted.  See screenshot of the SSID configuration interface below:
    wireless-network

There are some other things that would be nice to have, like:

  1. Multiple guest policy profiles
    Right now, you have one guest policy that is applied to all SSID’s that have the captive portal features enabled.  It could be nice to have multiple guest profiles.though I admit I can’t come up with a common use-case.
  2. Standardized PoE
    Ubiquiti uses a modified version of PoE.  They do ship a PoE injector with their devices, but it would be annoying to have to use these if I had a large number of AP’s and already had a PoE capable switch.  I’ve heard a UAP Pro mentioned on the Ubiquiti forums that sounds like it’ll use 802.3af.
  3. Gigabit
    All of the Unifi AP’s currently offered (UAP, UAP Mini, UAP Long Range, and an external model) have a single 100 Mbps port.  The 150 – 300 Mbps they claim to support support with 802.11n pretty useless with a 100 Mbps uplink.

I’ve heard of a UAP “Pro” mentioned on the Ubiquiti forums that will add a couple of these features (802.3af PoE, and gigabit interfaces).

Overall I love Unifi.  If your environment can live with these annoyances, frankly I don’t see any reason why you would use anything other than Unifi. 

The company I work for uses Ubiquiti AirOS devices for a short point to point connection.  I also have set up Ubiquiti AirOS devices at the UNC Project in Malawi for point-to-point links there between satellite clinics.  In both of these cases they perform fantastically.

I’m excited to see where Ubiquiti will go in the future – they have some really great products.  Find more details on Unifi at Ubiquiti’s website.

Vyatta to DD-WRT: OpenVPN Site to Site

July 17th, 2011

I recently changed my site-to-site VPN configuration.  Previously, I had been running my remote OpenVPN endpoints on CentOS with Quagga and OpenVPN installed.  Now that I have a couple Vyatta VM’s routing traffic on the internal network I’ve been wanting to use those as the VPN endpoints.  This simplifies things a big, and lets me remove Quagga from the two CentOS boxes.  I’ve also been wanting to switch my OpenVPN to bridge mode as opposed to routed mode.  OpenVPN doesn’t currently support IPv6.  Using bridge mode allows me to configure the L3 aspects outside of OpenVPN.

Rather than bridged the OpenVPN interface directly to one of the VLAN’s on each end of the tunnel (which would result in broadcasts traversing the WAN, and hosts on both locations being on the same Layer 3 network, not desirable in this case) I’m not actually “bridging” the VPN tunnel interfaces to other interfaces.  Instead, I created a bridge interface on each end of the connection with only the OpenVPN tunnel interface included in it.  I assigned the IP addresses on the bridge interfaces.  Here’s a diagram on how the interfaces are set up:

diagram2

I hadn’t used OpenVPN on Vyatta before, or OpenVPN in bridged mode, so this was a little new for me.  I generated a static key and copied it to all of the devices.  I have the OSPF costs on my internal Vyatta routers set so that one is always preferred.  (I also have that router preferred in VRRP, though this is unrelated to OpenVPN).  The Vyatta VM’s aren’t directly connected to the Internet, so I have OpenVPN listening on an internal address instead.  That internal address is then NAT’ed to a public IP on my edge router.  Ignoring the edge router, here’s a simplified diagram of the tunnels:

diagram1

Here is the configuration I used on the two Vyatta endpoints.  The only OpenVPN feature I’m using that Vyatta doesn’t have a configuration option for is the compression.  Thankfully, Vyatta let’s you manually specify OpenVPN configuration that they don’t support with “openvpn-option”.

intra-rtr-1:

bridge br0 {
        address 172.16.5.1/30
        description "L3 IF for APT site to site VPN"
        hello-time 2
        ip {
            ospf {
                authentication {
                    md5 {
                        key-id 1 {
                            md5-key myospfkey
                        }
                    }
                }
                cost 51
                dead-interval 40
                hello-interval 10
                priority 1
                retransmit-interval 5
                transmit-delay 1
            }
        }
        max-age 20
        priority 0
        stp false
    }
openvpn vtun0 {
        bridge-group {
            bridge br0
        }
        local-host 192.168.1.21
        local-port 1501
        mode site-to-site
        openvpn-option --comp-lzo
        protocol udp
        shared-secret-key-file /root/apt-rtr.secret
    }

intra-rtr-2:

 bridge br0 {
        address 172.16.6.1/30
        description "L3 IF for APT site to site VPN"
        hello-time 2
        ip {
            ospf {
                authentication {
                    md5 {
                        key-id 1 {
                            md5-key myospfkey
                        }
                    }
                }
                cost 49
                dead-interval 40
                hello-interval 10
                priority 1
                retransmit-interval 5
                transmit-delay 1
            }
        }
        max-age 20
        priority 0
        stp false
    }
 openvpn vtun0 {
        bridge-group {
            bridge br0
        }
        local-host 192.168.1.22
        local-port 1502
        mode site-to-site
        openvpn-option --comp-lzo
        protocol udp
        shared-secret-key-file /root/apt-rtr.secret
    }

I’m still using DD-WRT on the endpoint in my apartment. I’ll soon be switching this over to Vyatta as well, but until then I’m relying on some somewhat messy methods of using OpenVPN. I don’t trust the OpenVPN setup that the DD-WRT web GUI uses (for one, they use NAT’ing in the site-to-site connection…which makes no sense whatsoever), so I have a shell script on an attached flash drive that runs on the OpenVPN daemons. The included version of Quagga also does not work on DD-WRT (apparently they compiled it wrong), so I had to find an old statically compiled version. That’s what I’m using for OSPF on the DD-WRT box right now. For this new VPN setup, I just added a few lines to the shell script I already use for OpenVPN.

Anyway, here is the OpenVPN config on my DD-WRT router. It’s fairly simple. I actually have two separate OpenVPN daemons on the DD-WRT system. One that connects to each of my two internal Vyatta routers. The only thing different between them is the port that they connect to.

dev tap0
proto udp
remote my.vpn.endpoint 1501
resolv-retry infinite
nobind
secret intra-rtr.secret
comp-lzo

Here is the script I’m using on the DD-WRT router. It creates the bridge interfaces, and adds the OpenVPN interfaces to the bridge groups. It also assigns the IP’s on the bridge interfaces, starts Quagga, and starts the OpenVPN daemons. When starting the OpenVPN daemons, it also calls DD-WRT’s route down script, to make sure the NAT’ing that DD-WRT tries to do by default is removed from iptables.

openvpn --mktun --dev tap0
openvpn --mktun --dev tap1
brctl addbr br10
brctl addif br10 tap0
brctl addbr br11
brctl addif br11 tap1
ifconfig tap0 up
ifconfig tap1 up

ifconfig br10 172.16.5.2 netmask 255.255.255.252
ifconfig br10 promisc
ifconfig br10 up
ifconfig br11 172.16.6.2 netmask 255.255.255.252
ifconfig br11 promisc
ifconfig br11 up

zebra --daemon --config_file /jffs/opt/zebra.conf
ospfd --daemon --config_file /jffs/opt/ospfd.conf

( sleep 2 ; killall openvpn ; /tmp/openvpncl/route-down.sh ;
    /jffs/opt/ovpn --config /jffs/opt/openvpn-intra-rtr-1.conf --daemon) &
( sleep 3 ; killall openvpn ; /tmp/openvpncl/route-down.sh ;
   /jffs/opt/ovpn --config /jffs/opt/openvpn-intra-rtr-2.conf --daemon) &



Here’s the relevant OSPF configuration in Quagga on my DD-WRT router. Most of it’s fairly straightforward:

interface br10
 description L3 IF for site to site VPN - intra-rtr-1
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 myospfkey
 ip ospf cost 51
!
interface br11
 description L3 IF for site to site VPN - intra-rtr-2
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 myospfkey
 ip ospf cost 49
router ospf
 ospf router-id 192.168.17.1
 network 192.168.5.0/24 area 0.0.0.0
 network 192.168.6.0/24 area 0.0.0.0
 network 192.168.16.0/21 area 0.0.0.0
 area 0.0.0.0 authentication message-digest

I haven’t fully set up IPv6 yet. I’m planning on waiting until I replace my DD-WRT router with Vyatta to set all of that up. The version of Quagga I found that would run on DD-WRT doesn’t support OSPFv3. Vyatta does though.

Slow Performance with Vyatta on VMware ESXi

July 13th, 2011

I use a pair of Vyatta VM’s to route between VLAN’s at home.  One is running on ESXi, the other on VMware Server 2 (yeah, I really want to move that to ESXi, haven’t had time to move all of the services I’m running on the host onto VM’s).  They are redundant, I use VRRP on the user and server facing VLAN’s and OSPF on the VLAN facing my Cisco 1720 that’s connected to the Internet.  The VM running on ESXi is prioritized in OSPF and VRRP to make lifer simpler and troubleshooting easier.

Drawing2

I recently noticed how slow the performance was when accessing FTP and HTTP for one particular host.  From outside, a FTP session to this particular host would average maybe 10 KB/s.  One moderate size image hosted via HTTP on that host basically would not load because the connection was going so slow (1 KB / s in this case).  Off and on, I noticed slow performance connecting to other hosts.  For a while, I focused my troubleshooting on the host exhibiting most of the performance issues.  Oddly enough though, when I failed everything over to the other Vyatta VM running on VMware server, the performance was much better.  I was able to receive 40 – 50 KB / s with that FTP session.  Connecting the host directly to the VLAN between my Cisco router and the Vyatta VM’s also resulted in much improved performance.  Nothing obvious jumped out from tcpdump’s or logs on the Vyatta VM’s.

I started looking at what inconsistencies there were between the two VM’s, and between the interfaces on each VM.  These VM’s originally were made from Vyatta’s OVF template.  I must have added one additional NIC to each of them.  Two of the NIC’s on the ESXi Vyatta instances were VMXNET, and the third was E1000.  Eventually, after troubleshooting this issue for quite a while, I found this particular topic on the Vyatta forums.  When I changed the two VMXNET interfaces to VMXNET3, all of the performance issues on the ESXi Vyatta instance went away.  (The Vyatta instance running on VMware server still has the NIC’s on VMXNET and doesn’t appear to suffer performance-wise from it).  From other posts in that topic, it sounds like this may be related to a issue with the open-vm-tools included in past versions of Vyatta, that hadn’t been fixed in the OVF yet.

Another post on the Vyatta forum linked to a VMware KB article.  The KB article describes performance issues on Linux guests that forward traffic if “Large Receive Offload” is enabled.  I’m not sure if this is related to the issues I was having or not.  Currently, LRO is enabled on my ESXi Vyatta instance, and performance still seems OK.  So, I’m guessing that must be a separate issue from the one I was having.

CRL Expiration

June 17th, 2011

I have my own root certificate authority that I use to sign my personal SSL certificates (for my Exchange server, other internal web servers, etc.)  I’ve noticed that I’ve been getting warnings in Google Chrome, that the browser is unable to check the CRL (Certificate Revocation List).  To my knowledge, the CRL is basically just a list of certificates that have been revoked, that is signed by the CA, and stored in some accessible location (like on a web server).  These errors were confusing however, because I do have a CRL published where the SSL certs say it is, and the browser is able to fetch the CRL file stored there.  Being the perfectionist that I am, I had to solve this problem.

After looking into file extensions, MIME types, making the location accessible from the Internet, I was looking through my OpenSSL configuration and came across this line:

default_crl_days= 365                   # how long before next CRL

I had never generated a new CRL since when I tested it initially when set up all of my internal PKI stuff.  (I have maybe 15 SSL certs, all of which are on systems I control, so I’ve never had a reason to revoke one for real).  So, it was a couple years old at this point.  I regenerated the CRL, published it at the right location, and cleared the cache in Chrome.  After doing that, the CRL warning went away.

Not sure how I would have learned this had I not come across that line in the config.  You learn something new every day. :)

Auto-Reply for Groups in Exchange

February 21st, 2011

Today I was wanting to set up a auto-reply for a group in Exchange. So, when someone sends a message to that group, all of the members of the group will receive the message, and the sender will receive an auto-response (so that they know I received the message).

Distribution groups aren’t actually email accounts, so rules and out of office messages can’t be configured for them. The best solution I found was to have a fake user account set up that receives the message, and then have that account forward the message to the appropriate distribution group.

Unfortunately, Exchange’s built-in “out of office” messages will only send a auto-reply once to each sender. In some scenarios (if its being used truly for someone being out of office), this behavior makes sense. In other situations though, it would be nice to send a auto-reply every time a message is received. So, instead, I had to create a rule that when send an automatic reply. This can’t be done with OWA. OWA doesn’t give you all of the rule options that the actual Outlook client does.

Still however, this wasn’t working for me. When I sent a message to the “fake” user account, I saw the mailtip in Outlook 2010 with the autoresponse message in it. However, I wasn’t actually receiving an auto-reply. I had forwarding turned on on the server. This was actually forwarding the message before the rules configured on the account saw the message. It worked as desired after changing the forwarding setting to deliver the message to the fake account’s mailbox as well as forward it on to the distribution group.

So, if you want to configure an auto-reply on a group (that always sends a reply for each message it receives):

  1. Create a fake user account with the email alias you want people to send the messages to
  2. Create a distribution group that contains the members who will receive messages sent to the fake user. If users from outside the Exchange organization will send messages to this group, make sure that non-authenticated users are allowed to send messages to this group.
  3. Under the recipient configuration in Exchange, set the fake user account to forward messages to the distribution group. This is configured under the “Delivery Option” properties on the “Mail Flow Settings” tab. Make sure to check the box that says “Deliver messages to both forwarding address and mailbox”.
  4. Log in to the fake user’s account with the full Outlook client. Create a rule that will apply to every message that is received. Choose “have server reply using a specific message” for the action.
  5. One other thing to check if you’re having problems is to make sure that auto-replies are enabled for the Exchange organization. They may not be by default. In Exchange 2007 / 2010 this is configured under Organization Configuration -> Hub Transport -> Remote Domains. Right click on the * (default) remote domain and go to properties. The “Allow Automatic Replies” option in under the “Message Format” tab.

    Oliver

Netflow Monitoring Options

January 6th, 2011

One of the books I received as a gift this Christmas was Network Flow Analysis.  It introduced me to a bundle of Netflow related tools I hadn’t worked with before, flow-tools.  Previously, I have used nfdump.  Perhaps its just because of how it was introduced to me, but nfdump seems to be better suited for ad-hoc monitoring, rather than continually running monitoring.  Flow-tools on the other hand provides a handy script in /etc/init.d (at least on CentOS / Fedora).  Its easy to run it as a daemon.  The only tricky part of the install process was finding the configuration file for the daemon.  It is located at /etc/sysconfig/flow-capture in CentOS/Fedora, rather than in /etc/flow-tools/.

By simply running running the daemon on a Linux server at a site, and pointing the Cisco router to export its Netflow data to the server, I can store a history of network connections.  If we see an unusual period of heavy traffic, or a user complains about slow performance, we can go back and see what happened in each five minute increment.  By allowing us to find what the culprit was after-the-fact, this makes it much easier to troubleshoot rare or transient network performance issues.  Instead of guessing, or making excuses this allows a network administrator to dig into the details and determine what was actually happening at a network level.

When combined with our Squid proxy server logs, we can determine what URL’s the user was visiting if it was web traffic causing the slow-down.  Netflow will just show the destination IP and port the traffic was going to.  In many cases, this isn’t enough to determine what actual website the user was visiting.  If the user was downloading a large file, there’s a good chance it was hosted on a CDN (Content Delivery Network).  In this case, the IP will belong to the CDN, its reverse DNS address will likely be related to the CDN, and browsing to that specific IP address in a web browser probably won’t give much in the way of clues.  The proxy server log files can be searched for the specific IP, to determine what URL / website the user was actually accessing.

So far I’ve been pleased.  From seeing a period of high bandwidth usage we were able to track down someone downloading a bunch of MP3’s (over the clinic’s slow and high latency satellite connection).  A couple additions to the Squid configuration took care of that.

Exchange 2007/2010 Forwarding Rules Not Working

November 14th, 2010

Recently, I noticed that the forwarding rule I set up on my Exchange 2010 account at an organization I help provide IT support wasn’t working.  This was on a new Exchange install.  I wasn’t that worried about it, nobody contacts me at that address anyway, but it annoyed me a little.  Earlier this week I finally looked into it.

Apparently forwarding / redirects to external domain names is disabled by default in Exchange 2007 and 2010.  I never really realized this before, I’ve used redirect rules before in Exchange, but not on a new install (that I manage) before.  It seems like the Outlook client (or OWA) should be able to check for this, and give the user an error message when they try to create a forwarding rule that would be blocked by this setting.

To change this setting, open the Exchange Management Console, and drill down to the Organization Configuration -> Hub Transport.  Under the Remote Domains tab, open the Default domain. 

image

Then, on the format tab, check the “Allow automatic forward” box.

image

Alternatively, from the Exchange Management Shell (PowerShell rocks!), this will do the trick:

set-remotedomain -identity Default -AutoForwardEnabled $true

S3 Backup Scripts: Limiting Upload Rate

November 2nd, 2010

I’ve recently started toying around with Amazon S3 for doing some remote backups.  On Linux, I’ve been using S3 tools to access S3.  I’ve been fairly impressed with S3 and S3 tools so far.

There are a few extra things beyond what the s3cmd program (included in S3 tools) does that would be useful for me.  First, s3cmd will upload / download from S3 as fast as it can.  If you are transferring a large file, this can be problematic as it will saturate your connection for some time.  I was looking for an easy way to shape this traffic without doing traffic shaping at the network level (which could be tricky as S3 uses HTTP for transfers).  I found trickle, a pretty neat little app that does traffic shaping in userland.

Second, the ability to resume a failed transfer would be nice.  Last week I was in the middle of uploading a large file to S3 from the fairly limited speed connection I have here.  With trickle, it was going to take about 24 hours to upload the whole file.  Unfortunately, about 16 hours in to the transfer some bad storms here cut my connection off for a few minutes.  It was rather frustrating to have to start that transfer back at the beginning again.

I researched it a little bit, and it looks like S3 doesn’t really provide a mechanism to resume interrupted transfers.  The best way that I could think of to handle this and minimize the pain of a dropped connection is to split the larger archive into smaller pieces and upload them individually.  By doing this, if you have to start the transfer over, you only have to re-transfer the part of file.  I made my script split it into 20 MB pieces.  At the 20 KB/s rate that trickle lets through, this would only take 17 minutes or so to re-upload.  Splitting a file into smaller pieces reduces the amount of data that would have to be uploaded again if the connection is cut off.

I’ve put the scripts below.  s3-up.sh will first create a tar.gz (compressed archive) of the files specified for upload, then split the tar.gz file into 20 MB chunks and upload it at 20 KB/s.  s3-down.sh will download multiple files from S3, put them back together, and extract them.  It will try to decrypt them using the currently configured password for s3cmd.  You can specify a different decryption password (if you regularly change the encryption password you use for s3cmd for instance).

Feel free to tweak as needed.  You’ll need to have s3cmd installed, along with trickle to use these scripts.  If you are using CentOS, both of those are in the EPEL repositoryI make absolutely no guarantees that these scripts will work for you.  They have not been thoroughly tested.  They are not intended to be relied upon in a production environment.

s3-up.sh:

#!/bin/bash

DATE=`date +%Y-%m-%d`
if [ -z $1 ] || [ -z $2 ]
then
echo Arguments: files to upload, base name of file
else
mkdir /tmp/$$
tar -czf /tmp/$$/$2_$DATE.tar.gz $1
mkdir /tmp/$$/$2_$DATE
split -d -b 20m /tmp/$$/$2_$DATE.tar.gz /tmp/$$/$2_$DATE/$2_$DATE.tar.gz.
trickle -s -u 20 s3cmd put -e --recursive /tmp/$$/$2_$DATE s3://your-s3-bucket-name/backup/
fi

s3-down.sh:

if [ -z $1 ] || [ -z $2 ]
then
echo Arguments: file to get, like s3://your-s3-bucket-name/backup/bak1/*, where to extract, optional password
else
mkdir /tmp/$$
mkdir /tmp/$$/split
cd /tmp/$$/split

if [ -z $3 ]
then
s3cmd get $1
else
s3cmd get $1
for fn in /tmp/$$/split/*
do
mv $fn $fn-enc
gpg -d --verbose --no-use-agent --batch --yes --passphrase $3 -o $fn $fn-enc
rm $fn-enc
done
fi
cat  * > ../temp.tar.gz
cd ..
tar -xvzf temp.tar.gz -C $2
fi

Copyright 2009 Simpliciti Solutions