By Jeff Orloff

It was the day before the state’s standardized testing day, and I received a call from the assistant principal. At the school district where I was working, standardized testing is done mostly online, so it was certainly bad news when the assistant principal told me that half of the computers in the facility were not working. The school, located in a juvenile detention facility, had about 60 students using computers in eight  different rooms with three servers; a domain controller, an application server, and a media server for online courses that the students could take.

When I arrived at the school, one of the teachers showed me the strange problem. The teachers could not access any of the practice tests, retrieve documents, or access data from other network based applications. They could, however, get online and students could access their online courses — but the videos that delivered lectures were lagging.

Rogue Device to Blame

The computers were obviously attached to a network, since they were able to access the Internet. But running the simple IPCONFIG test on the computers showed a Class C network address opposed to the Class A block that was given out to all computers on the district network. Immediately, I thought that somehow our computers were connecting to the detention facility’s network. Checking one of their computers, I noticed that they, too, were using Class A IP addresses. Now I was starting to worry.

Clearly, something was on the network that was acting as a DHCP server. It would have been easy to ask the teachers if they had brought in a device that they shouldn’t have, but by this time everyone was gone for the day with the exception of myself, the administrator, and the one teacher who was helping me out. Using a laptop with RogueChecker installed on it, I was able to connect to the network and immediately find a server that was pushing out addresses to roughly half the campus. Now I just needed to find it.

RogueChecker in action

Using NetStumbler, I was able to look at the IP address of the server with the different wireless access points in the building. Sure enough, the server IP address of the rogue device shown in RogueChecker matched up with one found in NetStumbler. Using the signal strength indicator we could now narrow down our search to one wing of the building.

Identifying Rogue Devices

Sure enough, one of the classrooms had an off-the-shelf brand wireless router plugged into the network jack which was promptly removed. Once all the computers were restarted, we were able to restore access to network folders, data and most importantly the application that would run the assessment for the students the next day.

For a school this size, the process of finding the exact location of the rogue device was not that difficult a task. On a large secondary school, or university, the search would be more problematic and would take the efforts of many more people. In fact, one of the best methods I have seen to handle this task involves crowdsourcing.

The methodology is similar to this case. First the rogue device needs to be verified and then the location narrowed down using technology, generally more than one person searching for the device’s signal. Once you can eliminate a majority of the campus you need to enlist the help of as many willing participants as you can find to help search for the device by assigning each a geographic location that they are responsible for making sure that the assignments overlap as much as possible to ensure nothing is left unturned.

Spilt tunneling can also be used in conjunction with the local firewall that comes with the NCP client.  Rather than locking the user in to the tunnel as described earlier, one can also just use a shorter list of the subnets or hosts that can be reached from home via the VPN tunnel at the corporate side, and all other is simply dropped by the local VPN client’s firewall.  The user can then try to access expedia.com (our example from before), but it is simply dropped.

It all depends on how secure one wants to lock down this remote resource.  He or she can extend the full restrictive measures imposed on the corporate environment to the machine at home or on the road as if they’re still partaking in the central network, or choose to be less restrictive using a combination of split tunneling and firewall rules on the client.

It should be mentioned that Cisco gateways will most often ‘publish’ these ‘whitelists’ to the client during the negotiations, and so the ‘split tunneling’ list is populated automatically.  Other gateways don’t supply this, and so the client MUST either define it manually or automatically be locked in.

A helpful resource Rene recommends is Security Now podcast: episode 208

Follow this discussion on Twitter @VPNHaus

After spending a couple of weeks worrying that I’d have to be plugged directly into my router to connect to my work VPN network, with my Dell D830 Latitude laptop and Windows 7 64 bit, I finally chanced upon the solution. It turned out to be a device manager setting and potentially a setting in the BIOS on my D830 dell latitude (bios revision A14).

Follow the following steps if you are suffering the same issue yourself…

1. Changed the MTU setting on the VPN device…

2. Changed a setting in the bios, which dictated that the wifi connection should be turned off when another connection is available (i.e. LAN or 3G).

UPDATE: 23:15 15 January 2010: Actually I’ve just discovered the real route of my problems. Turns out that if my router (3com office connect adsl wireless 11g firewall router), assigns an ip address that is in use by one of the virtual server LAN IP addresses, on either wireless connection or LAN connection, then the VPN software fails to connect.

What actually happened was when I plugged another router into my firewall router, I got assigned 192.168.1.3 to my laptop wireless card, which wasn’t one of the entries in the virtual servers table, and that’s when it started working.

So if you have trouble connecting, double-check if you have conflicting IP addresses, or, drop us a line – help@ncp-e.com or @VPNHaus

An overlapping subnet is when you establish a connection from the VPN client to another network with the same ‘private IP address range’, and an ‘overlap’ occurs with the addresses.  I.e. the hotel router assigns your machine a ‘private IP address range’, i.e. 192.168.1.0, and this address matches the office’s.  When the client connects, it uses the source IP address it currently has, which is the home network.  The gateway sees this as an internal (local) address, and thus subnets overlap  and deny your VPN connection.

Here is a technical description NCP shared with us:

IPsec includes two negotiation phases; phase 1 authenticates and negotiates a secure channel to set up a Phase 2 tunnel.  Phase 1:  ‘ISAKMP/IKE’ takes place over UDP500.  Once the negotiations have taken place, one or more IPsec tunnel(s) is created in Phase 2 (between the two peers—client and the gateway.  Traffic is sent using ESP (Encapsulated Security Payload) Frames, which are not within UDP or TCP, except ESP = IP Protocol 50; something ‘parallel’ as it were to the aforementioned TCP or UDP.  However, if there’s a router or firewall in between that performs Network Address/Port Translation (aka Network Address Translation) these packets will either be dropped or modified (modified, meaning tampered with, therefore being dropped by the gateway or client).  Some routers/firewalls allow for ‘ESP Pass-through’, meaning these ESP frames will not be dropped and it’ll work.

99% of the time there is going to be NAT performed on the packets.  In order to circumvent this problem, the ESP frames are wrapped inside UDP packets which may be modified/touched by the routers.  Once they arrive at either the two peers, the outer (modified) UDP headers are stripped off, revealing the untouched ESP frames which then can be processed.  This UDP encapsulation is called NAT-Traversal or NAT-T.

Back to our original definition, IPsec uses UDP 500 and ESP frames, the latter may be encapsulated within UDP 4500 (or variable; other gateways sometimes use UDP10000).

We will follow up on this topic with solution in a later post—stay tuned.

OR, maybe the VPN client went into sleeper mode or was deactivated by user error (happens more than anyone likes to admit!). Most people find layers of security irritating and turn them off.

Without access to the actual device, it seems to be a setting issue or Hoff is testing a weak client (NCP’s solves all these issues – even an option to prevent user tampering).