Windows Server 2008 R2 : Server-to-Client Remote Access and DirectAccess - VPN Protocols

To establish a tunnel, both the tunnel client and tunnel server must be using the same tunneling protocol. Tunneling technology can be based on either a Layer 2 or Layer 3 tunneling protocol that corresponds to the Open System Interconnection (OSI) reference model. Layer 2 protocols correspond to the Data-link layer and use frames as their unit of exchange. PPTP and L2TP are Layer 2 tunneling protocols that encapsulate the payload in a PPP frame before it is sent across the Internet. Layer 3 protocols correspond to the Network layer and use packets. IPSec tunnel mode is a Layer 3 tunneling protocol that encapsulates IP packets in an additional IP header before sending them across the Internet.

Windows 7, Windows Vista, Windows XP, and Windows 2000 workstation VPN client and server computers support both L2TP/IPSec and PPTP by default. Both PPTP and L2TP/IPSec use PPP to provide an initial envelope for the data and then append additional headers for transport through the Internet. PPTP and L2TP also provide a logical transport mechanism to send PPP payloads and provide tunneling or encapsulation so that PPP payloads based on any protocol can be sent across the Internet. PPTP and L2TP rely on the PPP connection process to perform user authentication and protocol configuration.

There are a few differences between the three protocols. First, when using PPTP, the data encryption begins after the PPP connection process is completed, which means PPP authentication is used. With L2TP/IPSec, data encryption begins before the PPP connection process by negotiating an IPSec security association. In SSTP, the session is encrypted by SSL before authentication begins. In DirectAccess, communications are encrypted transparently before user data begins to flow.

Second, PPTP connections use MPPE, a stream cipher that is based on the Rivest-Shamir-Adleman (RSA) RC-4 encryption algorithm and uses 40-, 56-, or 128-bit encryption keys. Stream ciphers encrypt data as a bit stream. L2TP/IPSec connections use the Data Encryption Standard (DES), which is a block cipher that uses either a 56-bit key for DES or three 56-bit keys for 3DES. Block ciphers encrypt data in discrete blocks (64-bit blocks, in the case of DES). SSTP uses SSL with RC4 or AES. DirectAccess uses 3DES or AES.

Finally, PPTP connections require only user-level authentication through a PPP-based authentication protocol. L2TP/IPSec connections require the same user-level authentication as well as computer-level authentication using computer certificates. In contrast, SSTP and DirectAccess only require computer-level certificates for the VPN servers.

Table 1 compares some of the characteristics of the three tunneling protocols.

Tunneling Within a Windows Server 2008 R2 Networking Environment

For Layer 2 tunneling technologies, such as PPTP, L2TP, and SSTP, a tunnel is similar to a session; both of the tunnel endpoints must agree to the tunnel and must negotiate configuration variables, such as address assignment or encryption or compression parameters. In most cases, data transferred across the tunnel is sent using a datagram-based protocol. A tunnel maintenance protocol is used as the mechanism to manage the tunnel.

Layer 3 tunneling technologies generally assume that all the configuration settings are preconfigured, often by manual processes. For these protocols, there might be no tunnel maintenance phase. For Layer 2 protocols (PPTP, L2TP, and SSTP), however, a tunnel must be created, maintained, and then terminated.

After the tunnel is established, tunneled data can be sent. The tunnel client or server uses a tunnel data transfer protocol to prepare the data for transfer. For example, as illustrated in Figure 1, when the tunnel client sends a payload to the tunnel server, the tunnel client first appends a tunnel data transfer protocol header to the payload. The client then sends the resulting encapsulated payload across the internetwork, which routes it to the tunnel server. The tunnel server accepts the packets, removes the tunnel data transfer protocol header, and forwards the payload to the target network. Information sent between the tunnel server and tunnel client behaves similarly.

Figure 1. Tunneling payload through a VPN connection.

Point-to-Point Tunneling Protocol

The Point-to-Point Tunneling Protocol (PPTP) is a Layer 2 protocol that encapsulates PPP frames in IP datagrams for transmission over the Internet. PPTP can be used for remote access and router-to-router VPN connections. It uses a TCP connection for tunnel maintenance and a modified version of Generic Routing Encapsulation (GRE) to encapsulate PPP frames for tunneled data. The payloads of the encapsulated PPP frames can be encrypted and/or compressed. Figure 2 shows the structure of a PPTP packet containing user data.

Layer 2 Tunneling Protocol

Layer 2 Tunneling Protocol (L2TP) is a combination of the Point-to-Point Tunneling Protocol (PPTP) and Layer 2 Forwarding (L2F), a technology proposed by Cisco Systems, Inc. L2TP encapsulates PPP frames that are sent over IP, X.25, frame relay, and ATM networks. The payloads of encapsulated PPP frames can be encrypted and/or compressed. When sent over the Internet, L2TP frames are encapsulated as User Datagram Protocol (UDP) messages, as shown in Figure 3.

L2TP frames include L2TP connection maintenance messages and tunneled data. L2TP connection maintenance messages include only the L2TP header. L2TP tunneled data includes a PPP header and PPP payload. The PPP payload can be encrypted or compressed (or both) using standard PPP encryption and compression methods.

In Windows Server 2008 R2, L2TP connections do not negotiate the use of PPP encryption through Microsoft Point-to-Point Encryption (MPPE). Instead, encryption is provided through the use of the IP Security (IPSec) Encapsulating Security Payload (ESP) header and trailer.

IP Security

IP Security (IPSec) was designed as an end-to-end mechanism for ensuring data security in IP-based communications. Illustrated in Figure 4, the IPSec architecture includes an authentication header to verify data integrity and an encapsulation security payload for both data integrity and data encryption. IPSec provides two important functions that ensure confidentiality: data encryption and data integrity. IPSec uses an authentication header (AH) to provide source authentication and integrity without encryption and the Encapsulating Security Payload (ESP) to provide authentication and integrity along with encryption. With IPSec, only the sender and recipient know the security key. If the authentication data is valid, the recipient knows that the communication came from the sender and that it was not changed in transit.

Secure Socket Tunneling Protocol

Introduced in Windows Server 2008, SSTP was specifically developed to get around the difficulties of setting up VPN tunnels through corporate firewalls, which block many of the ports and protocols used by PPTP and L2TP. The SSTP tunnel uses the HTTP over SSL (HTTPS) protocol, which is widely supported for secure web traffic. SSTP uses port 443 for the connection.

The tunneling protocol functions by encapsulating the original IP packet with a PPP header and then an SSTP header. The SSTP header, the PPP header, and the original IP packet are all encrypted by the SSL session. Finally, an IP header is added to the packet and it is routed to the destination. The structure of the packet is shown in Figure 5.

Unfortunately, SSTP does not support tunneling through web proxies that require authentication. Another limitation of SSTP is that it does not support site-to-site connections in Windows Server 2008 R2, which both PPTP and L2TP do.