Good old VPN encryption: the most important and least understood component of any virtual private network.
Almost everyone has a basic understanding of what encryption is, but in case you’ve never heard the word, it’s the science of making information unreadable by anyone except the intended recipient. It’s one of the main applications for higher mathematics and a vital concept for living in our modern surveillance state.
The simplest encryption method is the one every child comes up with to conceal the personal information in their diary. Every character is replaced by another character, and only the author has the key that lists every substitution.
But what if you need to share the information among several people? What if an unauthorized stranger gets their hands on the key? How can you guarantee secure encryption to millions of users across the internet? And what does any of this have to do with your VPN? In this article, I’ve got the answers to all those questions and more.
What Is VPN Encryption?
If you’ve forgotten the purpose of a VPN, here’s a quick refresher. By connecting to a virtual private network of servers before moving onto the wider internet, you can hide your IP address and online activity from anybody who may be snooping.
The catch is that a VPN can’t just be a bunch of linked servers. I can connect to another server and use its IP address to browse instead of my own, but without encryption, it would be easy for any motivated interloper to link my activity back to my home device.
For a VPN service to work, it has to create something called an “encrypted tunnel.” That means your information is translated into code the moment it reaches the VPN server and isn’t decoded until it reaches its destination. It never leaves the tunnel.
Encryption is crucial, but it comes with trade-offs. Traffic encrypted by a VPN is slower, on top of the delay you already get from ping-ponging between servers. Engineering a VPN means constantly balancing performance with security.
How Does VPN Encryption Work?
Like everything else computers do, encryption is about starting with the proper set of instructions.
An encryption algorithm is a set of rules for scrambling data. A simple encryption algorithm might be something like “transpose every letter three steps forward in the alphabet.” You might also see this referred to as a cipher.
The other term you’ll need to know is key. A key is a set of data, separate from the encrypted data, that’s required to decode it. At least, it should be.
Have you ever solved a cryptogram puzzle? You start with a string of nonsense words, with each letter representing another. By analyzing how often the letters appear and in what patterns, you can figure out the code. A famous example of this process can be found in the Sherlock Holmes story “The Adventure of the Dancing Men.”
Cryptogram puzzles show us why encryption is such a deceptively simple task. Holmes, an unauthorized third party, is able to break the code without the key because he knows what encryption algorithm was used.
So here’s the question: how can we encode information so nobody but the recipient can read it, even if the interloper knows what algorithm we’re using?
Well, what if instead of one letter standing for one other letter, each pair of two letters stood for another pair of two letters? Now, instead of 26 possible keys, there are 676 possible keys (i.e. 262).
Already, it’s essentially impossible to break the code through Holmes-style frequency analysis. However, a hacker could still do a brute-force attack by writing a program to try all 676 solutions in a row.
So let’s bump it up even more and have each set of three letters stand for another set of three letters. Now there are 17,576 possible keys (i.e. 263). Our hacker’s life is getting a lot rougher, and you can theoretically go as high as you want to as long as you have enough room on the paper.
There’s an extra hurdle on the internet, though. Software doesn’t encode data as letters of the alphabet but as bits. Each bit has only two possible states: 0 or 1. If every string of three bits stands for three other bits, that’s only 23=8 keys. So how long does each key need to be in order to prevent hackers from brute-forcing it?
Three bits isn’t enough. Four isn’t even enough. But what about 256? That’s how long the keys are in AES-256 encryption, the official encryption algorithm of the United States Government. “AES-256” is the abbreviated name for Advanced Encryption Standard, 256-bit.
Any of the 256 bits can be either 0 or 1, resulting in 2256 = 1.16×1077 possible keys, or somewhere north of a trillion trillion trillion trillion trillion trillion trillion.
If you loaded all of Earth’s computers into a TARDIS, took them back in time to the Big Bang and had them start brute-forcing an AES 256-bit cipher, you might find the key sometime around the invention of TikTok.
There’s one more detail to understand, and it’s extremely important: key size is not the same as block size.
In other words, the key does not have to have the same number of bits as the block of data it represents. AES-256 encryption has a block size of 128 bits. That’s 2128 = 3.4×1038 possibilities for each block. It’s technically easier to brute-force a block than a key, but only in the way that it’s easier to swim to the moon than to Mars.
So why does this matter? Because enormous key sizes can be used as a form of security theater — putting people’s minds at ease by appearing secure without actually being so.
Blowfish, a symmetric encryption algorithm similar to AES-256 encryption, can have keys of up to 448 bits. But its maximum block size is 64 bits, small enough for modern computers to break. That’s probably why OpenVPN recently switched its default cipher from Blowfish to AES.
In fact, almost all VPNs nowadays use a protocol with AES-256 encryption built in, giving you the maximum protection currently available. A few choose other options, which I’ll explore more in “Encryption Protocols for VPN” section But first, I’d like to go over some of the ways the intended recipient can decode a modern encryption cipher.
Types of VPN Encryption
Nearly all VPN encryption is either symmetric or asymmetric. There may be other approaches, but you’ll be hard-pressed to find anybody using them outside of a university or a corporate R&D lab.
Symmetric encryption is also called single-key encryption because it uses the same key to both encrypt and decrypt the data.
Anything from basic decoder-ring substitution ciphers to the 256-bit hash from the previous section could count as symmetric encryption. No matter how complex it is, if you have the key, you can read the plaintext.
The single key is both the biggest strength and the main weakness of symmetric encryption. Single-key algorithms are simpler, so they can handle more data at higher speeds. But symmetric keys have to be exchanged securely, or the whole thing is pointless. Otherwise, hackers can get their hands on the key and crack the code.
AES is the most common single-key algorithm and one of the most popular encryption methods of any kind due to its adoption by the United States government. AES is followed by a number, either 128, 192 or 256, which denotes how many bits are in each key (see “How does VPN encryption work?”).
AES-256 encryption is the strongest AES algorithm. It’s often called “bank-grade” or “military-grade” security since it’s trusted by the U.S. government and most major banks.
The History of AES Encryption
To understand why AES is so important, it helps to know its origin story. Before AES, most encryption ran on the Data Encryption Standard (DES). DES only supported blocks of up to 64 bits and keys of up to 56 bits.
By 1997, computers were getting powerful enough to crack DES. The government, realizing it needed better security to keep up, held a contest to determine what encryption algorithm would be the new Advanced Encryption Standard. In 2000, the algorithm known as Rijndael was selected as the winner. We’ve called it AES ever since.
Several other algorithms made the finals, including RC6, which crypto nerds will tell you should have won. Do not engage with them.
Some symmetric algorithms are “stream ciphers.” Instead of encrypting whole blocks at once, these ciphers encode each bit based on the state of the cipher after encoding the previous bit. Theoretically, this makes them both faster and more secure, though the cryptanalysis of stream ciphers is still in its infancy. Examples include the Salsa algorithm and its successor, ChaCha.
The only other symmetric algorithm you’re likely to encounter is Blowfish, an alternative to AES that hasn’t yet been successfully cracked. It was formerly the default cipher for the OpenVPN protocol before new implementations switched over to AES.
Twofish, a newer version of Blowfish, is now recommended by Blowfish’s creator. It’s theoretically stronger but hasn’t received much attention from cryptographers yet.
Asymmetric encryption is also called public-key cryptography. Unlike symmetric encryption, it involves two keys: a public key and a private key. Let’s run through an example to understand how these two fit together. (As you read, keep in mind this is intentionally oversimplified.)
Imagine that John Doe, a secret agent, wants to send a message to his colleague, Jane Roe. The message is extremely sensitive, so John must be absolutely sure that only Jane will be able to read it.
Luckily, Jane gave John and everyone else at the Agency a set of instructions for how to pass her coded messages. Jane’s instructions go on for several pages. By the time he’s done running through them, John doesn’t know how he’d get the message back to the plaintext it started from.
He leaves the coded message for Jane at their usual dead drop. Back at her safe house, Jane decodes the message with a second set of instructions, one only she possesses.
The instructions Jane distributed to John and the other agents are her public key. She’s not broadcasting it to everyone, but it can be safely circulated to a few people she trusts.
The second set of instructions are Jane’s private key. It’s paired with her public key, but nobody else has access to it. As long as that’s true, Jane will be the only person who can decode messages that were encoded with her public key.
Why go to all this trouble when they could just use a single-key algorithm? Let’s see what happens when we introduce John and Jane’s rival agents, Ivan and Ivana.
The next time John leaves a message for Jane, Ivan watches and intercepts it. Of course, he can’t read it because it’s encrypted. But then Ivana shows up. She’s just stolen a copy of the instructions Jane handed around (the public key) and thinks they’ll be able to use it to read the secret message.
Unfortunately for them, they still can’t because they don’t have the private key. Without that, not even John can read the messages he encodes with Jane’s public key. They’re for her eyes only.
Replace all the pen-and-paper spycraft with advanced algorithms, and you’ll understand why asymmetric encryption is more secure than symmetric encryption. Each public key matches with a single private key to form a unique pair. There’s no need to secure public keys and no reason to risk exposing private keys.
You may have figured out the drawback already: all of this takes a lot longer. Large blocks of information are harder to encrypt asymmetrically, so they’ll transmit more slowly.
The RSA Algorithm
The best-known example of asymmetric encryption, and the one most relevant to VPNs, is the Rivest-Shamir-Adleman (RSA) algorithm, named after the three people who invented it in 1977.
RSA is based on prime factorization — the mathematical process of finding two prime numbers that multiply to make a given whole. It’s easy to multiply a pair of two-digit primes but much harder to factor them. With pen and paper, most people could multiply 37 and 59 in under a minute. But if asked to find the prime factors of 2,183, we’d probably resort to guesswork.
To once again oversimplify: in RSA, multiplying two primes is the public key that encodes the plaintext. Factoring the product of those two primes is the private key that decodes it. As in our secret agent example, decoding should always be much harder than encoding.
Over the years, RSA has been used for financial transactions, encrypted communication and many other applications that don’t transmit too much data at once. More than 40 years after its debut, RSA seems to be nearing senescence, but there’s one scenario in which it’s still vital: making secure connections with servers, including VPN servers.
Think back one more time to our secret agents. Only Jane can decode messages encrypted with her public key. From there, John can reason that if he gets an accurate reply to his message, it must have come from Jane.
Jane’s private key means that she and John have a secure communication channel, especially if he has a private key of his own. If Ivan and Ivana try to hijack the channel and send their own message, it will be obvious that Jane didn’t write it.
If two servers are each assigned a unique public-private key pair, they can communicate securely. That secure tunnel can be used to send a symmetric key without fear of interception, so the two servers can exchange information via faster single-key encryption.
This combination was first implemented as Secure Sockets Layer (SSL) and later refined into Transport Layer Security (TLS).
VPN Encryption Protocols
Now it’s time for me to explain VPN encryption protocols. After you connect to a VPN server, the VPN needs to communicate with the destination server on your behalf. Most destination servers are totally alien to the VPN and probably aren’t set up for privacy and security.
To keep your connection encrypted, the VPN uses a protocol — something like a “common language” in which it can “talk” to the destination server. At the heart of almost every VPN protocol, there’s an encryption algorithm.
Most VPN protocols use the combination of symmetric and asymmetric encryption we learned about at the end of the last section. Asymmetric encryption secures your connection to a server, while symmetric encryption encodes your data itself.
Here are a few of the most common VPN protocols currently available.
OpenVPN is an open-source VPN protocol that’s generally considered to be extremely secure and reliable. A veritable army of volunteers has spent over 15 years probing all its possible weak points.
For encryption, OpenVPN uses a library called OpenSSL that gives it access to every encryption cipher. It employs a version of TLS, using asymmetric encryption to establish a tunnel through which it transmits a symmetric encryption key. OpenVPN’s symmetric encryption uses AES-256 encryption.
The other great benefit of OpenVPN is its flexibility. It’s versatile enough to implement on any OS and platform, from desktop computers to smartphones and routers.
WireGuard is another open-source protocol that aims to update and replace OpenVPN. Almost everyone likes OpenVPN, but it does have a few issues, mainly with the massive size of its code. It’s highly secure and pretty fast. Still, it could be faster, and its thousands of lines of code hide a lot of potential vulnerabilities.
WireGuard is OpenVPN with less code and more gas. Instead of AES-256 encryption, it uses ChaCha20 for encryption. ChaCha20 is a stream cipher (see “Symmetric Encryption”) and helps WireGuard operate faster than OpenVPN.
The downside is that ChaCha20 has not been analyzed for vulnerabilities to nearly the same degree that AES-256 encryption has. Using WireGuard still carries a few risks. Many VPNs have accounted for these risks by building their own in-house protocols on the WireGuard framework. For example, NordVPN’s NordLynx is essentially WireGuard with an extra layer of obfuscation.
IKEv2 stands for Internet Key Exchange version two. It was developed by Microsoft and Cisco to be the fastest VPN protocol on the market, and it still is for the most part. Unlike OpenVPN and WireGuard, it’s not open-source.
Encryption through IKEv2 is based on IPsec, an alternative to TLS that was partly developed by the National Security Agency (NSA) in the United States. The NSA is basically the Emperor Palpatine of cryptography, so many people are understandably jumpy about IKEv2. Still, it is very fast.
L2TP/IPSec stands for Layer 2 Tunneling Protocol encrypted via IPsec. L2TP has no built-in encryption; it’s only used for “tunneling” or helping servers talk to each other. Its encryption comes from IPsec, with all the associated concerns (see IKEv2 above or just Ctrl+F for “Emperor Palpatine”).
SSTP stands for Secure Sockets Tunneling Protocol. It’s another protocol owned and operated by Microsoft. Like OpenVPN, SSTP uses SSL/TLS for encryption. But unlike OpenVPN, it’s inflexible, proprietary, and prone to delays. It’s not used very often these days.
Point-to-Point Tunneling Protocol
PPTPstands for Point-to-Point Tunneling Protocol. It’s extremely fast, but that’s because it communicates without any encryption whatsoever. Calling it “fast” is like calling a car “fuel-efficient” because it has no seat belts, airbags, cushions or doors. If a VPN supports PPTP, I instantly hate it a little bit.
Proprietary Encryption Protocols
Several VPNs have recently developed their own in-house protocols that can be used only on that VPN’s app. This typically involves taking an open-source protocol, such as WireGuard, and adapting it to make it faster, more secure or (hopefully) both.
A few examples:
- NordLynx is NordVPN’s exclusive protocol. Based on WireGuard, it uses the ChaCha20 stream cipher for encryption, plus some extra security features.
- Lightway is a protocol developed by ExpressVPN and released as open-source on GitHub. Instead of being based on an existing protocol, Lightway was built from scratch. For encryption, it uses wolfSSL, a library that gives it access to both ChaCha20 and AES-256 encryption.
- Chameleon was developed by VyprVPN explicitly for use in China. It builds on OpenVPN and uses that protocol’s standard AES-256 encryption.
Which VPNs Have the Best Encryption?
Right now, OpenVPN is the most trustworthy encryption protocol. It’s the perfect balance of speed and battle-hardened strength. Any VPN that supports OpenVPN is a winner in the encryption game. This includes my perennial favorites: ExpressVPN, NordVPN, Surfshark and Windscribe. You can read more about them in my Best VPN article.
The Future of VPN Encryption
The next great challenge to encryption is the rise of quantum computers. If I explained quantum computing in its entirety, this article would be twice as long, and I’d probably get it wrong. So let’s just say it means a massive jump in processing power.
Cryptanalysts have expressed concern about the possibility of hackers stealing encrypted data and hoarding it until they have access to quantum computers that are capable of breaking it. To prepare for a quantum future, a new field called post-quantum cryptography is on the rise.
The most secure encryption algorithms available today are based on difficult math problems like prime factorization. To beat quantum computers, cryptographers have to find an even harder math problem.
With several promising options, we can rest assured the battle between VPNs and hackers will continue for a long time to come.
Conclusion: VPN Encryption
VPN encryption is a complicated subject, but applying it in the real world isn’t so complex.
When you’re shopping for a VPN, you’ll see a lot about encryption. It’s bank-grade! It’s military-grade! Our encryption can slam a revolving door! Now that you know what terms like “AES-256” and “TLS” actually mean, you’re much better equipped to make the right decision.
Do you have any facts about encryption you’d like our readers to know? How about a favorite cipher I didn’t mention? Which of the VPN encryption protocols do you usually use? Sound off in the comments, and don’t forget to share this article around if it helped you out. Thanks for reading!