Practical lessons learned from working with Azure Virtual WAN

Working with Azure Virtual WAN, Mark knows how

For the last year I’ve been working with the relatively new Azure ‘umbrella-type’ networking solution: Virtual WAN. For those who are unfamiliar with it, it’s a managed service combining several connectivity aspects like hub-spoke topology, ExpressRoute, site-2-site & client VPN and Azure Firewall, making your life easier setting it up and managing it once it’s operational.

The last two years have seen Microsoft adding several functionalities and removing limitations of Virtual WAN. For example Azure Firewall became available in a Premium version, adding IDPS, Threat Intelligence, TLS inspection and URL & web categories filtering. Also the maximum number of BGP routes advertised over ExpressRoute was increased and source/destination NAT was made available on the VPN Gateway. All very practical and useful features for a maturing service. During this year however I found out several other important characteristics and limitations that can seriously impact your cloud network architecture, so I thought it wise to sum up and share these practical lessons learned. Hopefully it will help you choose between an architecture based on Virtual WAN, or more traditional Vnet hub-and-spoke.

Securing Hubs

The Virtual WAN Hub, the regional ‘container’ where all connectivity comes together, can be turned into a Secured Hub by enabling Azure Firewall. In addition to that, other Security Providers (Zscaler, iboss, CheckPoint) can be enabled within the hub, to provide their specific security services. Within Firewall Manager, the Security configuration setting can define on a connection level (Virtual Network, ExpressRoute, VPN) if internet or private traffic should be routed directly or has to pass through the Azure Firewall first. This way using Network and/or Application Rules, fine grained access control is possible.

Within Firewall Manager, instead of Azure Firewall, one of the other Security Providers can be selected for securing internet traffic. We encountered a situation where we wanted to secure internet access for Azure Virtual Desktops in one Vnet with Zscaler, and keep using Azure Firewall for securing internet traffic on all other connections. To our surprise this was not possible, when we wanted to choose Zscaler, it was selected for all Connections (Vnets, VPN’s, etc.). My recommendation here is that if you’re looking for a security integration like Zscaler or CheckPoint for part of your Azure workloads, do not go for the Hub integration but find another solution for securing specific workloads (there are several options).

One small AWS comparison

Every Cloud Service Provider has its unique strengths and features, that’s why I always hate these ‘which cloud is better’ posts. When services serve comparable purposes though, it can be useful to compare features to see which one or perhaps both will suit your architecture.

I’ve been working with AWS Transit Gateway for several customers, which fulfills approximately the same needs as Virtual WAN does; it bundles and simplifies your cloud network deployment, reducing the number of code rules and providing you with an overview. What Virtual WAN calls a Connection, each connected VPC, VPN or DirectConnect to the Transit Gateway is called an Attachment. A great feature of the Transit Gateway is that a custom route table can be created per Attachment, so this way networks can easily be isolated from each other. You don’t want a remote VPN connected site to connect to a Production VPC? Just don’t propagate the VPC routes to the VPN attached route table – simple as that; first basic security measure is in place.

In Virtual WAN however I was slightly disappointed to find out that for Vnet connections you can create and attach unique route tables, but all branch connections have to use the same route table. This forces you to start thinking of security measures in other places.

Branch-to-branch struggles

You may have a couple of remote offices that are connected to the Vwan Hub through a VPN connection, and those offices might also need to connect to each other. Another scenario is when an external party that’s connected through VPN not only needs to access some Azure services, but also services in the datacenter that’s attached via ExpressRoute. Fortunately for us there is one simple Vwan switch enabling this connectivity:

An important consideration however is that when you are using BGP on one or more of these branch connections, all known IP prefixes will be advertised, including those of the attached branch locations! So when you have multiple suppliers/partners connected through VPN and BGP, you are not only advertising your own Azure prefixes to those external parties, but also the prefixes of all other suppliers/partners!

Even when you are using static routes and not advertising any prefixes, traffic from a branch network with a destination of another known branch network, will be routed over that connection. Of course, the second external party would also have to have a static route defined to enable return traffic, still I would like to have better control from within the Vwan Hub which prefixes are advertised and routed. For now it’s also not possible to route traffic between branches through the Azure Firewall, that way traffic can be filtered where necessary, this is an upcoming feature.

Troubleshooting VPN’s

Elaborating a bit more on the branch-to-branch topic, the downside of a managed service like Virtual WAN is that you don’t have the detailed IPsec logging or insights you might need in some situations. For example when branch-to-branch between VPN’s isn’t working, how will you troubleshoot? Is traffic being received over one tunnel or blocked by the remote firewall? Is it being forwarded over the second? Am I seeing other errors that might cause miscommunication?

The Azure Portal has a Packet Capture feature for the VPN Gateway in Virtual WAN. Despite trying several options, I was only able to capture ESP and ISAKMP packets, not the TCP or UDP ‘inner’ traffic of the VPN. Fortunately there’s a simple PowerShell command you can use to achieve the desired result, that I would like to share.

Start PowerShell, if not already present install the required Azure modules:

install-module -name Az

Next connect to Azure and authenticate through the browser:

connect-azaccount

Set the packet trace parameters that you require, for example:

$a = "{`"TracingFlags`": 0,`"MaxPacketBufferSize`": 120,`"MaxFileSize`": 500,`"Filters`" :[{`"CaptureSingleDirectionTrafficOnly`": false}]}"

(note: the TracingFlags: 0 value ensures inside tunnel packets are captured)

Now start the packet capture, referring to the Resource Group and Name of your VPN Gateway, using the parameters defined earlier:

Start-AzVPNGatewayPacketCapture -ResourceGroupName "<rgname>" -Name "<vpngwname>" -FilterData $a

Perform your connectivity tests, and stop the packet capture:

Stop-AzVPNGatewayPacketCapture -ResourceGroupName "rgname" -Name "vpngwname"

You are now prompted for a SAS URL from a Storage Account Container, where the pcap file will be stored. Now download the pcap file from the Storage Account Container for analysis in for example WireShark, and voila there are your TCP and UDP connections for further investigation.

NVA integration

Contrary to Virtual Networks, the Virtual WAN Hub does not allow you to deploy any Network Virtual Appliance (NVA) you would like (routing, NGFW, IDS, SDwan, etc.). Only a select certified group of vendors have NVA’s available for deployment within a Hub (Cisco SDwan Cloud Onramp, Barracuda CloudGen, VMware SDwan, Versa Networks and Fortinet NGFW). If your preferred vendor or desired functionality is not in that list, you’ll have to deploy those NVA’s in a spoke Vnet, and connect that Vnet to the hub. In our situation it was for connecting SDwan branch offices.

The logical solution to this would be to use a Hub-Vnet Peering connection and create a User Defined Route in the Spoke Vnet pointing to the Azure Firewall, and for return traffic add a route to the Hub Default Route table with the NVA IP address as the next hop. It would look like something like this:

Unfortunately, combining the Azure Firewall and an NVA over a peered connection, and routing between these instances, does not work. It is not a Microsoft supported connectivity pattern. So we had to come up with something else.

We chose to deploy Virtual Network Gateways in the NVA Vnet, and create connections to the VPN Gateway in the Hub. It’s not ideal from a performance perspective, encrypting and decrypting traffic, sending it to the public Microsoft network and back again, however for SDwan connectivity in our situation the impact is very low, it might be different in your situation.

That’s it for now, I hope you found the insights interesting and helpful. Despite my critical remarks, Virtual WAN is still a great product. It has its characteristics though, which you need to be aware of when your Azure environment requires advanced networking features. Hopefully my experiences can help you in making the right decisions. Any feedback or additions are more than welcome!

Get in touch with Mark via our contact page or his LinkedIn.