Name resolution of Azure resources from on-prem/local environment using custom DNS server.
Context
While building a full-stack app on Azure, our team had deployed multiple private resources like Azure function apps, Azure Postgres database, Azure storage account.
Most of the resources we had were private to the VNET. It was easy to access these resources from a machine's (connected to Azure via P2S connection) CLI using IP addresses, but we felt the need to access those resources from the Azure portal's UI or using a resource's domain directly in the browser. And the problem was we were not able to resolve the DNS of the resources using the private DNS zones we had.
The crux of the problem
Most of the resources in Azure (function apps, storage accounts etc.) have a domain (and a public IP address) associated with it which is auto-assigned when we provision those resources. For example -
a function app with the name my-function-app-1 will have a domain name https://my-function-app-1.azurewebsites.net. This domain will also resolve to a public IP
> dig my-function-app-1.azurewebsites.net +short
20.49.104.17
Function apps' also have a different domain where we have access to the Kudu console for debugging purposes, and it resolves to the same public IP address as the function app.
> dig my-function-app-1.scm.azurewebsites.net +short
20.49.104.17
Kudu console accessible over public IP before enabling private endpoints
However, once we enable private endpoints on a function app we also get a private IP and our function app is not reachable over the public domain or IP. In the case of storage accounts with private endpoints enabled we can only access them if we whitelist our IP address.
Kudu console not accessible over public IP after enabling private endpoints
We need to find a way to resolve the domain to private IP for us to be able to access the private resources.
Things we are going to cover as part of this article
- How to resolve resources' domain to private IP using private DNS zone and a DNS forwarder?
- How to use that DNS forwarder to send queries over a P2S connection?
Solution
To access those resources over point-to-site without any whitelisting we have to use the private IP of that resource. Currently, the resources' domain is resolved to a public IP because our machine is using a public DNS server, for resolving it to private IPs we need to change the DNS server on our machine, for this, we are going to make use of DNS forwarder. We also have to make sure that we have respective DNS entries in our private DNS zone.
DNS forwarder
A DNS forwarder is a VM that acts as a DNS server and forwards our queries to 168.63.129.16 - which is Azure's internal DNS recursive resolver. The DNS forwarder VM may be attached with a public IP but since we don't want to expose our DNS server to the public, we will make this VM available over the point-to-site connection and will access it using private IP.
Deploying your DNS forwarder
The publicly available template deploys multiple resources which I didn't need for my setup as I already had a VNET and I also didn't need a public IP. So, I made a few changes to the ARM template to preset VNET and subnet configuration and remove public IP deployment.
You can find the configuration I used here -
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vmName": {
"type": "string",
"defaultValue": "dnsproxy",
"metadata": {
"description": "Name of the Virtual Machine."
}
},
"adminUsername": {
"type": "string",
"metadata": {
"description": "User name for the Virtual Machine."
}
},
"storageAccountName": {
"type": "string",
"metadata": {
"description": "The name of the storage account for diagnostics. Storage account names must be globally unique."
}
},
"forwardIP": {
"type": "string",
"defaultValue": "168.63.129.16",
"metadata": {
"description": "This is the IP address to forward DNS queries to. The default value represents Azure's internal DNS recursive resolvers."
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
},
"authenticationType": {
"type": "string",
"defaultValue": "sshPublicKey",
"allowedValues": [
"sshPublicKey",
"password"
],
"metadata": {
"description": "Type of authentication to use on the Virtual Machine. SSH key is recommended."
}
},
"adminPasswordOrKey": {
"type": "securestring",
"metadata": {
"description": "SSH Key or password for the Virtual Machine. SSH key is recommended."
}
},
"vmSize": {
"type": "string",
"metadata": {
"description": "Virtual machine size"
},
"defaultValue": "Standard_A1_v2"
},
"_artifactsLocation": {
"type": "string",
"defaultValue": "[deployment().properties.templatelink.uri]",
"metadata": {
"description": "The base URI where artifacts required by this template are located."
}
},
"_artifactsLocationSasToken": {
"type": "securestring",
"defaultValue": "",
"metadata": {
"description": "The sasToken required to access _artifactsLocation. When the template is deployed using the accompanying scripts, a sasToken will be automatically generated."
}
}
},
"variables": {
"ubuntuOSVersion": "18.04-LTS",
"asetName": "<Name of new availibility set which will be deployed as part of this template>",
"nsgName": "<Name of new NSG which will be deployed as part of this template>",
"vnetName": "<Name of the already existing VNET>",
"vnetAddressPrefix": "<VNET address prefix, value for this will look like - 10.50.0.0/24>",
"subNet1Name": "<Name of already existing subnet>",
"storType": "Standard_LRS",
"location": "[parameters('location')]",
"nicName": "[concat(parameters('vmName'), '-', 'nic')]",
"scriptUrl": "https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/demos/dns-forwarder/forwarderSetup.sh",
"linuxConfiguration": {
"disablePasswordAuthentication": true,
"ssh": {
"publicKeys": [
{
"path": "[concat('/home/', parameters('adminUsername'), '/.ssh/authorized_keys')]",
"keyData": "[parameters('adminPasswordOrKey')]"
}
]
}
}
},
"resources": [
{
"type": "Microsoft.Storage/StorageAccounts",
"comments": "Storage account for the VHD files for the VMs",
"name": "[parameters('storageAccountName')]",
"apiVersion": "2019-06-01",
"location": "[variables('location')]",
"sku": {
"name": "[variables('storType')]"
},
"kind": "StorageV2"
},
{
"type": "Microsoft.Compute/availabilitySets",
"comments": "availability set for creating a HA cluster, run the template multiple times to get multiple DNS servers",
"name": "[variables('asetName')]",
"apiVersion": "2019-12-01",
"location": "[variables('location')]",
"sku": {
"name": "Aligned"
},
"properties": {
"platformFaultDomainCount": 2,
"platformUpdateDomainCount": 2
}
},
{
"type": "Microsoft.Network/networkSecurityGroups",
"comments": "An NSG to prevent inbound traffic other than SSH, set sourceAddressPrefix to restrict access further or block all together (or remove the public ip) and ssh in from another vm",
"name": "[variables('nsgName')]",
"apiVersion": "2020-05-01",
"location": "[variables('location')]",
"properties": {
"securityRules": [
{
"name": "allow_ssh_in",
"properties": {
"description": "The only thing allowed is SSH",
"protocol": "Tcp",
"sourcePortRange": "*",
"destinationPortRange": "22",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "*",
"access": "Allow",
"priority": 100,
"direction": "Inbound"
}
}
]
}
},
{
"type": "Microsoft.Network/networkInterfaces",
"comments": "A single network interface on each DNS server",
"name": "[variables('nicName')]",
"apiVersion": "2020-05-01",
"location": "[variables('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/networkSecurityGroups/', variables('nsgName'))]"
],
"properties": {
"networkSecurityGroup": {
"id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName'))]"
},
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('vnetName'), variables('subnet1Name'))]"
}
}
}
]
}
},
{
"type": "Microsoft.Compute/virtualMachines",
"comments": "A stock Ubuntu server, a VM extension will add the DNS server to it later",
"name": "[parameters('vmName')]",
"apiVersion": "2019-12-01",
"location": "[variables('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/networkInterfaces/', variables('nicName'))]",
"[resourceId('Microsoft.Storage/StorageAccounts/', parameters('storageAccountName'))]",
"[resourceId('Microsoft.Compute/availabilitySets/', variables('asetName'))]"
],
"properties": {
"availabilitySet": {
"id": "[resourceId('Microsoft.Compute/availabilitySets', variables('asetName'))]"
},
"hardwareProfile": {
"vmSize": "[parameters('vmSize')]"
},
"osProfile": {
"computerName": "[parameters('vmName')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPasswordOrKey')]",
"linuxConfiguration": "[if(equals(parameters('authenticationType'), 'password'), json('null'), variables('linuxConfiguration'))]"
},
"storageProfile": {
"imageReference": {
"publisher": "Canonical",
"offer": "UbuntuServer",
"sku": "[variables('ubuntuOSVersion')]",
"version": "latest"
},
"osDisk": {
"caching": "ReadWrite",
"createOption": "FromImage"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]"
}
]
},
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true,
"storageUri": "[reference(resourceId('Microsoft.Storage/storageAccounts', toLower(parameters('storageAccountName')))).primaryEndpoints.blob]"
}
}
}
},
{
"type": "Microsoft.Compute/virtualMachines/extensions",
"comments": "The shell script to install Bind9 and setup the ACL and forwarders. If this step fails, check the logs in /var/log/waagent.log and /var/log/azure/* for details",
"name": "[concat(parameters('vmName'),'/setupdnsfirewall')]",
"apiVersion": "2019-12-01",
"location": "[variables('location')]",
"dependsOn": [
"[resourceId('Microsoft.Compute/virtualMachines/', parameters('vmName'))]"
],
"properties": {
"publisher": "Microsoft.Azure.Extensions",
"type": "CustomScript",
"typeHandlerVersion": "2.0",
"autoUpgradeMinorVersion": true,
"settings": {
"fileUris": [
"[variables('scriptUrl')]"
],
"commandToExecute": "[concat('sh forwarderSetup.sh',' ',parameters('forwardIP'),' ', variables('vnetAddressPrefix'))]"
}
}
}
]
}
After the deployment, you can try to SSH into this VM from your on-prem or local machine over P2S if your connection succeeds that means we are good to use this as our private DNS resolver.
Note - You may want to edit the allowed ACL goodclients in your DNS forwarder VM to allow point-to-site connections to use that as a DNS server. To do so ssh into your DNS forwarder and edit
/etc/bind/named.conf.optionsto add your point-to-site's CIDR range underacl goodclients.
sudo vi /etc/bind/named.conf.options
acl goodclients {
localhost;
localnets;
192.168.0.0/24;
};
...
Let's see this in action
This section assumes you have connected to your point-to-site already.
Whenever I try to connect to the machine over the public IP which is resolved by public DNS (1.1.1.1 in my case), I get 403. Carefully notice the IP address of the requested resource, it is a public IP.
Now let's change the DNS server to the forwarder's IP. Content of my /etc/resolv.conf looks like this -
# 10.56.4.1 is the IP of the DNS forwarder.
nameserver 10.56.4.1
After changing the DNS server and flushing the DNS cache I can access the private function app's Kudu console.
Similarly, we can also use this technique to access any private resource without the hassle of whitelisting. For instance, storage containers, key vaults. Just make sure to attach appropriate private endpoints to those resources.
© Yash Gupta.RSS