Original Post Read More
Many customers require Web Applications & APIs to only be accessible via a private IP address with a Web Application Firewall on the internet edge, to protect from common exploits and vulnerabilities. Azure Front Door provides global routing and WAF capabilities to satisfy this requirement.
Azure Container Apps ingress can be exposed on either a Public or Private IP address. One option is to put Azure Front Door in front of an ACA public endpoint, but currently there is no way (other than in application code) to restrict access to the ACA public IP address from a single Azure Front Door instance. Azure App Service Access restrictions supports this scenario, but unfortunately, there is currently no equivalent access restriction for Azure Container Apps.
To work around this limitation, Azure Private Link Service can be provisioned in front of an internal ACA load balancer. A Private endpoint (NIC with private IP in a virtual network) is connected to the Private Link Service and an Azure Front Door Premium SKU instance can then be used to connect to the private endpoint (known as a Private Origin in AFD). This configuration removes the need to inspect the value of the “X-Azure-FDID” header sent from AFD since only a single AFD instance is connected to the private endpoint, guaranteeing traffic to the ACA environment occurs only from that specific AFD instance. The overall architecture is captured in the diagram below.
In order to create this architecture, we will cover the high-level steps outlined below.
1. Deploy an internal Azure Container App environment
2. Create an Azure Front Door Premium instance, origin group & route
3. Create an Azure Private Link Service (PLS) instance
4. Deploy an Azure Container App instance
5. Finally, approve the private endpoint connection to PLS
All steps above have been codified into an Azure Bicep deployment and shell script. To deploy the sample, you will need an Azure subscription and Bash or PowerShell console with the Az CLI installed. The Bicep templates and scripts referenced in this article are available on my GitHub, here.
First, let’s review the Bash shell script used to deploy the Bicep template. I also included a PowerShell script in my GitHub repo, which is almost identical, if that’s your preferred shell.
#!/bin/bash
LOCATION=’australiaeast’
PREFIX=’frontdoor’
RG_NAME=”${PREFIX}-aca-rg”
# create resource group
az group create –location $LOCATION –name $RG_NAME
# deploy infrastructure
az deployment group create
–resource-group $RG_NAME
–name ‘infra-deployment’
–template-file ./main.bicep
–parameters location=$LOCATION
–parameters prefix=$PREFIX
# get deployment template outputs
PLS_NAME=`az deployment group show –resource-group $RG_NAME –name ‘infra-deployment’ –query properties.outputs.privateLinkServiceName.value –output tsv`
AFD_FQDN=`az deployment group show –resource-group $RG_NAME –name ‘infra-deployment’ –query properties.outputs.afdFqdn.value –output tsv`
PEC_ID=`az network private-endpoint-connection list -g $RG_NAME -n $PLS_NAME –type Microsoft.Network/privateLinkServices –query [0].id –output tsv`
# approve private endpoint connection
echo “approving private endpoint connection ID: ‘$PEC_ID'”
az network private-endpoint-connection approve -g $RG_NAME -n $PLS_NAME –id $PEC_ID –description “Approved”
# test AFD endpoint
curl https://$AFD_FQDN
The script first defines 3 environment variables used throughout the script – LOCATION, PREFIX & RG_NAME. Modify these as you see fit for your environment. A resource group is created using a reference to the $RG_NAME variable, then on line 11, the Bicep template is deployed to the resource group.
Once the deployment has completed, 3 deployment template outputs are collected and used as input to the private endpoint approval command on line 25.
Let’s break down the resources the deployed by the template.
Input parameters are defined to control the deployment location, prefix and container image uri.
param location string = ‘australiaeast’
param prefix string = ‘contoso’
param imageName string = ‘mcr.microsoft.com/azuredocs/containerapps-helloworld:latest’
var suffix = uniqueString(resourceGroup().id)
var vnetName = ‘${prefix}-vnet-${suffix}’
var frontDoorName = ‘${prefix}-afd-${suffix}’
var wafPolicyName = ‘${prefix}wafpolicy’
var workspaceName = ‘${prefix}-wks-${suffix}’
var appName = ‘${prefix}-app-${suffix}’
var plsNicName = ‘${prefix}-pls-nic-${suffix}’
var plsName = ‘${prefix}-pls-${suffix}’
var appEnvironmentName = ‘${prefix}-env-${suffix}’
var originName = ‘${prefix}-origin-${suffix}’
var originGroupName = ‘${prefix}-origin-group-${suffix}’
var afdEndpointName = ‘${prefix}-afd-ep-${suffix}’
var loadBalancerName = ‘kubernetes-internal’
var defaultDomainArr = split(appEnvironment.properties.defaultDomain, ‘.’)
var appEnvironmentResourceGroupName = ‘mc_${defaultDomainArr[0]}-rg_${defaultDomainArr[0]}_${defaultDomainArr[1]}’
A virtual network with two subnets is created. The ‘infrastructure-subnet’ is used by the ACA environment to host an internal Azure load balancer and the ‘privatelinkservice-subnet’ is used to host the Azure Private Link Service.
resource vnet ‘Microsoft.Network/virtualNetworks@2022-07-01’ = {
name: vnetName
location: location
properties: {
addressSpace: {
addressPrefixes: [
‘10.0.0.0/16’
]
}
subnets: [
{
name: ‘infrastructure-subnet’
properties: {
addressPrefix: ‘10.0.0.0/23’
delegations: []
privateEndpointNetworkPolicies: ‘Disabled’
privateLinkServiceNetworkPolicies: ‘Enabled’
}
}
{
name: ‘privatelinkservice-subnet’
properties: {
addressPrefix: ‘10.0.2.0/28’
delegations: []
privateEndpointNetworkPolicies: ‘Disabled’
privateLinkServiceNetworkPolicies: ‘Disabled’
}
}
]
virtualNetworkPeerings: []
enableDdosProtection: false
}
}
A log analytics workspace is created to host the ACA application & System logs
resource wks ‘Microsoft.OperationalInsights/workspaces@2021-12-01-preview’ = {
name: workspaceName
location: location
properties: {
sku: {
name: ‘pergb2018’
}
retentionInDays: 30
features: {
enableLogAccessUsingOnlyResourcePermissions: true
}
workspaceCapping: {
dailyQuotaGb: -1
}
publicNetworkAccessForIngestion: ‘Enabled’
publicNetworkAccessForQuery: ‘Enabled’
}
}
Next, an Azure Container App environment is deployed. Notice the ‘properties.appLogsConfiguration’ and ‘properties.vnetConfiguration’ sections where the Log Analytics workspace and infrastructure subnet are specified, respectively.
resource appEnvironment ‘Microsoft.App/managedEnvironments@2022-06-01-preview’ = {
name: appEnvironmentName
location: location
sku: {
name: ‘Consumption’
}
properties: {
vnetConfiguration: {
internal: true
infrastructureSubnetId: vnet.properties.subnets[0].id
dockerBridgeCidr: ‘10.2.0.1/16’
platformReservedCidr: ‘10.1.0.0/16’
platformReservedDnsIP: ‘10.1.0.2’
outboundSettings: {
outBoundType: ‘LoadBalancer’
}
}
appLogsConfiguration: {
destination: ‘log-analytics’
logAnalyticsConfiguration: {
customerId: wks.properties.customerId
sharedKey: listKeys(wks.id, wks.apiVersion).primarySharedKey
}
}
zoneRedundant: false
}
}
An Azure Container App is then provisioned into the ACA environment. In this example we are deploying the ACA HelloWorld application from the Microsoft Container Registry (mcr.microsoft.com/azuredocs/containerapps-helloworld:latest).
resource containerApp ‘Microsoft.App/containerApps@2022-06-01-preview’ = {
name: appName
location: location
identity: {
type: ‘None’
}
properties: {
managedEnvironmentId: appEnvironment.id
configuration: {
activeRevisionsMode: ‘Single’
ingress: {
external: true
targetPort: 80
exposedPort: 0
transport: ‘Auto’
traffic: [
{
weight: 100
latestRevision: true
}
]
allowInsecure: false
}
}
template: {
containers: [
{
image: imageName
name: appName
resources: {
cpu: ‘0.25’
memory: ‘0.5Gi’
}
}
]
scale: {
maxReplicas: 10
}
}
}
}
An Azure Private Link Service is deployed using a separate Bicep module file (./modules/pls.bicep). Notice that the ‘appEnvironmentResourceGroupName’ parameter expects the ‘MC_’ prefixed resource group name that’s automatically created when a custom virtual network is specified at ACA environment deployment time.
The PLS deployment will also create a NIC in the ‘privatelinkservice-subnet’, to which the Azure Front Door backend will connect later in the deployment.
param name string
param location string
param appEnvironmentResourceGroupName string
param loadBalancerName string
param subnetId string
resource loadBalancer ‘Microsoft.Network/loadBalancers@2022-07-01’ existing = {
name: loadBalancerName
scope: resourceGroup(appEnvironmentResourceGroupName)
}
resource privateLinkService ‘Microsoft.Network/privateLinkServices@2022-07-01’ = {
name: name
location: location
properties: {
autoApproval: {
subscriptions: [
subscription().subscriptionId
]
}
visibility: {
subscriptions: [
subscription().subscriptionId
]
}
fqdns: []
enableProxyProtocol: false
loadBalancerFrontendIpConfigurations: [
{
id: loadBalancer.properties.frontendIPConfigurations[0].id
}
]
ipConfigurations: [
{
name: ‘ipconfig-0’
properties: {
privateIPAllocationMethod: ‘Dynamic’
subnet: {
id: subnetId
}
primary: true
privateIPAddressVersion: ‘IPv4’
}
}
]
}
}
output id string = privateLinkService.id
output name string = privateLinkService.name
The Azure Front Door Premium instance and it’s dependent Endpoint, Origin, Origin Group and Route resources are now created.
Endpoint – defines a new publicly accessible Global AFD endpoint
Origin Group – defines the AFD load balancing and health probe settings
Origin – associates the ACA container app ingress hostname & header with the Azure Private Link Service
Route – binds the Endpoint to the Origin Group
resource frontDoor ‘Microsoft.Cdn/profiles@2022-11-01-preview’ = {
name: frontDoorName
location: ‘Global’
sku: {
name: ‘Premium_AzureFrontDoor’
}
properties: {
originResponseTimeoutSeconds: 30
extendedProperties: {
}
}
}
resource afdOriginGroup ‘Microsoft.Cdn/profiles/origingroups@2022-11-01-preview’ = {
parent: frontDoor
name: originGroupName
properties: {
loadBalancingSettings: {
sampleSize: 4
successfulSamplesRequired: 3
additionalLatencyInMilliseconds: 50
}
healthProbeSettings: {
probePath: ‘/’
probeRequestType: ‘GET’
probeProtocol: ‘Https’
probeIntervalInSeconds: 60
}
sessionAffinityState: ‘Disabled’
}
}
resource afdEndpoint ‘Microsoft.Cdn/profiles/afdendpoints@2022-11-01-preview’ = {
parent: frontDoor
name: afdEndpointName
location: ‘Global’
properties: {
autoGeneratedDomainNameLabelScope: ‘TenantReuse’
enabledState: ‘Enabled’
}
}
resource afdRoute ‘Microsoft.Cdn/profiles/afdendpoints/routes@2022-11-01-preview’ = {
parent: afdEndpoint
name: ‘route’
properties: {
customDomains: []
originGroup: {
id: afdOriginGroup.id
}
originPath: ‘/’
ruleSets: []
supportedProtocols: [
‘Http’
‘Https’
]
patternsToMatch: [
‘/*’
]
forwardingProtocol: ‘MatchRequest’
linkToDefaultDomain: ‘Enabled’
httpsRedirect: ‘Enabled’
enabledState: ‘Enabled’
}
dependsOn: [
afdOrigin
]
}
resource afdOrigin ‘Microsoft.Cdn/profiles/origingroups/origins@2022-11-01-preview’ = {
parent: afdOriginGroup
name: originName
properties: {
hostName: containerApp.properties.configuration.ingress.fqdn
httpPort: 80
httpsPort: 443
originHostHeader: containerApp.properties.configuration.ingress.fqdn
priority: 1
weight: 1000
enabledState: ‘Enabled’
sharedPrivateLinkResource: {
privateLink: {
id: privateLinkService.outputs.id
}
privateLinkLocation: location
status: ‘Approved’
requestMessage: ‘Please approve this request to allow Front Door to access the container app’
}
enforceCertificateNameCheck: true
}
}
Finally, we define a WAF policy and associate it with the AFD Endpoint.
resource wafPolicy ‘Microsoft.Network/FrontDoorWebApplicationFirewallPolicies@2022-05-01’ = {
name: wafPolicyName
location: ‘Global’
sku: {
name: ‘Premium_AzureFrontDoor’
}
properties: {
policySettings: {
enabledState: ‘Enabled’
mode: ‘Prevention’
requestBodyCheck: ‘Enabled’
}
managedRules: {
managedRuleSets: [
{
ruleSetType: ‘Microsoft_DefaultRuleSet’
ruleSetVersion: ‘1.1’
ruleGroupOverrides: []
exclusions: []
}
{
ruleSetType: ‘Microsoft_BotManagerRuleSet’
ruleSetVersion: ‘1.0’
ruleGroupOverrides: []
exclusions: []
}
]
}
}
}
resource afdSecurityPolicy ‘Microsoft.Cdn/profiles/securitypolicies@2022-11-01-preview’ = {
parent: frontDoor
name: ‘${prefix}-default-security-policy’
properties: {
parameters: {
wafPolicy: {
id: wafPolicy.id
}
associations: [
{
domains: [
{
id: afdEndpoint.id
}
]
patternsToMatch: [
‘/*’
]
}
]
type: ‘WebApplicationFirewall’
}
}
}
One the template has deployed successfully, the 3 template output parameters are collected and used as input to the ”az network private-endpoint-connection approve” Az CLI command to approve the Private Endpoint connection to the Private Link Service, on line 25.
Once approved, you will be able to access the Azure Container app via a browser using the AFD endpoint URL saved in the AFD_FQDN environment variable.
$ echo https://$AFD_FQDN
https://frontdoor-afd-ep-rczv4qasdrrms-akcehrf2dncngxfa.z01.azurefd.net