Ga naar inhoud

Deployment

The deployment of the app is configured in a DevOps pipeline, the configuration file of this pipeline can be found in azure-pipelines/azure-pipelines-deploy.yml. The pipeline builds one image containing all elements of the application (based on docker/app/Dockerfile) and pushes it to an Azure Container Registry (ACR). This pipeline is automatically triggered when a PR is merged into the master branch.

Azure setup

The Azure setup is split into two parts, one part that hosts web services (API/Portal) and one part that hosts the database and Fabric capacity.

Database + Fabric Capacity

Prerequisites:

  • Microsoft Entra ID user (from now on 'user_x') with a Power BI Pro license and the following roles:
    • Fabric Administrator
  • An Azure subscription on which user_x has the owner role and a resource group within this subscription;
  • The following Resource Providers enabled on the subscription:
    • Microsoft.Fabric;
    • Microsoft.PowerPlatform

Service Principal

  • Create an Enterprise Application in Microsoft Entra ID;
  • Create a secret for the application and saves it somewhere together with the TenantID and ClientID of the application.

Virtual network

  • Create an Azure Virtual Network;
  • Create a subnet called 'PowerBI' and delegate it to the service 'Microsoft.PowerPlatform/vnetaccesslinks'.

Database

  • Create an Azure SQL Server resource with SQL login and save the username and password somewhere;
  • Create an Azure Managed Identity and assign it to the create Azure SQL Server resource;
  • Create a Private Endpoint for the SQL Server resource in the networking tab and allow your IP to connect;
  • Create an Azure SQL Database resource inside the Azure SQL Server resource;
  • Connect to the SQL Database using the SQL Server credentials and run the following script to create a user that can only access the database and not the entire server.
CREATE USER metriek_db WITH PASSWORD = '<YOUR_PASSWORD_HERE>';
ALTER ROLE db_owner ADD MEMBER metriek_db;

Storage account

  • Create an Azure Storage Account;
  • Give the Service Principal the following roles:
    • Storage Blob Data Contributor;
    • Storage Blob Delegator
  • Give the Azure Managed Identity that is assigned to the Azure SQL Server the following roles:
    • Storage Blob Data Reader
  • In the networking tab enable 'Allow Azure services on the trusted services list to access this storage account.'.

Fabric Capacity

  • Create a Fabric Capacity;
  • Connect a Power BI workspace to the capacity;
  • Give the Service Principal the following roles:
    • Contributor
  • Go to 'Admin portal' in the Power BI Service and look for 'Tenant settings' -> 'Developer settings' and enable service principal access to API's;
  • Follow this guide to give the capacity access to the database.

Webservices

Virtual Network

  • Create a Virtual Network with default IP range 10.0.0.0/24 and add the following subnets
Name IP Range Delegated To
appservices 10.0.0.2/24 Microsoft.Web/serverFarms
workers 10.0.0.3/24 Microsoft.ContainerInstance/containerGroups

Container Registry (ACR)

Used to store any Docker images created in the build pipeline in Azure DevOps.

Storage Account

  • Create a Storage account and allow all subnets in the created Virtual Network to access it.
  • Create a blob storage called envs. This will be used to store .env files.
  • Create a file storage calles logs. This will be used by containers to write logs to a central file.

App Service

  • Create a App Service Plan and add the following App Services to it.
Name Image Env Path mappings (mounts)
app baa-bi:master ENV=prod
ROLE=app		 /envs -> The envs blob storage
/logs -> The logs file storage
workers baa-bi:master ENV=prod
ROLE=worker
WORKERS=3
ENABLE_WORKER_HTTP=true		 /envs -> The envs blob storage
/logs -> The logs file storage
scheduler baa-bi:master ENV=prod
ROLE=scheduler
ENABLE_WORKER_HTTP=true		 /envs -> The envs blob storage
/logs -> The logs file storage
Optional: flower baa-bi:master ENV=prod
ROLE=flower		 /envs -> The envs blob storage
  • Add all App Services to the appservices subnet in the Virtual Network.

Redis Cache

The Redis Cache forms the backbone for all communication between 'controller' processes (API/CLI) and workers.

Container Instance

  • Create a Container Instance with the following config:
  • Image: baa-bi:master
  • ENVS:
    • ROLE: worker
    • ENV: prod
    • AUTO_SHUTDOWN: 10
    • WORKERS: 10 (depending on the selected resources)
  • VNET subset: workers

Webservices -> Database

Only needed when the database is in a different tenant/subscription than the webservices.

To allow the webservices to access the database there has to be an Azure Private Link between the webservices subscription and the Azure SQL Server.

  • Copy the Resource ID of the SQL Server resource, this looks like: /subscriptions//resourceGroups//providers/Microsoft.Sql/servers/;
  • Switch to the Webservices subscription and create an Azure Private Endpoint, in the 'Resource' tab select 'Connect to an Azure resource by resource ID or alias' and paste the Resource ID, then enter 'sqlServer' in the 'Target sub-resource' field, in the 'Virtual Network' select the vnet that is connected to the VM that needs access to the database;
  • After the Private Endpoint is created, it needs to be approved by the database. Switch to the database subscription and navigate to the 'Networking' tab of the SQL Server resource. In the 'Private Access' tab the created link should appear with status 'Pending'. Approve it;
  • Switch back to the webservices subscription and navigate to the 'DNS configuration' page of the Azure Private Endpoint. Add a configuration and select 'privatelink.database.windows.net' as DNS zone, create it if it doesn't exist yet.

Webservices -> Database storage account

Only needed when the database is in a different tenant/subscription than the webservices.

To allow the webservices to access the storage account the database uses to insert bulk data there has to be an Azure Private Link between the webservices subscription and the Azure Storage Account.

  • Copy the Resource ID of the Storage Account resource, this looks like: /subscriptions//resourceGroups//providers/Microsoft.Storage/storageAccounts/;
  • Switch to the Webservices subscription and create an Azure Private Endpoint, in the 'Resource' tab select 'Connect to an Azure resource by resource ID or alias' and paste the Resource ID, then enter 'blob' in the 'Target sub-resource' field, in the 'Virtual Network' select the vnet that is connected to the VM that needs access to the database;
  • After the Private Endpoint is created, it needs to be approved by the storage account. Switch to the storage account subscription and navigate to the 'Networking' tab of the resource. In the 'Private Access' tab the created link should appear with status 'Pending'. Approve it;
  • Switch back to the webservices subscription and navigate to the 'DNS configuration' page of the Azure Private Endpoint. Add a configuration and select 'privatelink.blob.core.windows.net' as DNS zone, create it if it doesn't exist yet.