Skip to main content
warning

This module is an alpha integration and still work in progress!!

Microsoft Azure Setup

Architecture​

Cloud ex Machina leverages a three-layer stack described on the below diagram:

Vocabulary and preliminary notions​

The following definitions apply to the rest of this document:

  • Tenant: CXM Labs integrates at the tenant level. If you own multiple tenants, you will need to run this set up once for each of them.
  • Applications: Access to the assets, audit logs and billing exports is managed via AD Applications. You can decide to set up a single app, or three different applications depending of you level of control and breakdown of services you want. You can also re-use existing applications if you already have such integrations in place or decide to create them manually.
  • Components:
    • Asset Crawler: tracks all assets deployed in the environment and maintains a collection of the assets. The Asset Crawler can either be deployed in each subscription, or you can decide to use a Management Group instead.
    • Billing Export Crawler: tracks usage and cost of assets in the target cloud
    • Audit Log Crawler: tracks activity in the cloud that modifies assets

Asset Crawler​

The Asset Crawler provides an asset-level inventory of your cloud.

It requires access permissions installed in the Tenant and all subscriptions underneath it for the each Azure Tenant you have under management.

  • AD Application: Provides Read Only and Monitoring access across the decided scope

Billing Export Crawler​

The Billing Export Crawler aims at getting an accurate understanding of your Azure bills.

It is installed in the Billing Export Storage Account the Azure Tenant that you have under management and composed of

  • AD Application: authorizes reading files in the storage account that contains billing exports
  • Notifications: callback to Cloud ex Machina to notify of the availability of new export files

Cloud Audit Log Crawler​

The Cloud Audit Log Crawler helps understand historical and dynamic aspects of your cloud.

It is installed in the Azure Logging Storage Account and contains

  • AD Application: authorizes reading files in the storage account that contains cloud audit logs
  • Notifications: callback to Cloud ex Machina to notify of the availability of new export files

Setup​

Pre-requisites​

Required Software​

You will need the following dependencies to initialize Cloud ex Machina:

Deployment Configuration​

make sure to collect the following information:

  • Mandatory from your Azure Environment:
    • aws_notification_endpoints: URLs of endpoints for webhook notifications
  • Optional from your GCP Environment:
    • storage_account_name: name of the storage account(s) for billing exports and cloud audit logs.
    • application_id: For each application it is possible to reuse an existing app to avoid overpermissioning each of them

In addition, assemble credentials to configure Terraform providers for your Azure Tenant and subscriptions

Unpack Modules​

Unzip the artifact that Cloud ex Machina provided to you that contains all modules. Create a directory to host the configuration information with:

export CXM_ROOT="/path/to/cxm/root_folder"
mkdir -p ${CXM_ROOT}/modules ${CXM_ROOT}/config/azure
mv cxm-artifact.zip ${CXM_ROOT}/modules
pushd ${CXM_ROOT}/modules
unzip cxm-artifact.zip && rm -f cxm-artifact.zip
popd
pushd ${CXM_ROOT}/config/azure

Configure Terraform / OpenTofu​

Prepare a new Terraform configuration file main.tf that contains the following sections :

State Backend​

Configure this the way you usually do it.

Providers​

# Provider for the Azure Tenant
provider "azurerm" {
  features {}
}

# This alias is to deploy the Audit Log component
provider "azurerm" {
  alias = "audit_log"
  features {}
}

# This alias is to deploy the Billing component
provider "azurerm" {
  alias = "billing"
  features {}
}

Module Configuration​

# Enable the tenant
module "enable_tenant" {
  source = "../../modules/terraform-azure-enablement"
  providers = {
    azurerm = azurerm
  }

  all_subscriptions = true
  subscription_exclusions = [ "excluded_subscription_id", "excluded_subscription_id2" ]
  # To allow-list some subscriptions instead of deny-listing, set all_subscriptions to false and use
  # subscription_ids = [ "allowed_subscription_1", "allowed_subscription_2"... ]
  application_name  = "cxm_asset_crawler"
}

module "enable_billing_export" {
  source = "../../modules/terraform-azure-billing"
  providers = {
    azurerm = azurerm.billing
  }

  all_subscriptions         = true
  application_name          = "cxm_billing_crawler"
  location                  = "West US 2"
  log_retention_days        = 10
  aws_notification_endpoint = "https://aws.amazonaws.com"
}

module "enable_audit_log" {
  source = "../../modules/terraform-azure-audit-log"
  providers = {
    azurerm = azurerm.audit_log
  }

  all_subscriptions         = true
  application_name          = "cxm_audit_log_crawler"
  location                  = "West US 2"
  log_retention_days        = 10
  aws_notification_endpoint = "https://aws.amazonaws.com"
}

Using the Management Group​

You can change the configuration of the tenant enablement to set up access at the Management Group level instead. Use the following configuration block to set it up that way:

module "enable_tenant" {
  source = "../../modules/terraform-azure-enablement"
  providers = {
    azurerm = azurerm
  }

  use_management_group = true
  management_group_id = "SuperManagementGroupID"
  application_name  = "cxm_asset_crawler"
}

The rest of the configuration remains identical.

Using a pre-existing Storage Account for Activity Logs​

If you set up a new integration, Cloud ex Machina won't be able to retrace attribution of assets in the past. It is recommended to plug Cloud ex Machina on your existing audit log storage account to provide access to historical administrative logs:

module "enable_audit_log" {
  source = "../../modules/terraform-azure-audit-log"
  providers = {
    azurerm = azurerm.audit_log
  }

  all_subscriptions              = true
  application_name               = "cxm_audit_log_crawler"
  location                       = "West US 2"
  use_existing_storage_account   = true
  storage_account_name           = "superStorageAccount"
  storage_account_resource_group = "superStorageAccountRG"
  aws_notification_endpoint      = "https://aws.amazonaws.com"
}

The rest of the configuration remains identical.

Using a pre-existing Storage Account for Billing Exports​

If you set up a new integration, Cloud ex Machina won't be able to read your historical billing data. It is recommended to plug Cloud ex Machina on your existing billing storage account to provide access to past usage reports.:

module "enable_billing" {
  source = "../../modules/terraform-azure-billing"
  providers = {
    azurerm = azurerm.billing
  }

  all_subscriptions              = true
  application_name               = "cxm_billing_crawler"
  location                       = "West US 2"
  use_existing_storage_account   = true
  storage_account_name           = "superStorageAccount"
  storage_account_resource_group = "superStorageAccountRG"
  aws_notification_endpoint      = "https://aws.amazonaws.com"
}

The rest of the configuration remains identical.

Deploy​

Run

terraform init
terraform plan
terraform apply

and wait for the system to converge. Depending on the size of your estate, this can take from a few minutes to a few tens of minutes.

Share results​

The deployment will output between 1 and 3 JSON documents containing application secrets to use by Cloud ex Machina. Kindly share this back to enable the configuration of your solution.