Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compute best practices documentation update #1088

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from
Open

Compute best practices documentation update #1088

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a35e139
adding queries for compute
nteyan Nov 4, 2024
8fd6d21
App Service plans with no hosted applications
nteyan Nov 4, 2024
4cb78ce
Update docs/_docs/best-practices/compute/compute.md
nteyan Nov 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
171 changes: 137 additions & 34 deletions docs/_docs/best-practices/compute/compute.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,48 +18,36 @@ Discover essential FinOps best practices to optimize cost efficiency and governa
<details open markdown="1">
<summary class="fs-2 text-uppercase">On this page</summary>

- [Azure Kubernetes Service](#azure-kubernetes-service)
- [Advisor](#advisor)
- [Virtual machines](#virtual-machines)
- [App Service plans](#app-service-plans)
- [Azure Kubernetes Service](#azure-kubernetes-service)
- [🙋‍♀️ Looking for more?](#️-looking-for-more)
- [🧰 Related tools](#-related-tools)

</details>

---

## Azure Kubernetes Service
## Advisor

### Query: AKS Cluster
### List of cost recommendations for Compute
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to track "Advisor" as a service under every category? It feels like that should only fall under General, right? At least, unless we see some very targeted action that we can recommend based on the service-specific queries, right?


This Azure Resource Graph (ARG) query retrieves detailed information about Azure Kubernetes Service (AKS) clusters within your Azure environment.
This Azure Resource Graph (ARG) query retrieves a list of Azure Advisor recommendations specifically for compute resources. It filters the recommendations to include only those related to virtual machines, scale sets, and other compute services, providing insights into potential cost savings.

<h4>Category</h4>

Resource management
Cost optimization

<h4>Query</h4>

```kql
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: We should establish some KQL language standards. If you happen to know of any, we should document them in the wiki. No worries if you haven't.

resources
| where type == "microsoft.containerservice/managedclusters"
| extend AgentPoolProfiles = properties.agentPoolProfiles
| mvexpand AgentPoolProfiles
| project
id,
ProfileName = tostring(AgentPoolProfiles.name),
Sku = tostring(sku.name),
Tier = tostring(sku.tier),
mode = AgentPoolProfiles.mode,
AutoScaleEnabled = AgentPoolProfiles.enableAutoScaling,
SpotVM = AgentPoolProfiles.scaleSetPriority,
VMSize = tostring(AgentPoolProfiles.vmSize),
nodeCount = tostring(AgentPoolProfiles.['count']),
minCount = tostring(AgentPoolProfiles.minCount),
maxCount = tostring(AgentPoolProfiles.maxCount),
location,
resourceGroup,
subscriptionId,
AKSname = name
advisorresources
| where type == "microsoft.advisor/recommendations"
| where tostring (properties.category) has "Cost"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| where tostring (properties.category) has "Cost"
| where tostring(properties.category) has "Cost"

| where properties.shortDescription.problem has "underutilized"
| where properties.impactedField has "Compute" or properties.impactedField has "Container" or properties.impactedField has "Web"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move each or to a new line so it's more readable online. Web should be in the Web category and not in Compute according to FOCUS service categories.

Suggested change
| where properties.impactedField has "Compute" or properties.impactedField has "Container" or properties.impactedField has "Web"
| where properties.impactedField has "Compute"
or properties.impactedField has "Container"

| project AffectedResource=tostring(properties.resourceMetadata.resourceId),Impact=properties.impact,resourceGroup,AdditionaInfo=properties.extendedProperties,subscriptionId,Recommendation=tostring(properties.shortDescription.problem)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be spaces around =, after commas, and long lines should be broken up on multiple lines.

```

<br>
Expand Down Expand Up @@ -88,26 +76,60 @@ resources
| project id, PowerState, VMLocation, resourceGroup, subscriptionId
```

### Query: Virtual machine scale set details
<br>

This query analyzes Virtual Machine Scale Sets (VMSS) in your Azure environment based on their SKU, spot VM priority, and priority mix policy. It provides insights for cost optimization and resource management strategies.
### Query: List Virtual Machines deallocated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Query: List Virtual Machines deallocated
### Query: List deallocated virtual machines


This Azure Resource Graph (ARG) query identifies Virtual Machines (VMs) in your Azure environment that are in the 'deallocated' state. It retrieves details about their power state, location, resource group, and subscription ID.

<h4>Category</h4>

Resource management
Waste reduction

<h4>Query</h4>

```kql
resources
| where type =~ 'microsoft.compute/virtualmachinescalesets'
| extend SpotVMs = tostring(properties.virtualMachineProfile.priority)
| extend SpotPriorityMix = tostring(properties.priorityMixPolicy)
| extend SKU = tostring(sku.name)
| extend resourceGroup = strcat('/subscriptions/', subscriptionId, '/resourceGroups/', resourceGroup)
| project id, SKU, SpotVMs, SpotPriorityMix, subscriptionId, resourceGroup, location
| where type =~ 'microsoft.compute/virtualmachines'
and tostring(properties.extended.instanceView.powerState.displayStatus) == 'VM deallocated'
| where resourceGroup in ({ResourceGroup})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is {ResourceGroup} here?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used by the workbook to filter the results based on a parameter called ResourceGroup. We can remove it from this query.

| extend PowerState=tostring(properties.extended.instanceView.powerState.displayStatus), VMLocation=location, resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably standardize on only one property per extend for readability.

Suggested change
| extend PowerState=tostring(properties.extended.instanceView.powerState.displayStatus), VMLocation=location, resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)
| extend PowerState = tostring(properties.extended.instanceView.powerState.displayStatus)
| extend VMLocation = location
| extend resourceGroup = strcat('/subscriptions/', subscriptionId, '/resourceGroups/', resourceGroup)

| order by id asc
| project id, PowerState, VMLocation, resourceGroup, subscriptionId
```

<br>

### Query: List of virtual machines with their associated disks

This Resource Graph query retrieves a comprehensive list of all Virtual Machines (VMs) in your Azure environment, along with details of their associated disks. It provides insights into the storage configuration of each VM, helping you manage and optimize your storage resources effectively.

<h4>Category</h4>

Resource management

<h4>Query</h4>

```kql
Resources | where type == "microsoft.compute/virtualmachines"
| extend osDiskId= tostring(properties.storageProfile.osDisk.managedDisk.id)
| join kind=leftouter(resources
| where type =~ 'microsoft.compute/disks'
| where properties !has 'Unattached'
| where properties has 'osType'
| project OS = tostring(properties.osType), osSku = tostring(sku.name), osDiskSizeGB = toint(properties.diskSizeGB), osDiskId=tostring(id)) on osDiskId
| join kind=leftouter(Resources
| where type =~ 'microsoft.compute/disks'
| where properties !has "osType"
| where properties !has 'Unattached'
| project sku = tostring(sku.name), diskSizeGB = toint(properties.diskSizeGB), id = managedBy
| summarize sum(diskSizeGB), count(sku) by id, sku) on id
| project vmId=id, subscriptionId, resourceGroup, OS, location, osDiskId, osSku, osDiskSizeGB, DataDisksGB=sum_diskSizeGB, diskSkuCount=count_sku
| sort by diskSkuCount desc
Comment on lines +114 to +128
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Make sure the queries are formatted correctly.
  2. You're using a case-insensitive type filter on some lines and case-sensitive on the first where. Which is correct?
  3. Let's always use single instead of double quotes.
  4. Make sure table names are cased consistently. You have "resources" and "Resources".
  5. Doesn't KQL use order by instead of sort by?
Suggested change
Resources | where type == "microsoft.compute/virtualmachines"
| extend osDiskId= tostring(properties.storageProfile.osDisk.managedDisk.id)
| join kind=leftouter(resources
| where type =~ 'microsoft.compute/disks'
| where properties !has 'Unattached'
| where properties has 'osType'
| project OS = tostring(properties.osType), osSku = tostring(sku.name), osDiskSizeGB = toint(properties.diskSizeGB), osDiskId=tostring(id)) on osDiskId
| join kind=leftouter(Resources
| where type =~ 'microsoft.compute/disks'
| where properties !has "osType"
| where properties !has 'Unattached'
| project sku = tostring(sku.name), diskSizeGB = toint(properties.diskSizeGB), id = managedBy
| summarize sum(diskSizeGB), count(sku) by id, sku) on id
| project vmId=id, subscriptionId, resourceGroup, OS, location, osDiskId, osSku, osDiskSizeGB, DataDisksGB=sum_diskSizeGB, diskSkuCount=count_sku
| sort by diskSkuCount desc
Resources
| where type == 'microsoft.compute/virtualmachines'
| extend osDiskId= tostring(properties.storageProfile.osDisk.managedDisk.id)
| join kind=leftouter(
Resources
| where type =~ 'microsoft.compute/disks'
| where properties !has 'Unattached'
| where properties has 'osType'
| project
OS = tostring(properties.osType),
osSku = tostring(sku.name),
osDiskSizeGB = toint(properties.diskSizeGB),
osDiskId=tostring(id)
) on osDiskId
| join kind=leftouter(
Resources
| where type =~ 'microsoft.compute/disks'
| where properties !has "osType"
| where properties !has 'Unattached'
| project
sku = tostring(sku.name),
diskSizeGB = toint(properties.diskSizeGB),
id = managedBy
| summarize sum(diskSizeGB), count(sku) by id, sku
) on id
| project
vmId = id,
subscriptionId,
resourceGroup,
OS,
location,
osDiskId,
osSku,
osDiskSizeGB,
DataDisksGB = sum_diskSizeGB,
diskSkuCount = count_sku
| order by diskSkuCount desc

```

<br>

### Query: Virtual Machine processor type analysis

This query identifies the processor type (ARM, AMD, or Intel) used by Virtual Machines (VMs) in your Azure environment. It helps in understanding the distribution of VMs across different processor architectures, which is useful for optimizing workload performance and cost efficiency.
Expand Down Expand Up @@ -152,6 +174,87 @@ resources

<br>

### Query: Virtual machine scale set details
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be in the VMSS section? (Unless I missed it.)


This query analyzes Virtual Machine Scale Sets (VMSS) in your Azure environment based on their SKU, spot VM priority, and priority mix policy. It provides insights for cost optimization and resource management strategies.

<h4>Category</h4>

Resource management

<h4>Query</h4>

```kql
resources
| where type =~ 'microsoft.compute/virtualmachinescalesets'
| extend SpotVMs = tostring(properties.virtualMachineProfile.priority)
| extend SpotPriorityMix = tostring(properties.priorityMixPolicy)
| extend SKU = tostring(sku.name)
| extend resourceGroup = strcat('/subscriptions/', subscriptionId, '/resourceGroups/', resourceGroup)
| project id, SKU, SpotVMs, SpotPriorityMix, subscriptionId, resourceGroup, location
```

<br>

## App Service plans
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this to the Web category.


### Query: App Service plans with no hosted applications

This Resource Graph query identifies all App Service plans in your Azure environment that do not have any hosted applications. It helps you pinpoint unused resources, enabling you to optimize costs and improve resource management.

<h4>Category</h4>

Waste management

<h4>Query</h4>

```kql
resources
| where type =~ "microsoft.web/serverfarms"
| where properties.numberOfSites == 0
| extend Details = pack_all()
| project Resource=id, resourceGroup, location, subscriptionId, Sku=sku.name, Tier=sku.tier, tags ,Details
```

<br>

## Azure Kubernetes Service
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this back to the top. Services should be sorted alphabetically.


### Query: AKS Cluster

This Azure Resource Graph (ARG) query retrieves detailed information about Azure Kubernetes Service (AKS) clusters within your Azure environment.

<h4>Category</h4>

Resource management

<h4>Query</h4>

```kql
resources
| where type == "microsoft.containerservice/managedclusters"
| extend AgentPoolProfiles = properties.agentPoolProfiles
| mvexpand AgentPoolProfiles
| project
id,
ProfileName = tostring(AgentPoolProfiles.name),
Sku = tostring(sku.name),
Tier = tostring(sku.tier),
mode = AgentPoolProfiles.mode,
AutoScaleEnabled = AgentPoolProfiles.enableAutoScaling,
SpotVM = AgentPoolProfiles.scaleSetPriority,
VMSize = tostring(AgentPoolProfiles.vmSize),
nodeCount = tostring(AgentPoolProfiles.['count']),
minCount = tostring(AgentPoolProfiles.minCount),
maxCount = tostring(AgentPoolProfiles.maxCount),
location,
resourceGroup,
subscriptionId,
AKSname = name
```

<br>

## 🙋‍♀️ Looking for more?

We'd love to hear about any datasets you're looking for. Create a new issue with the details that you'd like to see either included in existing or new best practices.
Expand Down