Sam Speaks: 2018

Thursday, November 1, 2018

Azure CLI for listing VM images

Using az command line to List VM images

We need to have details of VM images to use them programmatically through ansible or other tools.

below I have listed a few examples of Azure CLI commands to list the Publishers, Offers, SKUs, Versions and Image details in general.

https://docs.microsoft.com/en-us/cli/azure/vm/image?view=azure-cli-latest

To list publishers

bash-3.2$ az vm image list-publishers -l eastus --query "[?starts_with(name, 'cloudera')]"
[
  {
    "id": "/Subscriptions/da35404a-2612-4419-baef-45fcdce6045e/Providers/Microsoft.Compute/Locations/eastus/Publishers/cloudera",
    "location": "eastus",
    "name": "cloudera",
    "tags": null
  }
]

To list all offers

bash-3.2$ az vm image list-offers -l eastus -p cloudera
[
  {
    "id": "/Subscriptions/da35404a-2612-4419-baef-45fcdce6045e/Providers/Microsoft.Compute/Locations/eastus/Publishers/cloudera/ArtifactTypes/VMImage/Offers/cloudera-altus-centos-os",
    "location": "eastus",
    "name": "cloudera-altus-centos-os",
    "tags": null
  },
...
  {
    "id": "/Subscriptions/da35404a-2612-4419-baef-45fcdce6045e/Providers/Microsoft.Compute/Locations/eastus/Publishers/cloudera/ArtifactTypes/VMImage/Offers/test-image",
    "location": "eastus",
    "name": "test-image",
    "tags": null
  }
]

To list all SKUs

bash-3.2$ az vm image list-skus -l eastus -p cloudera -f cloudera-centos-os
[
  {
    "id": "/Subscriptions/da35404a-2612-4419-baef-45fcdce6045e/Providers/Microsoft.Compute/Locations/eastus/Publishers/cloudera/ArtifactTypes/VMImage/Offers/cloudera-centos-os/Skus/6_7",
    "location": "eastus",
    "name": "6_7",
    "properties": {
      "automaticOSUpgradeProperties": {
        "automaticOSUpgradeSupported": false
      }
    },
    "tags": null
  },
......
  {
    "id": "/Subscriptions/da35404a-2612-4419-baef-45fcdce6045e/Providers/Microsoft.Compute/Locations/eastus/Publishers/cloudera/ArtifactTypes/VMImage/Offers/cloudera-centos-os/Skus/7_4",
    "location": "eastus",
    "name": "7_4",
    "properties": {
      "automaticOSUpgradeProperties": {
        "automaticOSUpgradeSupported": false
      }
    },
    "tags": null
  }
]

to list all images

bash-3.2$ az vm image list  -l eastus -p cloudera -f cloudera-centos-os -o tsv --all
cloudera-centos-os cloudera 6_7 cloudera:cloudera-centos-os:6_7:1.0.0 1.0.0
cloudera-centos-os cloudera 6_7 cloudera:cloudera-centos-os:6_7:1.0.1 1.0.1
cloudera-centos-os cloudera 6_7 cloudera:cloudera-centos-os:6_7:2.0.1 2.0.1
cloudera-centos-os cloudera 6_7 cloudera:cloudera-centos-os:6_7:2.0.4 2.0.4
cloudera-centos-os cloudera 6_8 cloudera:cloudera-centos-os:6_8:1.0.0 1.0.0
cloudera-centos-os cloudera 6_8 cloudera:cloudera-centos-os:6_8:2.0.1 2.0.1
cloudera-centos-os cloudera 6_8 cloudera:cloudera-centos-os:6_8:2.0.4 2.0.4
cloudera-centos-os cloudera 7_2 cloudera:cloudera-centos-os:7_2:1.0.2 1.0.2
cloudera-centos-os cloudera 7_2 cloudera:cloudera-centos-os:7_2:1.0.5 1.0.5
cloudera-centos-os cloudera 7_2 cloudera:cloudera-centos-os:7_2:1.0.7 1.0.7
cloudera-centos-os cloudera 7_2 cloudera:cloudera-centos-os:7_2:1.0.8 1.0.8
cloudera-centos-os cloudera 7_2 cloudera:cloudera-centos-os:7_2:2.0.3 2.0.3
cloudera-centos-os cloudera 7_2 cloudera:cloudera-centos-os:7_2:2.0.4 2.0.4
cloudera-centos-os cloudera 7_4 cloudera:cloudera-centos-os:7_4:2.0.7 2.0.7

to list details of a specific image

bash-3.2$ az vm image show -l eastus -p cloudera -f cloudera-centos-os -s 7_2 --version 1.0.2
{
  "dataDiskImages": [],
  "id": "/Subscriptions/da35404a-2612-4419-baef-45fcdce6045e/Providers/Microsoft.Compute/Locations/eastus/Publishers/cloudera/ArtifactTypes/VMImage/Offers/cloudera-centos-os/Skus/7_2/Versions/1.0.2",
  "location": "eastus",
  "name": "1.0.2",
  "osDiskImage": {
    "operatingSystem": "Linux"
  },
  "plan": {
    "name": "7_2",
    "product": "cloudera-centos-os",
    "publisher": "cloudera"
  },
  "tags": null
}

Wednesday, October 31, 2018

Create Azure VM using Ansible - With Extension

Extensions are similar to user data from AWS EC2 instances on Azure VM.
Below is an example of creating a virtual machine on Azure using Ansible.
Apart from creating a VM, it also demos creating an Extension, I am using a Custom Script as an extension so I can perform the Cloudera Director installation on the VM.

It can be used to run any shell script on the machine.

This play has below steps:

1. Create a public IP
2. Create a NIC
3. Create a Virtual Machine
4. Create an Extension

---
- name: Deploy Azure ARM template.
hosts: localhost
connection: local
gather_facts: false
vars_files:
- var.yml

tasks:
- name: Create a public ip address
azure_rm_publicipaddress:
resource_group: '{{ cd_resource_group }}'
name: '{{ cd_public_ip }}'
allocation_method: Static
domain_name: '{{ cd_vmname }}'
state: present
register: output_ip_address

- name: Dump public IP for VM which will be created
debug:
msg: "The public IP is {{ output_ip_address.state.ip_address }}."

- name: Create NIC
azure_rm_networkinterface:
name: '{{ cd_nic_name }}'
resource_group: '{{ cd_resource_group }}'
virtual_network_name: '{{ cd_vnet }}'
subnet_name: '{{ cd_subnet_name }}'
security_group_name: '{{ cd_security_group }}'
public_ip_address_name: '{{ cd_public_ip }}'

- name: Create VM
azure_rm_virtualmachine:
resource_group: '{{ cd_resource_group }}'
name: '{{ cd_vmname }}'
network_interfaces: '{{ cd_nic_name }}'
admin_username: centos
ssh_password_enabled: true
admin_password: xxxxxxx
# ssh_public_keys:
# - path: /home/centos/.ssh/authorized_keys
# key_data: <your-key-data>
vm_size: Standard_A4_v2
state: present
image:
offer: CentOS
publisher: OpenLogic
sku: '7.5'
version: latest a

- name: Create VM Extension
azure_rm_virtualmachine_extension:
name: '{{ cd_vm_extention }}'
location: eastus
resource_group: '{{ cd_resource_group }}'
virtual_machine_name: '{{ cd_vmname }}'
publisher: Microsoft.Azure.Extensions
virtual_machine_extension_type: CustomScript
type_handler_version: 2.0
settings: '{
"commandToExecute": "./director-install.sh",
"fileUris": ["https://raw.githubusercontent.com/samcloudera/random/master/director-install.sh"
]
}'
auto_upgrade_minor_version: true
state: present

Below is the playbook for deleting the Azure virtual machine.

It has below plays:

1. Deleting the extension
2. Kill VM
3. Delete the NIC
4. Delete the Public IP

---
- name: Deploy Azure ARM template.
hosts: localhost
connection: local
gather_facts: false
vars_files:
- var.yml

tasks:
- name: Delete VM Extension
azure_rm_virtualmachine_extension:
name: '{{ cd_vm_extention }}'
location: eastus
resource_group: '{{ cd_resource_group }}'
virtual_machine_name: '{{ cd_vmname }}'
state: absent

- name: Kill VM
azure_rm_virtualmachine:
resource_group: '{{ cd_resource_group }}'
name: '{{ cd_vmname }}'
state: absent
remove_on_absent:
# - network_interfaces
- virtual_storage
# - public_ips

- name: Delete network interface
azure_rm_networkinterface:
resource_group: '{{ cd_resource_group }}'
name: '{{ cd_nic_name }}'
state: absent

- name: Delete public ip
azure_rm_publicipaddress:
resource_group: '{{ cd_resource_group }}'
name: '{{ cd_public_ip }}'

state: absent