Configuring Storage IO Control IOPS Capacity for vCloud Director Org VDC Storage Profiles

Recently I began a small project to expose and control Storage IO Control in vCloud Director 8.20+. In order to leverage the capabilities there are a few things that need to be configured/considered. Before you begin you need to determine the capabilities (IOPS) of each of your datastores which is set as a custom attribute on each datastore and is exposed to vCloud Director as the “Available IOPS” for a data store. There are a few things to note before you begin:

  1. You cannot enable IOPS support on a VMware Virtual SAN datastore
  2. You cannot enable IOPS support if the Storage Profile contains Datastores that are part of a Storage DRS Cluster; all of the datastores in the Storage Profile must not be part of a Cluster; if any datastores are in a SDRS cluster you can’t leverage the SIOC in vCloud Director
  3. Each Datastore can have a value set between 200-4000 (IOPS)
  4. You need to have vSphere Administrator rights on the vCenters hosting the datastores to complete the below
  5. The tagged datastores must be added to a SIOC enabled Storage Policy which is mapped to vCloud as a Provider VDC Storage Profile
  6. The Organisational VDC Storage Profile can then have SIOC capabilities set against it using the REST API (or Powershell using my vCloud Storage Profile Module

Step 1. Set the iopsCapacity Custom Attribute

In order to expose SIOC in vSphere to vCloud Director custom attributes have to be added to the Datastores using the vSphere Manage Object Browser (MOB) as outlined in VMWare KB2148300 however it’s much easier to do this through the vSphere Client or vSphere Web Client.

  1. Logon to the vSphere H5 Client (https://<vCenter>/ui) and select the Tags & Custom Attributes from the menu and select Custom Attributes and click Add
  1. Enter the attribute iopsCapacity and select the Type Datastore and click Add
  1. Next select Storage from the main menu and select each Datastores which you wish to set SIOC capabilities to be exposed in vCloud and from the Actions menu select Tags & Custom Attributes > Edit Custom Attributes 
  1. Set the value for iopsCapacity and click OK
  1. Next; tag the datastores with a relevant tag and create a new Storage Profile with VMWare Storage IO Control provider for the SIOC enabled datastores

Step 2. Configure Storage Profiles in vCloud Director

  1. After this has been set on all the relevant data stores; logon to vCloud Director and select vCenters > Refresh Storage Policies 
  1. Add the Storage Profile to the relevant Organizations (Organizations > Organisational VDCS > Storage Policies)
  1. Review the Provider VDC and confirm that the IopsCapacity value shows a non-zero value when using the Get-ProviderVdcStorageProfile cmdlet (Open PowerShell and connect to vCloud Director and import the module Module-vCloud-SIOC.psm1 available from here)
  1. Set the Storage IO Control settings using the Set-OrgVdcStorageProfile cmdlet

$objOrgVDCStorageProfile = Get-OrgVdcStorageProfile -OrgName “PigeonNuggets” | ? {$_.Name -eq “SIOC”}
$objOrgVDCStorageProfile | Set-OrgVdcStorageProfile -SIOCEnabled $true -DiskIopsMax 1000 -DiskIopsDefault 100 -DiskIopsPerGBMax 100

The OrgVDC Storage Profile is configured for SIOC which is implemented in vSphere. SIOC as implemented in vCloud Director needs further work (manually tagging the datastores with capabilities and API only exposure is a bit rough) however the capabilities are beginning to be exposed; further configuration can be made on individual Virtual Disks via the API (hopefully I will get to this in the near future). Hopefully this is of some value for you. #LongLiveVCD

SIOC and Provider/Organization VDC Storage Profile Management in vCloud Director with PowerShell

Long time since my last post due to some major life events however thanks to some annoying Jet Lag I have managed to get some work done on a project I have been working on slowly over the past couple of months; development of some PowerShell cmdlets to expose and add support for updating VDC and Provider Storage Profiles/Policies in vCloud Director 8.20/9.0

The rationale for creating these cmdlets was twofold;

  1. There is currently no way to set the Storage I/O control parameters in vCloud Director outside of the API
  2. The Org VDC/Provider Storage Profiles are not readily exposed in PowerCLI which makes them a bit difficult to work with (need to combine API calls and vCloud Views)

Why would you want to use these cmdlets ? Two main use cases that I have;

  1. For orchestrating dynamic updates to the Org VDC Storage Profile limits; for example if you want to prevent Organisations from consuming all of your backend storage in a short period of time (and have limits set) but don’t want to have to manually update the limits/have clients calling asking why they can’t create a new VM or expand a disk these cmdlets can be used to adjust the Org VDC limits based on the available storage in the backend Provider Storage Profile as space is consumed/reclaimed
  2. If you wish to implement SIOC in vCloud in an Organization VDC Storage Policy, limit the IOPS available globally to that Storage Policy etc. and if there is “peak”/”off-peak” arrangement with a customer whereby there Storage Policies adjust based on Time of Day (e.g. Test Tier is throttled during 9am-9pm) this might assist
The code is available on GitHub here or below. The documentation in the PowerShell (get-help cmdlet -full) is more complete however below are a quick summary of the main user functions and how to use them;
  • Get-OrgVdcStorageProfile : Returns the Storage Policies/Profiles which are defined on the target Organisation Virtual Datacenter object.
  • Set-OrgVdcStorageProfile : Sets the properties of a provided Org VDC Storage Policies/Profiles.
  • Get-ProviderVdcStorageProfile : Returns the Provider VDC Storage Profile objects for the target organisation.
  • Set-ProviderVdcStorageProfile: Allows the settings to be adjusted on a Provider VDC Storage Profile

These cmdlets are a big rough and more work to do when time permits but have been tested on PowerCLI 6.5.1 and vCloud Director 8.20.1 and 9.0; I hope you get some value from these cmdlets and #LongLiveVCD


Powershell Cmdlets for Managing VMWare Photon Platform 1.2.1

VMware announced End of Availability (EOA) of Photon Platform on 6th October 2017 with the PKS Service due to launch soon. This is a product I started looking at in June for developing a strategy for offering Kubernetes-as-a-Service delivery to multi-tenant customers and if this was viable to develop into a Service Provider offering. I was not happy with the management tools available but there was a lot of potential for this to be a good fit.

I figured the best way to understand the product was to add some value by developing some PowerShell cmdlets from the REST API to manage the platform which quickly turned into a major exercise in-itself. The scope quickly got away from me part time developing this after work and other projects. With the product going end of life I have decided to publish what I have as-is (very incomplete, partial testing, needs work) as a code sample as if anyone is using Photon Platform at the moment this may be useful or at least of some benefit :) This was really useful for me to get better at coding against REST based API using swagger and some reverse engineering so although this was not how I wanted this project to end it has been valuable and I hope it is to someone else :)

There is a bunch of functions to explore (Get-Command -Module Module-Vmware-PhotonPlatform and Get-Help cmdlet) which should be documented (mostly) I have made an effort but beware the code quality :)

Available on Github here.

Basic functionality that generally has been tested/works is;

  • Host Management
  • Availability Zone Management
  • Cloud Image Management
  • Subnet Management
  • Quota Management
  • Tenant/Project Management