Configuring Cloud Ninja Metering Block

Greetings to all. We have recently published yet another exciting project to the Azure community at http://cnmb.codeplex.com at the end of the previous month. We will try to add more details on its usage as we find time here on our blog. This post’s topic is on configuring the block host.

The first decision is where to host the block. You can have a multitude of different combinations on the location, such as a Windows Service, an Azure worker role, or a web role. For the sake of simplicity, I will explain how you can configure to run it in a web role. We will continue this topic on a different post soon on how to continue this on the same role for the publishing part of the block.

Starting with Azure SDK 1.3, the web roles allow you to access the full IIS functionality. At the most basic level, the side effect of this is you will need to think about having two different hosts running your web role code. One host is the w3wp.exe process running your site’s code, and the other being the WaIISHost.exe process, hosting your RoleEntryPoint (which happens to be your WebRole class in the WebRole.cs file, if you have started with the Azure Web Role template in VS 2010).

Since our code is living in the .Net universe, each process, thus AppDomain need to grab their own respective .config files. Thus, there is a need to create a new file in your project called WaIISHost.exe.config, along your customary web.config.

Hmm, the team here looks like working on some new enhancements…

Anyhow, once you create the new file, you will need to think about 3 new points:

  1. Add the reference for the custom configuration sections.
  2. The common configuration for the block.
  3. The provider configurations

Let’s start with the configuration section references.

We do use the Patterns and Practices’ Unity Application Block (http://unity.codeplex.com/), so the Unity configuration sections, and also the Transient Fault Handling Application Block from Enterprise Library 5.0 Integration Pack for Windows Azure. So you will need to add those, along with the Metering block’s custom section reference. An example will look like this:














Following this will be the common configuration for the block within the custom configuration section marked between the tags:




The block uses a blob lease to determine the master for a given hosting environment who will collect the data and write to the repository. This method is required if you happen to be hosting the block in a role which runs in multiple instances. The BlockConnectionString attribute contains the Azure storage account connection string that contains this. The rest is automatic, the block creates its own container and the blob to put the leases on.

You can also specify a number of connection strings both for Azure storage accounts as well as SQL Azure connections if you want to consolidate all of your connection strings in one place and refer to them in the rest of the configuration section.

An example for this will be:












Now it is time to come to the providers’ configuration. Please note that you can specify a default collect interval for all of the providers here, but each provider can override this value at their own respective configuration elements. We support 5 providers out of the box. They are:

  1. Database size provider
  2. Database bandwidth provider
  3. IIS bandwidth provider
  4. Blob size provider
  5. Blog bandwidth provider

They all have a reference to the type that implements the provider as the Type attribute at the Provider element, and also a CollectInterval attribute. And each provider has it’s own particular configuration after that. Most of it after that almost repeats, but there are two important points about this.

  1. Bookmark support for parsing logs
  2. Tenant identifier pattern

We support keeping a bookmark of the last parsed log for both IIS logs and the blob storage analytics logs. So we need a repository connection string for both IISLogsParserBookmarkRepository and BlobBandwidthLogsParserBookmarkRepository attributes. They may point to the same storage account. The providers automatically create a table on those accounts where they start marking the last log file and the line they parsed.

The other point is telling the providers on how to identify a tenant ID. The attribute DefaultTenantIdPattern as the default setting for the provider and TenantIdPattern connection location specific one both have the same template. It is a BEGIN_REGEX{MIDDLE_REGEX}END_REGEX template, where the start, middle and end of the tenant ID patter are marked with the curly braces. You can basically put in any regex here, as long as it conforms to the expected patterns by the resources the providers are measuring.

There are also a couple of details for the IIS Log provider and the blob provider. The first one supports the location of the tenant identifier. The values can be UriStem, Hostheader and UriQuery. Those will translate to the column names on the IIS logs, as defined by the W3C standard column names. The default IIS log configuration on web roles, when collected through the Windows Azure Diagnostics (WAD) subsystem. The next one, about the blob size provider is to tell the provider where to find the blobs as the base container. It is provided by the TenantsBaseContainerPrefix attribute on the blob size provider.

Rest is almost repeating the same pattern. Specify a connection string, either explicitly, or refer to one of the common connection strings by their name. Following is an example how you can specify the settings for the various providers.


















































































Leave a Reply