I was looking for a solution to sync an Azure blob storage containing images to a third party solution that contained an API to upload images. my initial thoughts were to hook up Azure Functions to react on Azure Blob Storage triggers. One thing that is not possible with blob storage triggers is to act on delete. there is only a trigger on adding or changing files.

Luckily Microsoft announced a new solution called Event Grid a few months back. Event Grid is great for connecting events that come from azure resources (or custom resources) to things like Azure Functions or Logic Apps.

Event Grid also supports events for Blob Storage where you get events for adding, changing or deleting items. So how to get started?

Event Grid is easy to setup and exists out of 2 parts. Topics and Subscriptions. Topics are places where events are sent to by Azure resources or even custom publishers. Subscriptions can be made on topics and will receive the events from a certain topic.

Creating the Storage Account / Event Grid Topic

Azure Blob storage has an Event Grid topic built in so you don’t have to actually create a separate Event Grid Topic. At the time of writing Event Grid is only available in West Central US and US West 2 regions so if you create a Storage account there i’ll automatically also get an Event Grid Topic.

Let’s create the topic using Azure CLI

#create resourcegroup
az group create -n <<ResourceGroupName>> -l westus2

#create storage account 
az storage account create \    
   --location westus2 \ 
   --name <<NameOfStorageAccount>> \  
   --access-tier cool \    
   --kind blobstorage \    
   --resource-group <<ResourceGroupName>> \ 
   --sku Standard_LRS \

Or using the Azure Portal:

When we open the storage account in the azure portal we’ll see that in the left menu is an option called Event Grid. In this menu you can see a list of all subscriptions to this Event Grid Topic. Current we don’t have one so lets take a look on how we can do that.

Creating the Event Grid Subscriptions

Since a topic can have multiple subscribers let’s add 2 different subscribers. First we’ll create a very simple subscription that allows us to see what the events actually look like and after that we’ll take a look in adding an Azure Function that handles the events.

Simple test subscription using Requestb.in

Requestb.in is a website where you can request a simple URL that collects all http messages sent to that URL. This is a free service and will keep the last 20 messages for a maximum amount of 48 hours. As soon as we created our requestbin we’ll receive a URL that looks like something like this: https://requestb.in/1ckzahm1

we can add this URL as a web hook subscription to the topic we created earlier. this can be done either using Azure CLI or the Azure portal.

az eventgrid resource event-subscription create    
   --endpoint "https://requestb.in/1ckzahm1" \    
   --name requestbinsubscription \  
   --provider-namespace Microsoft.Storage \  
   --resource-type storageAccounts \  
   --resource-group <<ResourceGroupName>> \   
   --resource-name cloudinarysync

Or in the Azure portal:

 

when we upload a file to the storage account now we are able to see the event that was triggered by going to the requestb.in inspect web page. We’ll see the json payload containing the event details so we can use this later in our Azure function.

Example json for file adding and deleting looks like the following:

File Add/Change event

File Deleted event

So we’ve seen the easiest way to hook up the Storage account events using Event Grid. Let’s add an Azure function that actually does something with the events.

Azure Function

When you create a new Azure Function you’ll have to choose the trigger type. you can choose between several options here like a Http Trigger, Webhook Trigger or Event Grid Trigger.  When looking at the list you might think Event Grid Trigger is the way to go. I tend to disagree for now. The events sent from the Event Grid are just play POST http messages so choosing a webhook trigger or Http trigger works just as good and they are even easier to test locally. The thing that Event Grid Trigger adds is that it maps the json payload shown above to a typed object so you can immediatly use it without parsing the json yourself. Another downside from Http Trigger and Webhook trigger is that you’ll have to arrange security yourself since everyone who knows the url could call the webhook by default. Let’s look at both options.

If you are using the portal use the small link below “Get started on your own” called “Custom Function” to choose the trigger type.

Event Grid Trigger

When you open your event grid trigger (if you created it via the portal but also for precompiled functions that you uploaded there is a link on the top right called “Add Event Grid Subscription”. Click this to set up the Event Grid Trigger.

 

 

In the “Create Event Subscription” window select the “Storage Account” topic type and select your storage account. After pressing Create your Azure Function will be triggered after each change in the storage account.

 

Http / Webhook Trigger

Webhook or Http Triggers work almost the same way. in the Portal there is a link on the top right to get the Function URL. When you click this  you’ll see a popup with the endpoint that you should copy. after this adding the subscription works exactly the same as i described above for the requestb.in except now you’ll enter this URL you just copied.

Now we’ve set up the plumbing we can start writing our function. i’ll show you the code for a http trigger but an Event grid Trigger function would like almost the same. you can skip the parsing of the json there because you get a typed  object as parameter containing all information.

 

Pitfalls & Tips

So if everything is correct you should have a working Azure Function now. But how to track the events that are coming in? You could set up application insights and track usage yourself but a nice feature of Event Grid is that it has build in logging and metrics. The metrics itself are a bit hidden in the Azure Portal so i’ll explain how to find them.

The subscriptions itself cannot be found in the resource group of the topic. When navigating to the storage account and clicking Event Grid you can get a list of your subscriptions but no metrics.

For metrics of the subscriptions you have to go to your left menu in the Azure portal and click the > arrow for more resources. search for Subscriptions and here the Event Grid Subscriptions will show up here

 

when you go here you’ll get an overview of all events that were triggered by a Topic and the subscriptions connected to the topic. You can also see when the Event Grid did retries and which events completed successfully or failed. It took me a while to find this so hopefully this helps more people find it.

 

Another thing to note is that right after creation of the subscription it might take a while for the events to start firing. I don’t know if this is something that is related to the preview state of Event Grid or if it will always be the case.  In the end all events fired but it took a while for the first events to be fired. After it was running for a while the events were triggered really fast. even when i did a large amount of changes the events were fired within seconds.

Hopefully this is useful for you developers who want to react to triggers from Azure Storage.

Happy Coding!

Geert van der Cruijsen