Untangle your micro-service driven application with Azure API Management

-

In this blog I will introduce you with the basics of Azure API Management. Firstly, I will start by introducing an use case for API management: a merger of the webservices of two online web shops. Additionally, I will show you how to create the APIM instance, a product, operations and policies. This is preferably all done using ARM templates.

AZ-204 & Azure API Management

In my last blog I wrote about the renewed Azure AZ-204 exam. I had the intention to write some more blogs. I used the blogs as a way to learn for the exam, while sharing knowledge at the same time. Well, that did not go as planned.
I found that writing and studying did cost way more time than only studying. Who would have thought so.. Nonetheless, I have completed the AZ-204 exam, and I am going to continue sharing information about the topics.
In this blog I will show you Azure API Management, as it is part of the exam, but also because of the added value it offers to your micro-service driven application.

MotorcycleParts.com

We are building an online shop for a motorcycle parts dealer, called MotorcycleParts. It consists of multiple Azure Functions, including:

  • Product-service
  • (Shopping) Basket-service
  • Product-stock-service

The company recently bought a car part dealer called CarPartz. The CarPartz site is going to be incorporated into the MotorcyleParts site. CarPartz recently updated their whole backend for their shop, so the requirement is to reuse their backend in the current state.

API Management is going to help us to organize the calls to our functions, as well as incorporating the CarPartz backend into the MotorcycleParts site.

 

API management

API management, in short APIM, is placed in front of our backends. It gives us the possibility to present a single endpoint to our website, instead of multiple URLs for different Azure Functions. This way the website does not have to know about all the different backend service that may exist. All the routing is done by APIM.
It also allows us to switch one of the backends, for a totally different implementation. Due to APIM there is a low coupling between the front-end and the backends. APIM can also help us with IP restriction, rate-limiting, and more.

First off, we are going to create an APIM instance. This is possible via multiple ways: ARM templates, Azure CLI, Portal and Powershell. I am going to create the instance using an ARM template. I always choose ARM in favor of the other options since it give us the ability to use version control and it is less manual work, in other words, less error prone. The same can be said about Powershell and the Azure CLI when used in a script.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "publisherEmail": {
            "type": "string",
            "minLength": 1,
            "metadata": {
                "description": "Email owner service"
            }
        },
        "publisherName": {
            "type": "string",
            "minLength": 1,
            "metadata": {
                "description": "Name owner service"
            }
        },
        "location": {
            "type": "string",
            "defaultValue": "[resourceGroup().location]",
            "metadata": {
                "description": "Location to deploy to"
            }
        }
    },
    "variables": {
        "apimInstanceName": "apim-poc-prod-01",
        "apimProductName": "motorcyle-parts",
        "apimApiProductServiceName": "product-service",
        "getProductsOperation": "[concat(variables('apimInstanceName'), '/', variables('apimApiProductServiceName'), '/get')]",
        "getProductByIdOperation": "[concat(variables('apimInstanceName'), '/', variables('apimApiProductServiceName'), '/get-by-id')]"
    },
    "resources": [
        {
            "type": "Microsoft.ApiManagement/service",
            "apiVersion": "2019-12-01",
            "name": "[variables('apimInstanceName')]",
            "location": "[parameters('location')]",
            "sku": {
                "name": "Consumption",
                "capacity": 0
            },
            "properties": {
                "publisherEmail": "[parameters('publisherEmail')]",
                "publisherName": "[parameters('publisherName')]"
            },
            "resources": []
        }
    ]
}

As you can see above, I have created an ARM template that uses the bare minimum to create the APIM instance. The first parameter, publisherEmail, is required to receive notifications about your APIM instance. This email address is for example used to notify you when the creation of the instance is finished. The creation of an instance can sometimes take up to one hour.
For this blog I chose to create APIM in a consumption plan, since all the Azure Functions are also on a pay-per-use plan. The consumption plan has, in contrary to a normal subscription, some usage limits and, custom domain names are not supported. The documentation shows all the differences and the limitations that apply.

You can deploy this template, and after a while the APIM instance will be available. But without API definitions, APIM is not going to do anything. We can add API definitions through the portal. When you navigate to the APIM Resource -> APIs, it should look like this. But again, I like to automate as much as possible, so I will use ARM templates.

Adding APIs to APIM

As described in the introduction, the APIM instance should make four endpoints of the backend services available. Three new Azure Functions and an external system which is, for the sake of the difference, running on an Azure Kubernetes cluster. We are going to add the endpoints using the ARM template.

I like to start by creating a product in APIM. Using a product you can group one or more APIs. It is possible configure some stuff on product-level. In the example below you can see the product definition. For the product I only defined a name and a reference to the API that is going to be part of the product. The API on the contrary, contains more properties. The path defines the route we can call from the APIM perspective. The service-url defines the base url of the Azure Function.

{
    "type": "Microsoft.ApiManagement/service",
    "apiVersion": "2019-12-01",
    "name": "[variables('apimInstanceName')]",
    "location": "[parameters('location')]",
    "sku": {
        "name": "Consumption",
        "capacity": 0
    },
    "properties": {
        "publisherEmail": "[parameters('publisherEmail')]",
        "publisherName": "[parameters('publisherName')]"
    },
    "resources": [
        {
            "name": "[variables('apimProductName')]",
            "type": "products",
            "apiVersion": "2019-12-01",
            "properties": {
                "displayName": "[variables('apimProductName')]"
            },
            "dependsOn": [
                "[resourceId('Microsoft.ApiManagement/service', variables('apimInstanceName'))]"
            ],
            "resources": [
                {
                    "name": "[variables('apimApiProductServiceName')]",
                    "type": "apis",
                    "apiVersion": "2019-12-01",
                    "dependsOn": [
                        "[resourceId('Microsoft.ApiManagement/service', variables('apimInstanceName'))]",
                        "[concat(resourceId('Microsoft.ApiManagement/service', variables('apimInstanceName')), '/products/', variables('apimProductName'))]",
                        "[concat(resourceId('Microsoft.ApiManagement/service', variables('apimInstanceName')), '/apis/', variables('apimApiProductServiceName'))]"
                    ]
                }
            ]
        },
        {
            "name": "[variables('apimApiProductServiceName')]",
            "type": "apis",
            "apiVersion": "2019-12-01",
            "dependsOn": [
                "[resourceId('Microsoft.ApiManagement/service', variables('apimInstanceName'))]"
            ],
            "properties": {
                "displayName": "[variables('apimApiProductServiceName')]",
                "path": "product-service",
                "serviceUrl": "https://apim-poc-test.azurewebsites.net/api",
                "protocols": [
                    "http",
                    "https"
                ]
            },
            "resources": []
        }
    ]
}

Operations

One key ingredient is still missing, operations. Operations are the actual API calls that will be available from APIM. I have created two operations. One for retrieving all products, and a second to filter by id. The defined url template is appended to the API service-url we defined earlier.

Most properties of the operation are self-explanatory. The templateParameters, are the parameters that we expect to receive at the APIM endpoint. We can also define required or optional query parameters and headers. For POST requests we define an expected request body.

{
    "name": "[variables('apimApiProductServiceName')]",
    "type": "apis",
    "apiVersion": "2019-12-01",
    "dependsOn": [
        "[resourceId('Microsoft.ApiManagement/service', variables('apimInstanceName'))]"
    ],
    "properties": {
        "displayName": "[variables('apimApiProductServiceName')]",
        "path": "product-service",
        "serviceUrl": "https://apim-poc-test.azurewebsites.net/api",
        "protocols": [
            "http",
            "https"
        ]
    },
    "resources": [
        {
            "name": "get-all",
            "type": "operations",
            "apiVersion": "2019-12-01",
            "dependsOn": [
                "[concat(resourceId('Microsoft.ApiManagement/service', variables('apimInstanceName')), '/apis/', variables('apimApiProductServiceName'))]"
            ],
            "properties": {
                "displayName": "get_products",
                "method": "GET",
                "urlTemplate": "/products"
            }
        },
        {
            "name": "get-by-id",
            "type": "operations",
            "apiVersion": "2019-12-01",
            "dependsOn": [
                "[concat(resourceId('Microsoft.ApiManagement/service', variables('apimInstanceName')), '/apis/', variables('apimApiProductServiceName'))]"
            ],
            "properties": {
                "displayName": "get_product_by_id",
                "method": "GET",
                "urlTemplate": "/products/{id}",
                "templateParameters": [
                    {
                        "name": "id",
                        "type": "number",
                        "required": true
                    }
                ]
            },
            "resources": []
        }
    ]
}

Now we can call APIM, which will redirect our calls to the Azure Function. You can test your APIM implementation using any REST client, but also via the Azure Portal. Once you have deployed your ARM template, the defined operations are shown in the portal. Using the test tab, you can test your operation. For APIM development I prefer this over any other rest client, since it shows you an extensive trace of the retrieved request and how it is passed on to the backend service. In my case the call to retrieve all products is working just fine. Below you can see how the APIM url is translated to the correct Azure Function url.

Policies

As I tried to retrieve a single product, I came to the conclusion that the endpoint is not working yet. I forgot that the Azure Function expects the product id via a query parameter. The APIM get-by-id operation defines the product id as a route parameter. Since the path of the APIM request is just appended to the service url of the API, I am getting a 404 response. The Azure Function only has an endpoint on /products,  there is no endpoint /products/{id}.

In order to redirect the get-by-id request to the Azure Function, we need to introduce something to rewrite the calls to APIM. This is where policies come into play. Policies allow you to change the behavior of APIM. APIM policies are defined in XML. There are multiple types of policies:

Policy Type Usage
Inbound Statements to apply to the request
Backend Statements to apply before the request is forwarded to the backend service
Outbound Statements to apply to the response of the backend
On-error Statements to apply in case of an error

 

There are lots of policies you can create. For this example I am going to create a rewrite inbound policy. This means we are going to alter the requested path to another path. Below you can see how I have defined the policy. It is really basic. You can implement more advanced stuff, like IP-filtering and rate-limiting, as well. Keep in mind that it is possible to create policies in operation scope, but also in global, product and API scope.

<policies>
    <inbound>
        <base />
        <rewrite-uri template="/products?id={id}"/>
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

The last thing to do, is to add this policy to the ARM template. In order to add the policy, you need to join the lines so it becomes a single line, and escape the quotes. Now we can add another resource to the get-by-id operation, a policy type resource. In the value property we can provide the xml of the policy.

{
    "name": "get-by-id",
    "type": "operations",
    "apiVersion": "2019-12-01",
    "dependsOn": [
        "[concat(resourceId('Microsoft.ApiManagement/service', variables('apimInstanceName')), '/apis/', variables('apimApiProductServiceName'))]"
    ],
    "properties": {
        "displayName": "get_product_by_id",
        "method": "GET",
        "urlTemplate": "/products/{id}",
        "templateParameters": [
            {
                "name": "id",
                "type": "number",
                "required": true
            }
        ]
    },
    "resources": [
        {
            "name": "policy",
            "type": "policies",
            "apiVersion": "2019-12-01",
            "dependsOn": [
                "[concat(resourceId('Microsoft.ApiManagement/service', variables('apimInstanceName')), '/apis/', variables('apimApiProductServiceName'), '/operations/get-by-id')]"
            ],
            "properties": {
                "value": "<policies> <inbound> <base /> <rewrite-uri template=\"/products?id={id}\"/> </inbound> <backend> <base /> </backend> <outbound> <base /> </outbound> <on-error> <base /> </on-error> </policies>",
                "format": "xml"
            }
        }
    ]
}

Conclusion

We have just created an APIM instance and added the definition of two endpoints of one service. In the same way you can add APIs for the remaining Azure Functions and the Kubernetes cluster.

I have had some troubles fixing errors in my ARM template. I managed to resolve them using the specification for the ARM template which you can find here. Other great examples for the ARM templates are available in the Microsoft Azure Quickstart Github repo.

In retrospect, I did not like how verbose the ARM template became. I found the definition of the policy in the ARM template to verbose and error prone. I will look for a cleaner way to do so.