Configure QoS in API Gateway using ocelot and Polly

Introduction
API Gateway is an entry point for backend application. It maintains routing, authentication, logging, service discovery etc. Ocelot is used to design and develop API gateway for .net based application. QoS is generally configured in API gateway which provides different priorities for different applications, users or traffic. In this article, we will configure and discuss Quality of Services (QoS) using ocelot and Polly on ASP.NET Core web API project.

What is Quality of Service (QoS)?
QoS provides different priorities to different applications, users or data flow. We have already mentioned, Ocelot is used to design API Gateway and Ocelot uses Polly to achieve QoS.

The QoSOptions node contains three important properties.

• ExceptionsAllowedBeforeBreaking
  This value must greater than 0. It means that the circuit breaker will break after a certain number of exceptions occur. For example:

• DurationOfBreak
  This value specifies how long the circuit breaker will stay open after it is tripped. The unit of this value is milliseconds. For example: 5000 means 5 seconds

• TimeoutValue
  This value specifies that a request will automatically be timed out if it takes more than this value. The unit of this value is milliseconds as well. For example: 3000 means 3 seconds.

Tools and Technology used

• Visual Studio 2022
• .NET 6.0
• In Memory Database
• Entity Framework
• ASP.NET Core Web API
• Visual C#
• Ocelot and
• MMLib.SwaggerForOcelot

Implementation

Step 1: Create solution and projects.

• Create a solution name QoS.
• Add 2 new web api projects, name – Catalog.API, BFF.Web
  Here, BFF.Web project will act as API Gateway.

Step 2: Install nuget packages.

• Install following nuget packages in Catalog.API

PM> Install-Package Microsoft.EntityFrameworkCore.InMemory
PM> Install-Package Microsoft.EntityFrameworkCore.SqlServer
PM> Install-Package Microsoft.EntityFrameworkCore.Tools
PM> Install-Package Microsoft.VisualStudio.Web.CodeGeneration.Design

• Install following nuget packages in BFF.Web

PM> Install-Package MMLib.SwaggerForOcelot
PM> Install-Package Ocelot
PM> Install-Package Ocelot.Provider.Polly

Step 3: Organize Catalog.API

• Create CatalogItem model in Model folder

CatalogItem.cs

using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;

namespace Catalog.API.Model
{
    public class CatalogItem
    {
        [Key]
        [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
        public int Id { get; set; }
        public string Name { get; set; }

        public string Description { get; set; }

        public decimal Price { get; set; }

        public int AvailableStock { get; set; }

        public int RestockThreshold { get; set; }
    }
}

• Create DbContext class as CatalogContext in Db folder

CatalogContext.cs

using Catalog.API.Model;
using Microsoft.EntityFrameworkCore;

namespace Catalog.API.Db
{
    public class CatalogContext : DbContext
    {
        public CatalogContext(DbContextOptions<CatalogContext> options) : base(options)
        {

        }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            base.OnConfiguring(optionsBuilder);
        }

        public DbSet<CatalogItem> CatalogItems { get; set; }
    }
}

• Create SeedDataProvider class in Db folder

SeedDataProvider.cs

using Catalog.API.Model;

namespace Catalog.API.Db
{
    public class SeedDataProvider
    {
        public static void Initialize(CatalogContext catalogContext)
        {
            if(!catalogContext.CatalogItems.Any())
            {
                var catalogs = new List<CatalogItem>
                {
                    new CatalogItem
                    {
                        Name = "T-Shirt",
                        Description = "Cats Eye T-Shirt",
                        Price = 1000,
                        AvailableStock = 100,
                        RestockThreshold = 10
                    },

                    new CatalogItem
                    {
                        Name = "Samsung Mobile",
                        Description = "Samsung A30",
                        Price = 30000,
                        AvailableStock = 100,
                        RestockThreshold = 5
                    },

                    new CatalogItem
                    {
                        Name = "Meril Beauty Soap",
                        Description = "Beauty Soap",
                        Price = 40,
                        AvailableStock = 500,
                        RestockThreshold = 20
                    }
                };

                catalogContext.CatalogItems.AddRange(catalogs);
                catalogContext.SaveChanges();
            }
        }
    }
}

• Modify Program class as follows.

Program.cs

using Catalog.API.Db;
using Microsoft.EntityFrameworkCore;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.

builder.Services.AddControllers();

builder.Services.AddDbContext<CatalogContext>(opt => opt.UseInMemoryDatabase("CatalogDB"));

// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

var app = builder.Build();

// For Seed data generation
using(var scope = app.Services.CreateScope())
{
    var services = scope.ServiceProvider;
    var db = services.GetRequiredService<CatalogContext>();
    SeedDataProvider.Initialize(db);
}

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();

app.UseAuthorization();

app.MapControllers();

app.Run();

Here, the following line is used to configure in memory database

builder.Services.AddDbContext<CatalogContext>(opt => opt.UseInMemoryDatabase("CatalogDB"));

The following code snippet is used to initialize seed data

using(var scope = app.Services.CreateScope())
{
    var services = scope.ServiceProvider;
    var db = services.GetRequiredService<CatalogContext>();
    SeedDataProvider.Initialize(db);
}

• Add CatalogItemsController class in Controllers folder as follows.

CatalogItemsController.cs

using Catalog.API.Db;
using Catalog.API.Model;
using Microsoft.AspNetCore.Mvc;
using Microsoft.EntityFrameworkCore;

namespace Catalog.API.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class CatalogItemsController : ControllerBase
    {
        private readonly CatalogContext _context;
        private static int _count = 0;

        public CatalogItemsController(CatalogContext context)
        {
            _context = context;
        }

        // GET: api/CatalogItems
        [HttpGet]
        public async Task<ActionResult<IEnumerable<CatalogItem>>> GetCatalogItems()
        {
            _count++;
            if(_count <= 3)
            {
                // Sleep for 4 seconds
                Thread.Sleep(4000);
            }

            if (_context.CatalogItems == null)
            {
                return NotFound();
            }
            return await _context.CatalogItems.ToListAsync();
        }

        // GET: api/CatalogItems/5
        [HttpGet("{id}")]
        public async Task<ActionResult<CatalogItem>> GetCatalogItem(int id)
        {
            if (_context.CatalogItems == null)
            {
                return NotFound();
            }
            var catalogItem = await _context.CatalogItems.FindAsync(id);

            if (catalogItem == null)
            {
                return NotFound();
            }

            return catalogItem;
        }

        // PUT: api/CatalogItems/5
        // To protect from overposting attacks, see https://go.microsoft.com/fwlink/?linkid=2123754
        [HttpPut("{id}")]
        public async Task<IActionResult> PutCatalogItem(int id, CatalogItem catalogItem)
        {
            if (id != catalogItem.Id)
            {
                return BadRequest();
            }

            _context.Entry(catalogItem).State = EntityState.Modified;

            try
            {
                await _context.SaveChangesAsync();
            }
            catch (DbUpdateConcurrencyException)
            {
                if (!CatalogItemExists(id))
                {
                    return NotFound();
                }
                else
                {
                    throw;
                }
            }

            return NoContent();
        }

        // POST: api/CatalogItems
        // To protect from overposting attacks, see https://go.microsoft.com/fwlink/?linkid=2123754
        [HttpPost]
        public async Task<ActionResult<CatalogItem>> PostCatalogItem(CatalogItem catalogItem)
        {
            if (_context.CatalogItems == null)
            {
                return Problem("Entity set 'CatalogContext.CatalogItems'  is null.");
            }
            _context.CatalogItems.Add(catalogItem);
            await _context.SaveChangesAsync();

            return CreatedAtAction("GetCatalogItem", new { id = catalogItem.Id }, catalogItem);
        }

        // DELETE: api/CatalogItems/5
        [HttpDelete("{id}")]
        public async Task<IActionResult> DeleteCatalogItem(int id)
        {
            if (_context.CatalogItems == null)
            {
                return NotFound();
            }
            var catalogItem = await _context.CatalogItems.FindAsync(id);
            if (catalogItem == null)
            {
                return NotFound();
            }

            _context.CatalogItems.Remove(catalogItem);
            await _context.SaveChangesAsync();

            return NoContent();
        }

        private bool CatalogItemExists(int id)
        {
            return (_context.CatalogItems?.Any(e => e.Id == id)).GetValueOrDefault();
        }
    }
}

Step 4: Organize BFF.Web

In this stage, we are going to configure a gateway to communicate with other services using ocelot.

• Create a folder name - Routes.dev in root directory and add the following files. ocelot.catalog-api.json, ocelot.global.json, ocelot.SwaggerEndPoints.json in Routes.dev folder.

• Now modify the json files as follows.

ocelot.catalog-api.json

{
  "Routes": [
    {
      "DownstreamPathTemplate": "/{everything}",
      "DownstreamScheme": "https",
      "SwaggerKey": "catalog-api",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": "7282"
        }
      ],
      "UpstreamPathTemplate": "/catalog/{everything}",
      "UpstreamHttpMethod": [
        "GET",
        "POST",
        "PUT",
        "DELETE"
      ],

      "QoSOptions": {
        "ExceptionsAllowedBeforeBreaking": 2,
        "DurationOfBreak": 5000,
        "TimeoutValue": 3000
      }

    }
  ]
}

QoSOptions section of the above file basically configure QoS for Catalog service. The above configuration means that if the server does not response for 3 minutes, it will throw timeout exception. If the server throws two exceptions, it will not be accessible for five minutes.

ocelot.global.json

{
  "GlobalConfiguration": {
    "BaseUrl": "https://localhost:7205"
    //"ServiceDiscoveryProvider": {

    //  "Host": "localhost",
    //  "Port": 7205
    //}
  }
}

ocelot.SwaggerEndPoints.json

{
  "SwaggerEndPoints": [
    {
      "Key": "bffweb",
      "TransformByOcelotConfig": false,
      "Config": [
        {
          "Name": "BFF.Web",
          "Version": "1.0",
          "Url": "https://localhost:7205/swagger/v1/swagger.json"
        }
      ]
    },

    {
      "Key": "catalog-api",
      "TransformByOcelotConfig": true,
      "Config": [
        {
          "Name": "Catalog.API",
          "Version": "1.0",
          "Url": "https://localhost:7205/catalog/swagger/v1/swagger.json"
        }
      ]
    }
  ]
}

Note: I have added the following code block in CatalogItemController to produce timeout manually.

        [HttpGet]
        public async Task<ActionResult<IEnumerable<CatalogItem>>> GetCatalogItems()
        {
            _count++;
            if(_count <= 3)
            {
                // Sleep for 4 seconds
                Thread.Sleep(4000);
            }

            if (_context.CatalogItems == null)
            {
                return NotFound();
            }
            return await _context.CatalogItems.ToListAsync();
        }

Step 5: Run and test the application

Now run both web projects. In the BFF.Web, select Catalog.API-1.0 from swagger definition (“Select a definition on the top right corner”) and execute the api (CatalogItems) as follows.

When we visit the first time (or quickly second time), it tells us that circuit is breaking for 5000 ms. Look at the console of BFF.Web.

Then, the second time (quickly) it tells us that the circuit is open, and we cannot visit the service for 3 seconds as follows.

After 3 seconds, the service is accessible. If you execute now, you will see the output like below.

Source code