# Using Consul for Service Discovery with ASP.NET Core

One of the benefits of adopting a microservice architectural style is the ability compose applications by bringing together smaller units of functionality (aka services). Not only does it become easier to swap out implementations of an individual service, but it also becomes easier to scale that service too. For example, imagine you are running an e-commerce website. The holidays are coming up and there is going to be a huge increase in orders. Instead of creating copies of the entire website to handle the load wouldn't be great if you could just scale up the ordering service or the payment processing service? Then after the holiday season is over those services can be scaled back down. Being able to quickly scale horizontally make microservices a very attractive option.

For an application that's built this way, those services are going to need to be able to talk to each other so that data can flow from one end of the process to the other. Going back to the e-commerce example above, an ordering service might need to talk to the shipping service which talks to the inventory service, and so on. With the way that we typically build software today, the locations of these services would be put in a configuration file somewhere. The configuration file gets loaded up and the application can select which services it wants to talk to. However, when you're dynamically creating and destroying instances of a services, it becomes difficult to keep configuration files updated with the latest information. One way we can solve this issue is by implementing some form of service discovery strategy.

#### Service Discovery

The idea of service discovery essentially is trying to find an answer to the question of what services are available and how do I get to them. Two approaches that you'll often hear about are `Client Side` and `Server Side` service discovery. In this post, we will just focus on `Client Side`.

With `Client Side` service discovery, the consumer of the service has to retrieve a listing of service information from given location. This would lead us to believe that there must be somewhere to retrieve that service information from. The medium where service information is stored and retrieved is referred to as a `service registry`.  
![Client Side Service Discovery](https://cdn.hashnode.com/res/hashnode/image/upload/v1709341139166/2b7cface-64a1-43f3-b93c-f1ac4cbe0ca3.png)

As services go live, they will register some information about themselves into the `service registry`; IP address, port numbers, service names, etc. When a service goes down gracefully, it can deregister itself from the registry. At some point later, the consumer would query the `service registry` to find out about services are available for it to use. It can then cycle through the service information and distribute requests across service instances as it sees fit.

This pattern is fairly straight forward to implement. However, the `service registry` does introduce an additional piece for you to manage. The flexibility you gain from centralizing this configuration is often more than worth it though.

There are a few options for implementing a `service registry`. I've seen implementations using data stores like Redis or document databases. In the Linux world, tools like [ZooKeeper](http://zookeeper.apache.org/?ref=cecilphillip.com), [Consul](https://www.consul.io/?ref=cecilphillip.com) and [etcd](https://coreos.com/etcd/?ref=cecilphillip.com) are very popular. Let's see how we can use `consul` as a service registry.

#### Setting up Consul

`Consul` is a tool created by [Hashicorp](https://www.hashicorp.com/?ref=cecilphillip.com) that helps with the discovery and configuration of services in your infrastructure. It also has quite a few other interesting features such as heath checks, key/value storage and support for running in multiple data centers.

To get `Consul` on your machine, you can head over the [download page](https://www.consul.io/downloads.html?ref=cecilphillip.com) and grab a copy for the OS you are using. The zip file will contain the `Consul` command line executable that you can just run. It might also be available in your OS package manager. If you're using OSX for instance, you can use [homebrew](https://brew.sh/?ref=cecilphillip.com) and `brew install Consul` in the terminal. I like the package manager route because then you'll have the `Consul` command on your system path.

To quickly start `Consul`, enter the following into the command line:

    consul agent -dev
    

`Consul` should now be running in dev mode. In this state, all the data will be stored in memory and not on disk. This is fine for development or demos but definitely not what you want to do on your production machines. If everything went well, you should be seeing something like this:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1709341140027/877f3ef3-49e6-4af4-844e-a835a7b7e4de.png)

Open up your browser and head over to `http://127.0.0.1:8500`. You should now be seeing the `Consul` web UI. Here you can get some insight into what services are registered, their health status, and some other interesting information.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1709341141096/746e09db-413a-4914-810b-8462187272d5.png)

#### Registering a service

Now that the registry is up and running, let's put it to work. I have a Web API that I created with ASP.NET Core that I want to register. To get registration information into `Consul`, their HTTP API can be used directly, but instead I'm going to grab the [Consul NuGet package](https://www.nuget.org/packages/Consul?ref=cecilphillip.com) from PlayFab.

Here's what `Startup.cs` looks like:

    public void ConfigureServices(IServiceCollection services)
    {
        services.Configure<ConsulConfig>(Configuration.GetSection("consulConfig"));
        services.AddSingleton<IConsulClient, ConsulClient>(p => new ConsulClient(consulConfig =>
        {
            var address = Configuration["consulConfig:address"];
            consulConfig.Address = new Uri(address);
        }));          
        services.AddMvc();
    }
    
    public void Configure(IApplicationBuilder app, IHostingEnvironment env, 
                          ILoggerFactory loggerFactory, IApplicationLifetime lifetime)
    {
        loggerFactory.AddConsole();
    
        app.UseMvc();
        app.RegisterWithConsul(lifetime);
    }
    

In `ConfigureServices`, I'm registering an instance of the `ConsulClient` and binding a section of my configuration file to an instance of `ConsulConfig`. Also if you take a look at the `Configure` method, you will see that I added an extension method called `RegisterWithConsul` that I want to be called once whenever an instance of my API gets created. I'm also injecting an instance of `IApplicationLifetime`. More on that later. Let's take a look at `RegisterWithConsul`.

    public static IApplicationBuilder RegisterWithConsul(this IApplicationBuilder app,
             IApplicationLifetime lifetime)
            {
                // Retrieve Consul client from DI
                var consulClient = app.ApplicationServices
                                    .GetRequiredService<IConsulClient>();
                var consulConfig = app.ApplicationServices
                                    .GetRequiredService<IOptions<ConsulConfig>>();
                // Setup logger
                var loggingFactory = app.ApplicationServices
                                    .GetRequiredService<ILoggerFactory>();
                var logger = loggingFactory.CreateLogger<IApplicationBuilder>();
    
                // Get server IP address
                var features = app.Properties["server.Features"] as FeatureCollection;
                var addresses = features.Get<IServerAddressesFeature>();
                var address = addresses.Addresses.First();
                
                // Register service with consul
                var uri = new Uri(address);
                var registration = new AgentServiceRegistration()
                {
                    ID = $"{consulConfig.Value.ServiceID}-{uri.Port}",
                    Name = consulConfig.Value.ServiceName,
                    Address = $"{uri.Scheme}://{uri.Host}",
                    Port = uri.Port,
                    Tags = new[] { "Students", "Courses", "School" }
                };
    
                logger.LogInformation("Registering with Consul");
                consulClient.Agent.ServiceDeregister(registration.ID).Wait();
                consulClient.Agent.ServiceRegister(registration).Wait();
    
                lifetime.ApplicationStopping.Register(() => {
                    logger.LogInformation("Deregistering from Consul");
                    consulClient.Agent.ServiceDeregister(registration.ID).Wait(); 
                });
            }
               
    

The interesting part of this code is closer to the end where the registration is happening. Using an instance of `AgentServiceRegistration` (that's from the Consul NuGet package), I populate some metadata about the API and then register that information with `Consul`.

#### Deregistering the service

Whenever the service shuts down, it would be nice if it would tell our `Consul` service registry that it's not available anymore. To do that, we can leverage the `ApplicationStopping` event/trigger from `IApplicationLifetime`. At the end of `RegisterWithConsul` above, we make a call to `ServiceDeregister` and pass it the ID of the registration we want to remove.

When using these lifetime events on `IApplicationLifetime`, I'd recommend not doing too much work within your callbacks. Consider these events as an opportunity for quickly setting up and gracefully tearing down as needed. If any unhandled exceptions get thrown inside your callbacks, they will get swallowed and will never heard from again.

> If you want to learn more about `IApplicationLifetime`, I'd recommend checking out [Khalid's blog post](http://www.khalidabuhakmeh.com/looking-at-asp-net-cores-iapplicationlifetime?ref=cecilphillip.com) on the subject.

#### Consuming the registrations

On the client that needs to consume the registration information, you can simply create an instance of `ConsulClient` and query the registry. In the code below, I'm using tags to filter out the service instances that I'm interested in. You can always use the service name too if you wish.

    List<Uri> _serverUrls = List<Uri>();
    var consuleClient = new ConsulClient(c => c.Address = new Uri("http://127.0.0.1:8500"));
    var services = consulClient.Agent.Services().Result.Response;
    foreach (var service in services)
    {
        var isSchoolApi = service.Value.Tags.Any(t => t == "School") &&
                          service.Value.Tags.Any(t => t == "Students");
        if (isSchoolApi)
        {
            var serviceUri = new Uri($"{service.Value.Address}:{service.Value.Port}");
            serverUrls.Add(serviceUri);
        }
    }
    
    

The client can now manually load balance or failover its requests between the available service instances. One thing I like to do here is implement a retry policy with something like [Polly](http://www.thepollyproject.org/?ref=cecilphillip.com). After a given number of retries, the client will switch over to the next service.

#### Conclusion

Regardless of the tool you use to register your services, implementing service discovery will make managing your containers and microservices much easier. We covered one implementation of `Client side` discovery here where the service registers/degregisters itself as the instance starts up and shuts down. There are some other options that are just as easy to implement but each with its own trade-offs. If you're interested in seeing more samples, check out this [GitHub repo](https://github.com/cecilphillip/aspnet-servicediscovery-patterns?ref=cecilphillip.com).

In an upcoming post, I'll explore how to enable health checks with `Consul`.