Duende IdentityServer and OTel Metrics, Traces, and Logs in the .NET Aspire Dashboard

IdentityServer offers multiple diagnostic possibilities. The logs contain detailed information and are great for
troubleshooting, but we’re seeing a shift toward using OpenTelemetry to collect metrics, traces, and logs to help
monitor and troubleshoot applications.

This post will examine OpenTelemetry, its use
within Duende IdentityServer, and how to surface all necessary
telemetry signals in the .NET Aspire
Dashboard. Combining these powerful technologies gives you a world-class development experience that helps you
rationalize and implement solutions correctly.

What is OpenTelemetry?

OpenTelemetry, or OTel, is a set of tools, APIs, and SDKs that collect, send, and process telemetry data. It integrates
with many modern development stacks, including .NET. It provides a vendor-agnostic solution for collecting and analyzing
telemetry data from various sources, making monitoring and troubleshooting distributed systems more manageable.

OpenTelemetry is a large project, but users looking to implement it into their solutions should think about the three
signals offered by the toolkit: metrics, logs, and traces. Let’s discuss what these signals provide users and how you
may already use them in your applications.

A metric is a value within a particular unit of measurement. Values are numeric, with the unit of measurement expressed
in units over time, size, or frequency. Some metrics you might see in a .NET application might include “total number of
exceptions”, “HTTP request duration”, or “number of garbage collections”. These metrics can give you a holistic view of
the health of your application. While .NET ships with metrics out of the box, you can create and track custom metrics.
For example,
the Duende IdentityServer UI templates
come with a custom metric to track user logins.

private static Counter<long> UserLoginCounter 
    = Meter.CreateCounter<long>(Counters.UserLogin);

/// <summary>
/// Helper method to increase <see cref="Counters.UserLogin"/> counter.
/// </summary>
/// <param name="clientId">Client Id, if available</param>
public static void UserLogin(string? clientId, string idp)
    => UserLoginCounter.Add(1, new(Tags.Client, clientId), new(Tags.Idp, idp));

Logs are a constant stream of information produced from a running solution. These logs are typically stored from newest
to oldest but can depend on the storage mechanism. If you’ve built an ASP.NET Core application, you’ve likely seen logs
in the console output of the running application.

Request finished HTTP/1.1 POST https://localhost:5001/connect/token - 200 - application/json;+charset=UTF-8 34.9207ms

With recent .NET releases, you can log and store messages in a way entirely up to you. This separation is called *
Structured Logging*, and we’ll see how it works in our .NET example.

The final and arguably most helpful signal championed by OTel is tracing. A trace is a combined view and breakdown of
all work completed during the lifetime of a user request. It lets you see how a system’s parts contribute to the user
experience. Traces can help you diagnose performance issues or optimize for better performance.
The .NET documentation has an excellent tutorial for adding traces to your .NET solutions.
One point of confusion .NET developers may experience is that the naming conventions in .NET do not align with the OTel
naming conventions. As mentioned in the Microsoft Documentation:

OpenTelemetry uses alternate terms 'Tracer' and 'Span'. In .NET 'ActivitySource' is the implementation of Tracer and
Activity is the implementation of 'Span'. .NET's Activity type long pre-dates the OpenTelemetry specification and the
original .NET naming has been preserved for consistency within the .NET ecosystem and .NET application compatibility.

Now that you understand these signals, let’s talk about them in the context of .NET solutions, specifically .NET Aspire
and Duende IdentityServer.

What is Aspire?

.NET Aspire is a collection of tools that enhance modern app development by providing C# APIs, templates, and packages
for building observable, production-ready applications. It aims to help developers codify their solution architecture,
making it easier to reason about design choices and allowing them to evolve decisions over time.

The heart of any .NET Aspire solution lies in the AppHost project, which defines the relationship between web
applications, APIs, databases, or any other dependency. Let’s take a look at an example.

var builder = DistributedApplication.CreateBuilder(args);

var identityServer = 
    builder.AddProject<Projects.IdentityServer>("identityserver");

var apiService = builder
    .AddProject<Projects.Aspire_ApiService>("apiservice")
    .WithReference(identityServer)
    .WaitFor(identityServer);

builder.AddProject<Projects.Aspire_Web>("webfrontend")
    .WithReference(identityServer)
    .WithReference(apiService)
    .WaitFor(apiService);

builder.Build().Run();

This .NET Aspire solution has three major parts: the Duende IdentityServer host, an API Service, and the web application
frontend. The methods WithReference and WaitFor show how these projects depend on each other. These methods
automatically inject configuration from one service to another, allowing us to access important information. For Duende
IdentityServer, we’ll need the Authority URL used in the OpenID Connect configuration or JWT Bearer Authentication.

var configuration = builder.Configuration;
builder.Services.AddAuthentication()
    .AddJwtBearer(opt =>
    {
        opt.Authority = configuration["services:identityserver:https:0"];
        opt.TokenValidationParameters.ValidateAudience = false;
    });

When we start our .NET Aspire solution, we should see all our projects started and healthy in the .NET Aspire Dashboard.

Now that we have a running solution, let’s see how the .NET Aspire Dashboard gives us access to all the OTel signals
mentioned in the previous section and how we can get Duende IdentityServer signals for each category.

Duende IdentityServer in the .NET Aspire Dashboard

All three OTel signals are exposed and reported within all Duende IdentityServer implementations. However, developers
must enable many of these features to reduce the cost of potentially expensive telemetry. We’ll cover how to do this
shortly.

It is good practice only to enable the telemetry you will actively look at and act on; otherwise, you are wasting
valuable time and resources.

Structured Logs in Duende IdentityServer

If you use the ILogger interface in your ASP.NET Core applications, you’re already using structured logging. You only
need to ensure you follow best practices to get the most out of your logging.

logger.LogInformation("Say Hello to {Name}", "Khalid");

While the log message and value may look like a standard string formatting call, each placeholder is a key, and the
values are positional arguments. Invocations of logging methods mean the order of your arguments matters.

// wrong
logger.LogInformation("{LastName} {FirstName}", "Khalid", "Abuhakmeh");

// right
logger.LogInformation("{FirstName} {LastName}", "Khalid", "Abuhakmeh");

Additionally, you should pass all data on as arguments rather than leaning on string interpolation.

// wrong
var LastName = "Abuhakmeh";
var FirstName = "Khalid";
logger.LogInformation($"{LastName} {FirstName}");

// right
logger.LogInformation("{FirstName} {LastName}", "Khalid", "Abuhakmeh");

Duende IdentityServer already follows these best practices, and you should, too. Doing so will improve the logging
experience within your .NET Aspire solutions.

As you can see in the screenshot, while log messages appear in a tabular format on the left, Aspire’s dashboard breaks
down each log message argument into its parts in the log entry details view. This view gives you more information about
each entry than you could fit into a single text-based message.

In your Duende IdentityServer host, you can view logging information for everyday events, such as when a token is issued
or the discovery endpoint is retrieved.

Next, we will examine the traces section of the .NET Aspire dashboard and learn how to profile requests in your
solution.

Traces in Duende IdentityServer

Traces allow us to see the entire lifecycle of a user request across all the services used to complete it. To enable
traces in your .NET Aspire solution, you must first register the “Duende.IdentityServer” source in all your
dependencies if you want the Duende IdentityServer methods to participate in a trace.

Most Aspire solutions will have a shared ServiceDefaults project to share standard infrastructure code. In your
registration code, add the new source.

builder.Services
.AddOpenTelemetry()
.WithMetrics(metrics =>
{
    metrics.AddRuntimeInstrumentation()
        .AddBuiltInMeters();
})
.WithTracing(tracing =>
{
    if (builder.Environment.IsDevelopment())
    {
        // We want to view all traces in development
        tracing.SetSampler(new AlwaysOnSampler());
    }

    tracing.AddAspNetCoreInstrumentation()
           .AddGrpcClientInstrumentation()
           .AddHttpClientInstrumentation()
           // add the Duende.IdentityServer source
           .AddSource("Duende.IdentityServer");
});

Rerunning our .NET Aspire solution, we should start seeing traces originating from within our Duende IdentityServer.

By using traces, we can better understand how our system operates and what elements contribute to the overall
performance profile, which can help us make decisions to improve the user experience.

Let’s examine the metrics tab, which is the final signal for diagnosing overall performance and discovering the cause of
systemic issues.

Metrics in Duende.IdentityServer and the User Interface

Metrics are indicators defined by numeric values and a unit of measurement. To enable Metrics from the Duende
IdentityServer APIs and the custom user interface you implement, you’ll need to update the code in your
ServiceDefaults projects.

builder.Services
.AddOpenTelemetry()
.WithMetrics(metrics =>
{
    metrics.AddRuntimeInstrumentation()
           .AddBuiltInMeters()
           .AddMeter(
               // Stable counters from IdentityServer library
               "Duende.IdentityServer",
               // More counters from IdentityServer library
               "Duende.IdentityServer.Experimental",
               // Counters from the UI in our IdentityServer project.
               "IdentityServer"); 
})
.WithTracing(tracing =>
{
    if (builder.Environment.IsDevelopment())
    {
        // We want to view all traces in development
        tracing.SetSampler(new AlwaysOnSampler());
    }

    tracing.AddAspNetCoreInstrumentation()
           .AddGrpcClientInstrumentation()
           .AddHttpClientInstrumentation()
           .AddSource("Duende.IdentityServer");
});

The call to AddMeter includes two constants of Duende.IdentityServer and Duende.IdentityServer.Experimental. The
last value of IdentityServer comes from the Telemetry class found in the Duende IdentityServer host project under
the Pages directory. You must update this to match the project’s assembly name or the constant places in the
Telemetry class.

Navigating to the Metrics tab, we can select the Resource at the top of the page and monitor any metrics incremented
within our code.

It’s important to note that a category or metric will only appear if the application has logged a value. If you do not
see a particular metric, the code path that emits the value was not evaluated.

Conclusion

ASP.NET Core with OpenTelemetry is an excellent combination for debugging and performance-minded developers. Using .NET
Aspire, we can codify solution architecture while getting a great developer experience with the .NET Aspire Dashboard as
a visualization tool for all telemetry, including the telemetry from your Duende IdentityServer implementations.

We hope you enjoyed this post. You
can get this sample and others from our Samples GitHub repository. Feel
free to leave a comment or start a discussion in our community forum.