Minimal API test config overrides only apply after app has been built

[maisiesadler]

Describe the bug

Using .net 6 minimal APIs and WebApplicationFactory for testing, the test config override is only applied after .Build.
This means any interaction with config while configuring services using builder.Configuration will be using the app configuration, not test overrides.

To Reproduce

I've created a repo that shows the difference, here is the Program, and an Example test.

Test run

Further technical details

• ASP.NET Core version: net6.0

[martincostello]

I think the problem here is that you're binding the configuration into your service while the application is still in its bootstrapping phase (which is until Build()), so the configuration isn't yet "done". It's similar to the sort of issue where you need configuration to configure other things, like if you're say using Azure Key Vault, and need to use config to configure the Key Vault provider itself.

Does it work as you'd expect if you change it to this instead?

builder.Services.AddSingleton(() => new Options(builder.Configuration["AppSettings:Options"]));

I think if you resolve it lazily like that, when the lambda to get Options runs the Configuration would then have the extra settings in it from the tests.

[maisiesadler]

Hey Martin 👋

Yep that works - this might just be a gotcha rather than a bug since the Configuration is available on builder but it's not "Done" :(
It also works using .net 6 but WebHost.CreateDefaultBuilder rather than the new WebApplication.CreateBuilder
diff
test run

[martincostello]

Yeah it's probably more of a gotcha of the new model - the WebApplicationFactory code gets invoked as part of a callback that gets called before Build() returns back to the Program class, so the code that adds your test settings won't even have run yet when you register that dependency so it can't see your overrides as they don't exist yet.

[maisiesadler]

Makes sense. Will update the code and close the issue - thanks!

[halter73]

Reopening so we document this. This might also be worth trying to fix in .NET 7. Thanks for the great repro.

[davidfowl]

@halter73 are you gonna doc this?

[kristofferjalen]

Here's my question on SO on the same issue: https://stackoverflow.com/questions/69986598/with-webapplicationfactory-add-configuration-source-before-program-cs-executes

[twinter-amosfivesix]

I stumbled upon a workaround for this issue.

If you override WebApplicationFactory<T>.CreateHost() and call IHostBuilder.ConfigureHostConfiguration() before calling base.CreateHost() the configuration you add will be visible between WebApplication.CreateBuilder() and builder.Build(). From my testing it's visible in all the various places you'd look at the config (callbacks, etc.).

I have not tried it with adding a JSON file but I imagine it should work.

Note that calling IHostBuilder.ConfigureAppConfiguration() in this manner does not work. DeferredHostBuilder treats host configuration differently from app configuration. (Code part 1 - Code part 2)

The comments in DeferredHostBuilder.ConfigureHostConfiguration actually say "We can do this for app configuration as well if it becomes necessary." Perhaps it's necessary now. 😄

(The PR that made ConfigureHostConfiguration() work like this in DeferredHostBuilder.)

Example Program

var builder = WebApplication.CreateBuilder(args);

var result = builder.Configuration["Result"];

var app = builder.Build();

app.MapGet("/", () => result);

app.Run();

public partial class Program{ }

Example appsettings.json

{
    "Result" : "Normal Result"
}

Example Test

using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using NUnit.Framework;

namespace TestProject;

[TestFixture]
public class ProgramFixture
{
    [Test]
    public async Task Should_return_override()
    {
        var factory = new MyWebApplicationFactory();

        var client = factory.CreateClient();

        var result = await client.GetStringAsync("/");

        Assert.That(result, Is.EqualTo("Override"));
    }

    private class MyWebApplicationFactory : WebApplicationFactory<Program>
    {
        protected override IHost CreateHost(IHostBuilder builder)
        {
            builder.ConfigureHostConfiguration(config =>
            {
                config.AddInMemoryCollection(new Dictionary<string, string> { { "Result", "Override" } });
            });

            // This does not work.
            //builder.ConfigureAppConfiguration(config =>
            //{
            //    config.AddInMemoryCollection(new Dictionary<string, string> { { "Result", "Override" } });
            //});

            return base.CreateHost(builder);
        }

        // This does not work.
        //protected override void ConfigureWebHost(IWebHostBuilder builder)
        //{
        //    builder.ConfigureAppConfiguration(config =>
        //    {
        //        config.AddInMemoryCollection(new Dictionary<string, string> { { "Result", "Override" } });
        //    });
        //}
    }
}

[bvachon]

The above workaround did not work for me because I wanted the AddInMemoryCollection to override the existing config (ie: be added last). So here's what I came up with instead.

Program.cs:

    var builder = WebApplication.CreateBuilder(args);

    builder.WebHost.ConfigureAppConfiguration(builder =>
    {
        builder
            .AddInMemoryCollection(s_ConfigOverride);
    });
    
    ...

    public partial class Program 
    {
        static Dictionary<string, string> s_ConfigOverride { get; set; } = new();
    
        class ClearConfigOverride : IDisposable
        {
            public void Dispose() => s_ConfigOverride = new Dictionary<string, string>();
        }
    
        public static IDisposable OverrideConfig(Dictionary<string, string> config)
        {
            s_ConfigOverride = config;
            return new ClearConfigOverride();
        }
    }

UnitTest.cs

      [OneTimeSetUp]
      public void OneTimeSetup()
      {
          using var _ = Program.OverrideConfig(new Dictionary<string, string>()
          {
              { "MyConfig", "someValue" }
          });

          var application = new WebApplicationFactory<Program>()
              .WithWebHostBuilder(builder =>
              {
                  // ... Configure test services
              });
        }
        ....

[diegosasw]

Having the same issue.
I'm able to set the environment to "Test" for my functional tests but when reading settings at runtime (during tests), the appsettings.Test.json is being ignored when using minimal API

I use it like this:`

        var application =
            new WebApplicationFactory<Program>()
                .WithWebHostBuilder(
                    config =>
                        config
                            .UseCommonConfiguration()
                            .UseEnvironment("Test")
                            .ConfigureTestServices(ConfigureTestServices));

And I have the extension

public static class WebHostBuilderExtensions
{
    public static IWebHostBuilder UseCommonConfiguration(this IWebHostBuilder builder)
    {
        builder.ConfigureAppConfiguration((hostingContext, config) =>
        {
            var env = hostingContext.HostingEnvironment;

            config
                .AddJsonFile("appsettings.json", optional: true, reloadOnChange: false)
                .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: false)
                .AddEnvironmentVariables();

            if (env.IsDevelopment())
            {
                var appAssembly = Assembly.Load(new AssemblyName(env.ApplicationName));
                config.AddUserSecrets(appAssembly, optional: true);
            }
        });

        return builder;
    }
}

I don't understand why some of you suggest this is a gotcha rather than a bug.

If I set the webhost builder to use environment Test, I would expect the IConfiguration to read from the appsettings.Test.json.

Also, why do you say it applies only after app has been built?
I can see my builder.Build() happening before the IConfiguration is retrieved from the DI container and settings read

This is my Program.cs

using Rubiko.Logging.Microsoft.DependencyInjection;
using Sample.Extensions;

const string corsPolicy = "corsPolicy";

var builder = WebApplication.CreateBuilder(args);
var configuration = builder.Configuration;

// IoCC
var services = builder.Services;

services
    .AddMicrosoftFactoryLogging(configuration.GetSection("Logging"))
    .AddMicroserviceDependencies()
    .AddCors(corsPolicy);

var app = builder.Build();

// Http Pipeline
var assemblyPath = $"/{typeof(Program).Assembly.GetName().Name!.ToLowerInvariant()}";

app
    .UsePathBase(assemblyPath)
    .UseRouting()
    .UseCors(corsPolicy)
    .UseDeveloperExceptionPage()
    .UseOpenApi(assemblyPath)
    .UseAuthorization()
    .UseEndpoints(endpoints =>
    {
        endpoints.MapControllers();
    });

app.Run();

// See https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-6.0
public partial class Program { }

If anyone has a workaround, please let me know. But I will probably go back to the old fashioned Startup.cs and the

        var server =
            new TestServer(
                new WebHostBuilder()
                    .UseStartup<Startup>()
                    .UseCommonConfiguration()
                    .UseEnvironment("Test")
                    .ConfigureTestServices(ConfigureTestServices));

[humbe76]

This is blocking my effort to move to new ways of app init too.. Minimal code looks like a total anti-feature. To be able to show of a one-liner for hello world, we have to cope with a lot of new magic that complicates flexibility in unit testing, making it harder to make anything that is a real application..

Before .NET 6 we implement a main method, handle the command line arguments and call into web API init code. That way it is easy for us to make unit tests test everything and provide anything wanted from the unit test libraries into the application to mock, stub, and avoid running code that will blow up if not in production environment.

I'm unable to find any documentation of how to convert the minimal code to actual readable normal code which one can integrate with, and I have not been able to find a workaround to make a unit test to be able to provide input readable from the very beginning of the Program.cs file.

Going through a static variable of course works, but I want to avoid that as I want multiple unit tests to be able to run in parallel.

• Within the Program.cs one can not refer to this, so even using a static collection with this.GetHashCode() to index into it doesn't work.
• If I create members in a partial Program.cs implementation, I can't access them from the minimal code.
• There's no member in the auto-generated Program.cs file I can set something that is available at the top of Program.cs
• I've found no way to give it the command line arguments I want to test with.

All I've managed to do so far is inject a service that can be fetched after builder.Build() has been called, but that is too late..

[davidfowl]

We're going to tackle this in .NET 8, it'll require some new APIs to make it work well.

[kaylumah]

Inside protected override void ConfigureWebHost(IWebHostBuilder builder)

var config = new ConfigurationBuilder()
                .AddInMemoryCollection(new Dictionary<string, string>() {})
                .Build();
builder.UseConfiguration(config);

Appears to work as an alternative for the ConfigureAppConfiguration but not sure if that Is desired.

Guidance on this would be appreciated especially if it would be delayed to .NET8
Thanks
Max

[mikeblakeuk]

Sad times.

[ghost]

Thanks for contacting us.

We're moving this issue to the .NET 8 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[opolkosergey]

I've faced with similar issue in my tests. I need to use class inherited from WebApplicationFactory<Program> in my tests. The issue is that in Production scenario Program.cs initializes Serilog with MSSqlServer sink. So I have to have to provide connectionString - it isn't what I would like to do in tests. Part of my appsettings.json (located in project under test) is:
"Serilog": { "Using": [ "Serilog.Sinks.MSSqlServer" ], "MinimumLevel": { ... } }, "WriteTo": [ { "Name": "MSSqlServer", "Args": { "connectionString": "{connection_string}", "sinkOptionsSection": { ... }, "columnOptionsSection": { "customColumns": [ ... ] } } } ] }

This answer prompted me to find workaround for my case:

    internal class MyWebApplicationFactory : WebApplicationFactory<Program>
    {
        ...
        protected override IHost CreateHost(IHostBuilder builder)
        {
            builder.ConfigureHostConfiguration(config =>
            {
                config.AddJsonFile("appsettings.json");
            });
            return base.CreateHost(builder);
        }		
        ...
    }

where appsettings.json (located in test project) is:

{ "Serilog": { "WriteTo": [ { "Name": "Console" }, { "Name": "Console" } ] } }

Set this advanced settings for your test project

[JBastiaan]

So its really great that it is being worked on for .NET 8, but here we are on .NET 7. How is anyone supposed to make test configuration overrides work now? Are we just supposed to abandon the minimal hosting model until .NET 8? None of the above workaround work for me since im calling some extension method on startup that just requires a string value from config. Lazy loading won't help me, the overridden config value needs to be there when the app is starting. Did anyone find a workaround for this?

[diegosasw]

So its really great that it is being worked on for .NET 8, but here we are on .NET 7. How is anyone supposed to make test configuration overrides work now? Are we just supposed to abandon the minimal hosting model until .NET 8? None of the above workaround work for me since im calling some extension method on startup that just requires a string value from config. Lazy loading won't help me, the overridden config value needs to be there when the app is starting. Did anyone find a workaround for this?

I couldn't make it work either, so I abandoned minimal API and keep using the standard Startup + Program.

[cjbush]

So I was able to get this to work with a kind-of-hacky workaround. My Program.cs looks something like this:

var app = WebApplication.CreateBuilder(args)
                        .BuildDefaultConfiguration()
                        .ConfigureServices((services, configuration) => 
                        {
                            //configure IServiceCollection
                        })
                        .Build()
                        .ConfigureApplication(app => 
                        {
                            app.UseHealthChecks("/hc", new HealthCheckOptions
                            {
                                Predicate = _ => true,
                                ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
                            });

                            //define your other routes and what not
                        });
await app.RunAsync();

and I have the following extension methods defined:

public static class BootstrappingExtensions 
{
    public static WebApplicationBuilder BuildDefaultConfiguration(this WebApplicationBuilder builder)
    {
         if(builder.Configuration.GetValue("IsConfigured", false)) return builder; // this is the hacky/magic part
         //load appsettings.json or Azure App Config or whatever you want
         return builder;
    }
    

    public static WebApplicationBuilder ConfigureServices(this WebApplicationBuilder builder, Action<IServiceCollection, IConfiguration> configure)
    {
         configure(builder.Services, builder.Configuration);
         return builder;
    }

    public static WebApplication ConfigureApplication(this WebApplication app, Action<WebApplication> configure)
    {
         configure(app);
         return app;
    }
}

then in my test fixture I have:

[TestFixture]
public sealed class ProgramTest
{
    private sealed class ProgramFactory  : WebApplicationFactory<Program>
    {
        protected override IHost CreateHost(IHostBuilder builder)
        {
            builder.ConfigureHostConfiguration((configBuilder) => 
            {
                configBuilder.Sources.Clear();
                configBuilder.AddInMemoryCollection(new Dictionary<string,string>
                {
                    { "ConnectionStrings:MyDatabaseConnection", "<some test value>" },
                    { "IsConfigured", "true" }
                });
           });
        }
    }

    [Test]
    public async Task Program_HealthCheck_ShouldReturnHealthy()
    {
        using var application = new ProgramFactory();
        using var client = application.CreateClient();

        var response = await client.GetAsync("/hc");
    
        var report = await response.Content.ReadAsJsonAsync<UIHealthReport>(); //ymmv as far as json options goes here
        report.Should().NotBeNull().And.Match<UIHealthReport>(_ => _.Status == UIHealthStatus.Healthy);
    }
}

So the downside obviously is you're setting a test flag and checking for it in production code, but it does seem to work for overriding the config with minimal APIs until a real fix comes along in .NET 8.

[dj-nitehawk]

I have just discovered why any of the above was NOT working in my solution.

I was creating builder in my "Program.cs" like this: var builder = WebApplication.CreateBuilder();

I tried all of the above and in all different places (like ConfigureWebHost, CreateHost, etc.). Nothing worked.

Then I modified my Program.cs with this:

var builder = WebApplication.CreateBuilder(args);

And suddenly every method above was working. So by only adding args it is working (I guess it is identified as different method)

omg thank you so much!
i was losing my mind trying to hunt down why the environment set in WAF wasn't being picked up by the application.
all i had to do was add the args to WebApplication.CreateBuilder method.
so anyone reading this on .NET 7+ just make sure you create your app builder like this:

var builder = WebApplication.CreateBuilder(args);

[Rick-van-Dam]

Some ppl seem to imply/hope this is getting fixed in .NET 8. Can anyone confirm this is actually getting fixed in .NET 8? This is not just a gotcha but a breaking change in behavior between the old startup and the new WebApplication way of doing things. All the workarounds posted here seem to be pretty hacky and not a true solution to the problem.

[davidfowl]

It was not fixed in .NET 8.

[kriscremers]

I have just discovered why any of the above was NOT working in my solution.

I was creating builder in my "Program.cs" like this: var builder = WebApplication.CreateBuilder();

I tried all of the above and in all different places (like ConfigureWebHost, CreateHost, etc.). Nothing worked.

Then I modified my Program.cs with this:

var builder = WebApplication.CreateBuilder(args);

And suddenly every method above was working. So by only adding args it is working (I guess it is identified as different method)

OMG! 🤯 I spent nearly 2 days going through every possible permutation without success. This actually makes the WAF config work on dotnet 6! I had already resigned to using David's temporary workaround until I tried this.

Here's my case: I couldn't get AddUserSecrets to load anything, which was problematic since it contains our AWS configuration settings.

public class TestBase : WebApplicationFactory<Program>
{
    protected override IHost CreateHost(IHostBuilder builder)
    {
        builder.ConfigureHostConfiguration(configBuilder =>
        {
            configBuilder.AddJsonFile("appsettings.json", false);
            configBuilder.AddUserSecrets<Program>(true, true);
        });
        builder.ConfigureServices(ConfigureServices);
        
        return base.CreateHost(builder);
    }
    
    protected virtual void ConfigureServices(IServiceCollection services)
    {
    }
}

[tiaringhio]

I want to add my experience in case anyone finds it useful:

i'm using .net 8 with testcontainers an therefore need to add my configurations, i thought i should clear the sources before adding news ones but i was wrong, just removing the line fixed the issue for my use case.

protected override IHost CreateHost(IHostBuilder builder)
{
    builder.ConfigureHostConfiguration(config =>
    {
        //config.Sources.Clear(); <-- removed this line right here

        var configuration = new ConfigurationBuilder()
            .AddUserSecrets(typeof(CustomWebApplicationFactory<TProgram>).Assembly, true, true)
            .Build();
        
        config.AddConfiguration(configuration);
        
        config.Sources.Add(new MemoryConfigurationSource
        {
            InitialData = new Dictionary<string, string>
            {
                {"ConnectionStrings:Database", _database.GetConnectionString().Replace("Database", "Initial Catalog")},
                {"ConnectionStrings:Storage", _storage.GetConnectionString()},
                {"ConnectionStrings:Redis:Host", _redis.Hostname},
                {"ConnectionStrings:Redis:Password", string.Empty}
            }!
        });
    });

    return base.CreateHost(builder);
}

[KennethHoff]

@davidfowl This was not mentioned in any Asp.Net Core 9 "what we are doing" posts. I'm assuming that's mostly because this is a small change/bug fix? This is still in the plans, right?

I'm asking since it's still in the .Net 8 Planning Milestone, despite it being launched months ago, and this doesn't feel like a "Fix in Servicing" kind of issue.

[Dreamescaper]

Is it still planned for dotnet 9?
It has net9 planning milestone, but not included in any net9 roadmaps.
Thanks!

[martincostello]

I'm fairly sure at this point it will have missed the boat for inclusion in .NET 9 given no one appears to have done any work on it at all.

[MrBliz]

This seems to be the best workaround so far to override settings.

Also the .ConfigureServices seems to be executed AFTER the Program.cs registrations, so it's a good place to remove/replace service registrations.

With those two things it seems I can get by using mnimal API with .NET 7 without sacrificing integration tests or messing around with environment variables

Example:

protected IntegrationTestBase()
{
    // Workaround for overrides in .NET 7 and minimal API. See https://github.com/dotnet/aspnetcore/issues/37680
    var testConfigurationBuilder =
        new ConfigurationBuilder()
            .AddInMemoryCollection(
                new List<KeyValuePair<string, string?>>
                {
                    new("CoolSection:Colors:0", "red"),
                    new("CoolSection:Colors:1", "black"),
                })
            .Build();
    
    var webApplicationFactory =
        new WebApplicationFactory<Program>() // It requires public partial class Program { } at Program.cs in main assembly
            .WithWebHostBuilder(webHostBuilder =>
                webHostBuilder
                    .UseEnvironment("Development")
                    .UseConfiguration(testConfigurationBuilder)
                    .ConfigureServices(TestServiceOverrides));
    // ...
}

protected virtual void TestServiceOverrides(IServiceCollection servives)
{
     // Replace services in DI container. Virtual method so that test classes inheriting from IntegrationTestBase can override services also
}

Let's hope .NET 8 brings some good IWebHostedBuilder methods that work for these purposes. Because ConfigureTestServices and ConfigureAppConfiguration right now are misleading.

Just dropping into say this worked for me.

[ZaqueuCavalcante]

Just use old school Startup and Program classes works for me

[scrocquesel-ml150]

I tried all the workaround but setting configuration at the host level create empty section value for each section.
For example, given

class TestServer : WebApplicationFactory<Program>
{
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.UseSetting("mysection:myvalue", "test");
    }
}

then, in the app

var builder = WebApplication.CreateBuilder(args);
bool isNull = builder.Configuration.GetSection("mysection").Value is null

will return false. The section has an empty value because the args passed to the application are

--mysection= --mysection:myvalue=test

How can we get rid of this empty value ? It causes issues with Azure SDK (Azure/azure-sdk-for-net#48368)

[grzegorz-wolszczak]

Maybe I don't know something (e.g. I should not call some functions in CreateHost(IHostBuilder builder) etc...), but why can't we just do this:

// Program.cs
var builder = WebApplication.CreateBuilder(args);
var result = builder.Configuration["Result"];
var app = builder.Build();
app.MapGet("/", () => result);
app.Run();
public partial class Program { }

// UnitTest1.cs 
public class WebAppFactory: WebApplicationFactory<Program>
{
   protected override IHost CreateHost(IHostBuilder builder)
   {
      // note, we are NOT calling builder.ConfigureAppConfiguration(...)
      builder.ConfigureHostConfiguration(builder =>
      {
         builder.AddInMemoryCollection(new Dictionary<string, string?>
         {
            ["Result"] = "Hello World!"
         });
      });

      return base.CreateHost(builder);
   }
}

public class UnitTest1
{
   [Fact]
   public async Task Test1()
   {
      var waf = new WebAppFactory();
      var client = waf.CreateClient();

      var response = await client.GetStringAsync("/");
      Assert.Equal("Hello World!", response);
   }
}

[grzegorz-wolszczak]

BTW: one question: Having code like that (explicit controllers)

[ApiController]
public class MyController : ControllerBase
{
   [HttpGet("hello")]
   public IActionResult Hello()
   {
      return Ok("Hello, World2!");
   }
}

public  class Program
{
   public static void Main(string[] args)
   {
      var builder = WebApplication.CreateBuilder(args);
      builder.Services.AddControllers();
      var app = builder.Build();
      app.MapControllers();
      app.Run();
   }
}

I've noticed that if in test I do that:

public class WebAppFactory : WebApplicationFactory<Program>
{
   protected override IHost CreateHost(IHostBuilder builder)
   {
      builder.ConfigureHostConfiguration(builder =>
      {
         //builder.Sources.Clear(); // <-- this breaks controllers
      });

      return base.CreateHost(builder);
   }
}

public class UnitTest1
{
   [Fact]
   public async Task Test1()
   {
      var waf = new WebAppFactory();
      var client = waf.CreateClient();
      var response = await client.GetStringAsync("/hello");
      Assert.Equal("Hello, World2!", response);
   }
}

then if I uncomment the builder.Sources.Clear() - test will fail because client will respond with 404 Not Found. Did anyone maybe happen know what is this the case ? (What does ConfigurationSources have to do with Controller routes?)

[voroninp]

Overriding CreateHost and configuring host does not work for us.
Configuration values are passed to the tested application as a set of command line parameters, but our application switches to CLI mode in this case.

[colinbobolin]

I've been trying to figure out a way to make this work off and on for months now, as I am eager to get started with .NET Aspire and many of the helper functions in Aspire utilize the WebApplicationBuilder interface. Overriding CreateHost has not worked for me either, and I've been trying to get a sense for how to manipulate the DeferredHostBuilder with no success.

[mikekistler]

Removing this feature from the .NET 10 milestone due to resource constraints.