This is another post in a series of articles on creating performant and scalable web APIs using ASP.NET core 2.0. In this post we’ll focus on tools that can help us load test our API to ensure it’s going to perform and scale when it goes into production. Performance and scalability issues are much easier and quicker to resolve before our API has gone into production, so, it’s worth testing our API under simulated demand before it gets there.

WebSurge

WebSurge is a load testing tool for APIs behind the firewall or in the cloud that is really simple and quick to use.

During development, when we think our API is nearly complete, we can use WebSurge to determine how many requests per second our API can handle. So, this is not testing specific user scenarios under load, this is just testing a single API end point under load to see how many requests per second it can handle.

This video gives a great overview on WebSurge.

We’ll use WebSurge frequently during this series of blogs to determine the improvement we can make by doing different things.

Visual Studio Load Testing

If we are lucky enough to have the Enterprise version of Visual Studio, we can use the load testing tool within Visual Studio. It is more flexible than WebSurge but it is more time consuming to write the tests. This is perhaps a good choice if we are writing tests that simulate specific user scenarios under load.

In order to create tests, we first add a “Web Performance and Load Test” project to our solution.

When producing the actual tests we could record the test using a IE plugin that integrates with Visual Studio’s load testing tool. This is great for load testing traditional server driven web applications. However, for testing web APIs, creating a c# unit test is much simpler and gives us a lot of flexibility.

Now that we have a test, we can put this under load. We do this by right clicking on the project and clicking “Add > Load Test …“. A wizard will help us create the load test. We will see that we can use a cloud based load or a load generated from our PC (and potentially other PCs in our infrastructure). The wizard lets us configure lots of stuff such as the test duration, the think times (if we are trying to simulate realistic workloads), the number of users and obviously the tests to run and how they are mixed up during the load. After we have completed the wizard, we can run the load test by clicking the “Run Load Test” icon.

After the load test has finished running, we get a summary of the results. The statistic that I’m most interested in is “Tests / Sec” which is similar to “Requests / Sec” in WebSurge.

BenchmarkDotNet

BenchmarkDotNet is a low level tool to help us understand how our code will run at scale. It’s particularly good at comparing different pieces of code that give the same result to determine which one is the most efficient.

To do this we need to put our code in a .NET Core console app and bring in the BenchmarkDotNet nuget package.

As an example, let’s test the fastest method of iterating through a list to find an item. We’ll compare a classic for loop, a foreach loop and a FirstOrDefault() LINQ statement.

The functions to be tested need to have the [Benchmark] attribute. We simply call BenchmarkRunner.Run() in Main to invoke the test.

class Program
{
    static void Main(string[] args)
    {
        var summary = BenchmarkRunner.Run<FastestFind>();
    }
}

public class FastestFind
{
    private readonly List<Person> people = new List<Person>();
    private readonly int listSize = 10000;
    private readonly string find = "Person1000";

    public FastestFind()
    {
        for (int i = 1; i <= listSize; i++)
        {
            people.Add(new Person()
            {
                Id = i.ToString(),
                Name = $"Person{i}"
            });
        }
    }

    [Benchmark]
    public Person ClassicFor()
    {
        for (int i = 1; i <= listSize; i++)
        {
            if (people[i].Name == find)
            {
                return people[i];
            }
        }
        return null;
    }

    [Benchmark]
    public Person ForEach()
    {
        foreach(var person in people)
        {
            if (person.Name == find)
            {
                return person;
            }
        }
        return null;
    }

    [Benchmark]
    public Person Linq() => people.FirstOrDefault(p => p.Name == find);

}

public class Person
{
    public string Id { get; set; }
    public string Name { get; set; }
}

To simulate what would happen in production, we should build the console app in “release” mode and call it from the command line:

dotnet FastestFindTest.dll

Here’s the results:

… the classic way is always best!

So, now we’ve got the tools we need to test and profile our ASP.NET Core Web API. In the next post we’ll get into the actual code, starting with our data access code.

Comments

Meziantou January 31, 2018

I’m aware the main subject of the post is not the benchmark itself, but you can use List.Find which uses a for loop. So, the performance are equivalent to the ClassicFor but much simplier to use :

[Benchmark]
public Person ListFind() => people.Find(p => p.Name == find);

Carl February 1, 2018 Thanks Meziantou, good point about List.Find()!

Corey Weathers February 1, 2018 Neat post. Thank you for putting this together. I’m super curious about one thing. For Visual Studio Load Testing, have you taken a look at web performance tests? They shouldn’t require any code for a basic test like the one demonstrated in the blog post.

Carl February 1, 2018 Thanks Corey – good point! I’ve used web performance tests in the past to test the performance of ASP.NET web forms or MVC web apps but struggled with it when testing a real web API. Simple GET requests (like the one in my example) are fine, I struggled with POSTs PUTs & DELETEs and chaining requests together with variables in URL parameters, request bodies and HTTP headers ….

kenji March 3, 2018 nice post! it’ll help me a lot!!

Ike March 18, 2018 Just stumbled on this series and enjoying it so far.

Regarding BenchMarkDotNet: You can now also run it from dotnetscript (instead of writing a full fledged program). https://www.strathweb.com/2018/03/lightweight-net-core-benchmarking-with-benchmarkdotnet-and-dotnet-script/

Mario June 19, 2018

Hey Carl,

Awesome article. Do you know by any chance how can we do the load testing in a Azure App Service? Not localhost. Thanks!

Carl June 20, 2018

Thanks Mario. With WebSurge and VS Load Test you simply point the URL you are hitting to the azure one.

Mario September 3, 2018

I’m getting an error that it needs the robot.txt but I’m not able to make it work.I tried using this https://github.com/stormid/Robotify.AspNetCore and placing manually the robot.txt on my wwwroot of my API and still can’t get it to work.

Carl September 3, 2018

Hi Mario, If you are using websurge, have you tried websurge-allow.txt in wwwroot with Allow: WebSurge inside. https://websurge.west-wind.com/docs/_58p0ov73a.htm