[MVC]Improve scalability in ASP.NET MVC using Asyn

ASP.NET applications run with a fixed-size thread pool. By default, they have 200 (or 250? I forget…) threads available to handle requests. You can make the thread pool bigger if you want, but it’s not usually helpful: more contention and overhead from thread switching will eventually actually reduce the server’s throughput. But what if most of your threads are actually sitting around doing nothing, because they’re actually waiting for some external I/O operation to complete, such as a database query or a call to an external web service? Surely things could be more efficient…

Well yes, actually. Ever since Windows NT 4 we’ve had a notion of I/O Completion Ports (IOCP) which are a mechanism for waiting for I/O to complete without causing thread contention in the meantime. .NET has a special thread pool reserved for threads waiting on IOCP, and you can take advantage of that in your ASP.NET MVC application.

The IHttpAsyncHandler, first introduced in ASP.NET 2.0, splits request processing into two. Instead of handling an entire request in one thread (expensive I/O and all), it first does some processing in the normal ASP.NET thread pool, as per any normal request, then when it’s time for I/O, it transfers control to the I/O thread, releasing the original ASP.NET thread to get on with other requests. When the I/O signals completion (via IOCP), ASP.NET claims another, possibly different thread from its worker pool to finish off the request. Thus, the ASP.NET thread pool doesn’t get ‘clogged up’ with threads that are actually just waiting for I/O.

Adding asynchronous processing to ASP.NET MVC

The MVC framework doesn’t (yet) come with any built-in support for asynchronous requests. But it’s a very extensible framework, so we can add support quite easily. First, we define AsyncController:

public class AsyncController : Controller
{
internal AsyncCallback Callback { get; set; }
internal IAsyncResult Result { get; set; }
internal Action<IAsyncResult> OnCompletion { get; set; }
 
protected void RegisterAsyncTask(Func<AsyncCallback, IAsyncResult> beginInvoke, Action<IAsyncResult> endInvoke)
{
OnCompletion = endInvoke;
Result = beginInvoke(Callback);
}
}

It’s just like a normal Controller, except it manages some internal state to do with asynchronous processing, and has the RegisterAsyncTask() method that lets you manage transitions across the gap between the two halves of IHttpAsyncHandler processing. Note: it doesn’t implement IHttpAsyncHandler itself; that’s the job of the IRouteHandler we have to set up. Unfortunately I had to reproduce most of the code from the framework’s MvcHandler, because I couldn’t just override any individual method and still get at the controller:

public class AsyncMvcRouteHandler : IRouteHandler
{
public IHttpHandler GetHttpHandler(RequestContext requestContext)
{
return new AsyncMvcHandler(requestContext);
}
 
class AsyncMvcHandler : IHttpAsyncHandler, IRequiresSessionState
{
RequestContext requestContext;
AsyncController asyncController;
HttpContext httpContext;
 
public AsyncMvcHandler(RequestContext context)
{
requestContext = context;
}
 
// IHttpHandler members
public bool IsReusable { get { return false; } }
public void ProcessRequest(HttpContext httpContext) { throw new NotImplementedException(); }
 
// IHttpAsyncHandler members
public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)
{
// Get the controller type
string controllerName = requestContext.RouteData.GetRequiredString("controller");
 
// Obtain an instance of the controller
IControllerFactory factory = ControllerBuilder.Current.GetControllerFactory();
IController controller = factory.CreateController(requestContext, controllerName);
if (controller == null)
throw new InvalidOperationException("Can't locate the controller " + controllerName);
try
{
asyncController = controller as AsyncController;
if (asyncController == null)
throw new InvalidOperationException("Controller isn't an AsyncController.");
 
// Set up asynchronous processing
httpContext = HttpContext.Current; // Save this for later
asyncController.Callback = cb;
(asyncController as IController).Execute(new ControllerContext(requestContext, controller));
return asyncController.Result;
}
finally
{
factory.DisposeController(controller);
}
}
 
public void EndProcessRequest(IAsyncResult result)
{
CallContext.HostContext = httpContext; // So that RenderView() works
asyncController.OnCompletion(result);
}
}
}

It handles requests by supplying an AsyncMvcHandler, which implements IHttpAsyncHandler. Note that during the first half of the processing, i.e. during BeginProcessRequest(), it makes a record of the current HttpContext object. We have to restore that later, during EndProcessRequest(), because we’ll be in a new thread context by then and HttpContext will be null (and that breaks various ASP.NET facilities including WebForms view rendering).

Using AsyncController

It’s now very easy to handle a request asynchronously. Define a route using AsyncMvcRouteHandler, instead of MvcRouteHandler:

routes.Add(new Route("Default.aspx", new AsyncMvcRouteHandler())
{
Defaults = new RouteValueDictionary(new { controller = "Home", action = "Index", id = "" }),
});

Then set up an AsyncController. In this example, we’re calling an external web service using the WebRequest class and its BeginGetResponse() method. That uses an IOCP, so won’t consume an ASP.NET worker thread while it waits:

public class HomeController : AsyncController
{
public void Index()
{
WebRequest req = WebRequest.Create("http://www.example.com");
req.Method = "GET";
 
RegisterAsyncTask(cb => req.BeginGetResponse(cb, null), delegate(IAsyncResult result) {
WebResponse response = req.EndGetResponse(result);
// Do something with the response here if you want
RenderView("Index");
});
}
}

And that’s it. The web request gets set up and started in the first half of the async processing model, then the thread gets released to serve other requests. When the web request signals completion, ASP.NET takes a different thread from its pool, and gets it to run the code inside the anonymous delegate, inheriting the request context from the first thread so it can send output to the visitor.

Just remember that you should only use async requests when you’re waiting for some operation on an IOCP. Don’t use it if you just going to call one of your own delegates asynchronously (e.g. using QueueUserWorkItem()) because that will come out of the ASP.NET worker thread pool and you’ll get exactly zero benefit (but more overhead).