Performance is one of the attributes of software quality. An application with performance levels meeting or exceeding the expectations of its end users can be called performant. Often the term performance is used synonymously with scalability, which is another software quality attribute.
Performance, an indication of the responsiveness of an application, can be measured in terms of latency or throughput. Latency is the time taken by an application to respond to an event, for example the number of seconds taken by a screen to show some search result, in response to a user clicking a search button. Throughput is the number of events that take place in a specified duration of time, for example number of orders processed by an order processing system in a minute.
Scalability is the ability of an application to handle increased usage load without any (or appreciable) degradation of the performance. Scalability also refers to the ability of an application to show improved performance in proportion to the addition of resources such as memory, CPU power, and so on. A performant application need not be scalable, and vice versa, but ideally your application should be both performant and scalable.
The topics of performance and scalability are vast, and when it comes to ASP.NET Web API, these topics typically cut across multiple technologies. For example, if your ASP.NET Web API uses SQL Server as the persistence store and Entity framework for object-relational mapping, and you host it in IIS (web hosting), you need to be performant on all the following areas: the .NET framework in general, ASP.NET, IIS, EF, SQL server and of course ASP.NET Web API. It is not possible to cover all these topics in a single book, let alone a single chapter. Hence, in this chapter I cover only a few important and must-know areas in ASP.NET Web API.
12.1 Creating Asynchronous Action Methods
In this exercise, you will create asynchronous action methods. The objective behind creating an asynchronous method is to handle more requests with the same number of threads in the thread pool. On the IIS server, the .NET framework maintains a thread pool. The threads from this pool are used to service the requests. The number of threads in a thread pool is finite. The number of threads can be increased to the physical limits, but additional threads do add overhead. The better approach will be to serve the requests with fewer threads, in an efficient manner.
When a request comes in, a thread from the thread pool is assigned to process the request. This thread busily works as the request is processed; and on completion of the request, the thread is returned to the pool to service some other request. This is similar to the operation of a post office or bank, 1 where you wait in line and are processed by an available teller. The teller is with you all the time during the transaction, irrespective of whether she is capable of handling someone else while she waits on some external entity to complete your request. Likewise, as the thread allocated from the thread pool services your request, there are times when a thread must wait for an external entity such as a result from a web service call becoming available. During this time, the thread does nothing but is still allocated to your request because the service call is a blocking call. For this reason, the thread cannot return to the pool. As the number of incoming requests exceeds the number of threads that are free in the pool, requests start queuing up, as is the case when you visit your bank during the lunch break. If you somehow return the thread to the pool and take the same or another thread when the result from the web service is available to resume processing, you will be able to process more requests with the same number of threads. In a restaurant, by contrast, a waiter takes the order, puts it in for the chef to work on, and goes to some other table to take orders. Eventually, when the food is available, the waiter appears to serve you the food, but she does not stand there staring at you, while you eat! The waiter just works on some other table, until the time you are in need of something. In the case of ASP.NET Web API, a processing mechanism similar to the restaurant model is possible with the use of asynchronous controllers.
Asynchronous action methods allow you to start a long-running operation, return your thread to the pool, and then wake up on a different thread or the same thread depending on the availability of threads in the pool at that time, when the operation is complete, to resume the processing. The async and await keywords of C# 5.0 makes writing asynchronous methods easy.
However, asynchronous methods are not suitable for operations that are CPU-intensive, generally called CPU-bound. Using asynchronous action methods on such CPU-bound operations provides no benefits and actually results in overhead from switching the threads. Using asynchronous methods for the operations that are network-bound or I/O-bound, as in the case of calling an external service or reading or writing a large chunk of data from the hard disk, is typically beneficial. This basically means that we should start with normal synchronous methods and switch to asynchronous methods on a case-by-case basis.
Listing 12-1. The ValuesController Class
using System.IO;
using System.Threading;
using System.Web.Http;
public class ValuesController : ApiController
{
public string Get(int id)
{
return ReadFile();
}
private string ReadFile()
{
using (StreamReader reader = File.OpenText(@"C: < Path >SomeFile.txt"))
{
Thread.Sleep(500);
return reader.ReadToEnd();
}
}
}
C:>ab -n 60 -c 60 http://localhost:35535/api/values/1
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.121.2.12 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient).....done
Server Software: Microsoft-IIS/8.0
Server Hostname: localhost
Document Path: /api/values/1
Document Length: 771 bytes
Concurrency Level: 60
Time taken for tests: 7.154410 seconds
Requests per second: 8.39 [#/sec] (mean)
Time per request: 7154.410 [ms] (mean)
Time per request: 119.240 [ms] (mean, across all concurrent requests)
Transfer rate: 9.36 [Kbytes/sec] received
Percentage of the requests served within a certain time (ms)
50% 3576
66% 4593
75% 5560
80% 5628
90% 6614
95% 7079
98% 7121
99% 7134
100% 7134 (longest request)
Listing 12-2. The ValuesController Class Converted to Async
using System.IO;
using System.Threading.Tasks;
using System.Web.Http;
public class ValuesController : ApiController
{
public async Task<string> Get(int id)
{
return await ReadFileAsync();
}
private async Task<string> ReadFileAsync()
{
using (StreamReader reader = File.OpenText(@"C: <Path>SomeFile.txt"))
{
await Task.Delay(500);
return await reader.ReadToEndAsync();
}
}
}
Note the following about this code:
C:>ab -n 60 -c 60 http://localhost:35535/api/values/1
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.121.2.12 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient).....done
Server Software: Microsoft-IIS/8.0
Server Hostname: localhost
Document Path: /api/values/1
Document Length: 771 bytes
Concurrency Level: 60
Time taken for tests: 0.599034 seconds
Requests per second: 100.16 [#/sec] (mean)
Time per request: 599.034 [ms] (mean)
Time per request: 9.984 [ms] (mean, across all concurrent requests)
Transfer rate: 111.85 [Kbytes/sec] received
Percentage of the requests served within a certain time (ms)
50% 560
66% 570
75% 575
80% 577
90% 579
95% 583
98% 583
99% 584
100% 584 (longest request)
12.2 Pushing Real-time Updates to the Client
If the data returned by a service has the characteristic of changing rapidly over time, for example stock quotes, the traditional technique is for the client applications to poll the service repeatedly at regular interval. A client application makes a request, waits for the response to be returned by the service, inspects the response to see if it has gotten anything interesting and acts accordingly, and repeats this process again and again.
Server-Sent Events (SSEs), on the other hand, allows a unidirectional persistent connection between a client and service established as a result of the initial request made by the client, with the service continuing to push data to the client continuously through this connection, until the time the client drops the connection. This is more efficient than polling, but one consideration is the potentially higher number of open connections.
In the following exercise, you will use the PushStreamContent class to push real-time updates to the client. You will create a console application client as well as a JavaScript-based client to receive the updates from your web API. For the JavaScript client, you will use the Server-Sent Events (SSE) EventSource API, which is standardized as part of HTML5 by the W3C. All modern browsers support SSE, with Internet Explorer being a notable exception. We will use Google Chrome for this exercise.
SSEs have been around for a while but are somewhat eclipsed by later APIs like WebSockets that provide a richer protocol for bidirectional, full-duplex communication. Two-way channel is required for some scenarios but there are cases where a unidirectional push from the server to the client is sufficient. SSEs are better suited for this purpose. Also, SSEs just use the traditional HTTP. That means they do not require any special protocols or opening ports in the firewall and so on to implement your solution.
Listing 12-3. The Quote Class
public class Quote
{
public string Symbol { get; set; }
public decimal Bid { get; set; }
public decimal Ask { get; set; }
public DateTime Time { get; set; }
}
Listing 12-4. Changes to ValuesController
using System;
using System.Collections.Concurrent;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Threading;
using System.Web.Http;
using Newtonsoft.Json;
using Performant.Models;
public class ValuesController : ApiController
{
private static readonly Lazy<Timer> timer = new Lazy<Timer>(
() => new Timer(TimerCallback, null, 0, 2000));
private static readonly
ConcurrentDictionary<StreamWriter, StreamWriter> subscriptions =
new ConcurrentDictionary<StreamWriter, StreamWriter>();
public HttpResponseMessage Get()
{
Timer t = timer.Value;
Request.Headers.AcceptEncoding.Clear();
HttpResponseMessage response = Request.CreateResponse();
response.Headers.Add("Access-Control-Allow-Origin", "*");
response.Content = new PushStreamContent(OnStreamAvailable, "text/event-stream");
return response;
}
// More code goes here
}
Listing 12-5. The Callback Method
private static void OnStreamAvailable(Stream stream, HttpContent headers,
TransportContext context)
{
StreamWriter writer = new StreamWriter(stream);
subscriptions.TryAdd(writer, writer);
}
Listing 12-6. The Timer Callback
private static void TimerCallback(object state)
{
Random random = new Random();
// Call the service to get the quote - hardcoding the quote here
Quote quote = new Quote()
{
Symbol = "CTSH",
Bid = random.Next(70, 72) + Math.Round((decimal)random.NextDouble(), 2),
Ask = random.Next(71, 73) + Math.Round((decimal)random.NextDouble(), 2),
Time = DateTime.Now
};
string payload = "data:" + JsonConvert.SerializeObject(quote) + " ";
foreach (var pair in subscriptions.ToArray())
{
StreamWriter writer = pair.Value;
try
{
writer.Write(payload);
writer.Flush();
}
catch
{
StreamWriter disconnectedWriter;
subscriptions.TryRemove(writer, out disconnectedWriter);
if (disconnectedWriter != null)
disconnectedWriter.Close();
}
}
}
We will now create a console application client to receive events from our web API.
Listing 12-7. The Program Class
using System;
using System.IO;
using System.Net.Http;
using System.Text;
class Program
{
static async void RunClient()
{
HttpClient client = new HttpClient();
HttpResponseMessage response = await client.GetAsync(
" http://localhost: 35535/api/values ",
HttpCompletionOption.ResponseHeadersRead);
using (Stream stream = await response.Content.ReadAsStreamAsync())
{
byte[] buffer = new byte[512];
int bytesRead = 0;
while ((bytesRead = await stream.ReadAsync(buffer, 0, buffer.Length)) != 0)
{
string content = Encoding.UTF8.GetString(buffer, 0, bytesRead);
Console.WriteLine(content);
}
}
}
static void Main(string[] args)
{
RunClient();
Console.WriteLine("Press ENTER to Close");
Console.ReadLine();
}
}
Press ENTER to Close
data:{"Symbol":"CTSH","Bid":71.44,"Ask":72.70,"Time":"2013-04-27T14:22:21.642857 2+05:30"}
data:{"Symbol":"CTSH","Bid":70.80,"Ask":72.15,"Time":"2013-04-27T14:22:23.654972 3+05:30"}
data:{"Symbol":"CTSH","Bid":70.15,"Ask":71.60,"Time":"2013-04-27T14:22:25.668087 5+05:30"}
Figure 12-1. Using Google Chrome to browse
Listing 12-8. /Home/Index View
@section scripts{
<script type="text/javascript">
if (!!window.EventSource) {
var source = new EventSource(' http://localhost: 35535/api/values '),
source.addEventListener('message', function (e) {
var data = JSON.parse(e.data);
var content = data.Symbol + ' Bid: ' + data.Bid +
' Ask: ' + data.Ask + ' ' + data.Time;
$('#messages').html(content);
}, false);
source.addEventListener('error', function (e) {
if (e.readyState == EventSource.CLOSED) {
console.log("error!");
}
}, false);
}
else {
alert('It is almost time to upgrade your browser!'),
}
</script>
}
<div id="messages"></div>
Now, we have both the console application and the web application running side by side and receiving server-sent events.
12.3 Implementing Simple Web Caching
In this exercise, you will implement a simple web-caching mechanism. The term caching has different connotations. If you have ASP.NET experience, you will know of output caching and application data caching. The former is about storing the resource representation in the server or intermediaries or the user agent, while the latter is about storing the frequently-used application data in the server. Web or HTTP caching is defined in the HTTP specification http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.
A Web cache is a cache of the web server responses such as the web pages, the images, and the style sheets, for later use. The purpose of web caching is to reduce the number of round trips, the network bandwidth, and the web server resource utilization. End users also perceive better performance. The web cache can be in a web browser, if one is involved, or any of the intermediate servers such as that of the ISP or any proxy servers in between. Expiration and validation are the two primary mechanisms associated with the caching. The expiration mechanism allows a response to be reused without checking with the server, thereby reducing the round trip while the validation mechanism minimizes the bandwidth usage.
What is cached need not always be a file such as an image or a CSS. Even ASP.NET Web API responses can be cached. An example of such a scenario would be a web API returning a master list such as list of codes that changes infrequently. By default, the ASP.NET Web API framework marks the response not to be cached by setting the value of the Cache-Control header to no-cache. The Cache-Control: max-age directive specifies the duration in seconds a cache can be used before it expires.
Listing 12-9. The Employee Class
public class Employee
{
public int Id { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
}
Listing 12-10. The CacheAttribute Action Filter
using System;
using System.Net.Http.Headers;
using System.Web.Http.Filters;
public class CacheAttribute : ActionFilterAttribute
{
public double MaxAgeSeconds { get; set; }
public override void OnActionExecuted(HttpActionExecutedContext context)
{
if (this.MaxAgeSeconds > 0)
{
context.Response.Headers.CacheControl = new CacheControlHeaderValue()
{
MaxAge = TimeSpan.FromSeconds(this.MaxAgeSeconds),
MustRevalidate = true,
Private = true
};
}
}
}
Listing 12-11. EmployeesController
using System.Collections.Generic;
using System.Net;
using System.Net.Http;
using System.Web.Http;
using WebCaching.Models;
public class EmployeesController : ApiController
{
[Cache(MaxAgeSeconds=6)]
public HttpResponseMessage GetAllEmployees()
{
var employees = new Employee[]
{
new Employee()
{
Id = 1,
FirstName = "John",
LastName = "Human"
},
new Employee()
{
Id = 2,
FirstName = "Jane",
LastName = "Taxpayer"
}
};
var response = Request.CreateResponse<IEnumerable<Employee>>
(HttpStatusCode.OK, employees);
return response;
}
}
Listing 12-12. The /Home/Index View
@section scripts{
<script type="text/javascript">
$(document).ready(function () {
$('#search').click(function () {
$('#employees').empty();
$.getJSON("/api/employees", function (data) {
$.each(data, function (i, employee) {
var now = new Date();
var ts = now.getHours() + ':' + now.getMinutes() + ':' + now.getSeconds();
var content = employee.Id + ' ' + employee.FirstName;
content = content + ' ' + employee.LastName + ' ' + ts;
$('#employees').append($('<li/>', { text: content }));
});
});
});
});
</script>
}
<div>
<div>
<h1>Employees Listing</h1>
<input id="search" type="button" value="Get" />
</div>
<div>
<ul id="employees" />
</div>
</div>
The greatest thing about web caching, unlike other caching techniques such as caching application data in the server, is that the server-side code is not doing any work at all. No work is clearly better than less work! Web caching can clearly give your web API a great performance boost, if done correctly. However, you must never cache sensitive data, especially in a public cache such as one maintained by a proxy.
Figure 12-2. The ETag header and response
Listing 12-13. The EnableETag Action Filter
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Web.Http.Controllers;
using System.Web.Http.Filters;
public class EnableETag : ActionFilterAttribute
{
private static ConcurrentDictionary<string, EntityTagHeaderValue>
etags = new ConcurrentDictionary
<string, EntityTagHeaderValue>();
public override void OnActionExecuting(HttpActionContext context)
{
var request = context.Request;
if (request.Method == HttpMethod.Get)
{
var key = GetKey(request);
ICollection<EntityTagHeaderValue> etagsFromClient = request.Headers.IfNoneMatch;
if (etagsFromClient.Count > 0)
{
EntityTagHeaderValue etag = null;
if (etags.TryGetValue(key, out etag)
&& etagsFromClient.Any(t => t.Tag == etag.Tag))
{
context.Response = new HttpResponseMessage(HttpStatusCode.NotModified);
}
}
}
}
public override void OnActionExecuted(HttpActionExecutedContext context)
{
var request = context.Request;
var key = GetKey(request);
EntityTagHeaderValue etag = null;
bool isGet = request.Method == HttpMethod.Get;
bool isPutOrPost = request.Method == HttpMethod.Put ||
request.Method == HttpMethod.Post;
if ((isGet && !etags.TryGetValue(key, out etag)) || isPutOrPost)
{
etag = new EntityTagHeaderValue(""" + Guid.NewGuid().ToString() + """);
etags.AddOrUpdate(key, etag, (k, val) => etag);
}
if(isGet)
context.Response.Headers.ETag = etag;
}
private string GetKey(HttpRequestMessage request)
{
return request.RequestUri.ToString();
}
}
Listing 12-14. Changes to EmployeesController
public class EmployeesController : ApiController
{
[Cache(MaxAgeSeconds = 6)]
[EnableETag]
public HttpResponseMessage GetAllEmployees()
{
// Method is unchanged from the Listing 12-11
}
[EnableETag]
public void Post(Employee employee)
{
// It is okay to do nothing here for this exercise
}
}
Listing 12-15. HTTP Request and Response Messages
Initial GET Transaction
REQUEST
GET http://localhost:57925/api/employees HTTP/1.1
Accept: application/json, text/javascript, */*; q=0.01
Host: localhost:57925
RESPONSE
HTTP/1.1 200 OK
Cache-Control: must-revalidate, max-age=6, private
Content-Type: application/json; charset=utf-8
ETag: "0488a8df-c021-4cfa-88a9-78aa782c9cba"
Content-Length: 98
[{"Id":1,"FirstName":"John","LastName":"Human"},{"Id":2,"FirstName":"Jane","LastName":"Taxpayer"}]
Subsequet GET Transaction (After the cache expiry)
REQUEST
GET http://localhost:57925/api/employees HTTP/1.1
Accept: application/json, text/javascript, */*; q=0.01
If-None-Match: "0488a8df-c021-4cfa-88a9-78aa782c9cba"
RESPONSE
HTTP/1.1 304 Not Modified
Cache-Control: must-revalidate, max-age=6, private
Listing 12-16. A GET Transaction after Invalidation
Request
GET http://localhost:57925/api/employees HTTP/1.1
Accept: application/json, text/javascript, */*; q=0.01
If-None-Match: "0488a8df-c021-4cfa-88a9-78aa782c9cba"
Response
HTTP/1.1 200 OK
Cache-Control: must-revalidate, max-age=6, private
Content-Type: application/json; charset=utf-8
ETag: "ddc8000d-3170-44c4-b154-7961ccc594ba"
Content-Length: 98
[{"Id":1,"FirstName":"John","LastName":"Human"},{"Id":2,"FirstName":"Jane","LastName":"Taxpayer"}]
The EnableETag filter accesses the dictionary using the URI as the key . This is acceptable only for the most simplistic requirements. The resource identified by the URI of http://localhost:57925/api/employees can have multiple representations. We currently cache one and assume that is the only representation possible. For example, you can have an XML and JSON representation for this resource, based on the Accept header. Also, you can have multiple language-based representations using Accept-Language. Based on the Accept-Encoding header, you can have gzip and deflate representations, and with Accept-Charset, you can have UTF-16 and UTF-8. Most of these are out-of-box and you can have your own custom values as well. If you think about the permutations and combinations, caching can get really complex, provided your web API is leveraging all these features.
The HTTP specification defines another header, Vary, which can be used by the server to indicate the set of request header fields the resource representation in the response depends on. In other words, the key must be based on all the headers in the Vary header. For example, a web API that supports different media types will specify a Vary header like so: Vary: Accept. The intermediaries and the browser must cache multiple representations for this URI based on the value in the Accept header.
The objective of this exercise was to introduce you to web caching but not to produce a production-strength caching mechanism. CacheCow, an open source implementation of HTTP caching, available in GitHub (https://github.com/aliostad/CacheCow) will be a good start, if you are interested in implementing a full-blown mechanism.
Summary
Performance is an indication of the responsiveness of an application. It can be measured in terms of latency or throughput. Latency is the time taken by an application to respond to an event, while throughput is the number of events that take place in a specified duration of time. Another quality attribute that is often used interchangeably with performance is scalability, which is the ability of an application to handle increased usage load without any (or appreciable) degradation of the performance. The topics of performance and scalability are vast, and this chapter covered only a few important areas in ASP.NET Web API, namely asynchronous action methods, pushing real-time updates to the client, and web caching.
Asynchronous action methods allow you to start a long running operation, return your thread to the pool, and then wake up on a different thread or the same thread, to resume the processing. The async and await keywords of C# 5.0 makes writing asynchronous methods easy. However, asynchronous methods are not suitable for CPU-bound operations. Using asynchronous methods for the operations that are network-bound or I/O-bound, as in the case of calling an external service or reading or writing large chunk of data from the hard disk, is typically beneficial.
If the data returned by a service has the characteristic of changing rapidly over time, the traditional technique is for the client applications to poll the service repeatedly at regular intervals. Server-Sent Events (SSE), on the other hand, allows a unidirectional persistent connection between a client and service established as a result of the initial request made by the client, with service continuing to push data to the client continuously through this connection, until the time the client drops the connection. ASP.NET Web API supports the PushStreamContent class to push real-time updates to the client. The Server-Sent Events (SSE) EventSource API, which is standardized as part of HTML5 by the W3C can be used by a JavaScript client to receive the updates from web API. All the modern browsers except IE support SSE.
A Web cache is a cache of the web server responses for later use. The purpose of web caching is to reduce the number of round trips, the network bandwidth, and the web server resource utilization. End users also perceive better performance. The web cache can be in a web browser, or any of the intermediate servers such as that of ISP or any proxy servers in between. Expiration and validation are the two primary mechanisms associated with the caching, making use of the Cache-Control and ETag response headers. The expiration mechanism allows a response to be reused without checking with the server, thereby reducing the round trips while the validation mechanism minimizes the bandwidth usage.
1 “C# 5, ASP.NET MVC 4, and asynchronous Web applications” by Steve Sanderson, Tech Days 2012.