ticehurst.com

Streaming large blobs through ASP.NET

2022-01-30T00:00:00+00:00

Recently I spent way too much time with a colleague digging into an issue that occurred on one of our services that serves largely static content through an ASP.NET Core service. The service is a required intermediary in order to check authentication and do some other logic. It had been working fine for months with the assets we had, but suddenly starting failing with some larger downloads.

On further inspection, it was noticed that it was failing on assets over 8MB, which up to recently all our assets had been less than. When we pointed directly at the ASP.NET origin server instead of through the CDN, the downloads also worked fine.

The CDN in use is Azure Front Door. After some digging around in the docs, we found the below comments on the page at https://docs.microsoft.com/en-us/azure/cdn/cdn-large-file-optimization for the Azure CDN from Microsoft (which we use).

Azure CDN Standard from Microsoft uses a technique called object chunking. When a large file is requested, the CDN retrieves smaller pieces of the file from the origin. After the CDN POP server receives a full or byte-range file request, the CDN edge server requests the file from the origin in chunks of 8 MB….

…This optimization relies on the ability of the origin server to support byte-range requests; if the origin server doesn’t support byte-range requests, requests to download data greater than 8mb size will fail.

The last word seemed to identify the issue. This was a surprising find. If you read about the feature on that page for other Azure CDN caches, such as the Akamai one, you find statements such as:

This optimization relies on the ability of the origin server to support byte-range requests; if the origin server doesn’t support byte-range requests, this optimization isn’t effective.

That’s what I’d expect. Fall back to unoptimized behavior, not just fail a perfectly valid request!

Anyways, getting to a satisfactory resolution required quite a bit of research, and digging into some ASP.NET internals and nuances of the HTTP specs. This is a long write-up, as it’s the second time I’ve had to wrestle with this topic, so there’s a good chance I’ll need it as a future reference. Hopefully you can find some valuable or interesting information here too. (Else just skip to the end for the code!)

HTTP Range Requests

First, what are HTTP Range Requests and what support does ASP.NET Core have for them?

HTTP Range requests are defined in the RFC at https://httpwg.org/specs/rfc7233.html. If you are not familiar with Range requests, the page at https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests gives a good overview. Read that then come back.

Out of the box ASP.NET Core supports range requests with some minor limitation. For example, it does not support requesting multiple ranges in one request, such as Range: bytes=0-50, 100-150. This is valid per the spec, but ASP.NET Core only supports requesting a single range. See the code in RangeHelper.cs which contains:

    if (rawRangeHeader.Count > 1 || rawRangeHeader[0].IndexOf(',') >= 0)
    {
        logger.LogDebug("Multiple ranges are not supported.");

        // The spec allows for multiple ranges but we choose not to support them because the client may request
        // very strange ranges (e.g. each byte separately, overlapping ranges, etc.) that could negatively
        // impact the server. Ignore the header and serve the response normally.
        return (false, null);
    }

To start from scratch and experiment with range-requests on static files, spin up a simple static site with the below code: (This is ALL the code you need to write a web server with ASP.NET Core 6!)

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.UseStaticFiles();
app.Run();

For these tests I dropped a file at ./wwwroot/large.txt containing a million lines of text, each 32 bytes long, with the text on each line just stating the current line number.

Using Postman to make a Range request starting on line 1001 (i.e., starting 32000 bytes in) for the next 1024 bytes (which equals 32 lines), the request/response is shown below.

GET /large.txt HTTP/1.1
Range: bytes=32000-33023
Accept: */*
Cache-Control: no-cache
Host: localhost:7098
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
 
HTTP/1.1 206 Partial Content
Content-Length: 1024
Content-Type: text/plain
Date: Sat, 29 Jan 2022 05:58:46 GMT
Server: Kestrel
Accept-Ranges: bytes
Content-Range: bytes 32000-33023/32000000
ETag: "1d814d51dc99300"
Last-Modified: Sat, 29 Jan 2022 05:57:34 GMT
 
1001 is the current line
1002 is the current line
1003 is the current line
...
1031 is the current line
1032 is the current line

All good. Changing the range header to include multiple ranges shows this isn’t supported, and it simply returns the whole file as if no range was requested.

GET /large.txt HTTP/1.1
Range: bytes=32000-33023,64000-64100
Accept: */*
Cache-Control: no-cache
Host: localhost:7098
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
 
HTTP/1.1 200 OK
Content-Length: 32000000
Content-Type: text/plain
Date: Sat, 29 Jan 2022 06:04:55 GMT
Server: Kestrel
Accept-Ranges: bytes
ETag: "1d814d51dc99300"
Last-Modified: Sat, 29 Jan 2022 05:57:34 GMT
 
1 is the current line         
2 is the current line         
3 is the current line
...
999999 is the current line    
1000000 is the current line

Getting a 200 OK rather than a 206 Partial Content is perfectly valid per the spec, which states:

Range requests are an OPTIONAL feature of HTTP, designed so that recipients not implementing this feature (or not supporting it for the target resource) can respond as if it is a normal GET request without impacting interoperability.

Sending the file contents

Digging through the ASP.NET Core 6 code, you can see static files are served by the StaticFileMiddleware class in src\Middleware\StaticFiles\src\StaticFileMiddleware.cs. The meat of the work happens in method StaticFileContext::ServeStaticFile at src\Middleware\StaticFiles\src\StaticFileContext.cs.

To send the actual file it ends up in StaticFileContext::SendRangeAsync. Here it has some more interesting notes regarding behavior if the range is not satisfiable. Namely, the Content-Range header in the error response should include an asterisk and the content length.

// 14.16 Content-Range - A server sending a response with status code 416 (Requested range not satisfiable)
// SHOULD include a Content-Range field with a byte-range-resp-spec of "*". The instance-length specifies
// the current length of the selected resource.  e.g. */length
ResponseHeaders.ContentRange = new ContentRangeHeaderValue(_length);
ApplyResponseHeaders(StatusCodes.Status416RangeNotSatisfiable);

For satisfiable ranges, after computing the ranges there it effectively ends up calling:

await _context.Response.SendFileAsync(_fileInfo, start, length, _context.RequestAborted);

This ends up in SendFileResponseExtensions.SendFileAsyncCore in src\Http\Http.Extensions\src\SendFileResponseExtensions.cs and calling:

var sendFile = response.HttpContext.Features.Get<IHttpResponseBodyFeature>()!;

try
{
    await sendFile.SendFileAsync(fileName, offset, count, localCancel);
}

This then ends up in SendFileAsync in src\Http\Http\src\SendFileFallback.cs which does a basic File stream operation to the Kestrel response stream (the variable destination below) with a 16KB buffer, after seeking to the range offset.

const int bufferSize = 1024 * 16;

var fileStream = new FileStream(
    filePath,
    FileMode.Open,
    FileAccess.Read,
    FileShare.ReadWrite,
    bufferSize: bufferSize,
    options: FileOptions.Asynchronous | FileOptions.SequentialScan);

using (fileStream)
{
    fileStream.Seek(offset, SeekOrigin.Begin);
    await StreamCopyOperationInternal.CopyToAsync(fileStream, destination, count, bufferSize, cancellationToken);
}

When the await on the last statement above completes, the response has been sent. Everything unwinds and the streams (file and response) are disposed.

So generic static file serving works fine for HTTP range requests. But in my problematic scenario we weren’t serving static files. Our service fetches a blob from Azure Blob Storage and pipes it back to the client. How should this work?

Streaming Azure Blob Range-Requests

First you need to add the Azure Blob Storage NuGet package. We were using version 12.9.1, so the .csproj file contains:

<PackageReference Include="Azure.Storage.Blobs" Version="12.9.1" />

To add the ASP.NET Core controller endpoints to return the blob content, the program.cs is updated with two additional lines to look like the below:

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddControllers();
var app = builder.Build();
app.UseStaticFiles();
app.MapControllers();
app.Run();

And a Controller.cs file with the below contents is added:

using Microsoft.AspNetCore.Mvc;

[ApiController]
public class Controller : ControllerBase
{
    [HttpGet("/asset")]
    public IActionResult GetAsset()
    {
        return Ok("testing");
    }
}

If you run a PostMan request for /asset now, you should see network traffic such as:

GET /asset HTTP/1.1
Accept: */*
Cache-Control: no-cache
Host: localhost:7098
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
 
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: Sat, 29 Jan 2022 07:28:01 GMT
Server: Kestrel
Transfer-Encoding: chunked
 
testing

It’s interesting to note that without the Content-Length header being set, it defaults to Transfer-Encoding: chunked. This makes sense if the content length isn’t known in advance.

Now update the controller to return a file fetched from Azure Blob Storage. To start with, just use the most rudimentary approach possible of downloading it all into memory synchronously and then returning it. Update Controller.cs to contain the below. (Your appsettings.json, or preferably secrets.json, should contain a BlobUri setting with the connection string for your Azure Blob Storage account):

using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Specialized;
using Microsoft.AspNetCore.Mvc;

[ApiController]
public class Controller : ControllerBase
{
    BlobContainerClient blobContainerClient;

    public Controller(IConfiguration config)
    {
        string blobUri = config.GetValue<string>("BlobUri");
        blobContainerClient = new BlobContainerClient(blobUri, "assets");
    }

    [HttpGet("/asset")]
    public IActionResult GetAsset()
    {
        var blobClient = blobContainerClient.GetBlockBlobClient("medium.txt");
        var content = blobClient.DownloadContent();
        Response.ContentType = "text/plain";
        return Ok(content.Value.Content.ToStream());
    }
}

Then request “https://localhost:7098/asset” in Postman (or the browser) and verify it works. (Note: Here I have uploaded a ‘medium.txt’ file which is 1000 lines (32,000 bytes) for easier testing, as the 32MB responses tend to make some tools a little slow to work with. I put this into a container named assets on the Azure Storage account).

Trying requesting a range, and you should see output similar to:

GET https://localhost:7098/asset HTTP/1.1
Range: bytes=1024-2047
Accept: */*
Cache-Control: no-cache
Host: localhost:7098
Accept-Encoding: gzip, deflate, br
Connection: keep-alive


HTTP/1.1 200 OK
Content-Type: text/plain
Date: Sat, 29 Jan 2022 17:37:09 GMT
Server: Kestrel
Transfer-Encoding: chunked

7d00
1 is the current line         
2 is the current line         
...
999 is the current line       
1000 is the current line      

0

Here it is just fetching the entire blob are returning it. The controller needs to return a File with a stream that is seekable in order to be able to satify a request for a range within in. Change the controller method to just the below:

[HttpGet("/asset")]
public IActionResult GetAsset()
{
    var blobClient = blobContainerClient.GetBlockBlobClient("medium.txt");
    var content = blobClient.DownloadContent();
    return File(content.Value.Content.ToStream(), "text/plain", enableRangeProcessing: true);
}

Now you should see the expect request/response for just a portion of the blob!

GET https://localhost:7098/asset HTTP/1.1
Range: bytes=1024-2047
Accept: */*
Cache-Control: no-cache
Host: localhost:7098
Accept-Encoding: gzip, deflate, br
Connection: keep-alive


HTTP/1.1 206 Partial Content
Content-Length: 1024
Content-Type: text/plain
Date: Sat, 29 Jan 2022 18:19:00 GMT
Server: Kestrel
Accept-Ranges: bytes
Content-Range: bytes 1024-2047/32000

33 is the current line        
34 is the current line        
...
63 is the current line        
64 is the current line

The File class being returned by the controller here is ultimately using much of the same code as the static file middleware show before. It effectively expects to be given a seekable stream, and if range processing is enabled, will use the headers in the request to figure out the correct partial response to return. Nice!

Remaining problems

This still has several problems however. Chief amonst them being that the method is synchronously downloading the entire blob from storage, no matter how small a range we are requesting. Using Fiddler to observe traffic to blob storage, the below can be seen for the above range request. (I’ll show all headers here, but trim them for later examples).

GET https://example.blob.core.windows.net/assets/medium.txt HTTP/1.1
Host: example.blob.core.windows.net
x-ms-version: 2020-08-04
Accept: application/xml
x-ms-client-request-id: 7dda7476-1f79-4ee7-9a44-f75572772f2e
x-ms-return-client-request-id: true
User-Agent: azsdk-net-Storage.Blobs/12.9.1 (.NET 6.0.1; Microsoft Windows 10.0.19044)
x-ms-date: Sat, 29 Jan 2022 18:19:00 GMT
Authorization: SharedKey bi...0=


HTTP/1.1 200 OK
Content-Length: 32000
Content-Type: text/plain
Content-MD5: LLqzda6Tv79dBTuN6Bi7Aw==
Last-Modified: Sat, 29 Jan 2022 08:02:17 GMT
Accept-Ranges: bytes
ETag: "0x8D9E2FDAB6167FD"
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 90018034-b01e-002e-2f3c-15ff8c000000
x-ms-client-request-id: 7dda7476-1f79-4ee7-9a44-f75572772f2e
x-ms-version: 2020-08-04
x-ms-creation-time: Sat, 29 Jan 2022 08:02:17 GMT
x-ms-lease-status: unlocked
x-ms-lease-state: available
x-ms-blob-type: BlockBlob
x-ms-server-encrypted: true
Date: Sat, 29 Jan 2022 18:19:00 GMT

1 is the current line         
2 is the current line         
...
999 is the current line       
1000 is the current line

Also, this route makes no distinction for a HEAD request, (which it currently doesn’t support, but we’ll fix that next), where you wouldn’t want to fetch the entire blob as no response body should be returned.

You could weave the HEAD logic in with the GET logic, but it’s quite distinct, so it’s cleaner just to separate it out into its own method. Per the spec, it doesn’t need to handle Range requests any differently to a non-Range request, so just indicating the content length and type, and indicating support for range-requests (with the header Accept-Ranges: bytes) should be sufficient.

[HttpHead("/asset")]
public async Task<IActionResult> HeadAsset()
{
    var blobClient = blobContainerClient.GetBlockBlobClient("medium.txt");
    var props = await blobClient.GetPropertiesAsync();

    Response.Headers.ContentLength = props.Value.ContentLength;
    Response.Headers.ContentType = props.Value.ContentType;
    Response.Headers.AcceptRanges = "bytes";

    return Ok();
}

Now Fiddler shows the request and response as expected:

HEAD https://localhost:7098/asset HTTP/1.1
Accept: */*
Cache-Control: no-cache
Host: localhost:7098
Accept-Encoding: gzip, deflate, br
Connection: keep-alive


HTTP/1.1 200 OK
Content-Length: 32000
Content-Type: text/plain
Date: Sat, 29 Jan 2022 21:15:44 GMT
Server: Kestrel
Accept-Ranges: bytes

And importantly, the entire response from the request to Blob storage for the GetProperties call is just a few headers:

HEAD https://example.blob.core.windows.net/assets/medium.txt HTTP/1.1
Host: example.blob.core.windows.net
x-ms-version: 2020-08-04
Accept: application/xml
User-Agent: azsdk-net-Storage.Blobs/12.9.1 (.NET 6.0.1; Microsoft Windows 10.0.19044)
Authorization: SharedKey bi...4=


HTTP/1.1 200 OK
Content-Length: 32000
Content-Type: text/plain
Content-MD5: LLqzda6Tv79dBTuN6Bi7Aw==
Last-Modified: Sat, 29 Jan 2022 08:02:17 GMT
Accept-Ranges: bytes
ETag: "0x8D9E2FDAB6167FD"
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-version: 2020-08-04
x-ms-creation-time: Sat, 29 Jan 2022 08:02:17 GMT
... <a few other x-ms-* headers, but no body> ...

Now for the real challenge. Efficiently and correctly streaming the blob contents, including range-requests.

Here was my first attempt:

[HttpGet("/asset")]
public async Task<IActionResult> GetAsset(CancellationToken ct)
{
    // Tell the client we accept range-requests
    Response.Headers.AcceptRanges = "bytes";

    var blobClient = blobContainerClient.GetBlockBlobClient("medium.txt");

    var ranges = Request.GetTypedHeaders().Range?.Ranges;

    // If it's not a range request, or an unsupported one, just return the entire asset
    // We support only one range that must have a start position
    if (ranges == null || ranges.Count != 1 || !ranges.First().From.HasValue)
    {
        // Be sure to pass the cancellation token so the blob request is aborted if this request is.
        var blob = await blobClient.DownloadStreamingAsync(cancellationToken: ct);
        return File(blob.Value.Content, blob.Value.Details.ContentType);
    }
    else 
    {
        // Calculate the range to request from blob storage
        RangeItemHeaderValue range = ranges.First();
        long? length = range.To.HasValue ? range.To.Value - range.From!.Value + 1 : null;
        Azure.HttpRange azureRange = new (range.From!.Value, length);

        var blob = await blobClient.DownloadStreamingAsync(range: azureRange, cancellationToken: ct);
        // TODO: Handle errors such as out of range

        return File(blob.Value.Content, blob.Value.Details.ContentType, enableRangeProcessing: true);
    }
}

In requesting a range from 1024-2047, the underlying blob storage request was perfect:

HTTP/1.1 206 Partial Content
Content-Length: 1024
Content-Type: text/plain
Content-Range: bytes 1024-2047/32000
Accept-Ranges: bytes
Date: Sat, 29 Jan 2022 21:58:03 GMT

33 is the current line        
34 is the current line        
...
63 is the current line        
64 is the current line

However the response to the request from the controller was a 200 OK, not a 206 Partial, which is incorrect.

GET https://localhost:7098/asset HTTP/1.1
Range: bytes=1024-2047
Connection: keep-alive


HTTP/1.1 200 OK
Content-Type: text/plain
Accept-Ranges: bytes
Transfer-Encoding: chunked

400
33 is the current line        
34 is the current line        
...      
63 is the current line        
64 is the current line        

0

One thing to keep in mind is that the response to a range request is not guaranteed to be for that range. Per the spec, it’s perfectly valid for the server it ignore the Range header and return the full resource in a 200 - OK response.

Another important consideration; even if it does return a range, it may not be the range asked for. The spec states that if any part of the range is satisfiable, then that should be returned. This is indeed the case with the Front Door service behavior I was trying to support. When Front Door requests a resource it asks for the first 8MB in a Range header. However if the resource is smaller then 8MB, say 2KB, the response will be a 206 Partial response with the header Content-Range: bytes 0-2047/2048, indicating that it returned all the bytes for a 2KB resource.

Which Blob API to use

There are several different APIs that can be used when fetching a blob with the Azure SDK. The difference between some of them, and when to use them, is outlined in the GitHub issue at https://github.com/Azure/azure-sdk-for-net/issues/22022#issuecomment-870054035

As it states, DownloadContentAsync is for smaller downloads, which is not what we are dealing with here. DownloadToAsync writes to a stream or a file, but doesn’t allow you to specify just a range to fetch.

OpenReadAsync does give you a seekable stream, which means you could potentially just pass it to the File helper as the return value. However all you get here is a stream, so accessing other properties on the blob such as the content-type etc. is not possible, thus you would need to make another call to GetPropertiesAsync to fetch that info as done above for HEAD requests, and having an extra round trip to Azure Storage on each request is undesireable.

That leaves DownloadStreamingAsync, which accepts a range to fetch, and also has a return type which exposes all the Blob properties and HTTP response details we might need.

With that, one approach is to look in the request to see if a range was requested, if so pass that range to DownloadStreamingAsync to fetch only that range from Blob storage. When the response is returned, whether a 200 OK or a 206 Partial Content, return the corresponding status, headers, and content stream to the initial request.

The Controller return value

The File(...) return value helper used above is not suitable here. This expects to be given a stream for the entire resource, and if range processing is enabled will seek within it. The stream we have here is (potentially) partial, and also is not seekable.

I spent a while looking at the various controller return value helpers in ASP.NET Core, but ultimately that got a little complex and may have hidden subtleties, so I went back to basics. The return value from a controller is typically an IActionResult, and all this needs to do is process the response when its ExecuteResultAsync method is called. Implementing a BlobStreamResult class for this that simply set the status and headers, and then streams the content back is simple enough.

Cancellation

An important consideration that is often overlooked is cancellation. For each request to ASP.NET Core, there is a cancellation token attached. In general usage requests will often be terminated early, be it for cancelling a download, navigating the browser elsewhere before it finished loaded, the app crashing, or even potentially in some types of attacks. If this cancellation token is not passed on to the calls to blob storage, then even if the client aborts the request after a couple of packets, the Controller will still continue to download the full blob.

I wrote up a simple client to test this as shown below, which requests a large asset from the controller, and then after fetching the first 4KB of data kills the request.

var client = new HttpClient();
var request = await client.GetAsync("https://localhost:7098/asset/large.txt", HttpCompletionOption.ResponseHeadersRead);
var buffer = new byte[4096];
var stream = await request.Content.ReadAsStreamAsync();
var result  = await stream.ReadAsync(buffer, 0, 4096);
request.Dispose();

Using Wireshark to monitor traffic, and WITHOUT the CancellationToken being passed to the Blob API calls, the service still goes on to fetch all 32MB from Azure Storage.

After chaining through the CancellationToken from the request to the Blob API calls, when running the same test the service connection to Azure Storage gets reset and the download aborted after about 200KB.

Another important hygine consideration is to ensure the Blob stream is disposed once the response is sent, else the blob stream and resulting connection remain active until garbage collected.

The Final Code

After all the above, the below is where I ended up. This seems to satisfy all the requirements for streaming immutable blobs from Azure Storage through an ASP.NET Core controller, while supporting Range-Requests in an efficient and correct manner. At 130 lines of code it’s not too complex.

This does have a few pre-conditions to be aware of should you choose to reuse it:

This is for streaming immutable resources, hence the etags and cache-control headers being set to specific values, and request headers like if-modified-since being largely ignored. If serving mutable resources then the code will need to be updated to support that.
This does expect that Blob Storage calls always return Content-Length and Content-Type headers, and not something like a chunked encoding response. The docs do state the response has these headers, so I’m going to make that assumption here. (And it my testing that is the case).

Hopefully this was informative and useful. It was certainly a lot more involved that I expected!

using Azure;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using Azure.Storage.Blobs.Specialized;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Net.Http.Headers;
using System.IO.Pipelines;

public class BlobStreamResult : ActionResult
{
    int _statusCode;
    BlobDownloadStreamingResult _blobResult;

    public BlobStreamResult(int statusCode, BlobDownloadStreamingResult blobResult)
    {
        _statusCode = statusCode;
        _blobResult = blobResult;
    }

    override public async Task ExecuteResultAsync(ActionContext context)
    {
        try
        {
            var response = context.HttpContext.Response;
            response.StatusCode = _statusCode;
            response.Headers.ContentType = _blobResult.Details.ContentType;
            response.Headers.ContentLength = _blobResult.Details.ContentLength;

            if (_statusCode == StatusCodes.Status206PartialContent)
            {
                response.Headers.ContentRange = _blobResult.Details.ContentRange;
            }

            // Be sure to cancel flowing the stream if the request is aborted
            CancellationToken ct = context.HttpContext.RequestAborted;
            await _blobResult.Content.CopyToAsync(response.BodyWriter, ct);
        }
        finally
        {
            // Clean up the blob stream when done. A TaskCanceledException (or
            // for some other reason) may occur in the above block.
            _blobResult.Dispose();
        }
    }
}

[ApiController]
public class Controller : ControllerBase
{
    BlobContainerClient blobContainerClient;
    const string ImmutableEtag = "\"immutable\"";
    const string CacheControl  = "public, max-age=604800";

    public Controller(IConfiguration config)
    {
        string blobUri = config.GetValue<string>("BlobUri");
        blobContainerClient = new BlobContainerClient(blobUri, "assets");
    }

    [HttpHead("/asset/{assetId}")]
    public async Task<IActionResult> HeadAsset(string assetId)
    {
        try
        {
            var blobClient = blobContainerClient.GetBlockBlobClient(assetId);
            var props = await blobClient.GetPropertiesAsync();
            Response.Headers.ContentLength = props.Value.ContentLength;
            Response.Headers.ContentType = props.Value.ContentType;
            Response.Headers.AcceptRanges = "bytes";
            Response.Headers.ETag = ImmutableEtag;
            Response.Headers.CacheControl = CacheControl;
        }
        catch (RequestFailedException ex) when (ex.Status == StatusCodes.Status404NotFound)
        {
            Response.Headers.Clear();
            return NotFound();
        }

        return Ok();
    }

    [HttpGet("/asset/{assetId}")]
    public async Task<IActionResult> GetAsset(string assetId, CancellationToken ct)
    {
        Response.Headers.AcceptRanges = "bytes";
        Response.Headers.ETag = ImmutableEtag;
        Response.Headers.CacheControl = CacheControl;

        // If they have any "immutable" tagged content, then it hasn't changed
        if (Request.Headers.IfNoneMatch.Contains(ImmutableEtag))
        {
            return StatusCode(StatusCodes.Status304NotModified);
        }
        // Note: An "if-range" header may also be present which means if the representation is unchanged,
        // send me the part(s) that I am requesting in Range; otherwise, send me the entire representation.
        // Being that we only serve immutable content, the representation will be unchanged, so ignore this.

        HttpRange blobRange = default; // A default value results in requesting the full resource

        // We support a range request if it specifies exactly one range with a from position specified
        var ranges = Request.GetTypedHeaders().Range?.Ranges;
        if (ranges != null && ranges.Count == 1 && ranges.First().From.HasValue)
        {
            RangeItemHeaderValue range = ranges.First();
            long? length = range.To.HasValue ? range.To.Value - range.From!.Value + 1 : null;
            blobRange = new(range.From!.Value, length);
        }

        var blobClient = blobContainerClient.GetBlockBlobClient(assetId);
        try
        {
            var blob = await blobClient.DownloadStreamingAsync(range: blobRange, cancellationToken: ct);
            int statusCode = blob.GetRawResponse().Status;                
            return new BlobStreamResult(statusCode, blob);
        }
        catch (RequestFailedException ex) when (ex.Status == StatusCodes.Status416RangeNotSatisfiable)
        {
            // When DownloadStreamAsync throws, we don't have the return value to get headers from.
            // Fetch the properties again to get the Content-Length to return in Content-Range.
            var props = await blobClient.GetPropertiesAsync(cancellationToken: ct);
            Response.Headers.ContentRange = $"bytes */{props.Value.ContentLength}";
            return StatusCode(StatusCodes.Status416RangeNotSatisfiable);
        }
        catch (RequestFailedException ex) when (ex.Status == StatusCodes.Status404NotFound)
        {
            Response.Headers.Clear();
            return NotFound();
        }
    }
}

HttpClient in Unity

2021-08-27T00:00:00+00:00

On a code review of some Unity code where HttpClient was being used, a comment was made to “Consider using HttpClientFactory”. Those three words resulted in many hours of intrigue and frustration, which I will document here.

Why we have HttpClientFactory

First, some background: Why does HttpClientFactory exist? There’s a long and sordid history to this.

An (in)famous post titled “You’re using HttpClient wrong and it is destabilizing your software” at https://www.aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/ outlines the first problem. Namely, although HttpClient implements the IDispose interface, you shouldn’t be spinning them up and disposing them regularly. By default this will create new network connections for each HttpClient instance, and then leave them in a TIME_WAIT state for a couple mins when disposed. Beside the fact creating new connections is expensive (with the TCP handshake and SSL overheaded on each new connection), every network connection you leave in this state is consuming resources, and at the extreme end you get “socket exhaustion”, where you literally run out of TCP/IP sockets to allocate.

So the outcome of that article is that you should allocate a static HttpClient and just continue to use it, thus reusing the network connections is allocates. This however, results in another serious problem as outlined in “Singleton HttpClient? Beware of this serious behavior and how to fix it” at http://byterot.blogspot.com/2016/07/singleton-httpclient-dns.html. Basically the problem with using a single HttpClient and continually reusing the connections, is that DNS changes will now never get picked up. Various performance and reliability solutions on the internet (such as Azure Front Door) use DNS changes to route clients to different endpoints depending on load and health. Some deployment features such as A/B testing or switching staging slots can also be implemented via DNS changes.

The solution to this is to use HttpClientFactory to get a new HttpClient on demand. Under the covers it handles persisting connections for a couple of minutes (even when the HttpClient is disposed) and also spinning up new connections regularly to pick up any DNS changes. You can read more about how to use it on the doc page Use IHttpClientFactory to implement resilient HTTP requests.

So just use that right?

Unity and .NET

(Note: My specific Unity project is using the .NET 4.x profile, so those are the APIs this post will cover)

Unity doesn’t use the “official” .NET 4.x Framework. It also doesn’t use .NET Core. It’s version of .NET is from the Mono project. (Actually a fork of this project, but that’s not important for now). As can be seen on their doc page .NET Profile Support, Unity support .NET 4.x and .NET Standard 2.0 profiles. Unfortunately HttpClientFactory was added in .NET Core 2.1 (which is .NET Standard 2.1).

But all is not lost. HttpClientFactory itself is contained in the NuGet package “Microsoft.Extensions.Http”, and if you look on NuGet at https://www.nuget.org/packages/Microsoft.Extensions.Http/, you can see this supports .NET Standard 2.0. However it has a loooooong dependency graph, depending on other packages for Logging, Options, DependencyInjection, etc. Beside the pain of getting NuGet packages into a Unity application, requiring the manual download and unzipping of packages (see this documentation page), there is another pain point.

HttpClientFactory and DI

The only way to use HttpClientFactory is via dependency injection, which also requires you to use the .NET hosting model. The only way to get an IHttpClientFactory interface (needed to create the HttpClient instances) is via dependency injection, by calling the extension method AddHttpClient on an IServiceCollection, and then having the hosting model create the instances where it is needed.

You can see on the GitHub issue at https://github.com/aspnet/HttpClientFactory/issues/55 where ways to use HttpClientFactory without depending on Dependency Injection and the associated hosting model have been requested, but at the moment that is still required.

I did not want to add a long list of NuGet packages and restructure the Unity app to use an IServiceProvider and Dependency Injection, so HttpClientFactory was off the table.

Mono thwarts other avenues

If we revisit the Uber-issue on GitHub, and specifically this comment you’ll notice the below:

Notice the ”..not working reliably across platforms..” bit. Sadly, Mono is one of those platforms. Using ILSpy on the Mono version of System.Net.Http.dll versus the other .NET implementations, it can been seen there are many things that are simply not implemented. Sadly, ServicePoint.ConnectionLeaseTimeout is one of them.

This left but one option: Figure out how all this fits together and write a custom solution.

How HttpClient works in Mono

The HttpCient class itself doesn’t create and destroy network connections. The class relationships that map between an HttpClient instance and a network socket is very convoluted. I ended up mapping out and diagramming over 20 class types, and that was just to get a high level understanding. The crux of the issue though is that HttpClient is a very thin wrapper, and delegates much of its work to HttpClientHandler, which creates a network connection “group”. When you create a new HttpClient via its default constructor, it creates its own HttpClientHandler, and then when you call Dispose() on this HttpClient, it calls Dispose() on its HttpClientHandler, which closes the network connection group. You can see this code at this line in the Unity Mono fork

HttpClient itself largely just delegates to HttpMessageInvoker, and it can be seen from its constructor that you can provider an HttpMessageHandler (which HttpClientHandler derives from), and also tell it not to dispose this message handler when it is disposed. This is the key to sharing connections across HttpClient instances.

So that seemed easy enough. Just create all the HttpClient instances using one HttpClientHandler for a couple mins to share the connections, then periodically start using a new HttpClientHandler to cycle through the connections and pick up DNS changes. (This is effectively what HttpClientFactory does).

But that then left me with a few questions, namely:

So when should the HttpMessageHandler be disposed? How do you know when its connections are no longer in use (i.e. requests/responses have completed). You don’t want to kill the connections while requests are still in progress.
If I return just the response stream from a method, and the HttpClient is diposed, and the active HttpClientHandler has been replaced, will garbage collection potentially run and kill the (possibly in flight) connections?

Cut to the chase

Long story short, while calling Dispose() explicitly on the HttpClientHandler kills the connection group, that class doesn’t have a finalizer (a destructor, in C# parlance). So when the garbage collector cleans up an HttpClientHandler, no code is invoked, and Dipose(false) is not called to clean up unmanaged resources.

The connections themselves are managed by a ServicePointScheduler, which tracks when an operation completes on a connection and will then close it when idle. See this code in the Mono repository

Effectively HttpClient and HttpClientHandler are just factories to create HttpResponseMessage objects (and corresponding HttpRequestMessage) against a specific ConnectionGroup. They do not hold any references to these objects, and refer to the ConnectionGroup by name only, as seen below.

As long as you don’t explicitly Dispose the underlying HttpClientHandler, but instead provide your own and simply stop using it and leave to the GC to clean up when done, all is good.

This allows for both the HttpClient to be disposed (which doesn’t dispose the provided HttpClientHandler), and for the HttpClientHandler to be garbage collected when no longer referenced, without killing any in use connections. The ServicePoint manages the active connections, and will clean them up when no longer used.

Implmenting a UnityHttpClient with this approach allows for code such as the below, which returns only a stream, and the connection will remain active until the response is completed, even after the HttpClient and the underlying HttpClientHandler have long gone. The class will start using a new HttpClientHandler periodically to allocate new connections.

    async Task<Stream> GetHttpStream()
    {
        using var client = UnityHttpClient.Get();
        var stream = await client.GetStreamAsync("https://localhost:5001/api/blob");
        return stream;
    }

Using the excellent Memory Profilers Package in Unity, from the stream returned above, we can see it reference down to the underlying network socket. Also shown is the ServicePoint which holds a reference to outstanding operations and current network connections, (as well as idle connections to be cleaned up after the idle timeout). References of interest have been underlined.

Writing the Unity HttpClient factory

With the above in mind, how should the UnityClientFactory look? Below is the implementation I came up with. Note this allows for “named” clients to explicitly get a new set of connections (as outlined in the comments).

public class UnityHttpClient
{
    private const int handlerExpirySeconds = 120;
    private static readonly object factoryLock = new object();

    private static readonly Dictionary<string, UnityHttpClient> factories =
            new Dictionary<string, UnityHttpClient>()
            {
                { string.Empty, new UnityHttpClient() }
            };

    public static HttpClient Get() => factories[string.Empty].GetNewHttpClient();

    // Each unique HttpClientHandler gets a new Connection limit per origin, so create
    // a new "named" client factory to get a new handler (used by each HttpClient from
    // that factory), and thus new set of connections.
    //
    // For example, if you have a few long-running requests, you might choose to put
    // them on their own handler/connections so you don't block other faster requests
    // to the same host.
    public static HttpClient Get(string name)
    {
        UnityHttpClient factory;
        lock (factoryLock)
        {
            if (!factories.TryGetValue(name, out factory))
            {
                factory = new UnityHttpClient();
                factories.Add(name, factory);
            }
        }
        return factory.GetNewHttpClient();
    }

    private HttpClientHandler _currentHandler = new HttpClientHandler();
    private readonly Stopwatch _handlerTimer = new Stopwatch();
    private readonly object _handlerLock = new object();

    private UnityHttpClient() { }

    private HttpClient GetNewHttpClient() =>
            new HttpClient(GetHandler(), disposeHandler: false);

    private HttpClientHandler GetHandler()
    {
        lock(_handlerLock)
        {
            if (_handlerTimer.Elapsed.TotalSeconds > handlerExpirySeconds)
            {
                // Leave the old HttpClientHandler for the GC. DON'T Dispose() it!
                _currentHandler = new HttpClientHandler();
                _handlerTimer.Restart();
            }
            return _currentHandler;
        }
    }
}

Testing

To verify the code behaves as desired, I created a simple web service that has two REST APIs. The first /api/item generates small and fast responses, and the second /api/blob for large and slow responses. This simple web server can be written in a few lines of C# code using top-level statements (as introduced in C# 9). The entire ASP.NET Core 5 project code is shown below:

using System;
using System.Diagnostics;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Hosting;

int requestId = 0;

var builder = Host.CreateDefaultBuilder().ConfigureWebHostDefaults(webBuilder =>
{
    webBuilder.Configure(app =>
    {
        // Fast and small JSON response
        app.Map("/api/item", app =>
        {
            app.Run(async context =>
            {
                int itemId = Interlocked.Increment(ref requestId);
                byte[] body = Encoding.UTF8.GetBytes($"}");

                context.Response.StatusCode = 200;
                context.Response.ContentLength = body.Length;
                context.Response.ContentType = "application/json";

                await context.Response.BodyWriter.WriteAsync(body.AsMemory());
                await context.Response.CompleteAsync();
            });
        });

        // Large and slow binary response
        app.Map("/api/blob", app =>
        {
            app.Run(async context =>
            {
                Stopwatch chunkTimer = Stopwatch.StartNew();
                const int contentLength = 1024 * 1024 * 10; // 10 MB
                const int chunkSize = 1024 * 16;            // in 16kb chunks
                const int responseTimeMs = 10000;           // Response to take 10 sec

                Debug.Assert(chunkSize % 128 == 0 && contentLength % chunkSize == 0,
                  "chunkSize must be multiple of 128 and divide into contentLength");

                context.Response.StatusCode = 200;
                context.Response.ContentLength = contentLength;
                context.Response.ContentType = "application/octet-stream";

                int bytesWritten = 0;
                while (bytesWritten < contentLength)
                {
                    StringBuilder content = new(chunkSize);
                    while (content.Length < chunkSize)
                    {
                        string txt = $"{bytesWritten + 128} bytes including this line"
                            .PadRight(127, '-') + "\n";
                        content.Append(txt);
                        bytesWritten += 128;
                    }
                    byte[] chunk = Encoding.UTF8.GetBytes(content.ToString());
                    Debug.Assert(chunk.Length == chunkSize);

                    await context.Response.BodyWriter.WriteAsync(chunk.AsMemory());

                    // Delay each chunk so the request takes the desired time
                    double percentComplete = (double)bytesWritten / contentLength;
                    long durationSoFarMs = chunkTimer.ElapsedMilliseconds;
                    int delayMs =
                            (int)(percentComplete * responseTimeMs - durationSoFarMs);
                    if (delayMs > 0) await Task.Delay(delayMs);
                }

                await context.Response.CompleteAsync();
            });
        });
    });
});

builder.Build().Run();

Using dedicated connections

Below shows the code to test the service. Of particular interest are lines 6 & 7, where the requests to the large and slow endpoint can either use the default handler, or be configured with their own handler. For the initial tests we’ll put everything on the same handler.

This code sends a new request every second. Note the condition if (reqId % 3 == 0) {...} that causes every third request to be to the slow API.

public class MyDownloader : MonoBehaviour
{
    async Task<Stream> GetHttpStream()
    {
        // *** Switch which line is commented out below to use a dedicated handler ***
        // using var client = UnityHttpClient.Get("slow");
        using var client = UnityHttpClient.Get();

        var stream = await client.GetStreamAsync("https://localhost:5001/api/blob");
        return stream;
    }

    async Task<HttpResponseMessage> GetHttpRequest()
    {
        using var client = UnityHttpClient.Get();
        var req = await client.GetAsync("https://localhost:5001/api/item");
        if (req.StatusCode != System.Net.HttpStatusCode.OK)
        {
            throw new ApplicationException("Error");
        }
        return req;
    }

    bool doRequests = false;
    int requestId = 0;
    Stopwatch timer;

    void Start()
    {
        LogMsg($"Starting downloader.");
        timer = Stopwatch.StartNew();
    }

    void LogMsg(string msg) =>
        Debug.Log(msg + $" ThreadId: {Thread.CurrentThread.ManagedThreadId}");

    void Update()
    {
        if (!doRequests) return;

        // Every second start a new request
        if (timer.ElapsedMilliseconds >= 1000)
        {
            timer.Restart();
            int reqId = Interlocked.Increment(ref requestId);
            Task.Run(async () =>
            {
                if (reqId % 3 == 0)
                {
                    // One in three requests is for a large/slow item.
                    Debug.Log($"Request {reqId} starting for slow blob. " +
                            $"ThreadId: {Thread.CurrentThread.ManagedThreadId}");

                    var stream = await GetHttpStream();

                    // On a separate thread, read the stream to completion.
                    _ = stream.CopyToAsync(Stream.Null).ContinueWith(_ =>
                          LogMsg($"Request {reqId} completed for slow blob."));
                }
                else
                {
                    LogMsg($"Request {reqId} starting for fast json.");
                    var request = await GetHttpRequest();
                    LogMsg($"Request {reqId} completed for fast json.");
                }
            });
        }
    }

    void OnMouseDown()
    {
        doRequests = !doRequests;
    }
}

Running this code as-is we can see that pretty soon even the “quick” requests are getting backed up waiting for the slow requests to complete. The initial “fast” requests number 1, 2, 4, and 5 all complete within the expect 1 sec time. But once the second “slow” request starts (request 6), we see that all requests start taking a while to complete, as the two connections per origin limit is consumed with running the slow requests.

Log with one handler

If we look at the object in memory, we can see in the scheduler that the default two connections are allocated, and there are a queue of operations waiting to run.

Objects with one handler

Adding an additional handler

If we uncomment line 6 and comment out line 7, so that GetHttpStream is using its own dedicated handler group (named “slow”), we see that now all the fast API requests are completing immediately, even as the slow API requests start to queue up.

Log with slow handler

The objects in memory show us that there are now two ConnectionGroup objects contained within the ServicePointScheduler, and that the group named HttpClientHandler1 (the first group created for the fast requests) has an emtpy queue - even with slow requests waiting. Meanwhile the group created for the slow requests, HttpClientHandler2, has a couple items in the queue (as well as operations currently in progress).

Objects with slow handler

This shows that being able to provide a “named” connection group can avoid contention and blocking for requests on other groups.

Certificates and OpenSSL

2021-05-14T00:00:00+00:00

OpenSSL is a useful tool for working with certificates, however its usage is often obscure and challenging to get right. This is a cheat sheet for common operations.

Installation

I find it easiest to work with OpenSSL at the Bash prompt. As I’m on Windows, this means using the Windows Subsystem for Linux. With this running, install at least OpenSSL v1.1.1f via the following:

sudo apt-get install -y openssl
# Check the version installed
openssl version

The file formats

When dealing with certificates, certificate requests, and private keys, you’ll generally be working with PEM format files. These are basically text files with a header and footer surrounding some base64 encoded data. The file extensions are typically .key for private keys, .csr for certificate requests, and .crt for certificates. The contents will appear similar to the below, with the text in the header and footer around the base64 data indicating the content.

-----BEGIN CERTIFICATE-----
MIIGNzCCBR+gAwIBAgIRAMXBvoYhfBEw5+K+kj/8nI4wDQYJKoZIhvcNAQELBQAw
gY8xCzAJBgNVBAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAO
BgNVBAcTB1NhbGZvcmQxGDAWBgNVBAoTD1NlY3RpZ28gTGltaXRlZDE3MDUGA1UE
    ...lines omitted for brevity...
XvKsEcO2TvY+Dcxff5jggYAdrv4IHSkA33L9zAtvFjt3NO/H3d3qxBPE2CSbBmWw
djPNIn7JiaYbhSyz+mBXcxhnPAGqDVh0NyOI+kfgsLNol8Y8QFUxlNPRZNRSt+Hf
eT6Rt49bqgXOgv8=
-----END CERTIFICATE-----

The above is for a certificate (.crt). You will also see “BEGIN CERTIFICATE REQUEST” (.csr) and “BEGIN PRIVATE KEY” (.key).

Another format you may have to work with is PKCS#12, which is typically in a .pfx file. This is a binary format for storing a certificate chain and private key in a single, encrypted file, and is often used if working with Windows or the Azure AppService to upload a certificate for a web server.

Generating self-signed certificates

For local testing, it is common to generate a “self-signed” certificate. As this isn’t signed by a known and trusted authority, it will need to be manually placed in the “Trusted Root Certification Authorities” certificate store on the computer to be used without displaying an error when browsing to a site using it. This is still useful in development.

The quickest way to generate a self-signed certificate for example.com is via the below.

openssl req -newkey rsa:2048 -sha256 -nodes -x509 -days 365 -out example.crt -keyout example.key \
    -subj "/C=US/ST=Washington/L=Seattle/O=Example Inc/CN=example.com"

Breaking down the command line switches here:

req: This is a request for a new certificate
-newkey rsa:2048: Use RSA encryption with a 2048-bit key length. (Usually the default, but being explicit).
-sha256: Use SHA256 as the digest algorithm. (Again, usually the default).
-nodes: No DES. Do not protect the output files (including the private key) with a password.
-x509: Generate a (self-signed) certificate, not a request for a certificate.
-days 365: How long the certificate should be valid for.
-out <filename>: The file to write the actual certificate to. (Does not include the private key).
keyout <filename>: The file to write the private key to. (Keep this safe and secret!)
-subj <path>: The details for the org the certificate is for. “CN” is the “Common Name”, and the URL of the site.

If you leave off the -subj argument, you will be prompted at the command-line to provide these values.

Extensions

Typically there are two “extensions” specified in certificates. The first is the “Subject Alternate Name”, which is a list of other valid URLs this certificate covers. For example, typically a site will be accessible via both the “naked” domain name (example.com) and the www-prefixed version (www.example.com). You aren’t just limited to domain names here. You can also put names such as localhost, or even IP addresses such as 192.168.1.14.

The other common extension is the intended usage of the certificate. For web certificates, this is commonly set to “client authentication” and “server authentication”.

To specify all of this on the command-line, rather than in a config file, use the -addext flag. For example, to make the certificate valid for www.example.com, localhost, and 127.0.0.1, use:

openssl req -newkey rsa:2048 -sha256 -nodes -x509 -days 365 -out example.crt -keyout example.key \
    -subj "/C=US/ST=Washington/L=Seattle/O=Example Inc/CN=www.example.com" \
    -addext 'subjectAltName=DNS:www.example.com,DNS:localhost,IP:127.0.0.1' \
    -addext "extendedKeyUsage=serverAuth,clientAuth"

Installing the self-signed certificate as trusted

To install the certificate on the machine so it is trusted (on Windows):

In Chrome or Edge, navigate to chrome://settings
In the search box type manage certificates
Click the link to open the dialog and go to the Trusted Root Certification Authorities tab
Click Import, then Next, then browse to the .crt file created above.

Using the certificate on a web site

How you bind the certificate to an actual web site depends on the web server being used. If using ASP.NET Core and it’s provided “Kestrel” web server, then first generate a password protected .pfx file containing the certificate and private key. For the above example files, this command would be:

# Convert into a triple-DES password protected pfx file including the private key (you be prompted for a password)
openssl pkcs12 -export -out example.pfx -inkey example.key -in example.crt -des3 -password

In your appSettings.Development.json (or secrets.json) you would then bind this to the site with:

  "Kestrel": {
    "Certificates": {
      "Default": {
        "Path": "/somewhere/example.pfx",
        "Password": "TopSecret!"
      }
    }
  }

Certificate requests

To provide a real SSL certificate for a web site, you must first generate a certificate request, and then provide this to a “trusted authority” to issue the certificate (which they do by signing the request). This is not too different to the above, with a couple of extra steps.

Generate a certificate request

Generating the certificate request itself is actually easier than generating a self-signed certificate. All you need to do is remove a couple of flags from the above command-line. Firstly, remove the -x509 flag, as we are not generating an actual certificate. Secondly, remove the -days 365 flag, as the certificate authority will set how many days the issued certificate is valid for. You’ll also want to change the output filename from .crt to .csr to indicate it is a request, not a certificate.

Many certificate authorities will also automatically generate the subject alternate names (including the “naked” domain if a “www” domain is requested), and set the intended usage of the certificate, so the -addext flags aren’t needed. The final command being something like:

openssl req -newkey rsa:2048 -sha256 -nodes -out example.csr -keyout example.key \
    -subj "/C=US/ST=Washington/L=Seattle/O=Example Inc/CN=www.example.com"

Again, keep the .key file safe and secret (the authority never sees the private key). The contents of the .csr file are what will be provided to the authority. (Typically the contents are cut and pasted into a web form when purchasing the certificate).

Prove domain ownership

Before any (trustworthy) authority issues your certificate, they are going to verify that you own the domain you are requesting a certificate for. Typically this is done by you either registering a DNS record in the domain to prove you own it, or via an email sent to an admin account for the domain (e.g. admin@example.com) which will contain a verification link.

Create the certificate bundle

Once the authority is happy you own the domain, they will send you an email containing a set of certificates. The primary certificate is the certificate you requested, now signed by the authority. The other certificates are the certificates in the trust chain all the way back to the root certificate. Your web server needs all the certificates in this chain, as well as the private key, to enable SSL connections.

Some web servers expect all the certificates in a bundle. This is simply a concatenation of all the text files (recall they are base64). For example, the below concatenates the certificates I got from my certificate authority (https://comodosslstore.com/). The first is my domain certificate, the next two are intermediate certificate authorities, and the last is the root certificate authority.

cat www_example_com.crt SectigoRSADomainValidationSecureServerCA.crt USERTrustRSAAAACA.crt AAACertificateServices.crt >> ssl_bundle.crt

As was done for the self-signed certificate, for use with ASP.NET Core’s Kestrel server (or with Azure AppService) this should be packaged along with the private key into a password protected .pfx file.

openssl pkcs12 -export -out example.pfx -inkey example.key -in ssl_bundle.crt -des3 -password

This is then bound to the web server via the settings as for the .pfx file for the self-signed certificate. (Or via the Azure Portal for Azure AppService - see https://docs.microsoft.com/en-us/azure/app-service/configure-ssl-certificate#upload-certificate-to-app-service).

Bonus material

Below are several other commands I’ve found handy when working with certificates with OpenSSL.

# To view the details of a private key
openssl rsa -text -noout -in example.key

# To view the details of a certificate request
openssl req -text -noout -in example.csr

# To view the details of a certificate
openssl x509 -text -noout -in example.crt

# To verify the private key, certificate request, and certificate match
# This extracts the public key from each file and reports the sha256 hash of it
openssl pkey -pubout -in ./example.key        | openssl sha256
openssl req  -pubkey -in ./example.csr -noout | openssl sha256
openssl x509 -pubkey -in ./example.crt -noout | openssl sha256

Debugging ASP.NET Core in VS 2019

2021-04-06T00:00:00+00:00

This post details how I built ASP.NET Core 5 locally for much richer debugging of my web app, allowing me to step seamlessly through my code and the ASP.NET product code.

Motivation

I had been struggling to understand the way cookie authentication and OpenID Connect middleware interacted with each other, the authentication server, and my web app itself. To help understand this better, I wanted to step through the ASP.NET code locally. Being that ASP.NET Core is open source and source & symbol indexed (i.e. it can fetch the source automatically), this just works - to a degree. The trouble is that the version that ships is a release build, with lots of stuff optimized away by the compiler. This makes debugging a real pain. Here I will detail how I set up a local “Debug” build of ASP.NET Core 5 for better debugging.

Get the bits

The Build from source page in the ASP.NET Core repo goes over this in more detail, but in a nutshell, on Windows you:

Clone the repo via git clone https://github.com/dotnet/aspnetcore
To debug code corresponding to the 5.0.4 release, git checkout v5.0.4
Get all the submodules in sync via git submodule update --init --recursive
Ensure you have the required depenencies installed.
- Windows SDK version 17134 must be installed, else you get odd errors when building.
- VS 2019 with the necessary components to build native and managed code must be installed.
- Java should be installed with the JAVA_HOME enviornment set correctly. (I already had this installed via an Android Studio installation).
- Node.js should be installed - see https://nodejs.org/en/
- Install the Yarn package manager via npm install -g yarn
- You’ll also want the latest PowerShell release installed. See https://aka.ms/powershell-release?tag=stable

Building the bits

With the above done, restore the necessary packages and build locally via the below. Note that I had to rerun the build several times before it failed with various Yarn errors. (Looking through the issues, this seems to be a common problem)

.\restore.cmd
.\build.cmd -all -pack -arch x64

Once the build has completed successfully, the next step is to activate an environment in which you can use the local build. Open a PowerShell command-prompt at the source root and run . .\activate.ps1. (Note the leading dot. This is important).

At this point if you run dotnet.exe it should be using the version in the repo. (You can verify this by running where.exe dotnet, which should list the local version first).

Using the bits

The biggest challenge was how to actually use the locally built ASP.NET Core (and related packages) when running in Visual Studio. Especially as the package I was most interested in, Microsoft.AspNetCore.Authentication.OpenIdConnect, is shipped as a separate NuGet package (even if built as part of the main repo). The key to this was putting a NuGet.config file in the project root with the below contents (my ASP.NET repo is in C:\src\aspnetcore locally):

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <config>
	    <add key="globalPackagesFolder" value=".\.packages" />
	</config>
    <packageSources>
        <clear />
        <add key="AspNetDev" value="C:\src\aspnetcore\artifacts\packages\Debug\Shipping" />
        <add key="NuGet.org" value="https://api.nuget.org/v3/index.json" />
    </packageSources>
</configuration>

This does a couple of things:

Uses a different global packages cache location, as the default one has probably already populated with release versions of packages I want to debug.
Added the local Debug build folder for packages as a NuGet source. (Local folders are checked before web locations).

If you had been working on your web project already, delete the ./bin and ./obj folders under it to ensure any prior binaries are removed. Then run dotnet.exe build, which should restore the packages from the local build, (where it can), and then build the project.

Debugging the bits

The final piece was to have this work in Visual Studio 2019 for debugging. (VS Code works too, however I find for advanced debugging VS 2019 has a lot of additional nice features).

You can’t just open VS normally. You need to open from the activated environment. As I have VS 2019 Enterprise installed to the default location on my machine, this was done from the PowerShell prompt with:

& "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\devenv.exe"

Another issue I noticed is that if Visual Studio builds the project, it would often restore packages with release versions, not the local versions I configured above. (I assume it has different cache logic somewhere). To avoid this, keep the activated environment command prompt open, and build with dotnet.exe build whenever needed. (I also modified the Visual Studio “Tools / Options / Projects and Solutions / Build and Run” setting to prompt when out of date, rather than build automatically on run, to avoid any accidents).

The result

With that, I was able to F5 my web app in Visual Studio and step into the various ASP.NET methods with ease, with no call stacks or locals optimized away! It’s really helped piece together some of the internals for me.

Manifest-free ETW in C++

2019-11-04T00:00:00+00:00

ETW is a great technology when all the stars line up, but various iterations over time have had various limitations, and it often isn’t the easiest technology to work with.

One of the main challenges previously was the need to ship a manifest with your instrumented code, and installing the manifest when installing the app would often require custom actions and admin privileges for the installer.

With the new TraceLogging framework, “manifest-free” ETW became a thing, and instrumenting and distributing an app become simpler.

The struggle

Recently I was trying to include some ETW events in a project built with Clang/LLVM. I was attempting to use the “manifest-free” TraceLogging approach, however the inclusion of the Windows SDK <TraceLoggingProvider.h> header was causing compile errors in Clang++. (Which may have since been fixed). This led me to try and decipher what this header was doing exactly, and how to recreate this functionality just using standard C++.

Note: My project requires using C++14, hence not using some features of C++17 (such as std::string_view) which could have simplified some of the code below.

My first approach to understanding the TraceLogging header was to repeatedly “Go to definition” on the macros being used and try to decipher them, but this quickly became intractable. There are a lot of non-trivial macros that cross-reference and nest in various ways.

The next approach was to compile a simple use-case outputting the pre-processed source file to see what this got converted to. This reveals the extent of the processing these macros do. For example, when using the <TraceLoggingProvider.h> header, this simple line to log an ETW event:

TraceLoggingWrite(g_hMyComponentProvider, "My1stEvent",
  TraceLoggingInt32(42, "MyIntField"),
  TraceLoggingString("some values", "MyStrField"));

Got turned into the below by the pre-processor. Note the various uses of __pragma, __declspec, and __annotation. And even this code still calls into other helper methods from the header (e.g. the _Tlg* calls):

do {
  __pragma(warning(push))
  __pragma(warning(disable:4127 4132 6001))
  __pragma(warning(error:4047))
  __pragma(execution_character_set(push, "UTF-8"))
  __pragma(pack(push, 1))

  typedef _TlgTagEnc<0> _TlgTagTy;
  enum { _TlgLevelConst = 5 };

  static struct {
    CHAR _TlgBlobTyp;
    UCHAR _TlgChannel;
    UCHAR _TlgLevel;
    UCHAR _TlgOpcode;
    ULONGLONG _TlgKeyword;
    UINT16 _TlgEvtMetaSize;
    _TlgTagTy::type _TlgEvtTags;
    char _TlgName[sizeof("My1stEvent")];
    char _TlgName0[sizeof("MyIntField")];
    INT8 _TlgIn0;
    char _TlgName1[sizeof("MyStrField")];
    UINT8 _TlgIn1;
  }
    __declspec(allocate(".rdata$zETW1"))
    __declspec(align(1))
    const _TlgEvent = {
      _TlgBlobEvent3,
      11,
      _TlgLevelConst,
      0,
      0,
      sizeof(_TlgEvent) - 11 - 1,
      _TlgTagTy::value,
      ("My1stEvent"),
      ("MyIntField"),
      TlgInINT32,
      ("MyStrField"),
      TlgInANSISTRING
    };
    TraceLoggingHProvider const _TlgProv = (g_hMyComponentProvider);

    if ((UCHAR)_TlgLevelConst < _TlgProv->LevelPlus1 && _TlgKeywordOn(_TlgProv, _TlgEvent._TlgKeyword)) {
      EVENT_DATA_DESCRIPTOR _TlgData[2 + 1 + 1];
      UINT32 _TlgIdx = 2;
      (   _TlgCreateDesc<INT32>(&_TlgData[_TlgIdx], (42)),
        _TlgIdx += 1,
        _TlgCreateSz(&_TlgData[_TlgIdx], ("some values")),
        _TlgIdx += 1,
        __pragma(warning(disable:26000)) __annotation(L"_TlgWrite:|" L"23" L"|" L"g_hMyComponentProvider" L"|" L"\"My1stEvent\"" L"="   L"|" L"\"MyIntField\"" L"="  L"|" L"\"MyStrField\"" L"="),
        _TlgWrite(_TlgProv, &_TlgEvent._TlgChannel, 0, 0, _TlgIdx, _TlgData)
      );
    }
    __pragma(pack(pop))
    __pragma(execution_character_set(pop))
    __pragma(warning(pop))
} while (0);

Under the covers

Reverse engineering this further, and stepping through the code, it was surprising to see how simple everything ultimately is. (Note: I found afterwards that much of this info is documented in the TraceLoggingProvider.h file - starting at a helpful line 1804). Basically to log a “manifest-free” ETW event you need to:

Register the provider as usual, using the Win32 API EventRegister.
Construct the EVENT_DESCRIPTOR for the event as usual, though ensure the event is logged to channel 11. (The Id and Version are also less important for manifest-free events).
Construct the EVENT_DATA_DESCRIPTOR array as usual for the event, prefixed with two additional descriptors:
- One to provide the “provider traits” (basically the provider name).
- One to provide the event metadata (i.e. the event & field names and types).
Call the usual Win32 API to log the event. (e.g. EventWrite or EventWriteTransfer)
If desired, when done unregister the provider as usual with the Win32 EventUnregister API.

That’s effectively it for simple ETW events (I’m ignoring things like ETW Activities or logging fields containing arrays or custom structs in this write up). Ultimately, just three methods from kernel32.lib are called from user code.

Packing data without pragmas and structs

Constructing the event metadata is where things get tricky. This is the bulk of the goop in the preprocessor output above, using __pragma(pack(push, 1)) to ensure fields are tightly packed, and then inline declaring a constant instance of a struct to describe the event metadata. Modeling this in standard C++ wasn’t trivial. (And arguably the resulting “template metaprogramming” isn’t easier to read than the macros, which is unfortunate, but it is standard C++).

To get the metadata as a series of tightly packed bytes at compile-time, I wanted to use constexpr variables in C++. It turns out to be surprisingly tricky to take a string literal in C++ (as the event and field names will be represented) and use them in constexpr functions, especially as being limited to C++ 14 meant I couldn’t use std::string_view.

To manipulate a string literal in a constexpr function, the best approach I could find was to reference it as an array of characters of known size (the template parameter N in the code below). To make it easier to index into the character array, it was also helpful to have the compiler create an index sequence along with it (idx below), e.g.

template <size_t N, typename idx = std::make_index_sequence<N>>
constexpr auto MakeStrBytes(char const (&s)[N]) {
    return str_bytes<N>{s, idx{}};
}

The str_bytes type constructed above is a constexpr value consisting of the characters of the string. The implementation for this is shown below. A second constructor is also shown which allows for the joining of two character arrays into a sequence of bytes.

template <size_t N>
struct str_bytes {
  template <std::size_t... I>
  constexpr str_bytes(char const (&s)[N], std::index_sequence<I...>)
      : bytes{s[I]...}, size(N){};

  // Concatenate two str_bytes
  template <std::size_t s1, std::size_t s2, std::size_t... I1, std::size_t... I2>
  constexpr str_bytes(const str_bytes<s1>& b1, std::index_sequence<I1...>,
                      const str_bytes<s2>& b2, std::index_sequence<I2...>)
      : bytes{b1.bytes[I1]..., b2.bytes[I2]...}, size(N) {}

  char bytes[N];
  size_t size;
};

With this in place, we can not only join strings, but also bytes containing other values. For example, to append a byte indicating the type of the field to the name, I implemented the below function:

// Help function to ease joining instances
template <std::size_t s1, std::size_t s2>
constexpr auto JoinBytes(const str_bytes<s1>& b1, const str_bytes<s2>& b2) {
  auto idx1 = std::make_index_sequence<s1>();
  auto idx2 = std::make_index_sequence<s2>();
  return str_bytes<s1 + s2>{b1, idx1, b2, idx2};
}

// Join a field name with a byte indicating the field type.
template <size_t N>
constexpr auto Field(char const (&s)[N], uint8_t type) {
  auto field_name = MakeStrBytes(s);
  const char type_arr[1] = {char(type)};
  return JoinBytes(field_name, MakeStrBytes(type_arr));
}

There are a few other helper methods to ease constructing the ETW event metadata, as shown in the source file at https://github.com/billti/cpp-etw/blob/master/etw-metadata.h. This includes a variadic template function to simply pass along all the fields with the event name to construct a constexpr byte array of the metadata, for example:

constexpr static auto event_meta = EventMetadata("my1stEvent",
    Field("MyIntVal", kTypeInt32),
    Field("MyMsg", kTypeAnsiStr),
    Field("Address", kTypePointer));

The result of the above is a constant byte array representing the metadata for the event, constructed at compile time.

With constructing the metadata out of the way, the next step is to ease the work needed to implement a provider and log events. Every event requires an event descriptor to describe the event id, level, keywords, etc. Again, this data is static, so a helper method can easily make this a compile-time constant:

// Besides the event id, everything else can default to 0.
constexpr auto EventDescriptor(USHORT id, UCHAR level = 0,
                               ULONGLONG keyword = 0, UCHAR opcode = 0,
                               USHORT task = 0) {
  return EVENT_DESCRIPTOR{id,
                          0,  // Version
                          kManifestFreeChannel, // Channel 11
                          level,
                          opcode,
                          task,
                          keyword};

An ETW base class

The TraceLoggingProvider.h header does provide a callback mechanism to efficiently track if the provider is enabled, then check this before logging any events. This can be easily implemented in a base class also, along with registering and unregister the provider.

An interesting feature of Clang (from GCC) is the builtin that supports indicating that a condition is likely (or unlikely), named __builtin_expect. This allows for optimization such as moving code unlikely to be executed to the end of the function. For example, as the provider will usually NOT be enabled, the check could be written as:

// GCC/Clang supported builtin for branch hints
#if defined(__GNUC__)
#define LIKELY(condition) (__builtin_expect(!!(condition), 1))
#else
#define LIKELY(condition) (condition)
#endif

  bool IsEventEnabled(const EVENT_DESCRIPTOR* pEventDesc) {
    if (LIKELY(this->is_enabled == false)) return false;
    return (pEventDesc->Level <= this->current_level) &&
           (pEventDesc->Keyword == 0 ||
            ((pEventDesc->Keyword & this->current_keywords) != 0));
  }

Once various functions are inlined and optimized, this resulted in the below CPU instructions at the location in the user code where the ETW logging call was made. Basically, only 3 additional instructions, and no branch, are executed when logging is not enabled. This can also reduce the number of instructions fetched into the instruction cache during execution, if the code after the main body of the function is never executed.

;; ... code in the function prior to the ETW tracing call

;; Start of check if the provider is enabled
488b05d64f0000       mov     rax, qword ptr [mydll!my_provider] ;; Load the provider instance
803801               cmp     byte ptr [rax], 1                  ;; First byte is the 'enabled' field
0f844f020000         je      mydll!sort_array+0x2a3             ;; Jump to the tracing code if enabled
;;... rest of the function body
c3                   ret
;; Remaining instructions to do actual tracing are after the mainline code's ret (target of 'je' above)
80784a04             cmp     byte ptr [rax+4Ah], 4
0f82a7fdffff         jb      mydll!sort_array+0x54 (00007ff6`66a71124)
0fb75048             movzx   edx, word ptr [rax+48h]
;; ... remaining tracing code, then a jump back to the code after the tracing call

To implement a method to write an event to ETW, derived classes then simply implement one function for each event to log, and call the base class LogEventData method to do all the work. e.g.

void MyProvider::Log3Fields(INT32 val, const std::string& msg, void* addr) {
  constexpr static auto event_desc = EventDescriptor(100);
  constexpr static auto event_meta = EventMetadata("my1stEvent",
      Field("MyIntVal", kTypeInt32),
      Field("MyMsg", kTypeAnsiStr),
      Field("Address", kTypePointer));

  LogEventData(&event_desc, &event_meta, val, msg, addr);
}

The LogEventData implementation takes a pointer to the compile-time data representing the event metadata, and uses a parameter pack to handle the event data, e.g.

  template <typename T, typename... Fs>
  void LogEventData(const EVENT_DESCRIPTOR* p_event_desc,
                    T* meta, const Fs&... fields) {
    if (!IsEventEnabled(p_event_desc)) return;

    const size_t desc_count = sizeof...(fields) + 2;
    EVENT_DATA_DESCRIPTOR descriptors[sizeof...(fields) + 2];
    // etc.

In general, the provider methods to log events should be as simple as possible, (for example, like the Log3Fields method shown above), and be inline methods implemented in the header file, which will allow the compiler to inline the tracing calls into the instrumented code as shown in the assembly above. (Link time optimization may also be able to do this, but I didn’t try it).

Performance

I’ve read ETW notes that state that on an average machine a provider should be able to log about 10,000 events/sec without too much overhead. In my simple testing, writing a trivial app to allocate, fill, and sort random arrays (i.e. mostly memory and CPU bound), I noticed on average about a 3% overhead (increase in execution time) with the provider enabled and recording around 10,000 events/sec over a large number of runs. This is on my Surface Book 2, which is a decent laptop, but is still just a laptop.

The below charts shows the execution time (in microseconds) over 40 runs with the provider enabled and disabled. The overhead is often less than the noise.

The difference between runs where the provider was disabled, or the provider was completely removed, was not observable. (i.e. Instrumenting the code is effectively free, but you pay a small overhead when a session is actually listening for events).

Sample code

For the complete project demonstrating how to use the C++ only approach to logging manifest-free ETW events, see project on GitHub at https://github.com/billti/cpp-etw.

Minimal Node.js native modules - N-API

2019-07-10T00:00:00+00:00

This is a continuation from part 1. It is recommended to read this post first.

In this section we are going to write a native module for Node.js using C++ and the technique outlined in the first post. Set this up as follows:

Create an empty directory to host the project and cd into it.
Assuming you have a 64-bit Node.js >= v10.3 on your PATH, run npm init vs-napi.

This runs the create-vs-napi npm package I created to get up and running quickly with a Visual Studio 2017 project for experimentation writing Node.js native modules using the N-API API. It is used here mainly to fetch the header files and import library needed for Node.js development, but if interest, open the project in Visual Studio and build (or just run msbuild <projectname> from the Developer Cmd Prompt), and then run node test.js to verify the native module works. Delete the output directories (obj and x64) afterwards.

For clarity, builds will be done via the command-line compiler rather than MSBuild and/or .vcxproj files. To make this easily repeatable, create a file named build.bat as shown below. Note that some options have been added over the prior post, mostly for generating debug info and specifying directory paths.

@ECHO OFF
SETLOCAL
if not "%VSCMD_ARG_TGT_ARCH%"=="x64" (
    echo Please build from an x64 Native Tools command prompt for VS 2017
    exit /b 1
)

:: Compiler options are:
:: - /O2  optimize for speed
:: - /GS- switch off extra runtime security checks (these need the CRT)
:: - /LD  build a DLL
:: - /Zi  generate debug info
:: - /I   add a include directory
:: - /D   preprocess defines
:: - /F?  specify output locations for .obj, .pdb, and final binary
SET CL=/O2 /GS- /LD /Zi /I"include\node" /D"UNICODE" /Fdx64\ /Fox64\ /Fe"x64\vsnapi.node"

:: Linker options are:
:: - /OPT           enable given optimizations (using the /Zi setting defaults them to off)
:: - /NODEFAULTLIB  do not link to the C runtime by default
:: - /ENTRY         use the given entry point (not the default CRT one)
SET LINK=/OPT:REF,ICF /NODEFAULTLIB /ENTRY:DllMain

:: This will use the CL and LINK environment variables for options
cl main.cc kernel32.lib lib\node\x64\node.lib

Now from an x64 Native Tools Command Prompt in the directory, run build - and the build will fail.

This is because the sample code doesn’t provide a DllMain entry point and depends on the C runtime. You can temporarily remove the /NODEFAULTLIB and /ENTRY options and the code will compile by linking statically to the CRT, then run node test.js and see that the module does work (and is around 90KB on my machine). Now add the linker options back to exclude the CRT, and we’ll fix CRT dependency.

Initialization

When using the C runtime library, various things are taken care of automatically under the covers. Chief amongst these is initialization. When building a Node.js native module certain macros expand to be globals that are initialized at startup. If not using the CRT, this will need to be done “manually”.

One of the key lines in the C++ code for the Node.js module is the line:

NAPI_MODULE(NODE_GYP_MODULE_NAME, init)

This expands in the preprocessor to the below (if NODE_GYP_MODULE_NAME was defined as vsnapi):

extern "C" {
    static napi_module _module =
    {
        1,
        0,
        __FILE__,
        init,
        "vsnapi",
        0,
        {0},
    };
    static void __cdecl _register_vsnapi(void);
    __declspec(dllexport, allocate(".CRT$XCU")) void(__cdecl * _register_vsnapi_)(void) = _register_vsnapi;
    static void __cdecl _register_vsnapi(void) {
        napi_module_register(&_module);
    }
}

This depends on some C runtime “magic” to call napi_module_register at load time with the _module structure, by putting a pointer to a function to run (_register_vsnapi) directly into a section (.CRT$XCU) the CRT reads when it runs at startup. (i.e. the entry point we no longer have).

Note: I highly recommend the Matt Godbolt talk The bits between the bits for an under the covers look at how this initialization works at startup when using the C runtime. The documentation on CRT Initialization also gives a good overview.

To do the same initialization at load time without the CRT, we’ll provide the DllMain entry function and run similar code directly when the module is loaded. After also adding the windows.h include to the top of the file, the entire main.cc file should appear as shown below.

#include <windows.h>
#include <node_api.h>

namespace vsnapi {

napi_value Method(napi_env env, napi_callback_info args) {
  napi_value greeting;
  napi_status status;

  status = napi_create_string_utf8(env, "hello", NAPI_AUTO_LENGTH, &greeting);
  if (status != napi_ok) return nullptr;
  return greeting;
}

napi_value init(napi_env env, napi_value exports) {
  napi_status status;
  napi_value fn;

  status = napi_create_function(env, nullptr, 0, Method, nullptr, &fn);
  if (status != napi_ok) return nullptr;

  status = napi_set_named_property(env, exports, "hello", fn);
  if (status != napi_ok) return nullptr;
  return exports;
}

//NAPI_MODULE(NODE_GYP_MODULE_NAME, init)
extern "C" {
  static napi_module _module;

  bool WINAPI DllMain(HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved) {

    if (fdwReason == DLL_PROCESS_ATTACH) {
      _module = {
        NAPI_MODULE_VERSION,
        0,
        __FILE__,
        init,
        "vsnapi",
        0,
        {0},
      };

      napi_module_register(&_module);
      return true;
    }
  }
}

}  // namespace vsnapi

Now when you run build, it should compile without error, and run as before when node test.js is run. The resulting binary is not using the CRT, and on my machine the binary size is around 3KB:

  3,072 vsnapi.node

Delay loading

There is a problem with the above module. The node.lib library it links to tells the program it will find the N-API functions it needs in the node.exe module. This can be seen by running dumpbin /dependents /imports x64\vsnapi.node which will contain a section of output similar to:

  Section contains the following imports:

    node.exe
             18000C218 Import Address Table
             180014498 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                        38B5 napi_create_function
                        38BE napi_create_string_utf8
                        390B napi_set_named_property
                        38FB napi_module_register
...

However in certain environments this is not the module exporting those functions. For example, in an Electron application, they are contained in a node.dll binary beside the electron.exe executable.

If you created the project skeleton as initially outlined, you will have noticed a win_delay_load_hook.cc file was also created. This is responsible for intercepting any calls to locate the node.exe module, and return the “correct” module as appropriate.

Note: If interested, you can see some discussion of this code on this pull request, and a quick summary of the problem/solution on this stack overflow post

Setting the option to delay load and hook the loading of the node.exe module requires linking to delayimp.lib however, and unfortunately that depends on the CRT. So that option is out. An alternate approach is possible that requires a little bit of code, but has some additional nice properties.

Explicit loading

A solution to the above problem is to explicitly load the N-API module and functions using the Win32 APIs GetModuleHandle() and GetProcAddress(). GetModuleHandle requires that the module is already loaded, but Node must be loaded as it is loading the extension. GetProcAddress has the benefit of feature detection via late binding to N-API functions, rather than failing to load the module is an import is not present.

Another benefit is that the node.lib library does not need to be linked in any more. In fact, with this approach N-API Node.js modules can be built with only two header files (currently) from Node.js: node_api.h and node_api_types.h.

This does mean each N-API function is called through an indirect pointer however. The below code shows how this could be done with a helper macro.

// TODO: Find a cleaner way than this
bool LoadNapiFunctions()
{
  // Handle to the .exe for the current process (e.g. node.exe in the mainline scenario)
  HMODULE nodeModule = GetModuleHandle(NULL);
  // See if this contains the APIs we want to use.
  FARPROC fn_addr = GetProcAddress(nodeModule, "napi_module_register");

  // If not, see if node.dll is present and contains the exports (e.g. Electron scenario).
  if (fn_addr == NULL) {
    nodeModule = GetModuleHandle(L"node.dll");
    if (nodeModule == NULL) return false;
    fn_addr = GetProcAddress(nodeModule, "napi_module_register");
    if (fn_addr == NULL) {
      // Couldn't find the module with the N-API exports
      return false;
    }
  }

  bool apiLoadFailed = false;

  // Macro to simplify retrieving of delay loaded N-API functions. Keep format in sync with DECL_NAPI_IMPL in napi.h
#define GET_NAPI_IMPL(fn_name)                      \
    fn_addr = GetProcAddress(nodeModule, #fn_name); \
    if (fn_addr == NULL) apiLoadFailed = true;      \
    p##fn_name = (decltype(p##fn_name))fn_addr;

  // Assign the addresses of the needed functions to the "p*" named pointers.
  GET_NAPI_IMPL(napi_module_register);
  GET_NAPI_IMPL(napi_create_function);
  GET_NAPI_IMPL(napi_set_named_property);
  GET_NAPI_IMPL(napi_create_string_utf8);

  // If any required APIs failed to load, return false
  if (apiLoadFailed) return false;

  // Fetch the optional APIs if present.
  GET_NAPI_IMPL(napi_add_env_cleanup_hook); // Not available on Node.js 8

  return true;
}

The declarations and definitions for the N-API function points are in a header file as below:

#include <node_api.h>

// NAPI_IMPL is defined before including this header in the one .cc file that defines the pointers
#ifdef NAPI_IMPL
  #define DECL_NAPI_IMPL(fn_name) decltype(&fn_name) p##fn_name
#else
  #define DECL_NAPI_IMPL(fn_name) extern decltype(&fn_name) p##fn_name
#endif

DECL_NAPI_IMPL(napi_module_register);
DECL_NAPI_IMPL(napi_create_function);
DECL_NAPI_IMPL(napi_set_named_property);
DECL_NAPI_IMPL(napi_create_string_utf8);
DECL_NAPI_IMPL(napi_add_env_cleanup_hook); // Not available on Node.js 8

The LoadNapiFunctions function is called at initialization in DllMain, and the N-API APIs are called through the p-prefixed function pointers (checking for NULL first for optional APIs), e.g.

void DoSomething() {
  // If the DLL loaded OK, this required function was present
  pnapi_create_function(/* ... */);

  // The below was optional and isn't present pre Node.js 8
  if (pnapi_add_env_cleanup_hook != NULL) {
    pnapi_add_env_cleanup_hook(/* ... */);
  }
}

A final wrinkle

While we’ve done our best to avoid dependencies on the CRT library in our code, the compiler itself can actually generate calls to CRT functions in the code it generates. There are two things you can do if and when this occurs:

Enable intrinsics in the compiler (on by default in /O2 builds). This means the compiler MAY replace some function calls (e.g. memcpy) with inline versions, avoiding the need to link to an implementation.
Provide your own version of the function the linker fails to find. Most simple CRT functions (e.g. malloc, memcpy, etc.) are relatively short and trivial to find examples of. (For example, see some of the provided functions/operators at https://github.com/billti/tsetwlog/blob/master/src/crt.cpp).

Conclusion

While not entirely trivial and obvious, it is possible to write a N-API module which does not depend on the C-runtime library. This can avoid some pitfalls with both dynamic linking (are the required DLLs present), and static linking (binary size and resource allocation).

This does mean common C-runtime functionality must be avoided or re-implemented, but if the module is simple enough (and platform specific) this can result in a small binary with minimal dependencies.

Thanks for reading!

Minimal Node.js native modules - the CRT

2019-02-08T00:00:00+00:00

I recently wanted to author a Node.js native module for inclusion into an application that only runs on Windows. I wanted to keep the size and dependencies as small as possible. I hit numerous pain-points. This is my journey.

Note: This first post will deal only with consideration in using the C runtime library. A follow-up post will discuss how to build a Node.js native module with the build options laid out below.

The C runtime library

Dealing with the C runtime library can be complicated on Windows. The standard cross-platform APIs you can write with are highly valuable, but on Windows you have two ways to use the CRT, and both have pros and cons - especially when it come to small binaries shipped via a package manager

Note: Many of the issues I hit regarding the CRT for Node.js binaries I subsequently found were also hit by Steve Dower with Python extensions for many of the same reasons. I highly recommend reading his blog posts on the topic for greater detail on the underlying issues.

Dynamically linking

This is the recommended approach, with minimal issues if the CRT DLLs are already present on the machine. If you are distributing a large package for install, then you can bundle the CRT redistributable into your installer to make sure. However, when you are trying to distribute a small package via npm - that should install with minimal install scripts, permissions, and package size - this isn’t a good option.

Recently an updated CRT has begun to be included on Windows systems. (See this blog post on the Universal C runtime). However this is not a solution as:

This only includes the C-runtime, not the vcruntime or C++ bits, so those will likely still need to be distributed
This updated system-level CRT still may not be present on a large number of Windows machines. (It is either applied as an update or a download for certain Windows versions).

This makes dynamically linking in the CRT a poor option for apps/packages that ship as a small standalone binary without a full Windows installer currently.

Static linking

Another option is to compile the needed CRT functionality directly into your binary. This eliminates all runtime dependencies on separate CRT binaries, but still has issues to contend with:

Certain patterns are then to be avoided, specifically those that may share state across binaries that use the C runtime library. Basically, certain functionality in the CRT depends on global/share state within the process, but if multiple binaries statically link the CRT, then each has its own copy of this state. (This link contains some details).
Loosely related to the above, each initialization of the CRT (which now happens in each statically linked copy), consumes some finite resource - the most problematic of these being thread/fiber-local storage. If you load enough binaries that statically link the CRT, you could potentially exhaust the limit, causing failures. Steve’s Python blog linked to above contains much greater detail on this.
Size. While in a real app, even a few MB might be trivial, I was aiming for a minimal package size. My module is very lightweight, and with dynamical linking to the CRT the binary was less than 20KB. But with static linking, even with minimal CRT usage, this ballooned to close to 100KB. (Which may still not be much compared to many npm packages commonly installed today, but still seemed excessive to me).

Note that building native Node.js modules using node-gyp today uses this static linking approach.

No CRT linking

There is another option: Don’t use the C/C++ runtime libraries. For the limited functionality my module required, and as I was only targeting Windows, this was a perfectly valid option. However it did come with some interesting challenges we’ll cover along the way. (Most notably: Initialization and delay-loading).

Building with the various CRT options

But first, let’s see how these options differ in compiler flags and resulting binary size. We’ll use a basic console application for simplicity.

To follow along, open a x64 Native Tools Command Prompt, (assuming you have Visual Studio 2017 or the C++ Build Tools installed), and create a file named myapp.cpp. Enter the below code, which is about the simplest “Hello, world” C++ program you can write.

#include <iostream>

int main() {
    std::cout << "Hello, world" << std::endl;
}

Compile this at the command line with cl /O1 /MT /EHsc myapp.cpp . The flags given here are:

/O1 fully optimize with a preference for minimal size
/MT statically link in the C runtime library
/EHsc enable C++ exceptions (makes little difference here, but avoids a build warning)

The output size this gives me is:

  224,768 myapp.exe

Nearly 225kb for “Hello, world”! For reference, compile again linking the CRT dynamically (changing the /MT option for /MD). On my machine, this results in:

  11,264 myapp.exe

So of that ~225KB, all but ~11KB is C/C++ runtime code that can be dynamically link in. To see the DLLs that the binary depends on and what it imports from them run the command:

dumpbin /dependents /imports myapp.exe

This will show the few DLLs this application will load at runtime, and the functions it will use from them. (Which is quite a few for such a simple application).

But as outlined above, this isn’t really an option for an npm distributed Node.js native module, so back to static linking with the /MT option for now.

Next to try getting rid of the C++ code and stick to plain old C. Change the code to the below:

#include <stdio.h>

int main() {
    puts("Hello, world");
}

Compile this with cl /O1 /MT myapp.cpp (note that exception support is not needed now). This now gives me a binary size of:

  97,280 myapp.exe

That’s a pretty significant saving, going from nearly 225kb to under 100kb by switch to plain C.

And finally, not using the C runtime at all, and just using the APIs provided by Windows. You give up numerous things here (not least portability and the ability to use C++ exceptions), but let’s give it a shot. Change myapp.cpp to:

#include <windows.h>

int main() {
    char msg[] = "Hello, world\n";
    DWORD written = 0;
    WriteFile(
        GetStdHandle(STD_OUTPUT_HANDLE),
        (void*)msg, sizeof(msg) - 1, &written, NULL
    );
}

The command line to compile now becomes a bit more complicated, namely:

cl /O1 /GS- myapp.cpp kernel32.lib /link /NODEFAULTLIB /ENTRY:main

Note that /MT is no longer needed, as we are not linking the CRT in at all. The additional options are:

/NODEFAULTLIB tells the linker it to ignore ALL the default libraries (e.g. the CRT)
/ENTRY:main tell the linker to run main on launch (rather than the usual CRT entry point)
kernel32.lib says we want to link to the kernel32.dll Windows system binary
/GS- disables additional runtime security checks (as some depend on the CRT)

With that, compiling on machine now results in a binary of:

  2,560 myapp.exe

A fully functional x64 Windows application at 2.5KB in size! You can run it and see "Hello, world" written to the console. At the developer command prompt you can run dumpbin /dependents /imports myapp.exe again and see that it is dependent on only one other DLL at runtime (kernel32.dll) and imports just two functions from it. Nice!

In the next post we’ll discuss how to build a tiny Node.js native module (again, for Windows only) using only the OS (and Node.js) APIs via the approach outlined above.

Continue on to part 2