Customizing the .NET Common Language Runtime

Why Customize the .NET Common Language Runtime?
Dan Buskirk
May 1, 2007

The user double-clicks on a .NET program. The CLR loads it and runs it. Works like a champ. Why would anyone need to customize the .NET Runtime?

In fact, there are several good reasons. Access to the debugging API is one. Low-level monitoring of things like garbage collection is another. But the most important reason for many developers is the creation of robust extensible applications. In addition to running their own code, extensible applications run code provided by the user. Any mishandling of memory or security could, in principle, damage the entire application. The user library which extends the application needs to be carefully managed. Experienced .NET programmers know that the unit of isolation for memory management and security within .NET is the AppDomain, much as the process is the unit of management within the operating system. The .NET 2.0 runtime provides new interfaces which permit developers to provide their own customized AppDomain management. While the CLR must be launched using unmanaged code, a custom AppDomain manager for version 2.0 can be written in C# or your favorite .NET language.

One example of an program which uses a customized CLR is Microsoft’s SQL Server 2005. If a stored procedure or function is implemented as .NET code, that code cannot be allowed to request memory from Windows. Any server expected to run 24/7 must manage its memory very carefully, and SQL Server cannot have memory being requested behind its back, so to speak. The customized CLR running .NET code within SQL Server routes all requests for memory through SQL Server rather than to the OS directly, thereby ensuring that SQL Server can manage its memory successfully. A service or application which loads the CLR into its own process to run managed code is said to host the CLR.

Of course, it is not only mission-critical server database systems which need to make use of such features. Imagine a small financial services firm which provides software for its customers’ use. Some of the customers are quite sophisticated and request special features. Of course, each customer requests a different special feature. One solution to this problem is to make the core application extensible. Special features can then be implemented as add-ins, vastly simplifying the management of the core application’s development.

This will be the scenario for our exploration of the .NET runtime and its customization. We will start by examining how the CLR loads and runs a simple program.

How Windows Loads a .NET Executable

All Windows executable files contain more than just compiled program code. The file start with a header, which provides the operating system with a great deal of information about the executable code. Headers can be viewed with the dumpin.exe tool that comes with Microsoft Visual Studio. (Dumpbin.exe may be found in the C:\Program Files\Microsoft Visual Studio 8\VC\bin directory.) To use this utility, you will first have to run the vcvars32.bat batch file that also comes with the Visual Studio and is found in the same directory. The batch file ensures that various environment settings are set. You can then run dumpbin /headers <filename> on the command line.

Towards the end of the output you will see an entry

RVA [size] of COM Descriptor Directory

In a native Windows executable file, this entry will have a value of 0. If the value is not 0, Windows knows that this is a managed file, and begins to load the Common Language Runtime. In Windows XP and later, the OS directly calls a function _CorExeMain, which is located in mscoree.dll. This function identifies the required CLR version, loads it, and executes the managed program file under the default settings.

One of our major motivations in developing hosting applications is to go beyond limitations imposed by those constraints.

Writing a Custom .NET Host

Writing a custom host is the first step to customizing the CLR. Writing a custom host even allows you to store your executable code in forms other than the classic "file", though we won't be exploring that possiblity in this series. As an example, SQL Server 2005 stores in managed code for functions, triggers and stored procedures in its system tables rather than access DLL files from the file system. If no dll file needs read when a managed stored procedure is called, SQL Server does not have to concern itselp with the security of such files.

To start with, we will create a simple host that merely runs a .NET executable file. Of course, such a program must be written in unmanaged code. Let's look at the key features of an extremely simple host. This sample is similar to that provided by Steven Pratschner in his MS Press book Customizing the Microsoft.NET Framework Common Language Runtime

In step (1) we are simply declaring declaring a variable to hold our CLR reference. While variable declaration is not usually very exciting, this declaration is very important. The variable type is ICorRuntimeHost. This is the older interface which was created for .NET version 1. The interface which provides newer and more powerful options is ICLRRuntime. However, more powerful often means more complex, and that is certainly the case here. We are using the older interface in this example to get a feel for how a host works; we shall soon move to the newer interface.

Step (2) is where the action is. We call the CorBindToRuntimeEx() function, which is defined in mscoree.dll. In this case, we insist on the .NET 2.0 runtime. We also want the server version, which is optimized to run server applications on symmetric multiprocessing machine. (If there is only one processor we'll always get the workstation CLR, even if we ask for server.) We supply classid values defined in the header files and we get back a reference to the CLR. CorBindToRuntimeEx() returns an HRESULT to indicate success or failure. That's right; this is all COM code. While COM is being banished from the toolbox of the application developer, it is alive and well at a lower level. The CLR itself is, in a very real sense, a COM application and we will use COM interfaces for all our interaction with it from the outside (i.e. unmanaged) world.

The parameter for CorBindToRuntimeEx() marked with (3) allows you to set flags which determine key features of the runtime at startup. For example, we can specify whether garbage collection should be concurrent or not. In this example, we are not using the flags at all; we accept the default by supplying NULL. Later, however, we shall explore how to load different code into different AppDomains, and this parameter will become very important.

In the code near markers (4) and (5) we ask the COM mechanisms to give us a reference to an AppDomain, and then use this pointer to launch our application. It is the code in this section which disappears in the newer .NET 2.0 ICLRRuntime interface. Let's take a quick peek at a simple use of the ICLRRuntime interface.

Next:  Page 2