Customizing the .NET Common Object Runtime - Part 2
Posted by Dan Buskirk on 06/1/07 | DotNet
Overview of AppDomains
In the first part of this series, we created a small custom host for the .NET CLR. Our goal is to include custom loading of AppDomains as a part of the host's function, but first we must take a little time to review the fundamentals of the AppDomain itself.
I doubt that anyone would like to return to the days of Windows 3.1, when any errant application would crash the entire operating environment. In 32-bit Windows, applications run in their own separate process, and the operating system prevents an action in one process from inadvertently stomping on something in another. An AppDomain is very much an extension of this notion. As several processes run within the operating system, the CLR controls one or more AppDomains within a single process. An AppDomain is the unit of isolation with the CLR just as the process is the unit of isolation within the OS. The .NET runtime takes this further, however. AppDomains are not just about memory access, they are also the unit of isolation for security as well. The most common application of multiple AppDomains within a single process is the ASP.NET engine. Individual web applications run in separate AppDomains. As you know, the web application is the unit of security within ASP.NET; this is easy for the CLR to enforce via AppDomains. Our goal is similar; we wish to isolate unpredicatble add-ins from each other and from the application itself.
Another very valuable characteristic of AppDomains is that they can be unloaded. Once loaded, the CLR itself can never be unloaded, except by shutting down the process itself. Neither can individual assemblies be unloaded. Therefore, if we are loading a large assembly and wish to unload it to recover memory, the only alternative is to load that assembly into a separate AppDomain. Once we are finished with the code, the AppDomain, and the managed code within it, can be unloaded.
When the default host is asked to load managed .NET code into the CLR, an AppDomain, not surprisingly called the default AppDomain, is created. The host creates an AppDomainManager, which loads security information, which then loads and runs the managed code. Additional AppDomains can be created entirely within managed code. Each one gets an instance of the AppDomainManager which then loads more security information and more managed code.
Before we continue the development of our custom host, we will build two small applications to illustrate the creation of AppDomains in managed code. The first will simply load and run some user code in a new AppDomain. The second will create a managed AppDomainManager to provide more fine-grained control over this process.
Creating an Application that Compiles and Runs User C# Code
Much of the available sample code that illustrates th creation of new AppDomains illustrates how to load code from an assembly file into that new AppDomain. This, of course, is the practical task most developers will need to perform. For illustrative purposes, and perhaps a little fun, we will take a slightly different tack. Our application will accept a C# script from the user, compile it, and then run it in a new AppDomain.
Using CodeDom to Compile C# Code at Runtime
The .NET Framework has always provided mechanisms for compiling VB and C# code at runtime. Indeed, code written for this purpose in versions 1.0 and 1.1 will still continue to work in .NET 2.0. However, Microsoft is in the process of redesigning some of the key classes in the CodeDom namespace, so we will make a short detour to review them.
In earlier versions, to get a compiler you would declare a variable of type ICodeCompiler, instantiate an object of VBCodeProvider or CSharpCodeProvider class, and then call the CreateCompiler method of the appropriate object. The process is more direct with .NET 2.0. We simply call the new CompileAssemblyFromSource method of the CSharpCodeProvider class and we are all set. In the example code which follows, we will accept a string as input, then wrap that string in a class and function declaration. After this "on-the-fly" class is compiled, we will load its assembly and execute our method. The net effect, from the user's perspective, will be the execution of his or her code snippet immediately. If you are using the Visual Studio, create a Class Libbrary project called "AppDomainTestLibrary" and define the following class:
1 using System;
2 using System.Text;
3 using System.Windows.Forms;
4 using System.Collections;
5 using System.Collections.Generic;
6 using Microsoft.CSharp;
7 using System.CodeDom.Compiler;
8 using System.Reflection;
9
10
11 public class CompileInvoke
12 {
13 public void RunIt(string CodeSnippet)
14 {
15
16 // (1)
17 CSharpCodeProvider cscp = new CSharpCodeProvider();
18 CompilerParameters cp = new CompilerParameters();
19
20 // (2) We are goind to let users create message boxes if they like
21 cp.ReferencedAssemblies.Add("System.Windows.Forms.dll");
22 cp.GenerateInMemory = true; // false means create an output file
23
24 // (3) We can save intermediate files if we like
25 //cp.TempFiles = new TempFileCollection(".", true);
26
27
28 //List<string> al = new List<string>();
29
30 // (4) our source code will be in one big string
31 StringBuilder sb = new StringBuilder();
32
33 sb.Append("using System;\n");
34 sb.Append("using System.Windows.Forms;\n");
35 sb.Append("public class hola\n");
36 sb.Append("{\n");
37 sb.Append("public void f()\n");
38 sb.Append("{\n");
39 sb.Append(CodeSnippet);
40 sb.Append("\n}\n}\n");
41
42 // one array element is like one source file
43 string[] theCode = { sb.ToString() };
44
45
46 // (5) Where the action is
47 CompilerResults cr = cscp.CompileAssemblyFromSource(cp, theCode);
48 if (cr.Errors.Count == 0)
49 {
50 Assembly a = cr.CompiledAssembly;
51 Type HolaType;
52 HolaType = a.GetType("hola");
53 Object it;
54 it = Activator.CreateInstance(HolaType);
55 MethodInfo mi;
56 mi = HolaType.GetMethod("f");
57 try
58 {
59 mi.Invoke(it, null);
60 // (6)The following lines report where the code is running, but otherwise add nothing
61 Console.ForegroundColor = ConsoleColor.Blue;
62 Console.WriteLine(AppDomain.CurrentDomain.FriendlyName);
63 Console.ForegroundColor = ConsoleColor.Gray;
64
65 }
66 catch (Exception Ex)
67 {
68 MessageBox.Show(Ex.Message);
69 }
70 }
71 else
72 {
73 Console.ForegroundColor = ConsoleColor.Red;
74 foreach (CompilerError ce in cr.Errors)
75 {
76 Console.WriteLine(ce.ErrorText);
77 }
78 Console.ForegroundColor = ConsoleColor.Gray;
79 }
80 }//RunIt
81
82
83
84 }//CompileInvoke Class
85
86
87
88
Figure 1.
In section (1), we create our two fundamental objects, the CSharpCodeProvider which will supply the compiler and the CompilerParameters object which we will use to control the conditions of compilation. In section (2) we set a parameter which adds System.Windows.Forms.dll as a reference. Obviously we can add whatever we need. For example, we could add a reference to the application itself, and allow users to control the program with .NET scripts. In this case, we tell the set the GenerateInMemory parameter to true, indicating that we do not wish to create an actual exe or dll file.
In section (4), we use a StringBuilder to create the actual source code for out class. When the code string is finished, we assign it to the first element of a string array called theCode. The compiler expects a string array as an argument, even if we are only compiling a single string as we are here. A multiple element string array would be the equivalent of compiling a project with multiple C# source files. Each element of the array must therefore have the appropriate namespace declarations etc.
In section (5) we actually do the work of compilation. We obtain a CompilationResults object from the CompilaAssemblyFromSource method, and then assign the CompiledAssembly reference to a variable of type assembly. This is a full-fledged assembly and contains our new class and method. However, the code which will run this new assembly does not know about the class or method, so we must do some more work. We call GetType to obtain the type of our "hola" class and we then use Activator.CreateInstance to instantiate it. A MethodInfo object is then used to invoke the "f" mthod which was defined in out source code. In this example we are keeping things simple, but in practice the function we are invoking might require parameters. In this case, the second argument of Invoke would be an Object array holding the parameters. After Invoke executes (and we're certain not to get an error, aren;t we?) a few lines of code print to the console the name of the AppDomain in which this code is executing. At present, this code does not provide us with any useful information. Later, if we run this code from a different AppDomain, this code will provide confirmation of that fact.
We will need a simple user interface to test this code. If you like, you can create a console application which loads a Windows Form with a RichTextBox control serving as a simple source code editor. Or, if you prefer, you can use the form found in the collector01.zip file. The form must contain a button or a menu item which calls our RunIt method and passes the contents of the RichTextBox containing the C# source code. The code might look something like this:
12 internal static void RunItHere(string CodeSnippet)
13 {
14 (new CompileInvoke()).RunIt(CodeSnippet);
15 }
Figure 2.
Loading .NET Assemblies into a New AppDomain
Loading .NET assemblies into a new AppDomain is more difficult to understand than it is to actually program. The conceptual difficulty lies in the fact that the code which gets the ball rolling runs in the first AppDomain, so the code loaded into the new AppDomain will not be directly accessible. The alternative method to RunItHere (figure 2) is RunItInNewAppDomain which we will design to create a new AppDomain and then load our CompileInvoke object into that new AppDomain. We will then have to find some way of calling the RunIt method, which as we have said cannot be called directly. Here is our alternative to the code in Figure 2:
17 internal static void RunItInNewAppDomain(string CodeSnippet)
18 {
19 // (1) Create the new AppDomain
20 AppDomain AD = AppDomain.CreateDomain("HeyThereBuddy"); // It's a friendly name
21 // (2) load our class into the AppDomain, instantiate CompileInvoke,
22 // and get a reference to a proxy for the CompileInvoke object
23 CompileInvoke mbro = (CompileInvoke)AD.CreateInstanceAndUnwrap("AppDomainTestLibrary", "CompileInvoke");
24 mbro.RunIt(CodeSnippet);
25 }
Figure 3.
Section (1) creates the AppDomain with a friendly name of our choosing (as opposed to a full name). Section (2) does quite a bit. It tells the new AppDomain to load AppDomainTestLibrary.dll and to instantiate a CompileInvoke object. We will need to cast the return of CreateInstanceAndUnwrap as the correct type. But the return value is not actually a reference to our new object. Since the code in the new AppDomain cannot be directly accessed, what we get is actually a reference to a proxy which will marshal any calls to the code in the new AppDomain. But how did the proxy get created? The answer is "It didn't!". If we tried to run this code, we would get a runtime error. In order to be accessed across AppDomains, a class must either be serializable or it must inherit from MarshalByRefObject. My CompileInvoke method is neither, but you probably already noticed that. Let's modify the CompileInvoke class:
10 // must either be serializable or inherit from MarshalByRefObject
11 public class CompileInvoke : MarshalByRefObject
12 {
13 public void RunIt(string CodeSnippet)
14 {...
Figure 4.
Now we should be able to test our application.
Of course, up till now we have been letting the AppDomain management tasks to the CLR itself. Now we will turn our attention to designing our own AppDomain manager. In .NET versions prior to 2.0, this usually involved writing unmanaged code in our host. Now, however, .NET 2.0 has provided a toolset to accomplish this task entirely in managed code. Appropriately enough, the new class provided for this purpose is called the AppDomainManager class.
Building Custom AppDomain Managers
The AppDomainManager Class
First, a warning. When you read about the AppDomainManager class in the Microsoft documentation, don't accidently focus on a different AppDomainManager, Microsoft.VisualStudio.Tools.Applications.Runtime.AppDomainManager. This is an internal sealed class for use by Office tools. We need System.AppDomainManager, which, as we have just said, is new for .NET 2.0. Let's take a look at the methods we shall use first.
It would be reasonable to assume that the CreateDomain method is used to create AppDomains, but this is not the case. AppDomains are, in fact, created by the AppDomainHelper class. While this may seem counterintuitive, it is an important design pattern. .NET code may call CreateDomain, but our AppDomainManager might choose to override this request. For example, our AppDomainManager might maintain a pool of existing AppDomains. Based on some criteria, perhaps security credentials, our manager might decide to return a reference to an existing AppDomain when client code requests the creation of a new one. If the code in CreateDomain decides that the creation of a new AppDomain is required, that code calls AppDomainHelper, which actually creates the new AppDomain and returns a reference to it.
Our custom AppDomainManager must also make security decisions. The CreateDomain method expects a reference to an Evidence object (System.Security.Policy.Evidence) as one of its arguments. The security information carried by this object will establish the so-called "top-of-stack" permission set, which will place limits on what code loaded by the AppDomain is permitted to do. Let's take a look at code for a test AppDomainManager, which doesn't add any new functionality but illustrates how a custom AppDomainManager is built and applied.
1 using System;
2 using System.Reflection;
3 using System.Runtime.Hosting;
4 using System.Security;
5 using System.Security.Policy;
6 using System.Threading;
7
8 namespace AppDomainManager02
9 {
10 public sealed class AppDomainManagerTest : AppDomainManager
11 {
12 // (1)
13 // Only fully-trusted code can be loaded in an AppDomainManager constructor
14 public AppDomainManagerTest()
15 : base()
16 {
17 Report(".ctor", ConsoleColor.Red );
18 return;
19 }
20
21 // (2) We don't do any work, just pass the request to the base class
22 public override AppDomain CreateDomain(string friendlyName, Evidence securityInfo, AppDomainSetup appDomainInfo)
23 {
24 Report("CreateDomain", ConsoleColor.DarkGreen);
25 return base.CreateDomain(friendlyName, securityInfo, appDomainInfo);
26 }
27
28 // (3) A CLR host can provide a custom security manager to fine-tune the
29 // security behavior of an AppDomain. This propery provides programmatic access
30 // to the HostSecurityManager. In this example, we simply return a reference
31 // to the a new instance of the default HostSecurityManager.
32 public override HostSecurityManager HostSecurityManager
33 {
34 get
35 {
36 Report("HostSecurityManager", ConsoleColor.DarkYellow);
37 return new HostSecurityManager();
38 }
39 }
40
41 /*
42 // CreateAppDomainHelper actually creates a new AppDomain
43 // this allows CreateDomain to either return an exisiting AppDomain from a
44 // pool or call CreateDomainHelper to create an actual new AppDomain
45 protected static AppDomain CreateDomainHelper (
46 string friendlyName,
47 Evidence securityInfo,
48 AppDomainSetup appDomainInfo)
49 {
50
51 }
52
53 public AppDomainManagerInitializationOptions InitializationFlags { get; set; }
54
55 public override void InitializeNewDomain(AppDomainSetup appDomainInfo)
56 {
57 Console.Write("Initialize new domain called: ");
58 Console.WriteLine(AppDomain.CurrentDomain.FriendlyName);
59 InitializationFlags =
60 AppDomainManagerInitializationOptions.RegisterWithHost;
61 }
62
63
64 */
65
66
67 // Report() is only for learning purposes
68 // Report() displays the current AppDomain location for whatever thread calls it
69 private void Report(string location, ConsoleColor color)
70 {
71 ConsoleColor original = Console.ForegroundColor;
72 Console.ForegroundColor = color;
73 Console.WriteLine("AppDomain: {0}, AppDomainManagerTest::{1}", AppDomain.CurrentDomain.FriendlyName, location);
74 Console.ForegroundColor = original;
75 }
76 private void Report(string location)
77 {
78 Console.WriteLine("AppDomain: {0}, AppDomainManagerTest::{1}", AppDomain.CurrentDomain.FriendlyName, location);
79 }
80 }
81 }
82
Figure 5.
Section (1) is the constructor. In this example, we only note the fact that the constructor executes. If you want to load assemblies in the constructor, it is imperative that these assemblies be fully-trusted, and the CLR will insist on it. To be loaded in the AppDomain constructor, an assembly must either be in the GAC or be marked as fully trusted in an array of strong-names provided by the invoking code.
Section (2) is where we actually create the new managed AppDomain object. This method will be called by the CLR when a request for a new AppDomain is made by some executing managed code. We have not yet discussed how the CLR knows to do this; this is coming up shortly.
In section (3) we provide a HostSecurityManager object when the CLR requests one. We could fine-tune our security be providing a custom HostSecurityManager.
The dll containing our AppDomainManager must be strongly named. Clearly, .NET needs to trust code that controls all security within all AppDomains. A key file is included in the zipped source files; you may wish to use this file for consistency.
As is usually the case, we will need a test platform to see if our code works. In this case we will use a "HelloWorld" program, to keep things as simple as practical. We will integrate a custom AppDomainManager with our progammable CodeDom application later on. Here is the source for our HelloWorld test:
1 using System;
2
3 public class HelloWorld
4 {
5 public static void Main()
6 {
7 Console.WriteLine("HelloWorld::Main() has begun.");
8
9 // say hello in this domain
10 new HelloWorldAux().HelloWorld();
11
12 // create a new domain and say hello over there
13 AppDomain newDomain = AppDomain.CreateDomain("Second AppDomain");
14 HelloWorldAux remote = newDomain.CreateInstanceAndUnwrap(typeof(HelloWorldAux).Assembly.GetName().Name, typeof(HelloWorldAux).Name) as HelloWorldAux;
15 remote.HelloWorld();
16 return;
17 }
18 }
19
20 public class HelloWorldAux : MarshalByRefObject
21 {
22 public void HelloWorld()
23 {
24 AppDomainManager currentManager = AppDomain.CurrentDomain.DomainManager;
25 if (currentManager == null)
26 Console.WriteLine("No Managed AppDomainManager");
27 else
28 Console.WriteLine("AppDomainManager: " + currentManager.GetType().Name);
29
30 Console.WriteLine("Hola, mundo");
31 }
32 }
33
Figure 6.
Just as before, we must create a class in anticipation of launching it in a new AppDomain. In this case, the class is called HelloWorldAux.
Before we can test our application, we must find some way of telling the CLR to use our AppDomainManager. Ultimately, we will want to program our unmanaged host to specify the manager and obtain a reference to it. Alternatively, we could change entries in the Windows system registry so that the CLR will always load our custom manager. Obviously, this is not the wisesst choice for the early stages of testing. The last alternative is to set environment variables in a command console. Any CLR loaded up by a process launched from that window will use our custom AppDomainManager. Others will not.
A batch file to set the environment variables is included in the zipped example files. If you prefer typing it yourself, it looks like this:
SET APPDOMAIN_MANAGER_ASM=AppDomainManager02,Version=1.0.0.0,Culture=neutral, PublicKeyToken=931530800db9d27a,processorArchitecture=MSIL
SET APPDOMAIN_MANAGER_TYPE=AppDomainManager02.AppDomainManagerTest
Obviously, each of the SET commands must be on a single line. The value for the public key presumes that you use the strong key file included with the project files. If you create your own key file, you will need to substitute your new public key.
By default, the CLR will not probe for the AppDomainManager assembly. This means that there are only two choices. The assembly must be located in the same folder as the exe file, or the assembly must be placed into the GAC.
After you copy HelloWorld.exe and AppDomainManager02.dll into the desired folder, we are ready to test. Open a command prompt window and run the SET statements above or run the envvars.bat batch file. Then type "HelloWorld" and hit <enter> like you normally would. You should see something like this:
Figure 7.
Now we need to understand exactly what we are seeing. After Windows recognizes HelloWorld for a managed executable, it loads the CLR, which creates a default AppDomain. The CLR then creates an instance of our AppDomainManager for that new AppDomain, and we see the constructor in our object report its execution in red. The CLR then makes several requests for HostSecurityManager objects; we see these in yellow. The CLR then loads and runs HelloWorld.exe. We see the execution of Main(). Main() the reports the AppDomainManager for its AppDomain, the default AppDomain.
The code in Main() then requests the creation of a new AppDomain by calling the CreateDomain() method of the AppDomain class. Note that Main() is not calling our AppDomainManager. The CLR calls the CreateDomain method of the AppDomainManager in the default AppDomain, giving it a chance to specify the characteristics of the new AppDomain about to be created. Indeed, as mentioned our custom AppDomainManager might even decide not to grant the request for a new AppDomain, and return a reference to an appropriate existing AppDomain instead. The CreateDomain() method reports its execution in green in the console. Only after this, if our managed code requests it, does the CLR create a new AppDomain and load into it a new instance of our AppDomainManager class. We see the instantiation of this class in the red report in the console. Once again, the CLR requests HostSecurityManager objects. At this point, the new AppDomain is ready for use and the HelloWorld() method executes in the new AppDomain. It is worth nothing that the HelloWorldAux class must be loaded in both AppDomains, or there would be no way for the default AppDomain to create a proxy to run the code in the second domain. We may ultimately wish to consider alternatives to this duplication.
Congratulations on getting this code running. In the next installment, we will add some security features, making a real custom AppDomainManager, and integrate this into our unmanged host application.
Source Code
The source code for all examples can be found here.
About Dan Buskirk:
Dan Buskirk had been a research scientist for many years when he left the university labs to participate in a startup venture to design and build databases for medical science. Since that time he has managed a consulting practice specializing in database design and development. Dan balances this with training on Microsoft SQL Server, Microsoft Analysis Services, and .NET programming. His interests include mathematical methods for data mining and computing methods for advanced data categorization on Windows clusters.
Want to learn more about .NET Training?