lyon-smith.org

C++ placement new and delete

john — Sun, 17 Feb 2008 00:44:13 GMT

I was trying to write some debug memory allocation routines the other day and I decided I would attempt to use placement new. I hunted down some useful references:

If you are not familiar with the syntax it goes like this:

extern void* ::operator new(size_t size, char* pLine, int line);
...
char* p = ::new(__FILE__, __LINE__) char[100];

The first line shows how to declare an override of the new operator with two placement arguments, in this case a char* and int which would allow you to record the location of the allocation.

The logical extension to this mechanism is to actually create a placement override that passes a function pointer or a class instance to use for the memory allocation, like this:

extern void* ::operator new(size_t size, Allocator* pAllocator);
...
char* p = ::new(pAllocator) char[100];

Seems all well and good, but how do we delete memory allocated using placement new? There must be a placement delete, right? Well, there is but as with most things in the committee designed C++ language it doesn't work the way you might expect.

You can indeed declare a placement delete operator overload:

extern void ::operator delete(void* p, Allocator* pAllocator);
...
::delete(pAllocator) p;  // This doesn't compile!
p->~ClassName(); pAllocator->Delete(p);

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }

The ::delete line doesn't compile. The fact is, you can't explicitly call placement delete, making it effectively useless in my opinion for many uses. You must explicitly call any class destructor and your allocator's Delete or Free method.

So why use is it to create placement new override? Well, it turns out that if you classes constructor throws an exception then the appropriate placement delete override will get called. How nutty is that! The compiler knows enough to use the placement delete properly, but there's not C++ syntax to expose it for general use!

The need to explicitly call destructors and clean-up memory is particularly annoying when allocating arrays of objects. You must loop through all your objects calling destructors, etc.. Good luck creating a macro or a template to simplify things.

In my opinion, creating placement delete syntax is entirely feasible for compiler writers. The compiler knows all about the types of the classes involved, whether they are arrays, etc.. Hopefully this idea will work it's way through a future C++ committee meeting.

Correct Comparisons for files and paths in .NET

john — Fri, 01 Feb 2008 20:36:23 GMT

So exactly how should you compare file and path names in .NET programs? Ordinal case insensitive comparisons are the correct approach according to this MSDN article.

CPU-Z and CPU Information

john — Thu, 17 Jan 2008 19:06:33 GMT

My colleague Sergey just showed me a useful tool for getting CPU information on your Windows hardware system called CPU-Z.

Personal Rapid Transit

john — Mon, 07 Jan 2008 17:34:00 GMT

This Personal Rapid Transit (PRT) proposal is still one of the coolest ideas for improving the commuting situation that I have seen. Microsoft has a really large corporate campus with buildings spread out over many square miles. Going to meetings in another building today involves driving yourself, taking a campus bus or walking. Given the choice I'd walk rain or shine, but often you don't have enough time to leave the extra 10 - 20 minutes at each end of the meeting. And some buildings are too far away to walk too.

In any case, I'd like to see a system like this adopted for an even wider audience than just one company. Wouldn't it be great to be able to get on one of these at the end of your road and shoot all the way to work or into town. It would beat the pants of today's grubby and inconvenient mass transit systems.

Dynamic Code Generation in .NET

john — Sat, 05 Jan 2008 23:08:00 GMT

I recently had the need/desire to dynamically generate some code in .NET. It's not something you tend to do a lot unless you are creating a compiler, so it was an interesting experience. In the end it didn't solve my problem, but it was educational none-the-less.

First I'll describe the scenario of the problem I was trying to solve using dynamically generated code, then I'll discuss how I went about generating the code and problems that hit and so on.

The problem has to do with assembly resolution in .NET. I wanted to launch a program B in a separate appdomain from program A. Program B is in a completely different directory than A. Program B ultimately wants to load assemblies that are not in the GAC and are not in the same directory as it or in any of it's sub-directories. One way around this problem is to play games with probing paths and .config files. That works but it doesn't work if the assemblies are on a different drive. What I decided I wanted to do was to inject an AssemblyResolve handler into program B and provide a search path from program A. I could have done this by writing the AssemblyResolve handler in C#, creating an assembly and then loading that assembly into program B and wiring up the handler. It would be cleaner (but a more work) to dynamically generate the assembly as needed. For entertainment value I went with the latter plan.

First, lets set up some test programs so we can establish the scenario. We need a test assembly that's just going to be the fodder for our experiment:

using System;

public class TestClass
{
    public void TestMethod()
    {
        Console.WriteLine("In TestClass.TestMethod");
    }
}

Now program B, TestLoader, which is basically going to take an assembly name on the command line, attempt to load it and create an instance of TestClass and call TestMethod.

using System;
using System.IO;
using System.Reflection;

public class Program
{
#if ASSEMBLY_RESOLVE
    public static Assembly OnAssemblyResolve(object sender, ResolveEventArgs args)
    {
        Console.WriteLine("Resolving assembly ");
        Console.WriteLine(args.Name);
        
        string[] searchDirs = ((string)AppDomain.CurrentDomain.GetData("SEARCHPATH")).Split(new char[] { ';' }, StringSplitOptions.RemoveEmptyEntries);
        int index = args.Name.IndexOf(',');
        
        if (index == -1)
            index = args.Name.Length;
        
        string name = args.Name.Substring(0, index);

        for (int i = 0; i < searchDirs.Length; i++)
        {
            string assemblyPath = Path.Combine(searchDirs[i], name) + ".dll";
            
            if (File.Exists(assemblyPath))
                return Assembly.LoadFrom(assemblyPath);
        }

        return null;
    }
#endif

    public static int Main(string[] args)
    {
#if ASSEMBLY_RESOLVE
        if (args.Length != 2)
        {
            Console.WriteLine("TestLoader <assembly-name> <path>");
            return 1;
        }

        AppDomain.CurrentDomain.SetData("SEARCHPATH", args[1]);
        AppDomain.CurrentDomain.AssemblyResolve += OnAssemblyResolve;
#else
        if (args.Length != 1)
        {
            Console.WriteLine("TestLoader <assembly-name>");
            return 1;
        }
#endif

        try
        {
            Assembly assembly = Assembly.Load(args[0]);
            Type type = assembly.GetType("TestClass");
            MethodInfo methodInfo = type.GetMethod("TestMethod");
            object obj = Activator.CreateInstance(type);
            
            methodInfo.Invoke(obj, null);
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
        }
        
        return 0;
    }
}

So what's going on here then? Firstly, ignore the code inside the ASSEMBLY_RESOLVE macro. What this code does without that is to try to load the assembly name given on the path. It's expecting a full or partially qualified strong name, not a file path.

Compile the test assembly code into a .dll in one directory call TestAssembly. Then compile program B in another directory call TestLoader as a .exe. Run it specifying TestAssembly as an argument. Of course the Load() generates an exception. Let's assume we decide to solve this problem by adding an AssemblyResolve handler. Recompile program B with /d:ASSEMBLY_RESOLVE defined. Now run it specifying TestAssembly ..\TestAssembly as arguments. You can see by examining the code and looking at the output how the AssemblyResolve event handler saves the day when the CLR cannot find the assembly using its normal probing locations (GAC, local directory, special sub-directories). Before we continue, recompile program B without ASSEMBLY_RESOLVE defined.

Now we move on to program A, TestLoaderWrapper, the wrapper that will dynamically "inject" the assembly resolve event handler. Imagine that TestLoader was some code that you did not have the source code for. We're going to try change it's assembly loading behavior externally.

Here's the code for TestLoaderWrapper. It's a bit scary:

using System;
using System.IO;
using System.Reflection;
using System.Reflection.Emit;

public class Program
{
    public static int Main(string[] args)
    {
        if (args.Length != 2)
        {
            Console.WriteLine("LoadTestWrapper <assembly-name> <search-path>");
            return 1;
        }
    
        AppDomain appDomain = AppDomain.CreateDomain("NewDomain");
        
        // Set the search path in the new domain using domain data
        appDomain.SetData("SEARCHPATH", args[1]);

        // Dynamically create an assembly containing our assembly resolver
        AssemblyName assemblyName = new AssemblyName();

        assemblyName.Name = "AssemblyResolver";

        string assemblyFile = assemblyName.Name + ".dll";
        AssemblyBuilder assemblyBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.RunAndSave);
        ModuleBuilder moduleBuilder = assemblyBuilder.DefineDynamicModule(assemblyName.Name, assemblyFile);
        TypeBuilder typeBuilder = moduleBuilder.DefineType("Handler", TypeAttributes.Class | TypeAttributes.Public);
        MethodBuilder methodBuilder = typeBuilder.DefineMethod("OnAssemblyResolve",
            MethodAttributes.Public | MethodAttributes.Static,
            typeof(Assembly), new Type[] { typeof(object), typeof(System.ResolveEventArgs) });

        GenerateAssemblyResolveMethod(methodBuilder.GetILGenerator());

        Type handlerType = typeBuilder.CreateType();
        MethodInfo onAssemblyResolve = handlerType.GetMethod("OnAssemblyResolve");

        // You cannot specify a path in the saved assembly, so we must copy it next to the assembly
        // we will be running so that the CLR loader can find it.
        string targetAssemblyPath = Path.Combine(@"..\TestLoader", assemblyFile);

        assemblyBuilder.Save(assemblyFile);

        File.Copy(assemblyFile, targetAssemblyPath, true);

        Delegate handlerDelegate = Delegate.CreateDelegate(typeof(ResolveEventHandler), onAssemblyResolve);
        EventInfo assemblyResolveEvent = typeof(AppDomain).GetEvent("AssemblyResolve");

        assemblyResolveEvent.AddEventHandler(appDomain, handlerDelegate);
        
        appDomain.ExecuteAssembly(@"..\TestLoader\TestLoader.exe", AppDomain.CurrentDomain.Evidence, 
            new string[] { args[0] });

        return 0;
    }

    public static void GenerateAssemblyResolveMethod(ILGenerator il)
    {
        Type consoleType = typeof(System.Console);
        MethodInfo write = consoleType.GetMethod("Write", BindingFlags.Public | BindingFlags.Static, null, new Type[] {typeof(string)}, null);
        MethodInfo writeLine = consoleType.GetMethod("WriteLine", BindingFlags.Public | BindingFlags.Static, null, new Type[] {typeof(string)}, null);
        Type appDomainType = typeof(System.AppDomain);
        MethodInfo getCurrentDomain = appDomainType.GetMethod("get_CurrentDomain");
        MethodInfo getData = appDomainType.GetMethod("GetData");
        Type assemblyType = typeof(System.Reflection.Assembly);
        MethodInfo loadFrom = assemblyType.GetMethod("LoadFrom", new Type[] { typeof(string) });
        Type resolveEventArgs = typeof(System.ResolveEventArgs);
        MethodInfo getName = resolveEventArgs.GetMethod("get_Name");
        Type stringType = typeof(string);
        MethodInfo splitMethod = stringType.GetMethod("Split", new Type[] { typeof(char[]), typeof(StringSplitOptions) });
        MethodInfo substring = stringType.GetMethod("Substring", new Type[] { typeof(int), typeof(int) });
        MethodInfo indexOf = stringType.GetMethod("IndexOf", new Type[] { typeof(char) });
        MethodInfo concat = stringType.GetMethod("Concat", new Type[] { typeof(string), typeof(string) });
        MethodInfo getLength = stringType.GetMethod("get_Length");
        Type pathType = typeof(System.IO.Path);
        MethodInfo combine = pathType.GetMethod("Combine", BindingFlags.Public | BindingFlags.Static);
        Type fileType = typeof(System.IO.File);
        MethodInfo exists = fileType.GetMethod("Exists", BindingFlags.Public | BindingFlags.Static);

        // Console.Write("Resolving assembly ");
        il.Emit(OpCodes.Ldstr, "Resolving assembly ");      // -> T
        il.EmitCall(OpCodes.Call, write, null);             // ->

        // Console.WriteLine(args.Name);
        il.Emit(OpCodes.Ldarg, 1);                          // -> T
        il.EmitCall(OpCodes.Callvirt, getName, null);       // -> T
        il.EmitCall(OpCodes.Call, writeLine, null);         // -> 

        // string[] searchDirs = ((string)AppDomain.CurrentDomain.GetData("SEARCHPATH")).Split(new char[] {';'}, StringSplitOptions.RemoveEmptyEntries);
        LocalBuilder searchDirs = il.DeclareLocal(typeof(string[]));

        il.EmitCall(OpCodes.Call, getCurrentDomain, null);  // -> T
        il.Emit(OpCodes.Ldstr, "SEARCHPATH");               // -> T, T
        il.EmitCall(OpCodes.Callvirt, getData, null);       // -> T
        il.Emit(OpCodes.Castclass, typeof(String));         // -> T
        il.Emit(OpCodes.Ldc_I4, 1);                         // -> T, I4
        il.Emit(OpCodes.Newarr, typeof(Char));              // -> T
        il.Emit(OpCodes.Dup);                               // -> T, T
        il.Emit(OpCodes.Ldc_I4, 0);                         // -> T, T, I4
        il.Emit(OpCodes.Ldc_I4, 0x3b);                      // -> T, T, I4, I4
        il.Emit(OpCodes.Stelem_I2);                         // -> T
        il.Emit(OpCodes.Ldc_I4, 1);                         // -> T, I4
        il.EmitCall(OpCodes.Callvirt, splitMethod, null);   // -> T
        il.Emit(OpCodes.Stloc, searchDirs);                 // ->

        // int index = args.Name.IndexOf(',');
        LocalBuilder index = il.DeclareLocal(typeof(int));
        
        il.Emit(OpCodes.Ldarg, 1);                          // -> T
        il.EmitCall(OpCodes.Callvirt, getName, null);       // -> T
        il.Emit(OpCodes.Ldc_I4, ',');                       // -> T, T
        il.EmitCall(OpCodes.Callvirt, indexOf, null);       // -> T
        il.Emit(OpCodes.Stloc, index);                      // -> 

        // if (index == -1)
        Label indexThen = il.DefineLabel();
        
        il.Emit(OpCodes.Ldloc, index);                      // -> T
        il.Emit(OpCodes.Ldc_I4_M1);                         // -> T
        il.Emit(OpCodes.Ceq);                               // -> T
        il.Emit(OpCodes.Brfalse, indexThen);                // ->

        // index = args.Name.Length;
        il.Emit(OpCodes.Ldarg, 1);                          // -> T
        il.EmitCall(OpCodes.Callvirt, getName, null);       // -> T
        il.EmitCall(OpCodes.Callvirt, getLength, null);     // -> T
        il.Emit(OpCodes.Stloc, index);                      // ->

        il.MarkLabel(indexThen);

        // string name = args.Name.Substring(0, index);
        LocalBuilder name = il.DeclareLocal(typeof(string));

        il.Emit(OpCodes.Ldarg, 1);                          // -> T
        il.EmitCall(OpCodes.Callvirt, getName, null);       // -> T
        il.Emit(OpCodes.Ldc_I4, 0);                         // -> T, I4
        il.Emit(OpCodes.Ldloc, index);                      // -> T, I4, T
        il.EmitCall(OpCodes.Callvirt, substring, null);     // -> T
        il.Emit(OpCodes.Stloc, name);                       // ->

        // for (int i = 0; ...) ...
        LocalBuilder i = il.DeclareLocal(typeof(Int32));
        Label loopStart = il.DefineLabel();
        Label loopEnd = il.DefineLabel();

        il.Emit(OpCodes.Ldc_I4, 0);                         // -> I4
        il.Emit(OpCodes.Stloc, i);                          // ->
        il.Emit(OpCodes.Ldloc, i);                          // -> I4
        il.Emit(OpCodes.Br, loopEnd);                       // ->

        // string assemblyPath = Path.Combine(searchDirs[i], name) + ".dll";
        LocalBuilder assemblyPath = il.DeclareLocal(typeof(string));
        Label ifEnd = il.DefineLabel();

        il.MarkLabel(loopStart);
        il.Emit(OpCodes.Ldloc, searchDirs);                 // -> T
        il.Emit(OpCodes.Ldloc, i);                          // -> T, I4
        il.Emit(OpCodes.Ldelem_Ref);                        // -> T
        il.Emit(OpCodes.Ldloc, name);                       // -> T, T
        il.EmitCall(OpCodes.Call, combine, null);           // -> T 
        il.Emit(OpCodes.Ldstr, ".dll");                     // -> T, T
        il.EmitCall(OpCodes.Callvirt, concat, null);        // -> T
        il.Emit(OpCodes.Stloc, assemblyPath);               // ->

        // if (File.Exists(assemblyPath))
        il.Emit(OpCodes.Ldloc, assemblyPath);               // -> T
        il.EmitCall(OpCodes.Call, exists, null);            // -> B
        il.Emit(OpCodes.Brfalse, ifEnd);                    // ->

        //    return Assembly.LoadFrom(assemblyPath);
        il.Emit(OpCodes.Ldloc, assemblyPath);               // -> T
        il.EmitCall(OpCodes.Callvirt, loadFrom, null);      // -> T
        il.Emit(OpCodes.Ret);                               // -> 

        // for (...; i < searchDirs.Length; i++)
        il.MarkLabel(ifEnd);
        il.Emit(OpCodes.Ldloc, i);                          // -> I4
        il.Emit(OpCodes.Ldc_I4, 1);                         // -> I4, I4
        il.Emit(OpCodes.Add);                               // -> I4
        il.Emit(OpCodes.Stloc, i);                          // -> 
        il.Emit(OpCodes.Ldloc, i);                          // -> I4
        il.MarkLabel(loopEnd);
        il.Emit(OpCodes.Ldloc, searchDirs);                 // -> I4, T
        il.Emit(OpCodes.Ldlen);                             // -> I4, I
        il.Emit(OpCodes.Conv_I4);                           // -> I4, I4
        il.Emit(OpCodes.Clt);                               // -> B
        il.Emit(OpCodes.Brtrue, loopStart);                 // ->

        // return null;
        il.Emit(OpCodes.Ldnull);                            // -> T
        il.Emit(OpCodes.Ret);                               // ->
    }
}

Yikes! That almost 200 lines of code! What this snippet shows you is that dynamic code generation in .NET is not cheap. Let's touch on the key points.

Stripping away the dynamic assembly generation, what this code wants to do is create a new appdomain and execute TestLoader in it Remember this is the test loader that would normally fail to load the test assembly.

We are going to take the test assembly and the search path in to TestLoaderWrapper and pass them to TestLoader. The test assembly is a command line argument, but the search path is passed through appdomain data, so we set that up with a call to SetData.

Now the heavy lifting. We start by creating an AssemblyBuilder, ModuleBuilder, TypeBuilder, MethodBuilder family of objects. It's just a lot of paying attention to parameters.

Then we call GenerateAssemblyResolveMethod to build the IL for the method. I won't bore you with a tedious breakdown; the ILGenerator methods are all very self explanatory and easy to follow. I do have some tips to share on how I went about generating the code and typing it in.

Firstly, .NET Reflector is your friend. The IL disassembly is a great way to get started with the correct IL. Simply open up the version of TestLoader that has ASSEMBLY_RESOLVE defined and start cribbing.

What you are going to find really quickly however is that the C#, VB .NET or other compiler usually generate very verbose code, that seemingly takes a lot more IL instructions than necessary to get the job done. What you have to remember is that IL is not meant to be executed directly, it's meant to be compiled (also known as the jitter). The jitter will usually do a good job of removing much of the IL redundancy. Actually, one technique for writing a jitter is to look for standard patterns of IL which can be optimized into native code sequence.

The other thing that the extra instructions help with is debugging, in particular the IL instruction boundaries where the IL stack hits zero entries are called sequence points and they typically map to a logical unit of source code that you can stop at in the debugger. You can safely examine program variables at that point because, for example, the program will not be halfway through evaluating a source code expression.

All this is irrelevant to us though because unless you do some extra work to generate .pdb information for your dynamic assembly you want be able to see much in the debugger. You can however insert a Break opcode into it, turn off Just My Code in VS and look at the native code that is generated for the method.

Getting back to the IL generation. I took the .NET Reflector output as a starting point and began munging it into maintainable code. As you can see grabbing methods and types to pass to IL instructions is just a question of calling Reflection API's. Notice that I comment the current stack after each instruction executes. T is a metadata token, I4 means a 4 byte integer. I break the IL generation down in to chunks where the IL stack returns to zero entries, for the reasons explained above and to help keep things readable.

What would really be awesome is if someone would write a .NET Reflector plug-in that would automatically generate Reflection Emit code for a method.

Almost there now. We can specify a different directory for the generated assembly. Don't know why, it just doesn't work, so we copy it next to the TestLoader executable. Why do we need to do this? Because the appbase (the root directory where the CLR will probe for assemblies) is the directory of the entry assembly for the appdomain, which is different by default for TestLoader than it is for TestLoaderWrapper because they are in different directories.

Then we create a delegate for the assembly resolve handler and attach it to the new AppDomain object and we are done.

I'll attach the source code for all this to the post for those who may stumble this way in search of help with this topic.

More .NET Custom Configuration Madness

john — Mon, 10 Dec 2007 00:46:00 GMT

I have to blog about this before I do anything else. I just found out that if you specify a configSection in your .exe.config file it must come as the first element in the configuration section! I had done this:

<?xml version="1.0" encoding="utf-8" ?> 
<configuration>
  <startup>
    <supportedRuntime version="v2.0.50727"/>
  </startup>
  <configSections>
    <section name="csr" type="Microsoft.Tools.CodeRunner.Tasks.CsrSection, Microsoft.Tools.CodeRunner, Version=2.0.0.0, Culture=neutral, PublicKeyToken=20af5850399c8643" />
  </configSections>
  <csr>
    <settings>
...
    </settings>
  </csr>
</configuration>

Calling Configuration.GetSection("csr") was resulting in a DefaultSection type an not a CsrSection type. Moving the startup section after the configSection fixed the problem.

That's 5 stars on the suck scale in my opinion.

The Smarter Emergency Kit

john — Tue, 30 Oct 2007 20:47:42 GMT

Wired Magazine did an article on emergency readiness. I could not for the life of me find it on their web site, so here's what they recommend:

The House Kit

Sealed in plastic trash can or chest. For use in catastrophic situations, like earthquake, fire, etc.. You should probably be able to get it into the back of your car.

A. First aid kit. Bandages, alcohol, cotton balls, antidiarrheals, anti-inflammatories (ibuprofen), prescription meds.

B. Clothes. Full change, including warm outer layers (wool or synthetic) and sturdy shoes for each person under your roof.

C. Plastic Sheeting. Fiber reinforced, laminated Poly. film, 6mm.

D. Zip Ties.

E. Water. Gallon per person for 3 days. 16 drops unscented bleach makes a gallon potable.

F. Food. Dehydrated soups, canned tuna and veggies, nuts and candy.

G. Flashlight. LED like Inova X5.

H. Protective wear. Waterproof, cut resistant Kevlar gloves and N95 face masks.

I. Tools. Crowbar, adjustable wrench, screwdrivers, staple gun, rope.

J. Matches. Waterproof, windproof in ziplock bag with dry tinder.

The Go Pack

Keep in backpack ready to go. For use when you need to leave at a moments notice.

K. Radio & cell phones. Cheap tranny is fine, but Eton Grundig FR300 is even better.

L. Cash. $500 in small bills.

M. Documents. Home insurance, contact numbers, medical insurance card, passport, bank account numbers, photos, local map, spare keys.

N. Mylar space blanket.

O. Clothes, etc.. Hat, socks, toiletries.

P. Food & water. Sports and nutrition bars, nuts trail, mix & water.

Q. Essential medicines. Sunblock, ibuprofen, contact lens kit.

R. Duct tape.

S. Signal device. Greatland Laser's Rescue Laser and Fox 40 whistle.

T. Multitool. Leatherman with can opener, knife blade, screwdrivers.

Top 25 Greatest Science Books of All Time

john — Tue, 30 Oct 2007 16:57:16 GMT

In December 2006 Discover Magazine did an article on the 25 greatest science books of all time. You can find them here. The list was interesting, but let's be honest how many of us have actually read anything by Charles Darwin? The reader's response was rather weak too. Personally I've only read about half a dozen of the books on the list. Perhaps there's an opportunity for someone to write a synopsis of all the books in one single easily digestible paperback.

I wrote this entry because I've been carrying around the torn out pages of the article in my notebook since last December. How sad is that.

Arrays in PowerShell

john — Tue, 23 Oct 2007 15:39:00 GMT

While I'm waiting for some files to copy around I thought I'd record my recent PowerShell learnin'.

Somtimes a command, like Get-ChildItem can return either an array of results or just a single one. You can use the @ symbol to cast the result into an array, so:

(gci -i *.txt).Count

Sometimes returns nothing, whereas:

@(gci -i *.txt).Count

Always returns the count of *.txt files.

Second thing. The .. is the syntax for ranges. So 1..10 is expanded into an array of the numbers 1 thru 10. Turns out you can also use arrays as indexes! Each item in the index array is used to index an item in the data array, thus returning a new array.

There's an example in the help that is confusing at first sight:

$a[0..-2]

Returns the first, last and second to last elements in the array in that order. Why? Because the range is expanded into a list, equivalent to:

$a[-2,-1,0]

Make sense?

If you write blogs with code in and you are using LiveWriter grab the Insert Code for Windows Live Writer plug-in. It's got some flaws, but it's a huge time saver.

Line Count with PowerShell

john — Tue, 23 Oct 2007 03:35:00 GMT

Starting to get the hang of PowerShell. Today I wanted to quickly count the number of lines in some source code. There's a simple implementation and then there's a better one. In the end I settled on:

$count=0;gci . -i *.cs,*.cpp,*.h -r | %{$count += [System.IO.File]::ReadAllLines($_.FullName).Count};$count

Which is quick to type and fast enough for me, although as the links show you can butter it up and put it in a .ps1 file if you want too.

Puzzle Time!

Here's a little PowerShell test that I have to admit I had to get some help with. Why does:

1,2,3 | %{$_}

Produce the output:

1
2
3

But this produces nothing:

$code={%{$_}}
1,2,3 | &$code

Remember, & executes a script block and % is just an alias for Foreach-Object.

Want the answer?

It's a question of scope. In the second example &$code is equivalent to &{%{$_}}. The extra {}'s puts $_ in a different script variable scope, one in which it has not been initialized. To get things working again, you need to reattach it to the pipeline input, like so:

$code={$input | %{$_}}
1,2,3 | &$code

More PowerShell Fun

john — Wed, 17 Oct 2007 18:33:00 GMT

Still struggling to gain traction with PowerShell. I've stopped myself running the CMD prompt by default, and I've installed it on all my test VPC machines.

Firstly, I recommend you download the Windows PowerShell Graphical Help File. It's an old school .chm file, but you can search it. Don't underestimate how useful this is when learning new stuff.

Also, I totally hate the magic behind env: prefixed variables. They are basically a secret back door to the processes environment variables. It seems that the the processes environment variables are mirrored into variables starting with $env: So $env:path gets you the PATH and $env:pathext gets you PATHEXT. This is not documented anywhere that I can find. What's important is that these environment variables are still important in many cases. So $env.path is still the search path for programs, and $env.pathext is still appended to any program you type without an extension.

I grabbed a copy of Windows PowerShell Unleashed from the Microsoft library. It's not a bad introduction. I don't think it's much of a long term reference book though. There are too many important details missing (like the one above).

Changing the Desktop Wallpaper Programmatically on Windows

john — Fri, 12 Oct 2007 20:39:23 GMT

My friend Scott asked me the other day how he might go about changing the desktop wallpaper programmatically. It's one of those things that is still buried in the native Windows Win32 API. The call you need is SystemParametersInfo and the action you need is SPI_SETDESKWALLPAPER.

Scott pointed out that this only takes a bitmap. On Windows XP, if you set the desktop wallpaper to a JPEG under Control Panel it actually converts it to a bitmap and then makes this call. With Windows Vista, the JPEG can be set directly.

The Singleton Pattern in .NET and C#

john — Thu, 11 Oct 2007 16:18:18 GMT

I just got bitten by a static initialization problem - again. There are two things you've got to be careful of in .NET when using statics (i) threading issues and (ii) initialization ordering issues. I'm not going to go into a great long discussion about the issues as it's been covered really well in other places. The rules are simple, but there are a couple of tricky wrinkles, and it's one of those areas that you don't code to all the time, so by the time you need to know the "really important thing" you've forgotten it. Anyway, I found Implementing the Singleton Pattern in C# to be the most helpful memory jogger. Bart de Smet's blog shows how to do deterministic initialization order by using the RuntimeHelpers.RunClassConstructor method.

Changing the Windows Log On Keyboard Layout to Dvorak

john — Thu, 04 Oct 2007 01:15:23 GMT

If you found this entry, you are either a Dvorak keyboard typist searching for help or you are a strange individual who reads my blog but never leaves a comment.

If you are the former then the link you are looking for is to this Microsoft Help & Support Article stupidly entitled Unable to Log On after Changing Keyboard Layout which will tell you how to change the default users logon keyboard layout to Dvorak.

Halo 3 Videos

john — Wed, 03 Oct 2007 23:13:18 GMT

Amusing Halo 3 videos are starting to permeate the web. If you are not a video gamer read no further. If you are and you aren't playing Halo 3 ... ARE YOU CRAZY! Go buy a copy. A cool feature is that you can take video of a game and publish export it to the web. Check out My WiLL Be Done for an amazing lucky shot. It almost looks like a homing missile.

Adventures with PowerShell

john — Wed, 26 Sep 2007 21:44:54 GMT

I've held off moving to Windows PowerShell. The reason is that I came up with a solution for running C# source directly from the command line as script a few years ago using CodeRunner .NET, and I've haven't had a pressing need for more functionality.

But I decided that the time has come to say goodbye to the DOS prompt. I'm going to bring the CodeRunner along too. Perhaps as a Cmdlet. This blog is a collection of useful links that I have found so far.

First stop. The official PowerShell web site. All the links to the latest downloads, etc..

The PowerShell Owners Manual looks promising, but not yet deliver the goods.

This Channel 9 Wiki has a great PowerShell quick start page.

First thing I wanted to do was change the PowerShell prompt, and I couldn't figure how.

And of course, last but not least the MSDN PowerShell documentation.

Dr. Who Theme Music

john — Tue, 04 Sep 2007 22:17:57 GMT

I'm a big fan of Dr. Who. I'm not going to apologize. It's campy British sci-fi, with low budgets, cheesy plots and terrible acting, but I love it. The current series, with David Tennant, is the best ever. In my area it's being shown on CBC. Catch it if you can.

I stumbled across this YouTube vid on a re-making of the classic Dr. Who theme tune back in the 80's. This was a time, as my friend Lucien informed me the other day, when there was no digital music generation. Everything was analog, and being able to tweak the dials and sliders well was as much playing the instrument as hitting the keyboard.

What I did on my vacation (part 1)

john — Tue, 04 Sep 2007 18:04:00 GMT

I just got back from my summer vacation. We spent a week in Puerto Rico (more on that in another blog) and a week at home. I always like to have a project that has nothing to do with computers, so I built a garden shed. I didn't use a plan or kit or anything, just scribbled a design on a sheet of paper, bought some wood and went for it. I was really pleased with how it came out. This was my first experience of putting gutters up, and it wasn't hard at all. What was hard was building the shed up against the house. I had to build the frame and the back wall with it pulled out from the house, and the push it back in before putting the roof on and the whole thing got to heavy. It's going to fun when it comes time to paint the house! The gutter was a safety precaution because I didn't want the water from the roof pooling next to the foundation of the house. I bought all the materials from Home Depot, and looking at my receipts it looks like I spent about $400 to build it. That's certainly cheaper than the $1000+ shed kits that you can buy, plus I got exactly the shed that I wanted.

The Fastest Stupid Car Ever

john — Sat, 01 Sep 2007 04:52:29 GMT

Sorry, I'm on a bit of a Top Gear binge at the moment. Watch the fastest stupid car ever - I laughed so hard I was crying.

Second Largest Underground Car Park

john — Sat, 11 Aug 2007 20:48:39 GMT

My company (Microsoft) is currently digging a really large hole just on the other side of the freeway from my office. It's reportedly going to be the second largest underground car park in the western hemisphere. Here's a recent TV spot about it. Parking can be a really big problem around Microsoft. Kudos to the company for spending the huge amount of money to do this just so that employees can have some open space to hang out in.

3 Really Great Top Gear Episodes

john — Mon, 06 Aug 2007 19:37:36 GMT

I watched three brilliant Top Gear episodes this weekend on You Tube.

The first was a review of the Ariel Atom. Watch Jeremy's face deform under the sub-three second acceleration. His comments about middle aged men on motorcycles cracked me up.

The second was the review of the Ferrari F430. What a beautiful piece of machinery.

Finally there was the face off between a Mitsubishi Evo VIII and a Lamborghini Mercielago. That one is for Rich.

Team System Web Access Power Tool Announced

john — Mon, 30 Jul 2007 23:05:43 GMT

OK, in my opinion this is a bit more than a power tool - it's an entire product. If you are using Team System you need to download and set up the Team System Web Access Power Toy that we just released. Team System Web Access (formerly known as TeamPlain) is a Web interface to Visual Studio 2005 Team Foundation Server. We are using it internally, and it's really sweet. Read Brian Harry's comments about it.

Getting Windows crash dumps

john — Fri, 27 Jul 2007 22:01:10 GMT

Every so often I need get a crash dump for a crashing native process, and I always forget where to find the tools. The easiest to use is ADPlus which is part of the Debugging Tools for Windows. The best write-up I've found on other alternatives and how to take crash dumps is from the ASKPERF blog.

Chris Sharma Rules!

john — Wed, 25 Jul 2007 03:38:43 GMT

Chris Sharma is certainly an amazing rock climber. He isn't massive or particularly graceful, but he has amazing determination and arm strength. Here is a selection of my favorite videos featuring him on YouTube.

Dreamcatcher. A beautiful traverse line on an overhanging lip with the crux right at the end.
Barcelona Competition Bouldering. Just complete thuggery!
La Rambla. A really bad camera angle, but about halfway through just look at how crappy those holds are!
Access Fund Boulder Project. This reminds me of how Johnny Dawes used to climb if anyone remembers him.

Lion vs. Alligator vs. Buffalo. Buffalo wins.

john — Wed, 25 Jul 2007 02:55:38 GMT

If you haven't already, watch this amazing amateur video on You Tube of a baby water buffalo getting attacked by a lion. The buffalo is then also attacked by an alligator. Amazingly the buffalo herd arrives and fights off both attackers, and the baby buffalo walks away!