.NET's bad relatives: On native dependencies

I have a love-hate relationship with native dependencies. As a developer i love quickly boosting my app with features from native code libraries using e.g. P/Invoke. However, when it comes to deployment we need to make sure that the native dlls will be 1) correctly deployed to the target environment and 2) correctly loaded at runtime.


Native binaries = Specific architecture (x86, x64, ...)

Since native dependencies are built towards specific chipset architectures (x86, x64, ...) you need to include all versions in your distribution. Then at runtime you need to check which version to load. There already are some managed projects doing a good job on this, for example SQLite, GDAL or our own VLC.Native.

Pick architecture at runtime

The most simple solution is to just copy all native dlls right beside your executables directory. This way the AssemblyLoader will automatically pick up the native dlls (For more details see this discussion on MSDN on how Windows locate dlls).

However, this only works if you are targeting the one specific architecture that the native binaries are built for. Otherwise you need to 1) distribute all native builds for x86, x64, ... and then 2) pickup the version macthing the archtecture at runtime.

The most simple way to do this is to prepend the PATH environment variable. For decades, software tools are deployed with 1) installers modyfying the PATH environment variable or 2) helper scripts starting a shell with modified PATH. Examples are GDAL, Ruby, Python or Git to just name a few.

However, we neither want to persistently change the users enviroment possibly messing up some existing applications nor to have him execute a batch file everytime before using our software, right? Instead, we can change the PATH enviroment directly in our app right at startup in the so called bootstrapping phase.

This i a nice way of picking up the right version that i saw first being used by the SharpMap team in their GDAL.Native on NuGet gallery. Recently, we copied this pattern for our VLC.Native package. We are deploying the native VLC dependencies to a common home directory with the x86 and x64 builds in subdirectories. Then at runtime, we check if we are running in 32 or 64 bits to the select the directory. Here is a snippet:

void CheckVlc()
{
    var vlcPath = Path.Combine(_homePath, "vlc");
    var nativePath = Path.Combine(vlcPath, GetPlatform());
    IsVlcPresent = Directory.Exists(nativePath);
    if (!IsVlcPresent)
    {
        Trace.TraceWarning("Cannot find VLC directory.");
        return;
    }

    // Prepend native path to environment path
    var path = Environment.GetEnvironmentVariable("PATH");
    path = string.Concat(nativePath, ";", path);
    Environment.SetEnvironmentVariable("PATH", path);

    //...
}

private static string GetPlatform() 
{ 
    return Environment.Is64BitProcess ? "x64" : "x86"; 
}

Use AppDomain.AppResolve with mixed managed/native dependencies

An more unusual case is depending on managed libraries that require separate builds for x86 and x64. If ever, you will find this mostly on commerical libraries porting some natives libraries to .NET using managed C++ or COM. Google for .NET libraries supporting RichText, PDF, Jpeg2000 or OCR features and you will probably find some examples. Generally, i would not recommend using such libraries, especially if they are commercial unless you have good reasons.

Anyway, in case you are going down that road you need to extend the above bootstrapping code by hooking up into the AppDomain.AppResolve event and select the correct assembly like

    //...
    Environment.SetEnvironmentVariable("PATH", path);

    AppDomain.CurrentDomain.AssemblyResolve += ResolveCustomAssemblies;
}

private static Assembly ResolveCustomAssemblies(object sender, 
  ResolveEventArgs args)
{
    var assemblyName = args.Name.Split(',')[0];
    var assemblyFileName = Path.Combine(LookupPath, assemblyName + ".dll");
    if (File.Exists(assemblyFileName))
    {
       var assembly = Assembly.LoadFrom(assemblyFileName);
       Trace.TraceInformation("Resolved assembly \"{0}\" from \"{1}\"",
         assemblyName, LookupPath);
       return assembly;
    }
    return null;
}

where LookupPath will usually be the path to the respective x86 or x64 subdirectory.

Minimize Footprint by factoring out native dependencies

So after all, distirbuting native binaries along with your app is fine unless you are deplyoing smart clients, i.e. using ClickOnce or Java WebStart. The big advantage of smart clients is auto-updating so they make quick iterations easy but need small footprints to be practical. Since native dependencies tend to make your app a big download this is a killer for smart client deployment.

With smart clients like ClickOnce each update is a full, isolated, sandboxed copy of your app. This makes most deployment taks extremely simple, like auto-updating or rolling back to a specific version. As a consequence, you want to keep the size of your app as small as possible so updates will be quick and smooth.

Fortunately, the native dependencies used in your app usually won't change much often, instead your app will, right? So what you can do is factoring out big constant binaries as a prerequisite. With ClickOnce, a bootstrapper package is a reasonable choice. You can also go funky using a plugin-based approach like NuPlug which aims to download dependencies on demand.

A typical use case might be a .NET client talking to an Oracle database backend so you need an Oracle Client like ODP.NET (which btw you can squeeze down to about 35 MiB). Assuming you already have ODP.NET as a prerequisite you can then take the following class

class PathBootsTrapper : IBootsTrapper
{
  private readonly string _path;

  public PathBootsTrapper(string path)
  {
    if (String.IsNullOrEmpty(path))
      throw new ArgumentNullException("path");
    if (!Directory.Exists(path))
      throw new ArgumentException("Directory does not exist", "path");
    _path = path;
  }

  public void Bootstrap()
  {
    var path = Environment.GetEnvironmentVariable("PATH");
    path = String.Concat(_path, ";", path);
    Environment.SetEnvironmentVariable("PATH", path);
  }
}

internal interface IBootsTrapper
{
  void Bootstrap();
}

and use it like this:

class Program
{
  static void Main(string[] args)
  {
    var settings = new Settings();
    pathToOracleClient = Environment.ExpandEnvironmentVariables(
      settings.PathToOracleClient);
    new PathBootsTrapper(pathToOracleClient).Bootstrap();

    // ...
  }
}

Final words: Go for delta-updates with Squirrel

Since there seems to be no active development on ClickOnce anymore you should look out for alternatives. Recently, Paul C. Betts et al. developed Squirrel.Wndows which supports delta-updates and can be used to deploy non-.NET apps as well. However, at the time of writing, rolling back to a specific version is not supported, But why would you want to? A newer version should always be better, right? With hello.squirrel, i did a quick evaluation on Squirrel and really liked it.