Microsoft's .NET Framework arrived with two languages, C# and Visual Basic. Then came the Iron Languages, .NET compatible implementations of Ruby and Python which Microsoft "let go of" in 2010. Nowadays Microsoft is again embracing Python. Here's an account of Python's still evolving relationship with .NET 

This trip down memory lane was triggered by the recent news of the release of a new version of Python.NET which begged the question of what had become of Iron Python, one of Microsoft's dynamic Iron-languages on the .NET platform.

First of all let's clear up the confusion arising out of having two versions of Python for .NET: Python.NET and IronPython.
While not as well known as IronPython, Python.NET predates it but had seen no updates in recent years and was still on the Python 2.x branch. Recently it emerged from apparent hibernation with the release of Python.NET 3.0.0, revitalizing what many assumed to be a dead project.

So how does Python.NET differ from Iron Python?

Python.NET integrates with the .NET Common Language Runtime (CLR) and provides a powerful application scripting tool for .NET developers. It allows Python code to interact with the CLR, and may also be used to embed Python into a .NET application. Using this package you can script .NET applications or build entire applications in Python, using .NET services and components written in any language that targets the CLR (C#, VB.NET, F#, C++/CLI).

On the other hand IronPython is an implementation of the Python Programming language written for the .NET framework. This makes it a pure managed-code implementation of the Python language which renders IronPython a first-class .NET programming language. IronPython can use .NET and Python libraries, and other .NET languages can use Python code just as easily. IronPython is built on top of the Dynamic Language Runtime (DLR), while the DLR itself is built on top of the Common Language Runtime (CLR) of the NET platform.

As evident, the major difference between IronPython and Python.NET is that the latter does not implement Python as a first-class CLR language, does not implement Python language on top of CLR and does not produce managed code (IL) from Python code. Rather, it integrates CPython engine with the NET runtime that allows the use of the CLR services from Python while still able to use existing Python code together with C-based extensions while maintaining native execution speeds for Python code.

So there's a subtle difference; the libraries written using C for CPython like numpy and pandas are not compatible with IronPython. For this reason, if a Python Script want to run on the .NET platform that also requires use of CPython libraries which are written in C like numpy, then you got to go with Python.NET rather than using IronPython.

So the general rule of what to choose depends on what packages you would want to use; if CPython then go for PythonNET. If, however, you want seamless integration with the .NET environment and language interoperation then IronPython is better, even though it does not yet have the capability of the Python scientific stack, that is interaction with numpy etc.

But IronPython users also have three extra advantages:

Firstly, there's no GIL Global Interpreter Lock.That effectively means that multithreaded programs can take full advantage of multiple processors, something that CPython and subsequently the PythonNET ones can’t do. If you want to know more about how the GIL hobbles Python then refer to Programmer's Python:Async by Mike James - see side panel

Secondly, that it is also easier to extend than CPython as IronPython is written in C# and C# is memory managed as well as C#'s types can be used directly in IronPython with no wrapping.

Thirdly, since IronPython is built on top of the Dynamic Language Runtime (DLR) of the .net platform, and the DLR itself is built on top of the Common Language Runtime (CLR), C#/F#/etc. code can directly invoke IronPython code through .NET's "dynamic" type. All that thanks to the DLR. 

But what is the DLR?

Put simply it is a runtime that sits atop the CLR and hosts dynamic languages. It makes implementing a new language, be it a dynamic, application or domain specific one, much easier to build since you can use ready made parts and leverage existing functionality; for example instead of implementing a GC you plug into the CLR's GC.

Then you can mix and match objects from different languages. For example, calling a IronPython library from C# and vice versa (or even call the .NET graphical library from within IronRuby)   is the groundbreaking issue here, not only connecting dynamic languages with dynamic languages but also dynamic with static ones.

At the heart lie the DLR expressions. These are the common-denominator that ties all languages in the DLR platform. To sum it up in a few lines: each language has its own parser which parses the source code and results into a AST (not optree;abstract-syntax-tree is more flexible).

For example, IronPython produces an AST and IronRuby produces its own AST as well. Both ASTs are then transformed into a DLR AST, which essentially IS the Expressions, so that everybody speaks the same language. The tree itself is composed by nodes that are objects which represent code. The idea is that representing code as a graph instead of bytecode/IL, allows walking it, manipulating and extending it (using the Visitor pattern), semantically transform it and apply optimizations to it. These Expression trees can be directly interpreted by the DLR runtime or be compiled into bytecode/IL for execution by the CLR.

DLR's underlying processes are uncovered in an interview I did back then with Fredrik Holmström in his attempt to port Javascript to .NET in its IronJS alter ego. He gives answers to questions that pertain to this day, like why on earth would someone get into the trouble of implementing a dynamic language on the .NET platform:

FH: I think the main advantage is just the DLR, I was investigating an implementation on top of the JVM at first, this was back in early 2010 if I remember correctly. The DLR gives you a lot of really great stuff for free.

NV: Like language interoperability and what else ?

FH: That is one thing for sure, but also just how solid the DLR code is and how big of a piece of the technology it solves for you (emitting the IL)

NV: DLR has a reputation of making it easy to create a new language implementation. Is that meant in the context of the language's implementer borrowing core facilities out of the box such as the CLR's GC or JIT?

FH: The DLR lets you turn your AST into what they call an Expression Tree, which is a high level, object oriented version, of IL. In reality, the expression trees are syntax trees, but they are compiled to IL by the DLR and while emitting IL isn't a complicated task, it's time-consuming to write code that does it. So, getting that for free, is a big thing.

NV: After emitting IL, is it compiled or interpreted by the DLR at runtime?

FH: Well, that's pretty much one process, you hand the DLR your expression tree, it gives you back a delegate which you can invoke. The delegate itself can either be interpreted by the DLR or compiled by the JIT; it's something you specify when you compile it.

At that time, the first fully fledged DLR incarnations of the popular dynamic languages Python, with IronPython, and Ruby with IronRuby, began to appear on the.NET platform. But that would not last long.Microsoft killed the dynamic languages project in 2010 as we covered in Microsoft's Dynamic Languages Are Dying:

Are the Iron languages dead? Neither IronRuby nor IronPython have exactly set the .NET development scene alight - despite initial enthusiasm. It might be they delivered too little too late.

One of the key Microsoft developers, who left the IronRuby team recently, has revealed some embarrassing details in a recent blog. The basic accusation is that Microsoft isn't committed to IronRuby.The IronRuby team is now down to just one member.Version 1.0 was only released in April (2010) and Visual Studio support has been slow in arriving - and with virtually no one working on the project it is going be slower in the future.

There have been suggestions that the once well supported project simply clashed with Microsoft's recent ASP .NET MVC developments. After all you don't really want two MVC frameworks in the same .NET development space and while IronRuby may just be a language it is natural to think of Rails when considering an MVC framework to use with it. Perhaps it was feared that comparisons between a .NET Rails and ASP .NET MVC might not have been flattering.

When you also throw in the fact that the dynamic language runtime (DLR) has allowed mainstream languages such as C# to acquire dynamic features there seems even less point in pursuing .NET versions of Ruby and Python. Hence it looked like everything boiled down to using the DLR for giving dynamic language capabilities to the .NET static languages.