Allen is a pretty talented guy, and he has a long history and deep insights into the history of Delphi. I can confirm (from outside the Embarcadero/CodeGear fold) that everything he says is exactly the way it is, even the motivation for .Net being that Microsoft planned to "yank" the Win32 platform out from under us Win32/VCL programmers, and force everybody onto new "managed" .Net runtime APIs, to the point that existing Win32 APIs would be a "penalty box" environment, a sandboxed subworld inside the real windows, the equivalent of the NTVDM, the virtualization layer that kept 16 bit DOS mode executable compatibility around in Windows even as recently as Windows 7 32 bit. You may remember that NTVDM only went away when we got to the 64 bit Windows world, because Win32 is a virtual layer inside the main windows environment, now known as "WOW" (Windows-on-Windows). As being at the root of the platform's APIs is key to Delphi being Delphi, the now mostly dead and buried "Delphi for .net" product seemed necessary to Delphi's continuing health and it having a future on a .Net-only Windows world, that never materialized.
If Embarcadero could ship a WinRT targeting version of Delphi, they would have already. But Microsoft is holding all the keys, as they have meticulously crafted a sandboxed, signed, Apple-App-Store rip-off mode inside Windows 8. But WinRT is just a "penalty box" inside Windows 8. Rather than enabling new features, it is primarily notable for what it takes away. In Windows 8 AppStore/WinRT applications, your application does not get to decide when it runs, and is more heavily "managed" by a power-consumption-optimizing user shell that is more similar to Windows Phone, iOS, and Android than it is to classical Windows programming. It's also heavily asynchronous, and if you haven't noticed code-samples for Windows 8 WinRT applications in C# make heavy use of the "magic"await keyword in C#. I am not slavering for such magic to appear in Delphi, because such magic has a dark underside. Just as .Net garbage collection has a dark underside.
Does ARC have a difficult and dark underside? Not really. But there are a few things to be aware of in reference-counting based memory management, things you probably already should know if you are a competent Windows COM/DCOM programmer, things like:
If Embarcadero could ship a WinRT targeting version of Delphi, they would have already. But Microsoft is holding all the keys, as they have meticulously crafted a sandboxed, signed, Apple-App-Store rip-off mode inside Windows 8. But WinRT is just a "penalty box" inside Windows 8. Rather than enabling new features, it is primarily notable for what it takes away. In Windows 8 AppStore/WinRT applications, your application does not get to decide when it runs, and is more heavily "managed" by a power-consumption-optimizing user shell that is more similar to Windows Phone, iOS, and Android than it is to classical Windows programming. It's also heavily asynchronous, and if you haven't noticed code-samples for Windows 8 WinRT applications in C# make heavy use of the "magic"await keyword in C#. I am not slavering for such magic to appear in Delphi, because such magic has a dark underside. Just as .Net garbage collection has a dark underside.
Does ARC have a difficult and dark underside? Not really. But there are a few things to be aware of in reference-counting based memory management, things you probably already should know if you are a competent Windows COM/DCOM programmer, things like:
- Reference counting cycles are something that happen when object A holds a reference to B and object B either directly holds a reference to A, or holds a reference to something else that holds a reference to A. A "directed acyclic graph" means "a bunch of dots with lines in between them, and each line has an arrow, sort of line a one-way street sign, and you could follow the arrows any way you like, but never end up back at a node you had previously visited." If your references are like that, they could in some way be considered a "tree" instead of a "spiderweb". In your "tree" of objects there are no cycles. If however your application has more of a spiderweb structure, then you will need to learn how to break these cycles.
- Breaking reference counting cycles is easy to do in a CLANG/LLVM based ARC scenario, such as Objective-C in XCode, and the same is true in Delphi on iOS. Weak references are the answer. Weak references are really cool. They are like a C++ smart pointer on steroids. The compiler and its code generation back ends provide some really strong guarantees that enable you to work with a really easy to use programming model. If the object pointed to by a weak reference still exists, you get that object. If it does not, you get a null (or nil if you like) object reference. If you were curious how this worked when you first heard about it, you might want to read up on the Apple Clang/LLVM documentation on Weak references, because the Delphi implementation and mechanics are almost 100% guaranteed to be the same, since they're based on the same LLVM code. Weak references are expensive. They slow your application down, but remember that correctness (not crashing) is always more important than performance. It would be a real mistake to pathologically avoid weak references where they are the correct solution just because they can introduce performance issues if abused or used too much. On the other hand, if you can avoid circular references completely in your design, then you avoid the problem of refcount cycles for free. Use your head, and choose on a case-by-case basis, and optimize after profiling and running and testing your code, not before.
Having used Objective-C with manual memory management (retain and release calls), and with garbage collection (which was crap, and is going away out of Objective-C), and now having used Objective-C with ARC, I am very happy to see ARC coming to Delphi.
I predict that when ARC comes to delphi on VCL Windows Desktop, a lot of old crap in your codebases is going to have to go away including:
- Length delimited string syntax like string[30] which is really the ancient shortstring type from TurboPascal
- A lot of old compiler flags to turn on dos-mode real48 compatibility, alias string to shortstring ({$H-})
- UPDATED When I first looked at ARC, I thought Variant Records might be incompatible with it, but actually upon reflection, I'm guessing that there will just be continued restrictions on the combination of Variant Records with Managed Types, which is already the case today. You can't put an AnsiString in a Variant Record for example. So you won't be able to put an ARC object reference into a variant record.
I will be really surprised though, if the trial-balloon that Embarcadero has floated about non-mutable zero-based strings will fly. I expect that is a deal breaker, as there are hundreds of millions of lines of otherwise clean code that still depend on Strings starting at position 1. Parsing strings with Pos() and Copy() and the lack of requirement to deal with a separate String and StringBuilder type are classic differentiators between a Delphi managed-heap-of-strings and a Java or .Net managed string environment. Changing these semantics is not a simple matter of adding or subtracting 1 from a string position, it is a symptom of a fundamental difference between the classic mutable Delphi AnsiString and UnicodeString copy-on-write behaviours, and the behaviour of the type that it seems is being chosen to replace it, which is in the end, something homeomorphic to the Cocoa/Apple immutable NSString type.
Homeomorphic means "same shape", or in this case, "if it walks like a duck, quacks like a duck, and plays poker as badly as a duck, then it is a duck". It looks like the easiest way to move Delphi onto an LLVM compiler backend and ARC is to adopt immutable NSString as part of Delphi's base way of doing strings. I think that's a mistake. I believe it is possible to preserve the appearance and semantic model that Delphi has always provided. Strings are (to a user of the language, not to the implementor) as simple to work with as integers and characters. This differs from most compiled languages. Strings in Delphi are as simple to work with as strings in Python or C#. I am not aware of any other language that is strictly statically typed, and runs without an interpreter, JIT or VM, and yet provides as simple a string model as Delphi. Don't even try to tell me that C++ has it, because while you could probably write a string class in C++ that only you even use, real world C/C++ codebases routinely mix 10 or 20 string-like types per codebase. Erasing that advantage would not only break how Pascal/Delphi string coding has worked since 1984, it would prevent people from moving their existing codebases up, and would slow Delphi adoption.
I expect that the new compiler and the old compiler will be co-hosted in a single Delphi IDE for the next four to eight releases of Delphi, and that eventually the classic non-LLVM compiler will go away. During that time, it's critical that all Delphi codebases get transferred into a form that allows them to move up. That means that ad-hoc hacks like "obj.Free" doing nothing, and, if this is going to fly, it also means that String semantics stay exactly where they are, with no attempt to even offer immutability or zero-based offsets is made to users. The last thing we need is a new {$H+} and {$H-} compiler switch to replace the old one.
A simplified, rational coding model that can be easily applied and used without any possibility of bad things happening silently in the background, and without requiring users to rewrite around immutability or zero-based strings is required for this move towards ARC to be a success.