Porting software is hard. A lot harder than it should be. That's not to say it shouldn't be hard, but the problem is that it is difficult for the wrong reasons. Those of us who find ourselves porting applications (that were not originally painstakingly designed up-front to be portable) inevitably spend a lot of time on the mundane mechanics of getting it to compile and run in a maintainable way, rather than the real work of adapting the app's functionality to the endemic properties of the target environment.
What are you talking about?
It's probably best to start out with some definitions, just to make sure we are all starting on page one.
Let's call an "Application" any deployable unit of software. So, it could be a classic app, library, or a service.
To "Deploy" an Application to an Environment is to make a unit of software available to install, run, and ultimately use, using the natural means of the Environment.
An "Environment" is any unique, emergent Deployment situation. This could be a broad as all Android phones vs Linux Desktop, or as specific as a particular model of a phone that has any behavioral difference with some other phone.
A "Platform" is an abstraction of an Environment. This is a subtle distinction from an Environment, but I would say the primary difference is: an Environment is emergent, and a Platform is deliberate. The Android SDK + Java is a Platform that you would code against, build, and Deploy with, whereas "all Android phones" is an Environment.
A "Standard Library" is a set of APIs that one can reasonably expect to be available across all Environments for a given language. This must necessarily be subjective, because a crisp, pedantic definition would exclude such a thing from actually existing in reality. Even Java, which has generally done the most to ensure its Standard Library really is standard across most Environments had things like J2ME which was a completely different API you had to code against.
An "Extension Library" is a set of APIs that require special consideration to be made available to an Application during Deployment. Using Extension Libraries generally make it harder to Deploy Applications, because each special case requires attention. Special cases like these are points of friction, and often sources of rapid bitrot.
A "Platform Abstraction" is a set of APIs that allow software to access features of the OS or hardware. This may or may not be part of the Standard Library. Often not, which is one of the many problems. POSIX is an example of a Partial Platform Abstraction that is very nearly part of the Standard Library. SDL is an example of a Partial Platform Abstraction that is an Extension Library.
A "Module" is a self-contained API definition for a specific subset of Platform facilities.
"Complete" means to have all the necessary or appropriate parts. Deciding what is necessary and appropriate is completely subjective, so this is intentionally a judgy, prescriptive word.
A "Complete Platform Abstraction" is a Platform Abstraction that is sufficient for an author to build a large majority of Applications portably and performantly without pulling in Extension Libraries or defining their own intermediate Platform Abstraction. This is an obviously subjective definition — balanced precariously atop other subjective definitions — so, it would need to be arbitrated by some respected body. I can only suggest a starting point, at best.
A "Partial Platform Abstraction" is any Platform Abstraction that isn't considered Complete by whoever decides such things.
Proposal: A Standard Platform Abstraction
What I am asking for, here, is a "Specification," published and maintained by a respected standards body, of a Complete Platform Abstraction. It would probably look something like the DOM API specification, in that it would specify function and class signatures, but ultimately leave the details of binding to each language and implementation to each Platform.
Most urgently, the Specification would first have to define what "Complete" means. For example, I claimed that POSIX wasn't Complete, but how would I justify that position? The Specification would need to declare what it means to be Complete, such that one could objectively measure compliance.
Completeness would require a process to gracefully evolve with the times. For example, VR adds a new surface of input and output hardware facilities that need to be abstracted, but is still evolving. Maybe bio-interfacing would be a new surface coming down the pike in the future.
The Specification would provide a list of defined functionality surfaces and organize them into Modules, each with its own sub-specification outlining the details of that Module. Ideally, each Module would be as independent of others as possible, so it could be implemented by a Platform and initialized by an Application a la carte.
The Specification must facilitate a way to test Capabilities. This should support the possibility of efficiently culling unsupported code at build-time, but the details of that depend very much on the language being bound to. At a minimum, it must provide some Module allowing apps to avoid calling unsupported APIs at runtime, even if the calling code has to ultimately be Deployed anyway. If I'm running an Application from an SSH, it shouldn't initialize the windowing system, even if the app can offer a GUI when run locally.
If this sounds a lot like what the W3C does for the web, that's deliberate. It's the only standards stewardship that I've seen really able to be both stable and agile.
Motivation
This proposal was originally motivated by my personal struggles with POSIX as a Partial Platform Abstraction. I've had to find ways to port very large, graphical, real-time Applications onto embedded Platforms. I've had to port open third party and proprietary libraries to extremely exotic hardware. Life is hard, but I guess that's why I get paid in real money-dollars instead of good vibes. But, everyone would prefer that these these tasks take less time from fewer people. Here are some broader reasons why this is needed now.
Windows, macOS, Android, iOS, Web, and Linux
These days, everyone is building Applications that need to run across all these very different Platforms. Interfaces between these Platforms are quite different, so it may not make sense to actually try to Deploy the same Application. But certainly it would be great to Deploy a single mobile Application to Android and iOS, and a single desktop Application to Linux, Windows, and macOS. And, you really want to be able to share business logic and other libraries between your Applications, even if you don't wholesale Deploy the same app everywhere.
There are solutions out there trying to tackle this problem. Java classically wanted to solve this problem, but it really hasn't worked out - it's mostly used on servers, and Android. Unity can Deploy to all these critical platforms, and game consoles as well, but it's really geared towards games, isn't free, and comes with some operational baggage that not everyone wants to carry. Haxe is a language that is designed to compile into other languages in a consistent way, and there are projects like OpenFL or Lime that provide a Platform Abstraction in Haxe for key platforms. All these solutions place severe limitations on which technologies you might want to use, and they all raise the abstraction level at some tangible cost. This is not a canonical list, just what I'm most familiar with.
Every Language needs a Platform Abstraction
I'm primarily coming at this with my C/C++ embedded experience, which I will acknowledge is crusty and old. And quirky. But, in some ways, C is still the most portable language. The vast majority of technology stacks ultimately have C (and POSIX or similar) at the very bottom. There are many, many traps, but one can write C/C++ that can run anywhere. It's just a lot of work.
But, every language should consider that they are limiting their potential user base by not providing a stable, clean, Complete Platform Abstraction. People work around it, usually by creating some native bindings (usually C/C++). These libraries often themselves are very targeted to one purpose, or tied to other independent libraries that weaken their broad utility. Since they aren't centrally maintained, they can fall behind the pace of the language development, or the development of the platform they are bridging to.
Wouldn't it be great if every language's platform abstraction was already familiar? If it could be built directly off an existing low-level C API? Portable abstractions and idioms could then be built on top of that.
Java is probably the closest to having a Complete Platform Abstraction in my view, but it comes with a load of Deployment constraints and performance baggage. Good luck getting your Java program onto the iOS App Store, or running on any embedded platform like a Smart TV. Also, the Android UI development APIs are completely different from the Desktop UI development APIs. They tried super-duper hard with Applets and then Java Web Start, but it's always felt like a second-class citizen for end-users.
We are Living in the Long Shadow of Legacy
The responsibilities of an operating system have changed over time. Grown. POSIX has grown, too. Hey, we have threads, now! But, with incremental, organic growth comes cruft, debt, and other challenges.
POSIX is big. Really big. You just won’t believe how vastly, hugely, mindbogglingly big it is. Much of it doesn't really apply everywhere, or gets way in the weeds of details for something that very few people need.
Yet, POSIX is still not "Complete." It doesn't let you open a GUI window, access audio devices, or other sorts of things that one would want from an OS abstraction. There's still no standard "name this thread" facility (the "np" in pthread_setname_np means "non-portable"). You want some random numbers? Open up /dev/urandom, and hope you are running on Linux.
POSIX is often not implemented at all (less often in recent times), only partially implemented, or implemented in a quirky or non-standard way. Windows. Android. iOS. Game consoles. It's a minefield of incompatibility. This is because POSIX is just too big and too hard to implement. The unification of files, sockets, and other I/O into file descriptors seems cool, but also is historically an implementation sore spot. Pipes. Signals. PIDs and TIDs. And then, even if you get all of POSIX right, you aren't done, because you still need SDL or something similar to get access to everything all applications have been doing for the past 20 years.
I know, I know, insert xkcd #927 here. But, sometimes, it works. Sometimes, just sometimes, you really do need to start over to cast off that yolk of legacy. I imagine that even if there was a way to get this thing moving, it would HAVE to coexist with other Platform Abstractions like POSIX and WinAPI for a long time. But, it could be an option for folks who really want to plan for portability.
Denouement
Until and unless a cross-organizational effort is made to solve it, making portable software is going to involve reinventing the wheel over and over again, or lashing yourself to the mast of very specific technologies that may or may not be relevant in a few years, and likely to come with other baggage. Maybe you are OK with the baggage for now. Maybe not in a few years.
I threw out some hints, but I didn't get into details of what I, personally, think would approximate a Complete Platform Abstraction. I have done this thinking as part of aforementioned work that I've done, and I'll drill down in an upcoming post, for those that are interested.
Revision History
- Added a definition for "Complete." Punched up the definition for "Complete Platform Abstraction."