Cue controversial intro: In a perfect world, all software would be expressed using domain specific languages (DSLs).
Half the developers who read that likely just snorted with derision. The other half probably did a metaphorical fist-pump, or a sage nod.
“So why don’t we use DSLs to develop all software?”
Put simply, DSLs seem to have an unfortunate (and inescapable) downside: designing, implementing and maintaining a DSL as well as the tools required to translate that language to executable code is not exactly easy or cheap, particularly since the more domain specific you make it, the smaller the target market becomes.
We seem to be faced with a dilemma: How do we (easily and cheaply, remember) get from a DSL (requirements a human can easily convey) to a binary (code a machine understands and can execute)?
How’s this for a crazy idea? Instead of separating the language from the compiler, let’s create a world where the language and the compiler are one and the same.
“Great. So you re-invented macros.”
Not quite. Macros still ultimately require and ‘talk’ (i.e. provide directives) to a compiler. What we are proposing is removing the traditional compiler altogether.
It’s ‘macros’ all the way down, my friend.
So who/what do the macros now ‘talk’ to?
Each other.
We call these ‘macros’ agents and together, they make up the World Compiler, a globally distributed system that ingests requirements (given at any level of abstraction) and returns native code. Each one of these agents is hard-wired to have special, compile-time metaprogramming powers that allows it to compile-as-a-service.
World Compiler
At their run-time, hundreds upon thousands of agents will come together to form a ‘hive compiler’ (a specific instance of agents engaged within the World Compiler) spanning as many layers of abstraction as needed, all the way down to CPU instructions.
Each agent compiles-as-a-service by contracting other compile-as-a-service agents that contract other agents and so on. Every iteration of contracts results in a collaborative translation of (n+1)-level requirements into (n)-level requirements, effectively extending the compiler a little further.
When agents who are at the finest scale of abstraction (the ‘byte agents’) are contracted, the hive compiler is fully formed. These byte agents then place a few binary CPU instructions and return them to their clients. These clients combine those bytes (via simple string concatenation) and return them to their clients, and so on.
As the hive incrementally disappears, the binary incrementally takes shape. Eventually, one long string of bytes — the binary itself — arrives back at the root client. Job done.
Enough neologisms, tell it to me straight
If you were to browse this network, you might find agents that provide services that sound similar to a library function; for example, decode::as-ascii-integer. However, when you ‘contract’ that decode::as-ascii-integer, you aren’t importing some library function. Instead, you are contracting that decode::as-ascii-integer agent to design and return a custom binary fragment of code, a fragment that will seamlessly interlock with other fragments returned by any other agents you contracted.
For example, by contracting:(1) contribute::program-core(2) _read::bytesequence_(3) decode::as-ascii-integer and (4) _write::integer … y_ou have just ensured the four compiled fragments returned by those agents will be joined together upon delivery. Essentially, you are the root node in that particular hive compiler.
Who builds the agents that build the software?
Each agent in this World Compiler is created and owned by a software developer somewhere around the world.
Each agent is paid every time it is contracted to join a new hive compiler. For example, a dev who has built a decode::as-ascii-integer agent currently receives about 4 cents every time another dev contracts their agent to deliver that compiled fragment of code.
The better the agent (relative to its competitors), the more contracts it will win, the more hives it will join, and the more payments it will collect for its owner.
“Ok, that’s great. But who on earth would want to be on other side? Who wants to pay per compile?”
Right now, you likely perform your daily programming tasks by researching, vetting, and then gluing together existing libraries. Due to financial constraints, these libraries are ‘wired’ to work in a one-size-fits-as-many-as-possible manner; a manner which then doesn’t fit perfectly with the other libraries you want to use in your project.
But agents in the World Compiler are incentivised by that crazy notion of income to make your life easier. If there is a market for a variation of agent, that variation will inevitably emerge, because behind every agent is an enterprising developer.
You, as root client, no longer have to spend your valuable time (and therefore money) on the tedious exercise of integration, and on finding tools to compile your source code to executable code (tools which come with their own dependencies). Instead, you pay agents to carry out (or eliminate altogether!) that gluing, and that compilation, and for a price orders of magnitude less than the price of your valuable time.
Cue controversial ending: Now that the source code is also its own compiler, the World Compiler becomes ‘abstraction-scalable.’ Its hive compilers can grow beyond traditional levels of abstraction — past HLLs and libraries — to any level at which it is easy for you to express your program requirements. And all without sacrificing run-time performance.
One day soon, any end-user will be able to use this system to build custom, large-scale applications. They will simply select a few specific agents from the relevant application domain, satisfy their requirements in some easily understood ‘language’ and hit BUILD. A short time later, those agents will each return their own fragment of compiled code. Job done.
You can take the World Compiler for a spin here. You can request an invite to become part of it here.