This post looks at the thought process behind modular address spaces within NetKernel and Resource Oriented Computing (ROC).
The Internet is a global information space. This is the design intent and it is reflected in it’s data structures and registries including URI syntax, DNS, MIME types, and port number assignments. Intranets and such, the lower-case internet, allow for local address-space which provide a neighborhood around a node possibly backed by the Internet. However this is a pretty static and limited structure limited to local overrides with fallback and complex network configurations.
By contrast, software languages have demanded much richer namespace abstractions. Most modern languages support scoping which provides a mechanism for information hiding and encapsulation by restricting access to variables and expressions. Static scoping based upon the syntax of the programming language being the most common form. In addition many languages support the concepts of classes, packages or namespaces orthogonal from scope. They often employ the ability to create a flat list or hierarchical set of named spaces. These namespaces can then be imported or referenced from other spaces providing mechanism for managing the inherent complexity in having a large number of parts.
It is clear that programming languages have developed much richer addressing mechanisms and the likely reason is the increased complexity at this scale. The inherent complexity of systems is on an inevitable upward trend that is unlikely to end any time soon. This is due to a number of factors including the increasing interconnectedness of systems, the relentless march of new functionality and the difficulty and time involved in unifying the old with the new.
So it was clear to us that ROC needed to support rich modularity and we needed to find a way to marry the addressing mechanism of the internet with that. There is obviously a concern here that in doing this we damage the desirable properties of the web. Interestingly none of constraints and principles of the web as described in Fielding’s thesis say anything about the necessity of a global information space. My guess is that it was done this way because Berners-Lee had a egalitarian vision of the web; that everybody should see the same thing. This was then re-enforced by the complexity of engineering more sophisticated solutions on a global scale. Of course today we see sophisticated content filtering firewalls but these don’t attempt to create a coherent subset of the webs address space, they simply put crude barriers on certain areas.
I am very happy with the solution we came to to support modularity. We introduced the concepts of space and endpoint. A space is a resolution mechanism for requests. Exactly how a space resolves requests is not constrained but typically it will have a list of endpoints that it will test in order to determine if they will handle the request. Again exactly how and endpoint specifies what requests it can handle is implementation specific. Example endpoints would include Servlet like services or bindings of a portion of a filesystem into the address space.
Now consider an implementation of an endpoint which acts as a proxy to another space. It achieves this by resolving all requests that would resolve within the other space and then forwards them. We now have modularity. We can construct patterns of spaces like libraries of resources which are aggregated by a parent space. We call this endpoint an import.
As a specialisation of an import we can have an endpoint that intercepts requests entering a space and the response returning from the endpoint in that space. This is analogous to Aspect Oriented techniques. It enables many things such as exception handling, security gatekeepers and quality of service auditors. We call this endpoint a transparent overlay. It is transparent because it doesn’t effect the address space, the same address space is exposed but the behaviour of the resources might be different.
A further specialization of a transparent overlay is a non-transparent overlay. Here the endpoint will resolve certain configured requests that differ from the space it delegates too. It then maps the requests it receives to requests into the delegate space. Non-transparent overlays include filters and general purpose address space transformation technologies.
These examples give a good idea of the modularity constructs that ROC enables. One thing that is missing from this discussion is how scoping is achieved. Scoping introduces relativity to addressing and opens the way some powerful patterns. I’ll discuss how ROC uses a variant of dynamic scoping in a later post.