Black Bag Operations Network

The first commercial software I ever worked on produced reports from data files. I had to open the file, parse its contents, write a new file, and then close all the files. Something like 80 percent of the program was just I/O processing.

The first data base system I worked on used indexed files. It was more hierachical than relational and unexpected system failures would have you rebuilding your tables and indexes for hours on end on a regular basis to recover from partial writes. The data storage routines were in a library and were pulled in by the linker. There was no concept of locking because you knew you were the only user of those files.

It was data processing with stone knives and bear skins and we liked it fine. OK, actually it sucked but at least we didn’t have to open and close the files or parse their formats ourselves. Application development went a little faster because we could focus less on file I/O and more on code that called the data storage routines and processing steps.

That was a step forward.

Later on we got Oracle and we experiemented with embedded SQL. Thats where you write bits of SQL write into your source code and some preprocessor turns the SQL into C code and function calls. It had the effect of removing the programmer one level away from the data processing and you could specify things in terms of set operations which let you get away from writing loops. One more set of bugs eliminated.

Of course, we now had to know three programming languages, C, SQL, and the weird embedded C/SQL/ProC dialect where the two collided. Code was structured into function libraries based on business functions. A library might contain the implementation of a number of important “edits”. Basically code that checked data to make sure it obeyed certain logical constraints. Since the libraries were reusable, so were the edits and thus business logic was mostly centralized and relatively easy to maintain. The libraries were then organized into “systems”. Billing System, Order Entry System, Payment Processing System.

Step forwards.

Object Oriented Programming arose as a means of refactoring code and it gained widespread acceptance primarily because it made the newfangled graphical user interfaces so much easier to write.

Unfortunately, OO also had the effect of turing the data processing world on its ear. Numerous articles from a number of OO “Methodologists” appeared bemoaning the so-called Object-Relational “Impedance Mismatch”. The idea being that, because the two processing and representation models were so different, the translation between the models became difficult and somewhat computationally expensive.

Several brave companies jumped into the breach with products designed to map database tables directly into user interface elements. Thus, client server was born. It was a huge mistake. Client-server was a two-tier solution. You had your data storage tier which was your relational database, and then there was the user interface tier. Client server encouraged user interface programmers to build the data edits and business logic right into the user interface. Thus, the same business rule would be embodied over and over again in various screens and other user interface elements.

Step backwards.

Meanwhile, the people using Smalltalk had come up with the Model View Controller (MVC) Architecture. The model represented the business logic and entities that was common to the business. It once again centralized the edits and business rules into a single code base that could be reused across a number of applications. The views represented the user interface elements and the controller mapped the data between the model and view and provided application specific actions.

Step forwards.

Mapping the model to the database was still a problem though. The most enlightened developers realized that it was possible to apply MVC principles with the relational database being the model and the business model taking the role of the view. The controller was known as an Object To Relational (O-R) Mapping layer. This specific mapping was relatively difficult to do and only a few companies produced really good mappers. Several others produced some rather bad ones. The good ones were exemplified by Persistence, TopLink, and NextStep’s Enterprise Object Framework (EOF). They allowed a developer to specify an object model, an entity relationship model, and mapping between them without writing any special code. Business logic was associated with business objects, database consistency edits were stored in the database, and application developers only needed to write views and application controllers on top of the model to provide new business applications.

Step forwards.

Since every computer platform had a different user interface programming API, every application had to be rewritten once for each platform. This made it difficult to mix Macintoshes, Wintel machines, and Unix/X Windows machines in an enterprise. There was also the difficulty in doing remote administration of thousands of clients every time a new version of an application was completed.

Enter the world wide web.

Because web browsers behave mostly the same on all platforms, it was possible to move all the application processing to a server and serve up views rendered in HTML. This had the advantage of eliminating the client configuration and distribution issues at a cost of a loss of interactivity. Web applications could only be forms based. Things like collaborative drawing tools were not good candidates for web applications but things like order entry and online banking were mostly forms based and a good fit.

Both the two tier and three tier approaches were taken to the web. The two tier had all the same problems it always had with scattered business logic located in the user interface layer. The three tier approach simply created a hierarchy of view objects that would “draw” themselves by emitting HTML. It was better but it was no longer possible for the application to be very interactive because of the nature of HTTP request response processing. So applications got dumber and slower.

Step sideways.

The two tier people caught hold of XML and XSL which appear to address some of HTML’s shortcomings. They began to look at web requests as document processing rather than events for a user interface. Except that databases aren’t really documents. So to get the database to participate in the XML oriented applications, new mapping models to go from relational databases to XML documents were created. Of course, unlike objects, XML documents are essentially inanimate data lacking behavior. The behavior is supposed to be provided by applying style sheets and transformations to the XML to produce new XML.

Giant step backwards.

Mapping relationally structured data into intelligent objects eliminates work and ultimately bugs because the objects enforce their own constraints and business rules. Mapping relationally structured data into hierarchically structured data has limited value other than as a streaming format. The data remains independent of the rules that transform the data and maintain its integrity.

There are lots of other inappropriate applications of XML. I’ll discuss more of them later.