Dataphor Frontend Future Thoughts

Just some meandering thoughts as I consider various possibilities for Dataphoria and the Windows Client.

Observation: it seems that the history of approaches for engineering application processes has vacillated between two extremes: Let's call them structured and event based.

In a structured approach we consider the whole. The most obvious example is a block of imperative programming code containing branching and looping constructs. There is a clear path or flow of execution through the components. Other embodiments of structured process engineering include traditional flow charting and more recently workflow management.

An event based approach, on the other hand, deals with single moments or points of decision. The actual flow of logic is more dynamic and elusive. Another way of saying this is that each aspect or component of the application is defined as autonomously as possible. An example of this approach would be component based programming characterized by Rapid Application Development (RAD) toolsets.

Scaled Complexity

Everyone either knows or learns quickly that the structured approach does not scale very well as the complexity of the processing increases. The extreme of this approach is the all too familiar spaghetti application, where the developer tries to sort out all of the logic and states in a seemingly unending series of branches and conditions. Anyone who has used flow charting has discovered that even simple realistic scenarios easily overwhelm the model when things like error handling or backtracking are considered.

The event based approach, on the other hand, scales very well with added complexity. Because the logic is composed of reusable, autonomous components, the myriad complexities of the overall process logic are broken down into small, manageable pieces. Consider of the organs and cells of the human body. Each has a distinct job to fill, and in combination these components group together to create a higher level construct. Its difficult to imagine a body being composed of only a single "machine", but if it were, such a thing would clearly be infinitely complicated.

Visibility

Here is where the structured approach wins out. People like imperative process logic, flow charts, and workflow engines because they can "see" what is happening. Just as it is difficult to discern how a body works (or somethings doesn't) by observation, it is also difficult to diagnose the happenings of a complex event based application. The organic aspect that makes it dynamic, extensible, and scalable also makes the event based application opaque and unpredictable.

State Management

State management in the event approach is an exercise in backwards thinking. The closer one gets to a "pure" event based approach, the more state becomes ethereal. This is often manifest when building asynchronous logic. Rather than write:

var X, Y
X := GetX();
Y := GetY();
if X = Y then DoSomething(X, Y);

The developer has to do something like:

State := { X, Y };
BeginGettingX(LetMeKnowX, State);

LetMeKnowX(Result, State)
State.X := Result;
BeginGettingY(LetMeKnowY, State);

LetMeKnowY(Result, State)
State.Y := Result;
if State.X = State.Y then
BeginDoingSomething(..., State);

Again, the fact that the logic is broken into pieces has its upsides, but also results in poor visibility.

The enticing idea that real world processes can be effectively modeled by drawing little arrows between boxes and such has led to countless tools of similar nature, but all of these end up bumping against the scaling limitations inherent in their oversimplifications. I think this is why programs are still written in programming languages, not using lines and boxes.

So the question I am asking myself is: can we have our events and see them too? I want the scalability, reuse, and dynamic nature of the component approach, but the visibility of the structured approach.

Note: this is a blog posting I drafted months ago, but never posted. Its pretty scattered and mostly useless, but I thought I would post it because I think the event based versus procedural issue has raised its head again with recent discussion about massive parallelization; how will we cope with 80 core processors?! As for user interfaces, I think this matter can be largely solved by looking at most "workflow" user interfaces as providing arguments for some process. If this doesn't make sense, sorry, I'm still trying to make it concrete myself.