Ten Ways to Sunday

Camtasia Studio Tips

2007-07-08T22:47:00.001-06:00

Camtasia is great software for recording audio/screen presentations. Here are some tips and standards we've used:

Recording

Change the recording options to record to ''AVI format'' (the CAMREC format can lose audio/video sync over a long recording)
Recorded movies should be 800x600 or less unless there is a specific reason to record larger. Using the smallest capture size possible improves the readability and reduces the file size.
All videos should start with an announcement of what is being demonstrated (e.g. "This is a demonstration of the Device Capture System introduced in Cashwise 3.7)
Don't record the application window; set one of your screens to 800x600 and record the screen. This ensures that any popup windows or menus remain within the recorded region.
If recording other than the primary screen, select 'region' and select then entire alternate screen
Hide any toolbars or other desktop clutter.
Run a short test to be sure that audio/video are working before recording a long demo

Storage

Files should be recorded to AVI format. Recommend settings:
Video Codec:    TechSmith Screen Capture Codec
Audio Codec:    MPEG Layer-3 Codec
Audio Format:    32 kBit/s, 24000Hz, mono, 3KB/s

Content

Before delving into details, give a high-level description of the purpose of the feature and a high-level description of the components of the feature.
Consider dividing the topic into multiple videos if it is possible that someone might wish to view topics separately
Try to keep videos under 10 minutes long
Don't script out the content. If necessary, create a brief outline to ensure each important facet is discussed. It's hard to listen to scripted speech.
If you realize that some topic is missing from a video, record an addendum rather than re-record the whole thing (unless there is a mess of addendums)
Keep a reasonably fast pace. A video says even more than a thousand words. The viewer can always pause or rewind.
Try to keep the content timeless; e.g. do not say things like "this is a new feature".
Try to spend more time on abstract concepts, gotchas and special cases that might not be readily apparent rather than mundane descriptions of the obvious.

General comment on the state of things

2007-06-04T22:39:00.001-06:00

I think the field is in its infancy, though some claim it mature. Years of overlapping nomenclature, competing standards, and just plain fuzzy thinking have left the software development industry in a rather confused state. So many mistakes of the past are being repeated, and the divide between the abysmal research community and the vast army of confused practitioners widens. But I am an optimist. To me, the situation presents great opportunity, and I hope to be a part of making a better future.

Web applications - now only 15 years behind!

2007-05-20T23:45:00.001-06:00

It seems to me that web applications are finally approaching where "rich client" applications were in the late 90s. People are finally pooling together abstractions and forming UI toolkits around them.

What would be really great would be if we could skip 15 more years and stop using HTML altogether.

Down with whiners... Microsoft

2007-05-15T21:44:00.001-06:00

I've pandered to, accommodated, supported, and even defended Microsoft for years now, but I'm through. To this point, they may have occasionally been the big bully, but more the special ed kid kind of bully than the vindictive Harvard graduate type. They've had a sort of unspoken culture of defensive litigation. Though they may have used some aggressive tactics, their overall approach has seemed to be: win the battle through providing better software. With their recent whining about how the open source world is stomping on all their B.S. patents, they have crossed the line. I don't like Microsoft any more. Go Linux! Go Mac!

Needed: voice tracking webcam

2007-05-10T23:29:00.001-06:00

We recently acquired Logitech's motion tracking webcam in order to better capture presentations, discussions, and training sessions. In case you are interested, the camera is the Logitech QuickCam Orbit MP.

The camera does what it advertises and does so pretty well. As the presenter moves around, the camera keeps the person within the field of view. As an aside I was a little surprised by the fact that this camera moves in small jolts rather than in a smooth manner. Though the resulting video doesn't feel very professional, I would rather have this behavior that see constant movement as the system tries to decide where to point. Perhaps with sufficiently sophisticated software, a gradual movement system would be better.

Where the camera doesn't do as well is when more than one person is involved. The camera basically seems to track the "biggest" movement, so in a presentation setting, if one person walks away from the center of the action, the camera basically follows them. As a result, in our usage we had to basically wave the camera down to keep the presenter visible.

Clearly this camera was not designed for this type of scenario, but it did get me thinking about how easy it would be to provide a solid solution to this problem. The answer is to put dual microphones on the camera, and have it rotate to point to the dominant source of audio. This would be ideal for many scenarios, from video conferencing, to training, to Q & A and presenting.

So, could somebody please make such a thing and not charge too much for it. Thanks. :-)

Example Oriented Programming

2007-05-06T23:02:00.001-06:00

I was letting my mind wonder a bit this evening about how far one could take the idea of example base programming in order to make the programming process more concrete. It seems natural to think of one or more concrete cases, then deduce abstractions from there, so why not allow the system to cooperate in the entire process. Let me try to explain what I mean.

Query By Example (QBE) is a heavily used and long established concept whereby the user provides an example of what data she seeks. One could argue that modern search engines are in fact a form of QBE. QBE was used successfully as a database query language in what used to be Borland's Paradox. In a database context QBE is usually done only in a user interface context through a set of fields with allow the user to "filter" the dataset. Paradox took this much further and defined a complete language in which to express even complex disjunctive queries. When working in Paradox and even afterwards, I would often build queries in QBE, then ask Paradox to translate them into SQL, as I found QBE to be easier and more natural in many cases.

Thinking about the analogue of QBE for programming, a common case comes to mind: test scripts. QA engineers often use software that allows them to record a series of actions against a piece of software, then go back to the resulting scripts and randomize or parameterize whatever aspects. This is a process of taking a concrete imperative process, and making it abstract and reusable.

The questions is: could there be value in this for "regular" programming? When we program, it seems we already undergo this exercise in our minds. It seems that we "play" algorithms in our minds, considering the various boundary conditions and possible variable states. If, rather than just thinking this way, we could state concrete "executions" to the system, then inform the system that we wish certain constants to be variable, then perhaps the system could enhance the process.

Still a "thought in progress" to be sure...

Enough software patent madness!

2007-04-23T14:52:00.001-06:00

Say patent, and one might get a nostalgic tear in their eye as they envision a brave garage inventor, who, protected by his patent, earns what's due him thanks to legal protection against large greedy corporations. Enter reality. Now picture large corporations able to easily crush or control small businesses. Picture large companies being relentlessly attacked for money even though such companies have made every effort to respect other's claims. Picture an environment where anyone wishing to innovate is all but unable to do so because such effort will invariably step on an almost infinite and unknown set of patent "toes".

To make things worse, there are those that are pushing for even more liberal patent laws, such as first-to-file, rather than first-to-invent. Something must be done to stop this! Patents are ruining the software industry and causing tremendous indirect damage to our country!

Software shouldn't even be patentable. Patent law prohibits the patenting of algorithms or abstract ideas. Anyone who works with software knows that software is precisely the materialization of the same. The only reason software patents began to be allowed is due to the ignorance of those in the legal, legislative, and especially judicial fields.

Software patenting should not be allowed if for no other reason than the impossibility of doing so accurately and fairly. Software is qualitatively identical to mathematical systems. In some ways, even mathematics would be a more feasible target for patenting due to the more rigorous and disciplined state of the field. Due to the abstract nature of software, the irrelevance of the related field of science, and the hardly existing engineering practices that surround the field of software, just the practice of attempting to precisely define terms is an entirely fruitless effort. I challenge anybody to raise a single software term that isn't open to ample and gratuitous interpretation and redefinition.

Showing prior art is a requirement of patenting, and this is impossible due to the abstract nature of software. Unlike a physical device where someone such as a judge can easily observe its characteristics, software, in its typical form ends up as a highly obfuscated, tremendously large set of machine codes. Even for more tangible software aspects, such as user interfaces, it is truly ridiculous to pretend that we can assess what is prior art, due to the volume of produced software and the lack of access to it. In my experience, software reinvention is far more common than invention; and I imagine that most of what I consider invention is due to my ignorance.

For something like a manufacturing process, the 20 years of exclusivity a patent affords may be a reasonable time for the benefactor to reap the rewards of innovation. The lifecycle of a typical product or good is measured in decades due to the processes and resource constraints involved. On the other hand, a typical software lifecycle is a matter of a few short years. 20 years represents an eternity in the software industry. In 20 years, the software industry will hardly be recognizable. Such an period for software is utterly ridiculous, doing nothing but stifling innovation.

Another major problem for software patents is that they are intrinsically "obvious" (another exclusion made by patent law). This goes back to the abstract nature again. Like math, there are certain, obvious ways to accomplish certain goals within a logic system. Again, software is qualitatively identical to math; only with software there are many more names for common concepts and there are an almost infinite set of constructions, nearly all of which would be considered "trivial" by mathematics. There are just a few fundamental things in the software industry that are novel, and if any of them had actually been patented it would have spelled disaster for the industry. The increasingly litigious industry now benefits from these fundamental concepts, adding very little but engineering effort to them, yet wants to horde every idea that appears.

When it comes to protection, software is more like music than it is like a manufacturing process. Imagine if it were possible for a big company to step in and patent middle C. Another small musician releases a new single, which looks like it will take off, only to be squelched by a big recording label who holds the patent for the "boom bip" beat pattern. Again due to the abstractness of software, developers, like artists, must have certain bounds to safely work within.

I have much more to say on the subject, but I must get back to real work. Speaking of my work though, let me mention that as a member of a small business attempting to build innovative software, I am certain that I am unknowingly stepping on turf improperly staked out by patents. With the ever increasing number of software patents, most of which are outright ridiculous, one simply cannot avoid it. If someone ever did decide to try to enforce one of these absurd patents against us, the process of invalidating the patent or showing its inapplicability would be extremely draining or worse. This prospect is no doubt depressing, and it absolutely stifles motivation to innovate. The unfortunate price for success is to be a big fat target to greedy, litigious, hoarders who would rather spend their time pulling from others than contributing their own. Read the news if you would like some examples.

I don't want to leave on a depressing note. I do not think the situation is hopeless, but we need to change some laws to get there. For that to happen, we need to help most people understand and agree that software patents are a patently bad idea (sorry).

There are ample arguments and resources against software patents on the web. Here is one of my favorites. Please share your favorite. Also, please sign this petition if you agree.

Dataphor Frontend Future Thoughts

2007-04-15T00:28:00.001-06:00

Just some meandering thoughts as I consider various possibilities for Dataphoria and the Windows Client.

Observation: it seems that the history of approaches for engineering application processes has vacillated between two extremes: Let's call them structured and event based.

In a structured approach we consider the whole. The most obvious example is a block of imperative programming code containing branching and looping constructs. There is a clear path or flow of execution through the components. Other embodiments of structured process engineering include traditional flow charting and more recently workflow management.

An event based approach, on the other hand, deals with single moments or points of decision. The actual flow of logic is more dynamic and elusive. Another way of saying this is that each aspect or component of the application is defined as autonomously as possible. An example of this approach would be component based programming characterized by Rapid Application Development (RAD) toolsets.

Scaled Complexity

Everyone either knows or learns quickly that the structured approach does not scale very well as the complexity of the processing increases. The extreme of this approach is the all too familiar spaghetti application, where the developer tries to sort out all of the logic and states in a seemingly unending series of branches and conditions. Anyone who has used flow charting has discovered that even simple realistic scenarios easily overwhelm the model when things like error handling or backtracking are considered.

The event based approach, on the other hand, scales very well with added complexity. Because the logic is composed of reusable, autonomous components, the myriad complexities of the overall process logic are broken down into small, manageable pieces. Consider of the organs and cells of the human body. Each has a distinct job to fill, and in combination these components group together to create a higher level construct. Its difficult to imagine a body being composed of only a single "machine", but if it were, such a thing would clearly be infinitely complicated.

Visibility

Here is where the structured approach wins out. People like imperative process logic, flow charts, and workflow engines because they can "see" what is happening. Just as it is difficult to discern how a body works (or somethings doesn't) by observation, it is also difficult to diagnose the happenings of a complex event based application. The organic aspect that makes it dynamic, extensible, and scalable also makes the event based application opaque and unpredictable.

State Management

State management in the event approach is an exercise in backwards thinking. The closer one gets to a "pure" event based approach, the more state becomes ethereal. This is often manifest when building asynchronous logic. Rather than write:

var X, Y
X := GetX();
Y := GetY();
if X = Y then DoSomething(X, Y);

The developer has to do something like:

State := { X, Y };
BeginGettingX(LetMeKnowX, State);

LetMeKnowX(Result, State)
State.X := Result;
BeginGettingY(LetMeKnowY, State);

LetMeKnowY(Result, State)
State.Y := Result;
if State.X = State.Y then
BeginDoingSomething(..., State);

Again, the fact that the logic is broken into pieces has its upsides, but also results in poor visibility.

The enticing idea that real world processes can be effectively modeled by drawing little arrows between boxes and such has led to countless tools of similar nature, but all of these end up bumping against the scaling limitations inherent in their oversimplifications. I think this is why programs are still written in programming languages, not using lines and boxes.

So the question I am asking myself is: can we have our events and see them too? I want the scalability, reuse, and dynamic nature of the component approach, but the visibility of the structured approach.

Note: this is a blog posting I drafted months ago, but never posted. Its pretty scattered and mostly useless, but I thought I would post it because I think the event based versus procedural issue has raised its head again with recent discussion about massive parallelization; how will we cope with 80 core processors?! As for user interfaces, I think this matter can be largely solved by looking at most "workflow" user interfaces as providing arguments for some process. If this doesn't make sense, sorry, I'm still trying to make it concrete myself.

Condition Propagation renamed: Proviso

2007-04-15T00:10:00.000-06:00

Everyone who works in abstracts knows the importance of names. I've had to try to explain the concept of Condition Propagation (see Actively managing missing information through Condition Propagation) several times now, and a major challenge has been the vague terminology involved. Specifically, the generic term condition, versus the formal concept I am trying to define.

In an effort to be clearer, I am giving the "conditions" in the scheme I formerly called "Condition Propagation" a new name: provisos. This term's primary use is in regards to stipulations in contracts and legal documents, and so I am confident that its use in the context of computer languages should be unambiguous.

I am still not sure if the concept of provisos as I have presented them is unique or not. I have observed several similarities to other previously raised concepts, but have yet to find a duplicate. I believe that provisos would eliminate many of the design and implementation challenges facing other languages and libraries.

I've begun to attempt to flesh out provisos more formally, and to work out open questions, such as what to do with unresolved provisos at the expressive/imperative boundary. For proviso resolution, I am toying with the following imaginary syntax: ? as a suffix resolution, followed by a conjunctive list of provisos (individually inverted if necessary); and ?* as an short hand for "? true" meaning any and all provisos.

value.IndexOf(',') ?* value.Length

This reads: the index of the substring ',' in "value" or the length of "value" if no such substring is contained. This is short-hand (in this case) for:

value.IndexOf(',') ? IndexOfFound value.Length

Here is possible syntax for proviso evaluation (just return the condition):

if value.IndexOf(',') ?? IndexOfFound then ...

??* would, as expected, be short-hand for evaluation of any and all provisos.

A final thought: Myself and others have asked the question, are provisos too complex? No matter how ideal or elegant an idea is, complexity is a tremendous obstacle. As an example, I've interviewed numerous developer prospects in my career and have observed that very few practitioners understand nulls in SQL. Most SQL users, of course, know to use "is null" rather than "= null" in where conditions, but asked why they do this, they almost never understand null propagation. SQL style nulls are sufficiently complex that almost nobody has come to understand let alone leverage them. In fact the battle cry among practitioners and theoreticians alike is "avoid nulls at all cost". Regardless, I believe that users could overcome the learning curve associated with provisos for at least the following reasons:

Similarity to familiar concepts such as exceptions, nulls, and preconditions.
Deep library integration. Unlike SQL nulls, users couldn't simply learn to work around them.
Deep compiler integration. Unlike other systems, the compiler would "help" enforce proper usage of provisos.
Fundamental simplicity. Ultimately provisos seem a simplifying concept, consolidating and eliminating otherwise competing design options.
Performance. It is not difficult to imagine scenarios where implementations of provisos could benefit from certain performance advantages.

Nice clarification of scientific "laws"

2006-08-29T14:15:00.000-06:00

This jem was in a newsgroup of Steorn.

Ok, I have seen a lot of BS posted on the forums regarding the misnomer 'Law' in science. Since I am an engineer, and have freshly studied mathematics, physics, and electromagnetics, I'm reasonably qualified to comment on this subject.

First off, a phenomenon is observed. Then more. Eventually a large body of phenomena are observed that are related. (Faraday, for example, took hundreds of measurements of electric and magnetic fields, but lacked the math background and education to compile a single governing theory.) This is called experimental physics.

Next, a theoretical physicisyt (and I don't mean that one is incapable of doing the other, but I submit that anyone who compiles theory or does theory work is a theoretical physicist, even a schoolchild who derives ohm's law), takes that pile of evidence and constructs a set of governing equations. In this case, that was done by James Clerk Maxwell, into Maxwell's equations.

Then others test the limits of those equations, either mathematically, (what happens when we push this parameter farther than before...) or experimentally, (what happens if we crank this dial really high), and the theory either stands or needs revision. Scientific theories explain real observations, nothing more. New and unaccounted for observations require the refinement of existing theories.

The term 'law' is a historical peculiarity from before the 20th century regarding theories that stood up under an extremely wide range of experimental circumstances. However, there is a difference between a scientific law and a mathematical theorem; specifically, that a mathematical theorem can be prooven in the rigorous definition of the word, and a scientific law cannot. That is the reason why the word 'law' has fallen out of favor with modern science, because there is always a measure of uncertainty, even with the most basic laws, take Newton's for example.

Anyone who is prepared to discount the possibility of a phenomenon existing because it violates 'law' X would be wise to do a little reading on the planet Mercury, or a GPS receiver. both sytstems violate newton's laws, but none has been issued so much as a warning.

Mercury's orbital velocity is just fast enough to have relativistic effects be measurable, and when Einstein's theory successfully accounted for those irregularities, it was a big boost to the credibility of relativity. (If you think Einstein's credibility was never questioned, read about the arguments between he and Heisenberg). More recently, the relativistic effects of the GPS system are sufficient that software is needed to correct for the errors in timing, which would prevent the system from working.
Therefore the argument that a device cannot work because it violates a particular law has been given two counterexamples.

Next we have the issue of the lack of closure. Scientists, when performing an experiment, try to isolate, control, or account for all possible variables in the experiment. However, scientific literature is rife with situations where problems arose because all variables weren't accounted for. (psychology is particularly infested with them). For example, with thermodynamics. If one were to take an alkaline battery and hook it up to a motor, and examine it using a classic heat + thermal energy method (such as carnot analysis used in jet engines), there would be no heat input, but the motor would be turning, and would appear to violate that concept of thermodynamics. However, once electrical energy is taken into account, and the current into and the voltage across the motor is measured, thermodynamics can be extended to handle effects that are neither thermal nor dynamic.

-"EEgradstudent"

Issues raised by polymorphism in relation land

2006-08-08T21:15:00.000-06:00

I continue to ponder how to merge the Relational Model (RM) and Object Oriented (OO) worlds in order produce a best-of language. The task is a challenge due to the deep and fundamental differences between OO and the RM. One approach I am exploring is to utilize the notion of multiple logical representations to allow both to co-exist without compromise. Doing this, though, means pinning down canonical isomorphisms between the two models.

One important concept from the OO world is polymorphism. In OO land, we identify a useful abstract concept like "Contact" and build a class around it. We then build classes for more specific types of contacts such as Organization and Person. A challenge for me has been to identify a clearly obvious equivalent to this design in relation land. This has proven quite elusive because there are several relational designs that seem to qualify. The design that seems most commonly used involves a "type" table of some sort followed by tables corresponding to each class. For the running example, a ContactType table, a Contact table, and of course the tables Organization and Person. Continuing the OO design, and for the purpose of introducing polymorphism into the discussion, let's give the Contact an abstract Name property. The Organization class then might just implement the name property as "pass-through" to some field, while the Person class might combine first, middle, and last name fields.

class Contact
{
  abstract string GetName();
}

class Person : Contact
{
  string FirstName;
  string MiddleInitial;
  string LastName;
  override string GetName()
  {
    return FirstName + MiddleInitial + LastName;
  }
}

class Organization : Contact
{
  string OrganizationName;
  override string GetName()
  {
    return OrganizationName;
  }
}

In the working relational design, arriving at polymorphic behavior is not so natural. If we could assume that the relational system for which we are designing has "code" as a data type, a GetName column could be placed in the ContactType table. To view contacts with their respective names, the Contact and ContactType tables could be joined and the GetName routine invoked in an extension operation.

table ContactType
{
  ID : ContactTypeID;
  Description : Description;
  GetNameOperator : operator(AID : ContactID) : String;
}

table Contact
{
  ID : ContactID;
  Type_ID : ContactTypeID;
  key { ID };
  reference Contact_ContactType ...
}

table Person
{
  Contact_ID : ContactID;
  FirstName : String;
  MiddleInitial : String;
  LastName : String;
  key { Contact_ID };
  reference Person_Contact ...
}

table Organization
{
  Contact_ID : ContactID;
  OrganizationName : String;
  key { Contact_ID };
  reference Organization_Contact ...
}

constraint MutualIncl ...

constraint MutualExcl ...

view ContactView
  Contact
    join (ContactType { ID Type_ID, GetNameOperator })
    add { GetNameOperator(ID) Name }
    { ID, Name };

This might provide similar behavior, but doesn't seem as natural. Why not? I am exploring this question; here are some somewhat random observations:

ContactType is constant like the other propositions formed by the application's "code". There will always be two rows (Organization and Person) in the ContactType table, whereas the other tables contain variable information. It may not seem to harm anything to use a variable to represent a constant, but this is effectively placing code with data, perhaps forming a part of the reason for the apparent strangeness. As an aside, a table constant could be defined in the Relational Model (RM) as is through a table constrained to equal a table literal, but it seems that a true table constant would be a natural feature. I have often wanted enumerations and/or constants in D4.
A Contact is an abstract entity, therefore it doesn't make sense to have just a Contact; a specific type of contact is required. With several somewhat awkward constraints, a relational design can enforce the notion of mutual inclusion and mutual exclusion [shown by D4 tutorials], but the system still lacks the "is a" information that is know to an OO system. Without this information, the system cannot automate this commonly occurring pattern. For instance, if a user attempts to add a contact, rather than furnishing a complicated error after the user has filled out the contact form, a more useful behavior would be to allow the user to choose the type of contact to add.
Strictly speaking, the Contact table is redundant unless it introduces it's own non-key attributes. The Contact table could exist as a view based on the union of Organization and Person. Given a sophisticated relational system, this could logically behave identically to the shown relational design. Given a really sophisticated relational system it could even physically behave identically, though Physical Data Independence (PDI) is a topic for another day. Given these seemingly equivalent designs, what is one to conclude about abstract types?
Key values and object references have different semantics. Consider these points where a relational system differs from an OO one: Having a contact ID value doesn't guarantee having a contact row. Operators can be defined against a contact ID, but operators must access global state to actually refer to the referenced row(s). If a contact's key is compound, operators against a contact must take multiple arguments. In short, the relational design uses explicit, surrogate values to reference global logical entities, whereas an OO design uses local implicit references.
The OO design encapsulates the notions of Contact, Organization, and Person into a scalar form. The relational design leaves these entities as the nebulous product of various components (tables, references, constraints, operators, etc.). The types have not truly been described to the relational system as logical entities. For better and worse, the relational design yields complete transparency. This transparency lends itself to perspective independence and the analytical abilities that follow. On the other hand, encapsulation has also proven itself the great simplifier.
What does it mean to have operators as a type in a relational system? What higher order issues does this raise?
Code is obviously data, but not all data exists at the same level of discourse. As mentioned previously, the ContactType table seems at the meta-data level with say, the Person table. This is a similar relationship to the one that exists between the objects in the database and the reflective "system catalog" table variables that describe them. So why does the ContactType table in the design exist at the same level as the rest of the "data" while other application operators and such go with the "catalog"? Is there some general mechanism needed in a relational system to describe meta-data?

More recently, rather than trying to tie the two while leaving the RM as is, I've been considering how the RM might be different with a more "liberal" type system. For instance, imagine something like this:

type Contact
{
  ID : ContactID;
  abstract GetName() : String;
}

type Person : Contact
{
  FirstName : ProperName;
  MiddleInitial : NameInitial;
  LastName : ProperName;
  override GetName()
  {
    return FirstName + MiddleInitial + LastName;
  }
}

type Organization : Contact...

type Entity //..for whom we are tracking contacts
{
  Contacts : table { Contact : Contact; key Contact.ID };
  ...
}

The "database", would then simply be a shared variable ("instance") of type Entity (call it Database rather than Entity if you wish). I will not pretend that such a proposal doesn't deeply affect many of the assumptions made by the RM; regardless I see no harm in exploring them so long as it is not done in ignorance of the implications. So what are some of the implications of the above example?

A key is defined on an expression, rather than an attribute list. This might seem like heresy, but under some constraints (functional, local, deterministic) I don't see this as a problem. Note that an expression as a key might make functional dependency inference tricky under certain circumstances, but the cases that are currently facilitated by an attribute list would be trivial. This change might be a useful generalization regardless of the other ideas shown.
The single attribute in the Contacts table is declared as the abstract type Contact. In the style of OO this means that the value, or "runtime" type, of each row may differ. At a glance the Person and Organization entities may seem to contain more information than the declared type (Contact) allows for, but as I discussed in a prior post, an alternative view is that there are fewer in the set of more specialized types, but the representations are more specific.
How about relational operators? How are we to do useful operations on a table with a single attribute? One option is to have the system provide alternate representations of the same type. In the case of the example, the system could automatically provide a representation of a table that looks like a table containing the unencapsulated contact. Such alternative representations might be more useful in light of operations such as projection. At the same time, generalizing some relational operators to support expressions rather than just columns may not be a bad idea either. For example:
```
select Contact join MailingList
  by Contact.GetName() = MailingList.Name;
```
(is this still an equi-join?)

Representations may or may not provide a way to avoid all of this and keep the RM and OO perspectives independent. Still pondering...

TTMs Subset types and OOP

2006-08-04T09:11:00.000-06:00

In The Third Manifesto, C. J. Date and Darwen criticize OOP type systems for placing more attributes in subclasses. Intuitively the criticism seems justified, because from a Cartesian product perspective, more attributes implies more possible values. Their point is then, how can a sub-type have more possible values than a super-type? It occurs to me, however, that there is a subtlety being missed. The domain of possible values may be fewer in sub-classes, so long as the additional attributes are considered as part of a more "detailed" representation. In other words, the original representation holds at the level of the super-type (which is utilized in polymorphism), but a more detailed representation may be used in sub-types. Such mangling of representations is not allowed by their suggested type system, but the isomorphism seems to fit situation for OOP. None of this is to say that the OOP model doesn't have its flaws, I only wish to point out that there may be multiple ways of interpreting OOP from the perspective of their type model.

Actively managing missing information through Condition Propagation

2006-07-27T08:24:00.000-06:00

Few dead horses have been beaten as severely as the "missing information" one, yet I shall now drag it out and resume the thrashing. When arguing matters of null, discussion usually centers on "optional" columns in tables, but I wish to start on constructs that seem to me to be related, yet are rarely if ever discussed as such. Consider the following:

Points[10];

As I see it, there are four primary ways in which a system may deal with the range issues surrounding the indexer:

Always give a point value even if this means interpreting memory beyond the actual items allocated for the array. This is the approach taken in some cases by many pointer based languages.
Implement the indexer as a condition check followed by exception propagation logic if the condition fails; in other words, at runtime stop processing at whatever level and raise an error. This is the approach taken by most modern languages.
Implement the indexer with a runtime range check and return the null non-value (extending the core logic system to include null).
Propagate the range condition as compiler meta-data and provide compiler level mechanisms for dealing with these conditions.

Option 1 is clearly problematic as it requires no less than perfect diligence to ensure correct results. In other words, the compiler and runtime are no help in ensuring that all range conditions are satisfied.

Option 2 has the benefit of ensuring range checks, but has disadvantages including:

Condition checks are often duplicated; the developer must check them to avoid exceptions, yet the runtime checks them again.
Exceptions only surface at run-time; no help is given at compile-time regarding a context's conditions.
Exception propagation can only be managed imperatively; there are no means, for example, to interpret exceptions within an expression.
What really is an exception anyway? In other words, if any exceptions are actually anticipated, are they really exceptions? So what is "planned" behavior and what is an exception. This sounds merely philosophical, but this plagues today's pervasive languages with practical problems.

Option 3 is basically the stance taken by most "database" languages such as SQL and current D4. The approach does have the advantage of allowing a more optimistic design approach by propagating the missing information through the expression, but:

All missing information is lumped into a single "null" non-value; there is no visibility regarding where the null originated or why.
Nulls are forcefully "converted" at certain language boundaries and cause violations at others.
No compile-time information is provided as to where nulls might originate and whether those points have been addressed.

Option 4 has been alluded to in various discussions, but to my knowledge no concrete proposals or implementations have been produced. It is this option that I wish to explore. The heart of the concept is to treat missing information as a primary concern in the development process; something recognized by the compiler and considered by the developer; like types, variables, and the rest of the language elements.

Under Condition Propagation missing information is explicitly noted and antecedently avoided. Rather than extend logic systems or stop execution at run time, the compiler in-effect ensures that expressions are never evaluated that do not satisfy the conditions. The logical implications of this are significant. Unlike null-like solutions, operators, and thus truth-tables, remain logically unaffected by the missing information. For example, the AND truth table remains in it's traditional, four entry, form. Each primitive operator may publish compiler meta-data regarding the conditions under which the operator is able to operate. With this information the compiler ensures that any conditions are resolved prior to evaluation. The compiler also propagates the condition meta-data to allow the developer to choose at what point prior to evaluation to resolve the condition. Logic, arithmetic, relational and other operations can all be defined in their true logical form, unaware of missing information because they will never be requested to deal with it.

For example's sake, assume a compiler supporting Condition Propagation and consider the case of Points[10]. The compiler "knows" that indexing into points introduces a condition--namely that there must be at least 10 elements in the array--and that the expression can only produce a value if that condition is satisfied. We'll call this condition PointsInRange. This information is noted by the compiler and propagated to the next operation in the expression. In the example, the next operation will be a call to Translate:

Translate(Points[10], 5, 5)
(translate returns a point shifted by the given x and y deltas)

The Translate call inherits the condition: PointsInRange. Stepping back, this makes sense because Translate in this context could not provide a value unless the points indexer provides a value. Let's now resolve this condition so that an unconditional result value is produced.

Translate(Points[10], 5, 5)resolve PointInRange as Point(0, 0)
// result when indexer is out of range: Point(0, 0)
(hypothetical syntax for illustration)

Allowing the condition to propagate to the Translate before being resolved has significant logical implications versus addressing the condition at the indexer, as in:

Translate(Points[10] resolve as Point(0, 0), 5, 5)
// result when indexer is out of range: Point(5, 5)

This type of propagation can be useful for many reasons and is often cited as an advantage of null based systems. Primarily, the propagation allows for an optimistic coding style while still allowing conditions to be resolved. Consider this example:

Parse(String : String) : Integer

In a language with exceptions, an unparseable string would typically be handled by throwing a runtime exception. In a null supporting language, the function might return null. Due to the rather restrictive nature of exceptions, and the realization that an unparseable string isn't really an exception but one anticipated outcome, another pattern is often advocated:

TryParse(out ID : UserID) : Boolean

This pattern does make the parseability of the string explicit as the return value, but suffers from being hopelessly imperative. Clearly the user ID cannot be used until a subsequent statement in a block. Note that this is the pattern recently utilized and advocated by Microsoft in the .NET Framework 2.0.

What happens if conditions are never resolved? There are several possible options, including:

The compiler could address unresolved conditions at expression boundaries. It could, for instance:
Generate code to throw an exception. Once out of an expressive realm, exceptions are more useful than they are within an expression.
Raise compiler warnings and/or errors.
The compiler could pass the condition meta-data on to imperative operations, for instance:
Assigning a conditional expression to an optional attribute could have the effect of clearing the optional attribute if the condition is not met.
A failed condition at run-time may turn the imperative operation into a no-op; or perhaps more generally:
Branch or switch based on conditions.

It seems possible that language support for Condition Propagation would significantly automate many processes developer's manually handle without such support. A simple example might be:

var i := Parse(SomeString)
  when not StringParsable then
  begin
    ShowMessage("Invalid integer value, try again");
    return;
  end;

In this example, the "when.." clause is part of the assignment, allowing branching for unresolved conditions in the expression. In the case of a string not being convertible to an integer, the assignment would not be performed and the conditional block would execute. The effect is that not just functional operators can be kept from having to encounter non-values, but imperative ones as well.

The language should also have the ability to operate on conditions directly, such as:

if Points[10] satisfies PointsInRange then ...
(Returns true if there are at least 10 points in the Points array)

The optimization possibilities seem immense for Condition Propagation. In this example the actual evaluation of the expression can be pruned from the execution plan. In this case, only the PointsInRange condition needs to be evaluated.

Note that regardless of the expression the compiler must perform to ensure a condition, the condition is always dealt with as a simple Boolean value. The compiler generates the actual code to evaluate the condition.

It should be possible to preserve both the value and the conditions of an expression in an imperative context:

var p := Points[10];
select p satisfies PointsInRange;

In the latest example, the compiler determines the type for p based on the expression Points[10], but what type is it that can preserve the PointsInRange condition? This is a trick question, as it isn't the type that is maintaining the condition, it is the variable. The "manual" variable declaration might look like this:

var PointsInRange : Boolean;
var p : Point when PointsInRange;

If p receives an unconditional value (p := Point(0, 0);) it is clear to the compiler that PointsInRange must be satisfied or it could not receive a value. The compiler can thus maintain the conditional nature of p, even in an imperative context. If single variables can be conditional, how about attributes within a tuple (AKA row, record, struct)? Let's call such a structure a sparse row:

row
{
  Name : String;
  HasHair : Boolean;
  when HasHair
  {
    HairColor : Color;
    HairLength : Length;
  }
  when not HasHair
  {
    IsScalpWaxed : Boolean;
  }
}

The Name and HasHair attributes are required, whereas the other attributes are conditional on the HasHair attribute. When leaving the expressive realm, it becomes important to make conditions explicit data rather than just compiler meta-data. Otherwise, the conditional information would be "hidden" and the resulting solution could not be deemed complete from a data management perspective. Note that conditional attributes like HairColor and HairLength in the above example are typically represented in systems without making the conditional relationship known to the system. Relational systems, do provide a means to describe such conditional relationships through cardinality relationships and constraints. It could be argued that because a relational representation of the design is possible, there is no need for the extra complexity of a sparse row. This might be true if the only useful representation of the information were relational, but this simply is not the case. The relational model itself builds upon tuples and scalar values yet in order for the Information Principal to hold, there must be relational representations of information for which there are also tuple and scalar representations. A Sparse Row can therefore be seen as another logical representation of relations having certain cardinality relationships. It is worth noting that a sparse row is also a much more concise short-hand for what would actually be a rather complex relational schema. I will save more discussion of this point for another day.

var c := MyPerson.HairColor;
MyPerson.HairColor := Color.Green;

"Reading" the value of a conditional attribute, such as HairColor in the example, introduces a condition to the compiler, but what happens when a conditional attribute is written to? The answer of course depends on how assignment is defined. A sophisticated language might support the notion of updating the condition to make it true. Updating through scalar expressions is trivial for cases such as an identifier reference and simple unary operations (such as NOT in the example). So the effect is that setting HairColor would ensure that HasHair was true. Other semantics are also possible so long as the language maintains the integrity of the structures.

The examples thus far have explored a few cases; Let's explore other missing information plagued areas where Condition Propagation might be applied:

Row extractors (the extraction of a single tuple from a relation value having only a single tuple). It is useful to define the extraction condition as two conditions: 1. The table must not contain more than one row; 2. The table must not contain zero rows. A relational compiler can detect that more than one row is possible via a check for the "empty" key (a key with no columns). Thus, row extraction would only need to originate a ">1" cardinality check condition if the source table was not this way, singleton. The other condition, that the source table not be empty, could also be suppressed if a corresponding constraint could be identified.
Divide by zero. In computer math, it is most accurate to describe the results of a divide by zero as "not a value". Condition propagation seems perfect for representing this because expressions that do not satisfy the conditions are never evaluated and hence will always have a value.
Outer joins. The row type of a table resulting from an outer join can simply be a sparse row with the outer columns conditioned on an "existence" attribute. Note that D4 already has the ability to include such an existence attribute as part of an outer join operation.
Aggregate Operators. Aggregate operators such as Sum, Min, and Count face two missing information problems. One problem traditionally has been the possibility of missing information in the source information. For instance, should Count count a null "value" or not. This problem is clearly addressed by Condition Propagation because like all other operators the aggregate can always assume "real" values to operate on. The other missing information problem faced by aggregate operators is an empty set as a source. What, for instance, is the average of an empty set? Condition propagation again seems to address this issues because aggregate operators that require non-empty sets can originate this requirement as a condition.

I know that those who closely monitor relational theory are likely squeamish at what might seem like "extensions" to the relational model in order to have tables based on such things as sparse rows. I can understand this reaction, but consider that, again, the "sparseness" is removed before the operators even operate. Thus, all relational operators will only be asked to operate on "regular" rows. Another way to look at Condition Propagation is that the ability for things to be conditional exists at a higher level of abstraction, another logic system if you will, super-imposed over a more "pure and simple", realm of logic in which values are never missing when they are expected.

On the topic of usability, a compiler supporting Condition propagation might "know" what conditions exist at each point in the code, but how is the developer to know these while coding? The compiler may throw errors and warning, but this information would be much more helpful while the developer is actually formulating statements. I see this problem being addressed in essentially the same manner that modern development environments aid developers in terms of data types. Conditions could be visualized by a mechanism similar to "intellisense", whereby the development editor automatically displays annotations indicating conditions.

Optimization was briefly mentioned earlier, but it might be worth discussing in a little more detail. Consider alone how many times applications perform duplicate bounds checking. Strongly typed runtime exception supporting languages, for instance, are performing range checks each time the developer accesses items in an array. This is in addition to the checks already being done by the developer. The only way to avoid this in such languages is to invoke the operation, catch any exception, and perform condition logic that way. In reality, this is not practical because a) exceptions are purely imperative; b) the said approach is very cumbersome; and c) exception propagation is complex by nature and thus the resulting solution is going to be dramatically slower than a condition check.

Though there are clearly code generation optimizations to be had, there seem to be even greater possibilities for logical optimizations. Nulls and exceptions introduce almost insurmountable obstacles to even basic logical optimizations. A tautology or contradiction check, for instance, fundamentally assumes 2 value logic (with emphasis on the word value).

Further research is obviously needed. There are likely to be many issues that arise during design and implementation of a concrete language based on Condition Propagation. Questions, for instance, might include: what happens to conditions in the presence of non-functional operations (side effects)? Must conditions be resolved before invoking such operators or can they safely propagate through them? Can all conditions be deterministic? How should the compiler derive names for conditions and keep them from colliding? Questions aside, the approach seems promising, and I look forward to exploring it further.

Virtual Appliances... poodoo

2006-06-27T15:30:00.000-06:00

What I wish to address is this:
"With the rise of virtualization, applications will transform into virtual appliances. Virtual appliances simplify application deployment by delivering the complete software stack—from OS to the application—as a single, integrated unit." -SDTimes Web Seminar Abstract

Lest I be seen using VMs and am accused of being hypocritical: Virtualization is great for configuration testing. There is no better way to test against Oracle or Windows 98 without having to install such icky things than a VM. "Virtual Appliances", however, are a different story. Though they may “seem” exciting, and may be a practical solution given today’s technology and today’s problems, I find the trend disturbing. Why? ...Because a) the entire approach is grossly inefficient; and b) this is just another chapter in the long history of industry repetition. Setting aside the inefficiency aspect for a moment, isn’t this form of “run anywhere” virtualization exactly what Java set out to accomplish? “Write once, run anywhere” was the mantra, but what happened?
As I see it, the blame lies in:
1) Improper or incomplete implementations. This was exacerbated by overly complex design and poor specification making implementation difficult;
2) Changes in the platform. The platform should really be more of a constant than a variable. Too many versions and too much change turned “write once, run anywhere” into “write once, run where you have exactly XYZ version of the runtime”;
3) Performance problems. Only recently did Java even enter the C performance ballpark. Who wanted to “write once, run slowly everywhere”?;
4) Politics. Sun didn’t seem to find it important to keep everyone on board (e.g. Microsoft).

Doesn’t it seem possible for machine-level virtualization to succumb to some of the same vulnerabilities?

Now for the inefficiency aspect: I see running a virtual machine for the purpose of hosting a single application roughly equivalent to building a city for one person. Layers upon layers of infrastructure for running multiple processes, hosting hardware, sharing memory, et. al. basically squandered on a single application. Now hardware vendors are spending much of their resources optimizing the hardware to support these inefficient monstrosities rather than further optimizing the abstractions we already have. How many technologies are being worked on to make VMs interact with each other now in ways that have long been possible between OS processes? I realize that I am being a purist, but this usage of virtualization just seems like such an admission of defeat. The touted benefit is simplicity, but I predict we will end up with more complexity.

As I see it, there are two problems that virtual appliance aims to resolve:
1) Dependency complexity
2) Portability
Can't we solve these problems fundamentally rather than resort to such grossly inefficient architecture?!

Normalized polarity

2006-05-05T16:49:00.000-06:00

Though strikingly obvious to me now, I've often sought a quick (expressive) way to determine the normalized polarity of a number. What I am esoterically calling a normalized polarity is: a function that is 1 for all positive numbers, 0 for 0, and -1 for all negative numbers. I have often found the normalized polarity useful in financial and graphics/layout contexts.

The now obvious solution is to treat zero as identity, otherwise divide the number by the absolute value of itself.

create operator NormalizedPolarity(AX : Decimal) : Integer
begin
result := if AX = 0 then 0 else (AX / Abs(AX)).ToInteger();
end;

Examples (using D4):
select NormalizedPolarity(-2.3); // -1
select NormalizedPolarity(554.1); // 1
select NormalizedPolarity(0); // 0

Sorry if this seems painfully obvious, just thought I would share in case...

Shakespear Programming Language

2006-04-25T15:23:00.000-06:00

Wow... and I thought VB was verbose!
http://shakespearelang.sourceforge.net/report/shakespeare/

How to destroy the Earth

2006-04-25T15:21:00.000-06:00

I thoroughly enjoyed this, though the author did present as fact, many of today's tenuous theories (e.g. animatter, microscopic black hole!).
http://qntm.org/destroy

Source Control Inheritance

2005-12-01T15:50:00.000-07:00

I like subversion, mostly because they defined a small set of primitive concepts and build on them. Too often the temptation to add exotic features leaves source control products overly complex. All that said, there are some facets of Subversion that are a bit clunky. My suggestion is to extend the paradigm to include the notion of inheritance. Multiple inheritance would be best. Basically rather than just copied branches, there would also be inherited branches. Specifically, a branch could be given one or more ancestors. Changes made to ancestors would automatically be applied to the branch. Granted much careful thought should be given before integrating such a radical change, I think that this capability could be provided as a layer above the existing primitives. If so, the automation afforded would be great!

Welcome

2005-06-06T12:40:00.000-06:00

My old blogging software was deleting my entires. Needless to say this can be quite disconcerting, so I was compelled to find a new blog host. I finally got around to actually doing this and so here we are. I am splitting my old categorized blog into two uncategorized ones: Ten Ways to Sunday - for technical/scientific stuff (tenwaystosunday.blogspot.com); Absolutes Are Evil - for everything else (absolutesareevil.blogspot.com). Enjoy!

The Horror of XML (or Insolence Part Deux or Fabian Pascal v. Slashdot)

2004-11-01T10:40:00.000-07:00

If you have followed the ongoing exchange between Fabian Pascal and the slashdot.org community, you might be interested in this link (click title). I try (mostly successfully) to stay away from such arguments. The problem with slashdot and the likes is that there simply is little to no discipline practiced by its posters; most seem to just write the first thing that comes to their mind. That might be okay for socializing (a big part of what slashdot is about), but a nearly impossible forum for any fruitful discussion of topics that require more than superficial thought.

Dataphor Too Oh Are Sea

2004-10-26T10:51:00.000-06:00

Grief, pain, and red eyes and we have a Release Candidate of Dataphor 2.0. We are close and we mean it this time. ;-) Give us a couple more weeks to clean-up the docs, fix a few more bugs and we will have a final 2.0 release.
Watch out for changes with row and column extractors (see the readme). The library upgrade may exhibit a problem with existing catalogs. If it does you may need to start with a clean catalog and re-register your libraries. If the upgrade doesn't work for you and you have to clear your catalog, there is a Frontend upgrade that you will want to run manually to upgrade any forms you may have built (right-click on the Frontend library and select Upgrades). QuickLookup is a container now, not a control and "...align" properties have changed to "...alignment"; the upgrade I mention automatically fixes existing forms to add a TextBox within the QuickLookup and rename the align attributes.
Here is an example of old vs new extractor syntax:
Old:
var LRateTypeRow := row from (RateType where (ID = (RateType_ID from LIncidentRow)) and (Jurisdiction_ID = (Jurisdiction_ID from LIncidentRow)));
New:
var LRateTypeRow := RateType[LIncidentRow.RateType_ID, LIncidentRow.Jurisdiction_ID by { ID, Jurisdiction_ID }];

Sunday deriver

2004-10-14T10:52:00.000-06:00

A few weeks ago we made a change to form derivation that seems to have fixed the few holes that remained in it. The problems were primarily related to multiple references with overlapping source columns. Today, the changes were validated as I was able to change a 20 line complex view into a three line simple one.
I am also fond of the fairly new Singular/Plural pseudo form-type tag qualifiers. Before, if I wanted a tag to apply to the add, edit, delete, and view forms (but not the browse and list ones) I had add a tag with each. Now, I can just use a "Plural" qualifier for browse/list and "Singular" for the others. Like this:
create table Customer { ... } tags { Frontend.Plural.Title = "Customers" };

Dataphor performance boost

2004-10-07T10:55:00.000-06:00

I just witnessed a data entry form that used to take 7-8 seconds to show, take <5 seconds to show the first time and <1 second thereafter. The reason: Plan caching and a shared shadow catalog for application transactions. In-editing server round trips have also been reduced (in some cases to 0) reducing server load and improve high-latency connection performance. This will all be in the next release, along with better behavior for the lookup controls and the incremental search control.

Always a bigger scope

2004-10-05T10:56:00.000-06:00

In the context of a DBMS, there is some collection of data that is persistent, shared, transacted, etc., which constitutes the database. The simplest complete logic system that has been presented for defining and managing this collection of data is the Relational Model (RM). In the RM, all data is ultimately represented by a set of named relation (table) variables. In both traditional 3GL and relational languages, there are also other, more narrow, layers of scope, which describe the context available to a particular portion of logic. Within an expression, there might be identifiers available from the expression, identifiers available from the containing block, as well as the set of global identifiers.
Let's now look at a scoping issue that commonly arises. In this example, let's say we have constructed an accounting system in either a 3GL or DBMS logic system (it doesn't matter which for this analogy, though it certainly would matter in actuality!). Assume that our original implementation encompasses accounting for a single company, and we are subsequently requested to provide accounting for multiple companies. One obvious solution would be to construct multiple instances of the accounting system. By doing this we are in essence defining another implicit level of context to which all data in the system is tied. Another approach would be to refactor the existing system, essentially pairing a company identifier with each existing predicate of the system. This would make the company dimension an explicit part of the system's propositions.
There seem to be advantages to each approach. The multi-instance approach has the advantage of simplicity. Not only does the system not have to be redesigned and reimplemented, the logic of the system does not have to deal with the complexity of multiple companies. A re-design of the system would be far-reaching and would require nearly every facet of the system to change to address the additional company attribute. This is so because the company in question is not part of the implicit context of the executing logic, but an explicit facet of the data. We can draw from this and other examples that increasing implicit context narrows the scope and reduces the domain of explicit data. This in turn, allows more assumptions to be made by the logic and therefore nets simplification. On the other hand, the fact that the data is implicit to the instance is also potentially a disadvantage. For instance, what can be done if a report spanning companies is requested? Also, does the set of postal codes really need to be per-company?
In the current example, the multi-instancing was provided at the border between logic systems (between the accounting system's logic system and that of its host environment--presumably an operating system). This change in logic systems is problematic at best, especially when one of the logic systems is as dramatically low-level as an operating system. It is not the fact that we have contextualized the accounting system that makes it potentially difficult for us to construct a report that spans companies, it is the fact that we have done so in a low level system. Can instance-type scoping then be accomplished within a single logic system? Object Oriented Programming (OOP) languages provide one example of just such a logic system. In OOP languages, instances ("objects") contextualize units of logic ("classes") such that maximal contextualizing ("encapsulation") occurs. This allows reduction of explicit data and simplification of code. For instance, I might have logic that manipulates accounts. If that logic can assume a particular company and even a particular account, then the job of creating debit or credit transactions becomes simpler. It also insulates logic from more general changes. On the other hand, there are times when the very nature of the logic at hand is to deal with broader levels of scope, such as multiple accounts or companies. This task might prove difficult in a logic system that assumes maximally narrow scoping (such as OOP languages).
Can we then extrapolate from all this that appropriate scoping is a facet of each portion of logic, not an explicit facet of the data? This assumption would allow logic to exist at any level of scoping. A ramification of this is that logically all data is ultimately part of one giant collection (sound familiar) within which we are taking some limited perspective. Otherwise the scoping would also be defining limits on the availability of data and would suffer from problems as a result (like not being able to build cross-company reports).
Can all data be considered global then? Let's take another multi-instancing example, but this time let's assume that the instances are on separate computers. Let's say we have an system that analyzes a collection of interstellar "noise" looking for patterns, and we wish to run this system on multiple machines in order to increase the overall capacity. How can we maintain a single logic system across all instance boundaries without including virtually everything that the machine can possibly access in that collection of global data? We surely couldn't copy every available digital fact onto a machine and then keep it in sync!
A powerful concept, "physical data independence" suggests that copying the internet to our machine isn't necessary in order to maintain a single logic system. I am suggesting that perhaps all data accessible to the machine could logically be part of the database, not physically (we are talking about logical topics anyway). As an naive example, imagine a query like this:

select WebPage where URI = 'http://www.alphora.com/';
Now I must point out that at we are treading on dangerous turf and must watch our step. There are quite a few restrictions and assumptions made for good reason in the context of a DBMS, and so we should carefully consider each of them before naively throwing a mishmash of non-deterministic mayhem into the definition of the database. Nevertheless I am going to ignore these considerations for the present time in order to remain in the realm of broad strokes.
Ending my post, I deduce thus far that a logic system that provides a complete computing environment (an operating system) would need to provide at least the following:

A mechanism whereby the scope for a given portion of logic can be explicitly narrowed or widened. I see this as more than just providing local scopes within operators and expressions. This item may not technically be a logical need, but I do suspect it would be a practical one for several reasons.
A global set of relation variables logically encompassing every possible fact available to the system.

What fun!

Watch out for SizeF.ToSize()

2004-10-04T11:17:00.000-06:00

I've been bitten several times lately from code that performs a MeasureString call, which returns a SizeF structure, then converts that SizeF to a Size using the ToSize() method. The problem is that ToSize() performs a standard round on the dimensions. When measuring a string, a ceiling is usually the desired behavior but there is no Ceiling() member method of SizeF. Ceiling() rather, is a static method of Size, so this:
s = g.MeasureString(Text, Font).ToSize();
becomes:
s = Size.Ceiling(g.MeasureString(Text, Font));
Just wanted to share this possible caveat.

More on key column updates...

2004-10-03T10:58:00.001-06:00

Upon speaking to Bryn, I found out that my last post wasn't quite correct. The alternative update mechanism is chosen if the update is to a key column and the update is not context literal (the meaning of that is another story). For example, an update such as:
update Customer set { SSN := "333-33-3333" } where SSN = "222-22-2222"
(assuming that SSN is all or part of a key) will not require the "doopty doo dance maneuver" and Dataphor detects this.

Database scoping

2004-10-03T10:58:00.000-06:00

I've been struggling with the concept of database scope for some time now and I think it's time to put some resulting thoughts down, a) because it will force me to formalize them somewhat, b) so they do not get lost in shuffle of my thoughts, and c) to hopefully build a base upon which myself and others might build.
Before starting, I must admit that I have done no formal research in this area and for all I know there is an entire body of research already encompassing these ideas. That said, I have not seen such a field of research, and so conclude that if there is a body of material that covers these matters, it is obscure and should be brought forth. These "thoughts" could easily turn into a book, so I will probably distribute this over several posts. I'll start in this post by defining the problem.
The problem is that of applying database principals and specifically the relational model beyond the scope of a "shared databank" or Database Management Systems (DMBSs) as we have come to know them. Many of the ideas surrounding such systems assume (and appropriately so) a central, monolithic set of data that is manipulated, analyzed and shared between users. The systems provide extensive services to users by providing an abstract "data model" which serves as the bases for the operations that the system performs. In this approach, however, the logic system embodied in the "data model" can only be used within the DMBS leaving the rest of the application logic to reside in other logic systems. This "other" logic system is usually that of a third generation language (3GL), perhaps augmented by a "component toolset".
Let me remind the reader of the many benefits of maximizing declarativeness in application development and data management, such as fast development, enhanced quality, and agility to change. It is therefore desirable to bring the highest degree of declarativeness to the entire application, not only the portion of the application involving shared data access. Because of this, the most we could ever hope to achieve in the pursuit of the perfect DBMS is the enhancement of one portion of the overall application.
To this end, we should seek a single logic system (I'll avoid the term Data Model as it may narrow our thinking) that can be used for data management and application development within a DBMS as well as without. At first glance at least one portion of the Relational Model seems not to fit if taken beyond DBMS architecture, namely the Information Principal. This principal states that all data ultimately resides in a named set of relations (tables). This leads to other questions: What would it be like for the server and the client to be relational? What would a relational operating system be like? I would like to explore such questions.

Extension and keys...

2004-10-01T11:07:00.000-06:00

I hadn't really given it much thought, but it turns out to be pretty important that a relational DBMS be able to property track keys that are sometimes added during an extension. A trivial example:

select TableA;
ID
---
1
2
3
4

create view V
TableA add { ID + 1 OtherID };

select V;
ID OtherID
--- --------
1 2
2 3
3 4
4 5

ID is of course the key for TableA, but what are the inferred key(s) for view V? A not so mighty DBMS might say just { ID }. Thanks to a recent enhancement, Dataphor will now detect (yes even in the presence of n operators as long as they are order preserving) the additional key { OrderID }.
Why does such a thing matter? Keys turn out to be critical in a system like Dataphor where we are automatically deriving user interfaces and updating through even the most complex views. This is something that might not be appreciated until you have built a large application in Dataphor.
Another nifty Dataphor picked up recently is the ability to update key columns in ways that would violate the key at the sub-statement level. For example:
update TableA set { ID := ID + 1 };
This may seem trivial, but isn't so simple to implement even in this simple case. The naive approach is to open a cursor on the table and update row at a time. In this case, such an approach may or may not work depending on the order chosen. In general, this naive approach will fail. Now, thanks to an enhancement, Dataphor will detect that a key is involved in the update... perform a doopty doo dance maneuver and... well... do the update!