|
1. Introduction
Computer software can be made smarter by communicating better with humans, in a realtime, conversational manner. And since humans are so visually-oriented, software tools that communicate visually are even smarter. When I say "smart", I'm referring to general intelligence as evolved in the biosphere to varying degrees over the last 3 billion years or so. I refer to the ability of an entity to sense and adapt to its environment (which can include other entities which it relies on, and with which it must communicate). All animals and plants communicate, whether through pollen, pheromones, or Venn diagrams. Compilers have traditionally communicated to humans by way of textual error messages, often dumped in one fell swoop as an ugly spew. But compilers have generally begun to insert themselves into the creative programming process, within IDE's ( |
Visualizing Nested Containment ![]() Figure 2. Two representations of nested containment: one is visual, and the other is textual |
Since text is one-dimensional, containment must be specified using pairs of delimiters.
The concept of containment is more naturally
visualized using pictures or 3D models, in which the stuff being contained is shown inside of circles, boxes, etc.
When containers
have containers inside of them, it becomes even harder to visualize in one dimension.
The same might be said of temporal information, such as music, cinema, and orally-narrated stories. It is not just the eye-brain that processes visual containment naturally: the ear-brain also processes containment through time. But more accurately, normal experience entails a sensorium of visual, tactile, and aural stimuli, which all contribute to our generating internal representations of containment. |
Figure 3 below illustrates
|
![]() ![]() ![]() Figure 3. Examples of L-Systems |
Could this be because such structural properties are inherently
higher-dimensional phenomena? Or is it just that we (visually-oriented, and pattern-recognition-oriented
humans that we are) are not well-equipped to see such structure in a 1-dimensional format? Many of
the difficulties in computer programming are due to the fact that software is, for the most part, linear and
sequential, and structural attributes such as nested containment are not easily seen or manipulated in
low-dimensional representations. This is the reason why we format our text, which, after all, occupies
a two-dimensional space. And it is no surprise that programmers
can be very religious about how one should format their code.
Putting Delimiters in a One Dimensional Space Many of the difficulties encountered by people who are new to programming are centered on the fact that human thinking is generally spatial, non-linear, imprecise, multi-modal, and inherently parallel. It takes work for new programming students to get used to the hyper-linear nature of code. And so programming students tend to make many errors associated with using delimiters ( brackets [ and braces ] of all kinds, [not just the { curly } ones ] ). Notions of simultaneity and containment are deep in our pre-language firmware. And so we easily think about and talk about things being inside of other things. And we format our code to reveal this to the eye. For the same reason that print media uses visual layout conventions to convey information in, say, a magazine, software programs are formatted to enhance human readability, and express hierarchy. Compilers, by contrast, are very non-human (by design), and are not good at seeing overall patterns. They do a splendid job at catching errors that we would never notice in a million years (thank goodness). But they do not have a human Gestalt on the code. And that's why we end up with the strange behavior described below. 2. When a Missing Curly Bracket = = a Spew of Strange Errors When I started programming in Java, many of the annoyances of C++ went away. But some remained. For example, if I forgot to add a curly bracket, say, at the end of a new block I had created, or, if I removed a block and forgot to remove a curly bracket, the compiler would come back with some strange errors - often a huge smelly dump of them. Similar error-spew happens when I forget a semi-colon. These days, I can pretty quickly guess that it is one of these kinds of errors because I can see the telltale signs: namely, the compiler is complaining about some code way the hell down near the bottom of my file - a part of the code I haven't touched in a long time. The telltale sign, in other words, is marked stupidity. My inner-voice says, "can't the compiler see that the error is right here, where I was just coding?" But that's just me thinking like a human and not like a computer. Bad, BAD me! Now, I have discussed this with many computer scientist friends. Many of them speak in defense of the compiler. The defense is that the compiler only knows how to traverse the code, line-by-line, in sequential fashion, and so, when it encounters a closing curly bracket, it automatically associates it with a specific opening curly bracket it encountered earlier, using the logic of nested containment as dictated by the language rules. This can often start a cascade of errors, because once the compiler hits a discrepancy, it can be thrown out of sync with the code's intended logic. We human programmers know perfectly well the context within which these curly brackets have meaning, and do not fall into the same trap while reading it. But the compiler doesn't know this context. Indeed it can't. Or can it? A Conversational Compiler Compilers can make suggestions. For instance, with a multi-pass compiler, after its first pass on the source, a compiler can say: "hey, I just detected a discrepancy in the nested ordering, or pairing, of curly brackets. And since you last successfully compiled, you have only changed this region of the code, and so, could it be, dear human-programmer, that you have a curly bracket error in this area?" I can easily imagine the programmer's response: "Ah! thanks. You're right - silly me". The programmer then fixes it, and avoids the dump of silly, meaningless errors. The irony here is that when a compiler spews such silly errors, it is actually spewing IT'S OWN ERRORS (from the human point of view). Also, IDE's can store information about the recent history of the code, and establish "good guesses" as to the programmer's intended block structure. For instance, a block of code delimited by curly-brackets may have persisted for over many iterations of successful compiling (using some heuristics). If the compiler then detects a discrepancy in curly brackets, it can use this history to make a reliable guess that the programmer inadvertently made a curly-bracket error. But this is tricky - because when the programmer actually does intend to re-structure the code, one would not want the compiler to intrude. This is the rationale for the compiler making "gentle reminders", which can be easily ignored. Yet another possible clue to programmer's intent is analysis of indentation, which follows some general conventions for visualizing blocks. This is used in a very precise manner in the language Let us consider "compiler intelligence" as being within the context of an IDE and the programmer who uses it, just as animal and human intelligence have meaning within the contexts in which they are embedded. A compiler may occasionally make an educated guess, a nudge here, a warning there, and express this to the programmer before it spews robot-like insanity. A superior IDE interacts with the programmer - tapping his/her innate intelligence, and flows with it. You've probably heard of the Theory of Multiple Intelligences. These include EQ ( But there's an aspect of all this that I haven't talked about yet: a text-editor with visual behavior. Let me explain what I mean by visual behavior - below. 3. Making Code Blocks Clearer In example 1 below, the placement of curly brackets uses a convention favored by many programmers, that is, to place the open-curly bracket at the end (right-side) of the fist line, and then to place the close curly bracket at bottom-left of the block, indented with the first line. This is sometimes referred to as the K&R style of |
Notice that the inclusion of these lines does not displace any of the original text. These lines merely
replace the curly-brackets, and occupy the spaces between existing lines of code, similar to the way one might
draw a line around some text on a paper printout of code.
Consider these colored lines as syntactically equivalent to curly-brackets, and that the text editor (like many text editors these days) is able to apply visual layout rules to the text within the context of the programming language (in this case, Java). What I'm showing you here then is a solution which is entirely on the visual text editor side, and of no concern to the compiler - as far as the compiler is concerned, these rectangles are just curly-brackets. The key is that the visual text editor forces the programmer to match up the curly brackets, by way of the fact that a rectangle has (by definition) four corners - the left two of which take the place of curly brackets. This is the kind of thing we see in Collapsing Blocks When Microsoft tries to be user-friendly, we end up with paper clips that dance and won't go away; puppy dogs that try to use cuteness to distract us from their horrible search UI; and Vista windows that do acrobatics. Even basic visual layout is awkward. In Visual Studio's text editor, one can collapse a block of code, and wrap it into a smaller box (as shown below). In my opinion, the collapsed block at right should be below the first line (unless the original block was previously at right). Also, notice that the empty line after the original block is gone, removing the natural chunkification that was there before. The controls at left are difficult to parse visually, and it is too easy to accidentally close a block when the intention is to click in the left gutter. |
![]() ![]() Figure 5. An example of a block of code collapsed using Microsoft Visual Studio's text editor. |
Microsoft has the right idea, though. I'm sure there are other text editors that allow chunks of textual info to be
collapsed in a similar way so that the code-reader can see structure at-a-glance. To me, this is a very interesting subject.
And I look forward to seeing more techniques along these lines.
Text Goes Postmodern Compare software programming to print media, like magazines. Over the last century, printed media have included more punctuation, text layout, images, font styles, and mashing of text and image into a post-modern blur. This evolution from old-school text as dry, left-brained, and linear, to visual and interactive is a natural trajectory for technology, society, and art. I believe that computing (meaning, all activities that we associate with using computers) will continue on its current trajectory from the old typewriter-ergonomics, to include more modalities, multimedia, and ergonomic variety. Software programming, then, will not suddenly break free of its textual origins and become visual/interactive as a result of some company like Borland or Google finally getting it right. Instead it will evolve, gradually over time, as it has been doing, and become more visual and interactive, in the same way that textual media has. 4. Conclusion I would not be surprised if there are already text editors that allow visual blocks to take the place of curly-brackets, and which can actually be parsed the same way as curly-brackets. This would have the advantage of making it easy for code-readers to see structure at a glance, and more importantly, insuring that delimiters always come in pairs. And I would not be surprised if there are also multi-pass compilers for modern languages which are able to do a gentle first-pass on one's code to detect discrepancies and to check-in with the programmer before doing The Spew. For sure, realtime, background compilers exist and they work in collaboration with text editors with visual behavior to reveal potential errors and warnings as the programmer codes. In short, programming environments are becoming more conversational and interactive. Basically, they are becoming more SOCIAL, as well as more VISUAL. And to me, that's intelligent. |