Arc v' May 2002

Some mornings, one just needs to be reminded that it's not a good idea to reach over an open gas stove flame (to get the kettle to put on said flame) while wearing a loose heavy terry cloth bathrobe.

Unless one really does intend to set ones self on fire.

Postscript: I'm OK though - the robe doesn't even bear any burn marks, and I did stop the flames before they got to my hair. Besides nothing really wakes you up like the sight of fire dancing across your arm. On mornings like this, though, I feel like Charlie Brown.

Charlie Brown:All I can make is cereal and maybe toast.

Linus:That's right Charlie Brown, I have seen you make toast before! You can't butter it, but maybe we can help you!

(Quote lifted from the Charlie Brown Thanksgiving. Speaking of Charlie Brown, flip through this tribute to Charles Schulz by a large section of top editorial cartoonists.)

Low End Mac published results of a survey about What we hate on the web.

An interesting one was:

Articles with unidentified authors
729 votes (164/245/173/111/36). 22.5% don't care, 57.3% dislike it somewhat, and 20.2% hate it when a site doesn't identify its writers.

This interests me because one of my favorite lunchtime reads is The Economist, which very very rarely identifies its writers. It doesn't bother me at all. At the same time, however, it is a rather solid and respectable magazine. In a sense, it's almost a big magazine blog. It has short articles, wide geographical and topical coverage, and is as timely as a printed weekly can be.

When it comes to blogs and other web sites, I haven't yet figured out when I care about the authors name and when I don't. In general, I trust The Economist. I think that anonymous correspondent reporting works out well for it, and I think it probably works out for similar styled web sites/blogs as well.

Half of our power is out, but the airport base station and the gateways are still on. So, todays teaser medicine consists of a "Python" translation of some UserTalk code I wrote (the first major script I wrote in Radio), runnable as an External Method in "Zope". Also included is the UserTalk code, as well as some DTML and ZPT output.

I show this because, personally, I loathe coming across HTML in code. Sorry Jon. Writing for templates increases possibilities for reuse, since the data producing code is independent from the code that displays it. While you can do this in pure python scripts (by having one script produce data, and have another script as a consumer that outputs HTML from within Python), I take a "why bother?" attitude. I've had to maintain that HTML generating code in the past - the long ago past in web time (1996). And I hated the maintenance chores that seemed to go along with even the most trivial HTML. Why do we keep reinventing old problems? The script-inside-HTML way is often as broken, in my mind, as the HTML-inside-scripts way. It always surprises me to see ASP/PHP code that looks like::

<table>
  <% for row in data {
    out.writeln('<tr>');
    out.writeln('<td>' + row.name + '</td>');
    out.writeln('</tr>');
  } %>
</table>

Why bother having the HTML abilities of ?SP/PHP languages at all? I can understand why it happens. I just don't think it should. ";->". But, in the words of Dennis Miller, "Of course, that's just my opinion, I could be wrong".

In any case, I present the Goldberg Variations:

import os
radioWwwFolder = '/Applications/Radio Userland/www'
def listOutlines():
	out  =[]
	path = os.path.join(radioWwwFolder, 'outlines')
	def visit(arg, dirname, fnames):
		for fname in fnames:
			if fname.endswith('.opml'):
				fn = os.path.splitext(fname)[0]    # Split extension off
				if dirname != path:
				    # Deal with subdirectories
				    fn = os.path.join(dirname[len(path):], fn)
				arg.append(fn)
	os.path.walk(path, visit, out)
	return out
 
ZPT = """
<ol>
 <li tal:repeat="item here/listOutlines">
  <a tal:attributes="href string:${item}.html"
     tal:content="item">Outline</a>
 </li>
</ol>
"""
 
DTML = """
<ol>
 <dtml-in listOutlines>
  <li><a href="&dtml-sequence-item;.html"><dtml-var item></a></li>
 </dtml-in>
</ol>
"""
 
USERTALK = """
on listOutlines() {
	local(out = "<ol>\n");
	local(path=user.radio.prefs.wwwFolder + "outlines");
	on visit(f) {
		f = file.fileFromPath(f);
		if string.hasSuffix(".opml", f) {
			f = string.popSuffix(f, '.');
			out = out + " <li><a href=\"" + f + ".html\">" + f + "</a></li>\n"};
		return true};
	file.visitFolder(path, 1, @visit);
	out = out + "</ol>";
	return out
}
"""

Jon Udell and Zope Newbies both write stuff along the lines of I decided again not to use them because they would add more complexity to my sites and wouldn't solve any problems -- I'm not about to start using Dreamweaver or GoLive any time soon. It's just me and Pepper or Mozilla. I don't have to work with any designers, or that the learning curve is too steep.

Personally, I think that the learning curve for Page Templates is much smaller than DTML. The only problem is in trying to think like DTML when you really shouldn't be. While there are helpful conversion guides, I do think it best to approach ZPT on a fresh project rather than trying to retrofit them into something existing (by either adding on to a site/project, or trying to replace existing DTML).

Page Templates have a wonderfully clean syntax in the TAL/TALES combination. There are very few TAL attributes, and their operations and operational order are clearly defined. TALES eliminates much of DTML's guesswork by eliminating the whole "is it an expression? a literal? a name lookup?" game. The METAL macro expression system is a little more complicated, but ultimately provides a clean way of componentizing HTML/XML, and can be effectively used to eliminate the standard_html_header/footer idiom (I strongly dislike the whole header/footer inclusion idea proffered by so many templating languages, which require you to maintain balance between two separate files/objects to try to give some notion of "standard look" to a site. METAL is much more along the lines of Dreamweavers and GoLive's Templates idea, where a page has editable "slots" between commonly shared data, and can thus be edited as a whole page).

There are areas that Page Templates break down, especially in comparison with DTML. First, Page Templates are purely about HTML/XML. They're not good for dynamically generating style sheets, Javascript, or SQL, although they can be used to do so (but as the old maxim goes, just because you can doesn't mean you should). Second, DTML has a strong history as a reporting oriented templating language. TAL and TALES don't really take on this responsibility, and rather hand it off to other modules like ZTUtils (Zope Template Utilities), a woefully underdocumented piece of the pie. The Page Templates approach is ultimately better, I believe, because it's more open to new components being used to handle batching and useful widgets, while with DTML you got what was there. The 'dtml-tree' tag has been woefully stretched in order to allow many variations in how an expanding tree might find its data. ZTUtils offers handy utility modules, classes, and functions to replace all of this and allow a developer to better handle how it all might work, but it's currently much harder to do things like dtml-tree in Page Templates.

But DTML has suffered from many problems over its lifetime. Until Python Scripts came to Zope, it was the primary way most people learned how to program Zope. It grew a lot of programming language style muscle, while retaining all of its reporting language heritage. Its namespace stack is a powerful but often untamed beast, and its full of grotesque shortcuts and workarounds. Anyone who's seen _[_['sequence-item']] knows what I mean. And anyone's who's used _[_['sequence-item']] probably had a momentary twinge of "ugh!" before moving on. DTML has gotten the job done. There are places where it continues to get the job done - the SQL specific tags are one of my favorite things about doing SQL in Zope. But DTML is full of accidental surprises - a common mistake is due to the fact that <dtml-var someObj> and <dtml-var "someObj"> can produce wildly different results.

I've been using DTML since early 1997, and it was a joy when I first used it - there was no Zope, there was no through-the-web editing, there were no DTML expressions, there was just a clean templating language meant to be combined with Bobo published objects. All real logic happened in Python, and display oriented logic could happen in DTML. It worked out well. But, as DTML has grown new appendages in an ad-hoc manner over the years since then, it's become more and more of a wild card. Page Templates in Zope, combined with Python Scripts give us back a lot of that early power. I think there's less harm in trying them than avoiding them. They're truly one of the few really clean extensions to Zope - well designed, well maintained. Members of PythonLabs really got involved and did an excellent job ensuring Pythonic standards were met, and early contributors like Todd Coram (who introduced me to Ruby) did an equally good job of driving the design of TAL/TALES to be a fairly open design specification, of which Page Templates are merely an implementation.

"Zope" has many delightful aspects that arise out of its very basic nature as a Web Object Publisher - that is, an ORB (Object Request Broker) that turns HTTP, FTP, WebDAV, and potentially other requests into Python object calls.

There's a really cool trick that arises out of Zope's SQL Methods and Acquisition, where you can go directly to a single result out of a relational database. It's generally known as direct traversal. What it does is basically turn URL Path pieces into parameters passed into a SQL Method. SQL Methods are part of Zope's relational database story - a separate DTML based SQL template (the SQL Method), posing as a callable Python/Zope object. SQL Methods make use of some extended DTML tags to ease dynamic SQL authoring. A simple, but rather powerful SQL Method might look like:

id: getPackage
params: package_id=""
 
SELECT package_id, name, price FROM PACKAGES
<!--#sqlgroup where-->
 <!--#sqltest package_id type=int multiple optional-->
<!--#/sqlgroup-->

When a 'package_id' parameter is passed, the entire 'sqlgroup where' block will render out into a full SQL 'Where' clause - if more than one value is passed in for 'package_id', the sqltest statement will even generate 'package_id IN ...' code. If no value(s) are passed in, the whole 'where' clause will be dropped (in this case - there's a lot of power in the dtml-sqlXXX tags).

The cool tricks come with direct traversal, where you can treat a SQL Result object like a local one. What this means is that a URL like:

../site/getPackage/package_id/2/menu.html

actually causes the getPackage SQL Method to be called, with the value '2' passed in to the package_id parameter. Traversal continues at this point, allowing menu.html to be acquired and rendered in the context of the result of 'getPackage'. Thus, if 'menu.html' were a Page Template, it could contain code like:

<h2 tal:content="here/name">Package Name</h2>
 
<p tal:content="here/price">package price...</p>
 
<a href="packages.html" 
    tal:attributes="href string:${container/absolute_url}/packages.html"
 >Return to package listing</a>

Notice the use of here, a page template context meaning the local context of where the template is being executed. The final line uses the 'container' context, which is where the template physically resides in Zope, to render a URL to get back to the package listing (placing the value of the TALES expression in the 'href' attribute of the anchor tag).

Since this example SQL Method has only one parameter, it can actually be configured to do Simple Direct Traversal, whereby the URL could be shortened down to:

.../site/getMenu/2/menu.html

and "Zope" would match the first value after the SQL Method in the URL to its 'package_id' parameter. Or, you could have URL's with many parameters, such as:

.../site/getMenu/menu_id/2/restaurant_id/4/menu.html

Which would pass in '2' to the 'menu_id' parameter, and '4' to a notional 'restaurant_id' parameter.

There's even more power that can be reaped by using Brains, classes (Python or ZClasses) that can be mixed in with the result to add extra computing power (or other tricks one might figure out to make a simple O-R Mapping system). And ever more power beyond that.

Today, I got to re-experience the benefits of working with Page Templates (a very cool "Zope" technology) via a designer. Page Templates take a fairly (but not totally) unique approach to server side HTML/XML markup - by using special tag attributes, via TAL (the Template Attribute Language). This allows server side markup to be added to a page while having little or no effect on typical design tool (GoLive, Dreamweaver) usage. TAL is coupled with TALES, an extensible system for expressions that can be used in TAL statements.

While I've been using Page Templates for quite some time, it's been primarily for my own work or for administration screens (where they've worked out quite well). Today, the designer who's been working on this project was ready to start uploading his work. After configuring Zope to turn uploaded HTML files into Page Template objects, I was able to open up Dreamweaver MX (although I probably could have and should have stuck with GoLive 6 - which has gotten very good at doing both Code + HTML and has nice local/remote file management - but I'm trying to give Dreamweaver a chance) and plug away at adding a few TAL attributes to the pages that needed them.

It's gone rather smoothly so far. There's a nice combination TAL/TALES expression to remove text from a rendered page, but keep it in the design one. For example:

<ul>
 <li tal:repeat="item here/getNames" tal:content="item/lastName">McKinley</li>
 <li tal:replace="nothing">Macinroe</li>
 <li tal:replace="nothing">Macintosh</li>
</ul>

Using the 'replace="nothing"' construct, I was able to keep his code basically as it was, while still allowing a loop to happen (basically treating his code as example content). This was especially handy in a situation where I had to effectively "rotate" data - where rows in a database had to be displayed as columns in a table. I was able to go down each row, marking up the first data cell to be repeated (actually the second cell in each row), add in a 'tal:repeat' and a 'tal:content' statement, and then add a 'tal:replace="nothing"' to the remaining cells on the row. What I'm left with is a page that still looks fine in Dreamweaver, still draws fine in the browser without rendering, and also renders beautifully. The rest of this very well designed page I left completely untouched. Very cool. This could not be done in DTML, and would be tricky (ie - require a very smart tool) with ASP/JSP/PHP and TEA/Cheetah style templates. And the fact that there are few pages that need server side markup (at this point), coupled with a very rich design, XSLT would have been grossly overkill.

Granted, this process has gone very smoothly because the new design is based off of an existing site that we're being paid to replace. The existing site has driven the database schema design (five tables, all in Gadfly), as well as the general content part of the new site - there are few surprises. The design has been shown to the customer independently of the dynamic nature of the site, and the dynamic parts shown have been rather well designed administrative pages (the customer was pleasantly surprised with them, and quite happy, and made only a single change requirement to them). So, at this integration point, it's just a combination of running GoLive/Dreamweaver in split Code/WYSIWYG mode, and quickly adding in a few TAL attributes into the settling design, while still allowing the designer to make tweaks. And while there are cases where things could still get messed up in regards to the placement and rendering of the Page Template, the chance is pretty low, especially this late in the design.

Ultimately, the designer has been quite happy with the new ZPT driven sites we've been doing, since the developers no longer wrest control away from him once a page is done (with DTML, a page would typically be diced into plenty of smaller parts, bad DTML tricks, and other what-nots in such a way that the designer would never recognize what had been done anyways). And, if something breaks, the Undo/History features of ZODB (the object database behind "Zope") offer good help towards fixing a problem before it gets too bad.

Very Nice.

For the first time in many years, I think I have an apartment I actually like. I've got a sixth floor corner apartment on South Temple in Salt Lake City, and it's actually about eight floors above street level. Looking west, down South Temple, I can almost pretend that I'm really back in Murray Hill / Park Avenue South in New York City. Even the traffic noise is there. And the rent, of course, is far cheaper. And the city is far less fun. sigh. I miss New York.

I'm only halfway moved in. I still need to do the furniture. Then, I can let my body heal.

AP: 1999 Report Warned of Suicide Hijack. That no one cared is no surprise. We weren't paying attention. After Sept 11, people found all kinds of reports on the Web, in public, that warned what Al Qaeda was up to. Hey we're back asleep again. The reports could be out there, but we wouldn't see them. The issue isn't with the government, it's with all of us. We don't want to hear bad news.  [Scripting News]

It's not just "not wanting to hear bad news", it's that reams and reams of information come into the CIA and FBI each day. And besides the fact that they're not equipped to handle it (there is a serious staff shortage), if they threw up a red flag every time something did look suspicious, it would eventually turn into the "boy who cried wolf" scenario. The same people that may be faulting the government now for not doing enough with the information that they had may have also faulted the government for sounding off too many vague warnings.

If the public is alerted too much, with no consequence, they stop believing the alerts. Just because we don't see a bridge being blown up doesn't mean that there wasn't a plot against it. For all of their faults, the FBI, CIA, and other such authorities actually do stop bad plots. There's no recognition of that, for many reasons varying from protecting the identity of the agents involved and the [bad-guy] targets they're after to the fact that images of a crashed plane or destroyed bridge rally many more people around the news sources than a picture of a fine bridge with the caption of "DID NOT BLOW UP TODAY", or a bulletin about "yeah, they caught some guy who was planning on blowing up blablabla". Regardless of how much television news may try to exploit its audience (the whole "if it bleeds, it leads" notion), there is a much deeper emotional connection to a large scale tragedy than there is to a seemingly small triumph.

And of course, the perception of tragedy among both the mass audience and those who feed it varies. Witness the response to the Columbine shooting, which was indeed terrible, in contrast to the response to violence in the so-called "inner city schools".

It should be remembered that plots to blow up significant infrastructure like the Jersey-Manhattan Holland Tunnel have been averted. There are many more attempts against the United States than we are aware. And I agree with Dave's posting - the issue is with all of us. We really don't want to hear the bad news (there are still plenty of people in the world with the means and the will to attack us). We seem to forget the good news too (there are also plenty of people fighting and fending off such attacks). It's terrible that the September attacks got through. And while I think that more can always be done to increase our intelligence and protection, in a free society such as ours, the mantra "you can't catch them all" will surely come up again. It doesn't matter WHAT we do. There will always be someone with enough will to break through. It's a sobering thought, but it's proven itself out before. Sadly.

[Personal note: the last time I was in New York was last July, a couple of months before leaving the east coast and a decent income behind. I was in the city often, and this time was there alone. It was extremely hot. I have two favorite memories from that trip - (1) seeing Memento at a midnight showing, while drunk (that didn't really work out ";->"), and (2), walking down Mulberry street and then onto Canal or Grand (I can never separate the two in my mind, no matter how many times I walk on them) between occasional downpours. As I walked along Grand/Canal, I constantly looked to the southwest to watch lightning flashes over Jersey silhouetted against the World Trade Center. I was impressed, that night, of the acts of both Man and Nature. It's still very hard and saddening to think that no one will ever see that image again. And the sadness of that still pales in comparison to the lives lost, and all of the connecting lives that have been affected.]

It's going to be a low-post day, and probably a no-post weekend. I came in the office this morning to grab breakfast and catch up on blogs and email, after which I'd go back home and pack (I'm moving into a new apartment tomorrow). However, I got roped into seeing Attack of the Clones again. Drat ";->". So, I'll start packing later.

Hopefully I'll have a good little network going in this new place in no time, and get my home iMac back onto the 'net.

Some more OmniGraffle stuff. ";->"

• OmniGraffle Wireframe Palette - A palette for doing "wireframes" of web page design.
• OmniGraffle and the Google API - using the Google SOAP API's with "AppleScript" to generate backlink relationship diagrams. Really nice use of the Google API's.
• OmniGraffle Extras - Download links to other Palettes and plugins.

James Davidson: my favorite by far is the ability to drag a Cocoa framework (say, /System/Library/Frameworks/Foundation.framework) onto the OmniGraffle application icon and have it generate a nice class diagram for you.

We just got out of Attack of the Clones. My friend put it best - "...the day Star Wars got good again".

A follow-up to yesterday's "Sortof Agile Modeling" post - look how pretty! This one is from initial musings on the structure of SCS.

More OmniGraffle output can be seen in my old Compound Elements proposal for Zope's CMF (content management framework).

These are great examples of OmniGraffle's usefulness - enough usable objects and palettes (with new ones being easy to make) to get a diagram going, enough free-form nature to express anything you want.

As Jon Udell puts it - it's like drawing on napkins. Only better!

I started work on a new application today. We sat around the white board coming up with objects/data, and I did some rapid modeling of the white board data into a rough UML class diagram using OmniGraffle 2.0. OmniGraffle is a fine choice for anyone (on Mac OS X) doing some form of Agile Modeling.

I've had a love-hate relationship with UML over the years, and especially with modeling tools. There are many problems (that have been acknowledged, and some tools even do their best to solve them), such as model and source code getting out of sync, or how do you effectively model a language like "Python"?. On top of these issues lie many of the issues that have plagued programming, especially in the age of internet development, for years - the need for speed (just hurry and get it done! document it later!), scope (a project is [often misconceived as] too small to spend time modeling}, or inability to map web based programming techniques into communicative diagrams (is a JSP page a class? A UI Component? Or is its rendered state a UI component? Huh?). And often, good development processes seem to be killed by the attempted formalization of a process.

This is why it's been good to see the rise of practices like "Extreme Programming", "Agile Modeling", Refactoring, and so on. While there is definitely value to a heavy process like the Rational Unified Process or Catalysis, small shops and teams just can't take on that kind of overhead most of the time.

So, like my story about "Testing Joys" where I was able to get over that often difficult first hump from idea to realization through unit tests, today I re-experienced the same thing with a quick model. And I emphasize re-experienced, because these feelings get lost quickly during the course of a project, where deadlines and other pressures often kill off even the best practices a developer tries to have.

So, "Agile Modeling". Today is actually the first time I've read about it at all, besides seeing the book title. And I don't claim to follow any of their practices fully. I just like the general idea. Modeling is about communication. Sometimes that communication is in the form of ideas of how a system could be shaped. Sometimes that communication is in the form of details about how a system is shaped. And often, a tool can be too rigid which gets in the way of the "shaping of ideas", or it can be too typecast to a particular language or target environment to accurately express the details of what has been/is being produced.

The latter bullet has gotten in my way plenty of times. Few tools support Python concepts. Fewer still support Zope-ish ones. At Zope Corp, we were never able to come up with a really good system that would let an engineer jump right in easily, even using a Python friendly tool like Object Domain. Furthermore, many UML tools available for the Mac OS are either pure Java, or some other cross-platform compilation, making them awkward to use on a Mac.

So where does my take on "Agile Modeling" come in? Usually, I need diagrams most for sketching out ideas about a schema design or workflow model. A tool like OmniGraffle makes this easy, since it's really a very basic diagramming tool. And that's its strength. Visio and Concept Draw are both expansive diagramming tools, with more flexibility than their Pure-UML/CASE tool counterparts, but they're still too heavyweight. The learning curve is probably too steep for a developer to pick up quickly enough to do something like capture a schema concepts off of a white board. And still other tools, like Illustrator are too design focused to be used quickly for technical diagrams. OmniGraffle seems to get it just right. It has plenty of built in Palettes of usable objects, which are often simple shapes or compounds of simple shapes. It includes a UML palette, with various UML objects and connectors. And, it's Mac OS X native, meaning most expected shortcuts (option-click-drag to duplicate) work as expected. Since it's not a die-hard UML tool, it doesn't try to do any sort of type-enforcement and it doesn't present rich dialog boxes or property inspectors to fill in special properties about a class or a method. The parts of a UML Class object are just boxes with text in them that are freely editable. This is very good for rapid, er, "Agile" modeling situations, like being in a meeting and trying to capture whiteboard notes that don't necessarily fit in an outliner, or working on a diagram while meeting a perspective client while he's talking to someone else. A fast, flexible tool that gives you just enough features to communicate a design to a boss, coworker, client, customer, or general public...where can it go wrong?

It can go wrong when it comes to handling large models. A pure UML tool, or advanced diagramming tool like Visio, can attach a lot of smart data to a visual representation. Behind a UML Class object on a diagram can be all sorts of data, such as public/private member settings, where the class resides, OCL constraints, relationships, etcetera. All of this data can be used in another diagram in the same model, but with different visual representations. And, in theory, the data could be used to generate code from models. Documentation can also be auto-generated from all of this extra data in a model.

But, since code and models can get out of sync with each other rather quickly, it's sometimes still best to go couple "Agile Modeling" practices with "Extreme Programming" - keep the code clean and clear enough so that it can communicate the grand design on its own, and use models to express fundamental relationships, schemas, activities, or other aspects of the system before, during, and after the development process. You rarely need anything heavier than that.

The ultimate point of this rambling post - the diagram I was able to throw together today based on our whiteboard meeting immediately drew expressions of "can I get a printout of that?". Since I was able to keep up with the meeting and design as it was happening, instead of wrestling with the tool, the results were on target even though the design was in flux throughout the meeting. Then, without too much effort, I was able to tweak the diagram after I got back to my desk to better communicate relationships between objects, with very little fuss from the program. And since it was so quick to start up, work on, and print out, I have no real strong ties to it - I know that changes to the design will be easy to record and reflect, and not take up too much of my time away from producing the software I'm expected to produce. This ultimately means that I'm more likely to either keep the diagram up to date, or discard it when it's no longer needed. This is in contrast to trying to enforce heavier process guidelines on myself, causing me to have to do big "hat switches" to shift from modeling mode, to coding mode, and back again - usually grumbling about "ugh, I have to update the model now..".

Apple, Xserve. It's really going to be a head trip if Apple can actually start invading Sun's territory. The (plotted) revenge of the NeXT continues. Now to see how well these pups do for sale.

I think they'll do moderately well. I'd love to have one here. We have a mix of pure-designers, designer-programmers, and programmer-sysadmins. The latter two breeds have fully moved to Mac OS X, and the pure-designers are finally moving there as well now that so many critical design tools have moved over (Photoshop 7, GoLive 6, Illustrator 10, Macromedia's MX line). As a workgroup server, these machines could kick ass. I also wouldn't be surprised to see them take over Sun boxes in rendering farms (particularly at Pixar, but that should be no surprise), or at the very least become a key piece of SAN design targeted towards the massive files and projects typically shared by graphic designers, animators, and other key parts of the so-called Core Mac Audience.

Props go to Tom Tomorrow for this one. I have to admit that I tended to watch The O'Reilly Factor on Fox News Channel when my cable service in Virginia expanded to include FNC. I was never quite sure what to make of it, but it was at least entertaining.

Saturday Night Live did a great parody of him. I can't remember what the guest on the show was talking about, but somehow Carbon Dioxide came up. The SNL Cast Member doing O'Reilly said "So you're telling me that Carbon and Oxygen atoms are bonding together? Frankly, I'm just not buying it."

Via Haughey, a great quote from Pascal inventor Niklaus Wirth: "People seem to misinterpret complexity as sophistication." Tatoo that on your forehead, post-it on your monitor, send it to the editors of XML.Com, and to the leaders of the W3C. [Scripting News]

A corollary is that people also misinterpret simpleness as elegance. SOAP has a lot of pros and cons as it's now being managed by a growing body of seemingly headless chickens, but at the same time - it's hard for me to find any good use for XML-RPC besides rudimentary (and poorly designed) blogging API's and cute little services like getStateName. Again, "Zope" has long served both REST-ian and Web-RPC ideas, long before they were ever a battle ground. It should be a better player in these services. Some of this is the fault of "Zope", but some (or more) is the fault of what I see as some design flaws in XML-RPC. See my story (which needs updating) "Zope and Web Services Story".

While a lot of people may attain simplicity, very vew attain elegant simplicity. It's the difference between "yeah, it's simple but it works pretty good" versus "it's so easy to use! and I can still do ...!"

There's some concern that Apple is not allowing the chat client vendors to access the system address book. If so, this is a repeat of the Sidhu mistake. It will end badly for the developers, but it will also end badly for Apple. [Scripting News]

There's been a lot of concern over the Address Book scheduled to appear in the next release of "Mac OS X". Either this Address Book is going to be so good that there will be no other address book possibilities on Mac OS X...Ever. Or, people are just being people and are creating their own FUD. Chant it with me, "Fear! Uncertainty! Doubt!".

Anyways, there's already an address book IN Mac OS X! And it's bundled!.

Yet, there are a number of third party address books available. They range from the Palm Desktop and Microsoft Entourage to Sbook5 (my current favorite) and various other PIM's of varying sizes.

What I think would be great would be for all of those tools to be able to read from a central address book (like a personal LDAP based address book), with the ability to synch up between personal machines (laptop and workstation) with, of course, palm synchronization. :)

I should note that I paid for my upgrade to Office v.X during MS's cheap upgrade opportunity, primarily to have Entourage. All of this while Apple's Mail and Address Book were available for free. Hmm, bundled apps being passed over by rather costly ones. And I didn't just blindly go with Entourage because it was "Microsoft" - I checked out other mail and PIM applications that were available for Mac OS X at the time, and wasn't really impressed with any of them.

I guess a lot of the issue is fear that iChat and the new Address Book are going to be nicely integrated and won't allow other clients access them. I doubt that. A couple of reports from the keynote referred to the address book as being a system wide database, available to all (when I first read that, I was hoping there would be something along the lines of the database capabilities of the BFS). And personally, I hope this is really the case. I personally would love to have all of my contact information finally actually truly under control.

It was a long day peppered with strange (and also long) intermissions. After four and a half years of using Zope/Principia, I finally used Sessions for the first time. This is one area where Zope strangely lagged behind other web technologies, but the resulting Session implementation in Zope 2.5.x is outstanding.

As I was reading about Lutris folding, I remembered reading Philip Greenspun's story about the bust up of Ars Digita after large amounts of venture capital came in. If you go to the page now, it's been basically removed due to some court settlement.

But, thanks to the Wayback Machine, the story lives on. It's a very interesting read, and a good lesson to be learned about seeking out venture capital - you might get the money you need, but the price can sometimes be higher than you foresee. I've been in companies that have nearly been destroyed by a group of bad investors, but the integrity of the founders and a collection of good investors were able to stand up to them and come up with a good plan to get the company back on focus.

Java server maker Lutris bows out [IDG InfoWorld]

It's been interesting, first watching Ars Digita be gutted, and now Lutris folds. Zope Corporation is still very much alive however, even without trying to make a $5000 product out of Zope to sell, like it seems some of the other companies tried to do with their open source app servers.

As I was going to bed, trying to read Wonderful Town, a collection of New York focused stories from The New Yorker and being somewhere in between being too tired and being an insomniac, I remembered a night a year and a half ago, stepping out of Lincoln Center a bit past midnight and taking a cab down to Orchard, one of my favorite bars in the city.

I don't know why, but there's a bit of a halo around that cab ride in my memory. The colors are a little off (white's are very sharp), like the night shots in Bringing Out The Dead. Beautiful night. I miss the east.

"Investment guru Warren Buffett offered a bleak prediction for the nation's national security, saying a terrorist attack on American soil is ''virtually a certainty.''" [AP]

The purpose of ootips is to allow developers to keep up-to-date with object oriented technologies, without having to read through high volume, low signal-to-noise sources. I try to extract the most useful and interesting posts from other mailing lists, newsgroups and web forums, and post about one article every week or two.

I've had this page bookmarked for some time and thought I'd pass it along. The site is a condensed version of some of the discussions and arguments about OOP that have taken place on UseNet over the years. [Lambda the Ultimate]

Looking back at Sam Ruby's article, Google's Genius, he mentions my favorite point in favor of SOAP/RPC:

Perhaps the most illumining part of Paul's essay is when he describes his optimized doSpellingSuggestion API.Ê In this case, he declares that XML is overkill for the job.Ê Unquestionably, omitting XML in some cases creates a tighter data stream.Ê It can also require custom marshallers and parsers to be written.Ê More tradeoffs to consider.

The second to last sentence is important. In my view, XML "scraping" is not much better than HTML "scraping". The Google API's were immediately usable in everything from Ruby to Python to AppleScript to Frontier/Radio, and more, with only a SOAP library needed - nothing specific to Google. The calls were Google specific, yes, but the client immediately understood the results of the call.

Even though it's a silly application, I would never have even attempted to write something like HyprSwank, as mentioned in "AppleScript Studio, XML-RPC, and Zope", if I had to parse the results out of my data structure myself. I'm not sure what XML parsing libraries even exist for AppleScript, but I'm sure if they do (or were to) exist, it couldn't be any easier to use than:

tell application "http://euc.cx/" to set link_list to call xmlrpc {method name:"hyprSWNK", parameters:{}}

Or in Python:

import xmlprclib, pprint
link_list = xmlrpclib.ServerProxy('http://euc.cx/').hyprSWNK()
pprint.pprint(link_list)
[{'sitename': 'mobileffe', 'link': 'http://www.mobileffe.com/'},
 {'sitename': 'Prada', 'link': 'http://www.prada.com/'},
 {'sitename': 'ACFny', 'link': 'http://www.acfny.org/'},
 {'link': 'http://www.peopleusedtodream',
  'sitename': 'People Used To Dream About The Future'},
 {'sitename': 'David Lynch', 'link': 'http://www.davidlynch.com/'},
 {'sitename': '.tiln', 'link': 'http://tiln.net/'},
 {'link': 'http://www.timecube.com/',
  'sitename': 'educated people are stupid cowards'},
 {'link': 'http://www.antigirl.com/',
  'sitename': 'absolut antigirl !(absolut cybele)'},
 {'sitename': 'no ~ type', 'link': 'http://www.notype.com/'},
 {'sitename': 'meta.am +=', 'link': 'http://meta.am/'},
 {'sitename': 'fals.ch', 'link': 'http://fals.ch/'},
 {'sitename': 'infotron', 'link': 'http://infotron.web.fm'},
 {'link': 'http://www.m9ndfukc.org/korporat/',
  'sitename': 'aaaaaaa n t i o r   p;uss'},
 {'sitename': 'nato0+55+3d', 'link': 'http://www.eusocial.org/'},
 {'link': 'http://www.angelfire.com/electronic/campmsp/',
  'sitename': 'themoonstealingproject'},
 {'link': 'http://artists.mp3s.com/artists/42/musiquemotpol.html',
  'sitename': 'musique:motpol @ mp3.com'},
 {'link': 'http://www.helmutlang.com/',
  'sitename': 'finest cotton armbags since 1997 (HL)'},
 {'sitename': 'One Bit Louder', 'link': 'http://mi.cz/obl/'},
 {'sitename': 'purple', 'link': 'http://purple.fr/'},
 {'sitename': 'OFRPublications', 'link': 'http://www.ofrpublications.com/'}]

While I could have written my own parser, written my own format, or spent time researching common XML formats and then scouring the web for an already written parser for my language or sitting down with a copy of "XML and Python", I chose not to.

For as much as I rag sometimes on some of XML-RPC's shortfalls, it's much better than doing all of that! It gives one common format for a common internet use, and has spread around to many languages. If I had to write my parser, or wait for someone to write one, or deal with DOM navigation, in order to deal with GoogleML, I'd be much less curious about "hmm, what can I write today to take advantage of that?".

A nice thing that Bobo got right from day one was to implement and handle marshalling of incoming HTTP requests in such a way that my code never has to go through a "is it a list? is it a string? is it a string I can convert to a list?" somersault again. Or at least, not often.

It's been nice working on some old school Zope development - nothing but SQL Methods, Python Scripts, and Page Templates. Alright, it's not that old (the latter two are fairly recent technologies for Zope, but they make an excellent replacement for DTML Methods). It's been a nice contrast to the more speculative development I've been doing which has consisted of lots of writing lots of my own little components. That framework is starting to be useful, but taking a break away from it to do some real work is nice.

I'm doing this all armed with Adobe GoLive 6 for Page Template authoring, and RBrowser 3 coupled with Apple's TextEdit for Python Script and SQL Method editing, all via FTP. There are a few trips to the management interface to reorganize things or set extra properties, but not too many. It's jobs like this that make me love Zope.

Earlier, I started writing up some thoughts in response to Jon Udell's Web Services post today, particularly the part about WSDL Autogeneration. Jon says:

The classes you'll want to annotate for such treatment should, in many cases, be abstractions of those that implement behavior.ÊCreating such abstractions is, of course, more work -- and the kind of work that doesn't easily yield to automation.

Sounds like Interfaces. Which is where my post was going earlier, but I found flaws in my thinking. While I think the following is possible, and may actually work, it will invariably get more complex than this:

wsdl = WSDLWriter.generateForInstance(obj, onlyInterface=IBlogger)

shrug. I'm too tired right now to explore this notion too much. The issues I started running into is how much to tie web service or security related metadata attached to an Interface (ie, abstraction) to an implementation. It's an issue they struggled with as Interfaces were starting to get heavily used for "Zope" (particularly in the growing Zope 3 architecture), and I believe such information has been moved to configuration files(?). I could be wrong.

Anyways, as long as we're going in the direction of generating WSDL out of code versus generating code out of IDL WSDL, things aren't all that bad.

Yet.

Apple's Jaguar Page is up. Something hypr-cool? Sherlock 3 is actually Karelia's Watson, a really cool app showing how a desktop application can make use of web available data without succumbing to Microsoft's "Everything IS a web page" type interface.

"The great thing about this arrangement is that I can substitute a different block generating function with zero effort, as sizeAndDigest doesn't need to know how its data is being generated. I could have done that with saved state and a callback to get the next chunk of data, but generators are a lot simpler." [Deadly Bloody Serious]

Steve Jobs: "We are here today to mourn the passing of OS 9."

I'm reading Maccentral's Live WWDC keynote coverage. Jaguar is sounding great.

There's a post on Lambda the Ultimate today about Generators and Abstraction, with reference to "Python". Ruby has the concept of generators built in from the ground up. Last summer, I came across Ruby and started playing around with it, and annoyed everyone around me (I was working for Zope Corporation who own PythonLabs) by going on and on about Ruby's object model and generators.

While I'm sure my little rants had little or nothing to do with the course of Python 2.2, it is nice to see Python picking up on simple generators, iterators, and the whole new classes thing. It has me excited about Python again.

An interesting show of the power of generators and iterators comes from PEP 279, The enumerate() built-in function. It's a new built in function that gives to all collections (anything iterable) functionality similar to dict.items() (more specifically, dict.iteritems()) which loops over the keys (index) and values (item) of a dictionary. This example, from the PEP itself, shows a good use of generators and iterators in action, keeping evaluation of a loop lazy while being able to index it:

def enumerate(collection):
    'Generates an indexed series:  (0,coll[0]), (1,coll[1]) ...'     
    i = 0
    it = iter(collection)
    while 1:
        yield (i, it.next())
        i += 1

I should pay attention to the comp.lang.python mailing list more often, but Usenet newsgroups seem to be a much bigger pain in the ass in these days of Broadband - it's hard to know whether your provider, or your providers provider, will really offer any usenet service.

Anyways, PEP 285 brings about True and False to the Python language as natural members for the first time. One of the areas this will help in is the reduction of code like this:

return not not foo #the twin 'not not' keywords cast foo into a boolean statement

and replace it with:

return bool(foo)

Which, while brief, avoids the headslam that boolean logic games like 'not not' can be. One place this will be especially beneficial is with XML-RPC and SOAP, and other means of communicating with languages that have (and expect!) booleans. It's been sufficient so far in Python to use any true value or any false value for boolean use, and 1 and 0 for basic usage, but that's hard to communicate over the wire. A Python function may return '1' as an integer value. But it can also be returning '1' to indicate truth (as in predicates like 'isinstance()').

So... Interesting. Never even knew it was up for debate, let alone accepted.

Looks like tomorrow..er, today.. I shift over to the seedy world of Independent Contracting. But, it should mean getting paid. And finally getting to do some real work for once. I've enjoyed doing speculative development on the SCS framework, sure, but I need a good "do this system in two days" challenge again.

A seemingly forgotten Web Service package for Python. Even its developer seems to have left it alone for a while. It was to be part of Web Service support for "Zope", which has yet to materialize.

Anyways, this is an interesting package to look at. It's the most complete of the few SOAP + Friends packages I've looked at for Python, with deep support for WSDL Reading and Writing among other things. It's powerful, flexible, yet still easy to use. And, it mostly works. Whether or not it continues to be developed, it's a good Python package design that's worth looking at.

Yale Law Top Ten New Copyright Crimes.

Forward: a few hours ago, after getting home from dinner with friends, I was reading more on Web Services, and found this article on Wrapping Web Service APIs on ORN by Stephan Figgins. It somehow led me back to an old Scripting News post that somewhat rankled me when I first read it. I'm finally posting my response. At 2AM, saturday morning.

Does Dave Winer get Web Services? This posting talks about the overhead involved in publishing a web service via .NET. I admit, my experience with Radio and Frontier is still very small. But, at least with "Radio", you have to write your Web Services script in a special folder, Web Services. How is that not overhead? Now, you're writing a single script adapter to another part of the system in order to publish it on the web services channel. This may make sense in an environment based on pure scripting, but what about real persistent objects that are already on the web - objects that are both Data and Behavior? "Zope", and Bobo before it, have been built on this concept from day one.

Many a year ago, Amos Lattier wrote an article called The Bobo Way. I'm going to quote his article here, because (as I've said before), this sounds like familiar ground:

For example if your app keeps track of company employees, you will probably want a separate Employee object for each employee. That way you can access these objects with URLs like:

/Employees/amos/set_menu?lunch=spam

This URL assumes that Employees is a mapping object that contains Employee objects. Employees["amos"] is an Employee object that represents the amos employee. The above URL calls amos' set_menu method with lunch=spam for arguments:

Employees["amos"].set_menu(lunch=spam)
Ed: this could also be Employees.amos.set_menu(lunch=spam)

For contrast lets look at a non-bobo way of doing the same thing:

/set_menu.py?empid=amos&meal=lunch&food=spam

It seems to me like an awful lot of XML-RPC interfaces are not much better than this last example. They're usually a single function that is handed some sort of key to operate on another object with, rather than going to that object directly. It defeats polymorphism, it defeats generalization, and ultimately it can lead to a large and unwieldy adapter layer, tucked away in a Web Services directory somewhere.

There are merits to this design. You are basically writing an Adapter pattern of sorts, mapping a Web Service friendly interface into whatever the internal system really has. And, if that's the only gateway into your system from the outside, you have some level of security by limiting the amount of access to a collection of XML-RPC dedicated functions instead of opening up access to every potential object.

But there are merits to the other design too. In theory, you can program Zope the same way on the inside as on the outside, so long as you have the credential to do so. You have instances of classes out there, on the web, already. .NET seems to be following this route somewhat. Doing Web Services by this route is scarier. Why? Security for one reason. And, you just don't want to turn every single method on the object to be callable over the web, or - at the very least - you want to attach conditions. This is what Microsoft is doing. Let's look at how a simple 'return hello' class for Zope looks, disregarding any Zope OFS framework stuff (making the class addable through the web interface). Putting an instance of this class in a Zope tree would yield the same results over XML-RPC as calls to this .NET example would:

from Security import ClassSecurityInfo
from Persistence import Persistent
from Globals import InitializeClass
 
class Hello(Persistent):
    security = ClassSecurityInfo()      #Used to do declarative security
 
    security.declareObjectPublic()      #Make these objects inherently public
 
    security.declarePublic('sayHello')  #And make this method public
    def sayHello(self, name):
       """ Say hello to 'name' """
       return "Hello %s" % name
InitializeClass(Hello)                  #This processes the security declarations

Now, it's about the same overhead as the Microsoft ASP.NET example. note: There's more overhead involved with Zope itself in order to fit the Hello class into the Zope framework, but this is purely an example. This could have been easily written as a Python Script object, where the security settings could be set through a web UI.. Why do all this? Because web services as API's are ultimately programmatically exposed. There are two ways this is done:

1. Declarations, either in code or configuration (ie, xml) files.
2. Writing separate handlers, and only exposing those to the web

The first is the "Zope" way, and apparently the .NET way. At some point, you're taking code that exists already and saying "yes, this is a web service".

The second seems to be the Radio way, where you write a script that calls into other existing code, and say "this script is the web service".

Sometimes, the amount of overhead is equal. But I think relegating all Web Services to being scripts and functions in a special folder or singleton object weakens their potential. Likewise, not all existing web-published code is ready to be made into a Web Service. Both ways are valid. But, there's got to be a better unification. Dave needs to understand the declarative way. It may look more verbose on a silly little Hello World example, but I have access to thousands of methods exposed via XML-RPC that are done so automatically via Zope's similar security declarations. That's a lot more than I'd have access to if I have to write those methods in a special place.

So, here's another verbosity example. Let's say there's the following bit of code in a Blog Entry class:

class BlogEntry(...):
   security.declareProtected('Edit Entries', 'editPost')
   def editPost(self, content, publish=0):
      """ Set the content of the post, and publishes it if publish is true """
      self._content = content
      if publish:
          self._publishedContent = content
      return "OK"      #simple return value

And, for simplicities sake, lets say we had a flat structure for all entries to make URL's easier, and we have an entry identified as '123'. So, to edit it via URL. Hmm, this is almost REST-like:

.../Posts/123/editPost?content=boy+howdy&publish:int=1

And do to it via BCI (ZPublisher.Client):

myPost = ZPublisher.Object('.../Posts/123', username='bob', password='bobby')
myPost.editPost(content='Boy howdy!', publish=1)

And XML-RPC in Python. Note that in order to do this, due to Zope's built in security, you either have to have a hacked XML-RPC library locally that generates authentication headers, of have RPCAuth by Nathan Sain (which I haven't had time to test yet) installed on Zope:

myPost = xmlrpclib.Server('.../Posts/123')
myPost.editPost('Boy Howdy!', 1))

These are very similar, but ZPublisher.Client was doing this long before XML-RPC. Why? Because Zope was already doing distributed objects (of sorts) on the web, translating HTTP requests into object calls. It only makes sense that there be some simple client lib.

Now, to implement this as a proper Blogger API, and do it the way that Radio style web services are done, instead of necessarily having all of those security declarations on the BlogPost class itself, I would instead have to write this:

def editPost(appkey, postid, username, password, content, publish):
  """ Lookup the post by its ID, verify that the user can access it,
      and set its content and published flag """
  thePost = Posts[postid]
  ## security assertions based on username/password would go in here, somehow.
  thePost.editPost(content=content, publish=publish)
  return 1

Now, this isn't too bad. But, it's a similar amount of overhead to turn an otherwise already existing published web method as a Web Service. Huh.

As people start exposing ever more valuable resources as Web Services, or even REST, it's no wonder complications arise. As we move beyond the 'getStateName' example and start getting into exchanging business data (Microsoft's heavily advertised 1 degree of separation), more care must be taken. I don't want everything I write automatically turned into a Web Service. I don't necessarily want to have to write extra adapters either.

To sum up and say it all over again, the way that "Radio" seems to promote for Web Services is easy. And simple. But you're effectively writing scripted adapters and bridges into other code, something that could turn out to be a maintenance nightmare. The way that "Zope" has promoted for years (even though it's not a terrific Web Services player, sadly) says that "Objects and their methods are already published on the web. No gateway scripts needed.", but with that comes the price of maintaining security declarations similar to C#.NET's ASPX's declarative Web Service calls. Dave calls the latter too much overhead. I say it's the other way around. It should be easy to write Web Services, but you should have to think about what you publish. Those declarations aren't much worse anyways than the long signatures verbose languages like Java can make you write (protected final synchronous int Bob()).

I found this article while browsing around today about using AppleScript Studio and Filemaker, and I finally had the nerve to do what I've been meaning to do for a long time - expose the hypr list of urls for euc.cx as a list of mappings, and start learning AppleScript / AppleScript Studio.

I did it. Pretty easily too. I followed the first half of the article, but instead of using Filemaker to fill in the data source, I used a new DTML Method on my site called hyprSWNK. I got that app going in about sixteen lines of code - not bad considering the verbosity of AppleScript. I spent the second half of the evening adding in support to open the URL of the selected row in the table. I had some problems, because OmniWeb was not responding to the AppleScript statement I was using, but eventually when I tried Internet Explorer, it worked fine. Pretty cool for a couple of hours work.