[<<] Industrie Toulouse

August 29, 2003

I might have a Pattern here. Ever since I started implementing the system outlined in the May post, "Component Relational Mapping?" I've been employing a variety of patterns. Most have come from Martin Fowler's book Patterns of Enterprise Application Architecture, others from the venerable "gang of four" book, Design Patterns. And I've noticed some of our own patterns start to emerge.

The key pattern is something I refer to as the Schema Validator. A Schema Validator takes a raw clump of data, preferably in some sort of key/value form, and does one of two things - return a new data object with all of the values in 'correct' format/typecast, or raise an exception with a collection of all failed validations.

It's this pair of responsibilities that make the schema validator such a powerful component. The ultimate responsibility of the validator is to give comfort to the developer or integrator that when data is passed to lower levels of the system that will be responsible for storage or messaging, that data is known to be "correct." That is - integers are integers, floats are floats, dates are dates, credit card expiration dates are credit card expiration dates. More than that, any further restrictions on the data, such as an integer range, string length, etc, have already been checked. It severely cuts down on the possibility that an exception will be raised down at the storage level for attempting to insert an invalid value.

I've come to rely heavily on this patterns as I've worked to abstract the storage layers of my web applications, typically layers which speak to an external database such as MySQL or OpenLDAP. It allows me to comfortably generate dynamic SQL statements, particularly DML statements (INSERT, UPDATE), without having intimate knowledge of the schema. These lower serializers or gateways can loop over the data that's passed in to them and then format it for the target data storage, especially important when dealing with date/time storage in SQL systems (which still seems to vary wildly between databases).

To date, my Schema Validator system has basically been the Formulator product for Zope, courtesy of Infrae. Formulator also gives me HTML Widgets for HTML form generation. This aspect is very nice (it cuts down on the amount of manual HTML writing one has to do to deal with forms), but it's the validation system that I absolutely depend on. Most of the time, this allows me to describe my data model once, and use that model to generate user interfaces and to ensure that the data I depend on is relatively strong.

I think work might start soon on the next generation of our system. Formulator has been very good to me thus far, but on a recent project I started to run into two issues that I think are going to present themselves again in the future:

  1. Multi-field Validation. Formulator goes on a field by field basis. In my systems, I need to add extra elements to the framework to do more complex validation that may impact/check multiple fields. It's preferable that these would execute inside the same 'validate_all()' command.
  2. Context/Security Sensitive Widget Display and Validation. On any level of granularity, I need to apply an extra set of rules about who can do what to a particular field. On the display side, this entails making certain fields read-only, or maybe not displayed at all. On the validator side, this could mean ensuring that certain values that the current user does not have access to are dropped out of the 'valid data' structure.
I'm also noticing a lot of duplicated code which I haven't had time to refactor. Mostly, this concerns manually passing the same validated data block to multiple Table Gateways - the components which actually take one of these validated data structures and prepare the data to be sent to storage. Instead of multiple manual calls in the same Controller/View component, there should be one. Ape uses an 'event' object that gets passed along to its gateways, and those gateways make notifications in the event object about which elements in the data block can now be ignored by other gateways in a chain. I like this idea a lot.

Ultimately - the patterns that we've started to deploy in our Zope apps since May have really started to yield stronger bases, and much stronger user interfaces. Many of these patterns come from Zope 3, P of EAA, and other places. The ideas are already out there, and many of them are out there for good reason.

J. Shell, August 29, 2003 10:07 AM, in Objects and the Web, Zope

August 28, 2003

I got this release months ago, but I'm the last one in the office and was looking for something to enjoy loud - Ryoji Ikeda's "Op."

Ryoji Ikeda is one of the best minimalist electronic composers out there. Many of his recordings are very cold, very very precise, beautiful, and sometimes scary. Many use exact pulses and frequences, carefully timed and panned. Some scatter clean cuts across audio. Some of his compositions have taken advantage of psychoacoustic principles - specifically written and recorded to change in tone as you move about the room.

"Op." completely stands out, as a result. While remaining purely Ikeda, he completely scrapped the computers and electronics, and applied his beautiful minimalism to strings. Slow moving, and very minimalist (in a much better fashion than well known so-called "minimalist" composers), it's just...well...striking. And beautiful. Some movements sit on the highest frequency I imagine he could get out of his performers, but the beauty comes from the natural fluctuations and interactions between string and bow, especially when some of these notes are held long enough for one to notice this effect.

With temperatures starting to cool down, I'm looking forward to that time when the apartment is free from the noises of heating and cooling devices. Hopefully I'll be wise enough to take advantage of that time to listen to releases like "Op." and friends.

J. Shell, August 28, 2003 07:41 PM, in Review, Sound Design

August 27, 2003

Monday night, for the first time in roughly ten years, I did homework. After two or three semesters of procrastination or forgetfulness, I finally squeezed in under the registration deadline and am now happily back in Math. Way back in math. sigh. I was hoping to test higher than I did on the placement test, but it turned out that my memory was scattered. Some operations and factorings I remembered very well. Others I was able to apply with a decent amount of effort. On others, I failed miserably.

Still, it's kindof fun to be back in school. Although my math homework (playing catchup for missing the first class) kept me up until one o'clock this morning. Happily, I have a shiny new TI 89 graphing calculator.

I didn't sleep well last night. After staying up finishing homework, I tried going to sleep, but tossed and turned for a while instead. I had a lot of nervous energy and couldn't get comfortable. I finally fell asleep sometime after 2:14 am (last time I remember glaring at the clock), only to be awakened by a very loud thunderclap at 4:04 am. Ugh.

I do love my new calculator though. It'll do until I have the smarts, time, and reasons to use Mathematica.

J. Shell, August 27, 2003 11:38 AM, in Etc

August 24, 2003

The second season, or "chapter" as they prefer to call it, of HBO's excellent series The Wire concluded tonight. Much of the investigation wrapped up, some of the bad guys got away, and some of the not-so-bad died off.

At the heart of this seasons story line was the character Frank Sobotka, who runs the IBS union on the docks. Sobotka is a man fighting hard to save the union, whom he says has been "dying every day for the past twenty years." He helps lift containers off of the cargo ships for a local smuggling ring as a line of extra money. When thirteen girls are found dead in one of those cans, the season kicked off.

Unlike many police dramas, The Wire spends a lot of time developing characters like Frank Sobotka who aren't necessarily bad guys, but on the wrong side of the law. Frank's concern is the union and the port. He wants desperately to return the port to its glory days and make it more competitive, and he's vested a lot of time and effort into that. But in all his years with the union, he has (of course) neglected his family. His son, Ziggy, and nephew, Nick, are also central characters to the story arc. Ziggy is a strange kid, and has gone through some interesting twists this season. Both he and Nick see the smuggling money going on around them and want a taste. Nick goes much further than Frank does away from the good side of the law, but his character is never a villainous one. He is, however, impressed by the money and magnitude of operation employed by the smugglers, and soon gets involved with selling drugs for them.

As I wrote earlier, I really appreciate how the HBO original series system works (and this probably applies to other original cable content as well - at least, pay channel content). Free from having to time episodes around ratings sweeps, free of commercial interruption (on the premium channels anyways), and free to do a shorter season that is likely to run all at one time, I think we're delivered with some very compelling stories. Time format seems pretty open as well - many of the 'hour' long shows sometimes clock in at 45-50 minutes (a lot of Sopranos episodes run in this time frame). While it's probably not encouraged for a show to go over their limit, except in special situations (like a Finale), the operational freedom to use a shorter cut, if it makes sense, again seems to deliver a better story. In any case, the character growth and emotional connections in The Wire were excellent in their portrayal. The story arc and overall timeline were enough to allow them all to be present, subtle, and honest. I only hope that sometime soon I can have the chance to watch the entire first season, in order, instead of the bits and pieces I've eked out over the past year.

With The Wire and Project Greenlight wrapped up for the season, and the summer Finale of Sex and the City coming up, we're in for two new all new series on HBO. The first is Carnivale. It seems similar on the surface to Something Wicked This Way Comes, a film I remember loving and fearing (but little else of). 1934, Dustbowl, Oklahoma, a traveling Carnival, an apocalyptic fight between good and evil as a result. At first, I wasn't terribly interested. But recent trailers have me very intrigued.

Also new is K Street, which was finally revealed today. This new series from George Clooney and Steven Soderbergh looks to be a real-time political "reality" show. In the trailer given today, Clooney stated that the intent is that they'll pick something out of the paper Sunday morning, spend two or three days filming/interviewing it, and cut it together into a show to be displayed the following Sunday. It's primary focus seems to be the political consultants in Washington DC. James Carville and Mary Matalin both seem to be very involved. I don't recall the others who spoke on the trailer. But it definitely looks interesting. And it's nice to finally know what this show is about, since there's still no section for it on HBO's web site.

And I have to say, I've become rather addicted to Curb Your Enthusiasm.

J. Shell, August 24, 2003 11:53 PM, in Review

Project management and communication continue to be a personal fascination, and one that I have covered many times in this weblog as I've gone through many tests of assorted bug trackers and document management systems, trying to find "the right one." For the time being, we've settled on Roundup, a standalone bug tracking application written in Python. Roundup is , not terribly difficult to customize, and is fairly flexible. And very fast. The application itself is very agnostic about its storage layer, and in our current setup we're still using basic DBM storage, which was very easy to set up and use. We've made some customizations in our instance (without touching the base code - always a good thing to avoid):

  • Simple Document Management: We needed a place to start keeping notes, documentation, tips, memories, requirements, etc. It's not as elegant or nice as a Zope solution might be, but it keeps all of this information in a single place. I added support for docutils and its reStructuredText format, including a directive to refer to other Roundup items (files, issues, projects, etc). It's decent. And I like that everything is kept closely together. But at some point, I really want to start evaluating a more powerful document management system like Silva, primarily to handle large documents such as contracts and proposals as well as all the little remembrance documents.
  • Project Management: A project is basically a collection of issues and documents. A project page in our tracker gives a list of all issues and their statuses. The status column on the issue list table is color coded, with lots of green being a good sign the project is finished. Projects follow a simple workflow (new/proposed/in-progress, etc). They've been a nice way for us to have a single tracker instance that encompasses everything in progress. Many Zope based trackers we deployed tended to have a project container with a bug tracker inside of it, which didn't allow for easy sharing (or even moving) of tasks.
Roundup's issue handling is excellent. There are many opinions out there that a weblog is a good place to do project management. This may be true. However, a decent bug tracking system gets you many of the same benefits, but with much more relevant metadata. The important thing one has to do is document their work - how they fixed a bug, how they implemented a feature, etc. A weblog makes good sense for that. But a tracker makes better sense - the conversation is both more open and more focused. Many authors can contribute to the discussion of a particular issue, but the issue (should) remain on the task at hand. To make this communication easy, it's helpful when a Bug Tracker allows mail-in responses to an issue. This way, communication between interested parties remains in the database of the bug tracking system, and not archived off in individual's email folders. When it's two months down the line and someone walks into your office asks "how did we fix that thing with the thing and the stuff?", you can bring the issue back up with a quick search, and hopefully have a good answer, such as, "oh yeah! We converted the value to megabytes at read-time, and rewrite it as bytes at write-time, and the user interface is none-the-wiser."

There is something to be said for the personal journaling experience of weblogs in a project, and this may make sense for some teams. I think it's just important to do the journaling - whether its in a CVS change log, a weblog, or an issue tracker. Or perhaps, a combination of the three. No matter how clever you think you are at solving a problem today, there is always some point in the future where you'll ask yourself "now why did I do it that way?" It's nice to have an answer.

J. Shell, August 24, 2003 10:40 PM, in Etc

August 21, 2003

All done with people, not computers. Really cool to watch. I give you the link of the day (windows media format).
J. Shell, August 21, 2003 03:10 PM, in Etc

A couple of years ago, while still working for Zope Corporation, I started to take a look at a programming language called Ruby. Ruby is an interesting language, and is best described as a mixture of Smalltalk and Perl. From Smalltalk, Ruby inherits a "pure" object oriented design, with a single class hierarchy and the notion that everything is an object/method. Even what some would consider as linguistic keywords are methods of the Kernel class/object. Unlike Smalltalk, however, Ruby uses more traditional imperative programming constructs, whereas Smalltalk does everything in message passing syntax. From Perl, Ruby picks up a lot of Unix shell programming features, first class regular expressions, all the joys of $_, $?, $:, and more. It also seems to have picked up the There's More Than One Way To Do It mentality of Perl as well, with a few different ways of spelling blocks, loops, and more. In Python, two people can enter the same algorithm and yield very similar looking code. In Ruby, two people can enter the same algorithm and yield radically different looking code, depending on how they like to spell certain things, whether they put their ifs at the beginning or end of the line, ie - next if a = b or if a = b then next. In Python, there's only one way to do it, if a == b: continue. Use of the return statement in Ruby is optional. If it's not used, the last value in the method is returned. Looking at the following code, I have no real clue as to what's being returned here (besides the possibility of 'nil'). With some hard looking, I can figure it out. I imagine that once one becomes familiar with Ruby, this would be understandable.

  def []( key )
    @table[key] or return nil
    @table[key].gsub(%r<\$([^/]+)>) { self[$1] }
  end
It's definitely not executable pseudocode (even without that regular expression in there), which is a rather common description of Python.

So while Ruby's influences are primarily Smalltalk and Perl, Python's influences include the ABC teaching language, Icon, Algol-68, and Modula-3. This is definitely where the import statement came from. Modules are my favorite feature about Python, especially after taking quick looks lately at Perl, Ruby, and PHP - all of which use multiple statements to load source code from one place into another. There is something inherently graceful and simple about Python's module system. A module is any Python file. Any named objects inside of that module are usable outside of it. They're accessed through dot notation.

  import foo

  def rebar():
    b = foo.bar()
    return b * b
Ruby, and some other languages, come up with yet-another-notation to access things inside of their modular namespaces, such as the double colon (which may then sometimes get used with dot notation as well).
  require 'foo'

  def rebar
    b = Foo::Bar.bar
    b * b
  end
Ruby also uses single (or double) prefix characters on names to excess, with a difference between name, $name, @name, @@name. The prefix characters denote scope, with @name being analogous to Python's self.name - looking up an instance attribute. There are other clever little prefixes used, such as colons, to do other operations. This creates a language full of cryptic single character command shortcuts, when in fact - none of them need to be used. Python, on the other hand, uses no such prefixes. The only characters heavily used in Python are the dot (object/module/etc traversal), and the leading underscore (sometimes overused), usually used to protect names or highlight special 'slots'. Ruby has a lot of cute and clever elements, and it really is a pretty good language. But it really makes me appreciate the zen simplicity of Python, wherein "explicit is better than implicit" and "there should be only one way to do it" are common mantras:
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
(execute import this in a Python interpreter, and this is what comes up).

I think that there's just a lot to appreciate about Python. It's not a perfect language, but it's getting better all the time. I staked my professional life on it in early 1997 and haven't had to look back since. I do like to stop and experiment with other programming languages on occasion, and you can read about my experiences with Ruby here. Interestingly, most of the really major things I liked about Ruby found their way into Python 2.2. Sitting down with another programming language for a while can open your mind up to other ways of solving problems, and I think I became a better Python programmer after spending time in Ruby land. It certainly made me better prepared for the new features that Python 2.2 sprouted.

I do think that Ruby is a cool language and it still has some pretty nice features that differentiate it from other languages in its field. But aside from some small shell scripts, I don't expect I'll ever do anything major in it. Most of my world is in Python right now. And when it comes time to sit down and figure out a new programming language, I really want to wrap my brain around Haskell in order to appreciate a really different way of doing things.

p.s. Speaking of early 1997 - the February 1997 issue of Byte had one of the first major magazine articles about Python, written by yours truely. :)

p.p.s., Andrew Kuchling wrote an excellent letter to Salon. One quote from it is this:

There are languages that are more compact than Python -- Lisp treats everything as a list, while APL used arrays everywhere and Tcl used to use strings -- but it's possible to go too far toward minimalism. People have different preferences, and I like Python's simplicity/complexity tradeoff, midway between Lisp's simplicity and Perl's complexity. Other people will find a different level of comfort.
Ruby and Smalltalk's system of pure object/messages are another degree of linguistic simplicity, on or near the level as Lisp. But while Ruby does employ a more imperative structure than Smalltalk, this still contributes to some funny looking syntax. In the next paragraph, Andrew mentions Modula 3's design goals:
his idea of simplicity may have come from Python's Modula-3 influence; if you read the introduction to the Modula-3 language definition at http://www.research.compaq.com/SRC/m3defn/html/intro.html, you'll see that the designers of Modula-3 limited themselves to a specification that could be printed in 50 pages. Python is similarly simple; almost all of the language's features can be explained in tens of pages, and suggestions for new features are often rejected because they'd complicate something too much.
All I can say is - I'm glad the Python community rejected PEP 308 (ternary operators for Python), especially after seeing this bit of Ruby code:
  x = if a<0 then b else c
That just doesn't read right. One of the proposed syntaxes in PEP 308 was to make it like this in Python:
  x = b if a < 0 else c
which reads a bit more easily... but only a bit. Again - I'm glad this whole thing was rejected, and by the community at large.

J. Shell, August 21, 2003 01:13 PM, in Python

August 18, 2003

Mac.Ars is back with another great issue. The August 18th edition discusses Apple's focus, and whether it might hurt the company. I differ with the author on this point:

Is this strategy of becoming all things to all people a good one? Can an IT purchasing manager take a company seriously which sells singles and servers? No other PC manufacturer is reaching into as many markets at the same time as Apple which sells blade servers, high-end UNIX workstations, commodity PCs, laptops, mp3 players, operating systems, productivity applications, webcams, WebObjects, and music (Dell perhaps comes close, but most of their offerings are truly third party).
No other PC manufacturer? How about Sony and Microsoft? Granted, Microsoft is not a PC manufacturer (depending on how one views the XBox), and I don't believe that Sony has entered the server market (or is even a serious contender for IT purchases, with the possible exception of Vaio laptops). But both companies have far reaching product lines.

Sony is not just a reseller of music - it has its own labels (including a fairly well respected classical label), and their own movie studio. It sells robot dogs, home and car stereos. And there are also some of the best looking non-macintosh computers out there. Hell - I've even been tempted by certain Vaio desktops recently. Other Japanese companies have an even broader range of product offerings - some companies are happy to supply you with a violin, a motorcycle, a synthesizer, and a wave runner, all under the same brand.

And then there's Microsoft. The same company that wants to sell you numerous editions of Windows Server 2003 also wants to sell you (or your children) interactive Barney. (All jokes about big dumb beasts who continually insult the intelligence of their intended audience have already been made, thank you). Microsoft makes BARNEY dolls and expects to be taken seriously? I mean - it's BARNEY! Maybe they're just trying to dumb up their next generation of users. And I say all of this out of pure contempt for Barney. Microsoft... I actually have no major beef with them at present. But Barney... Ugh. Turning away from that, however, and looking at Microsoft's other interests and holdings, we find not only media outlets, but they've recently started to dabble in legal digital music selling as well.

My point, up to here, is that it's not uncommon for a company to have a wide product offering. The Mac OS X platform is just as viable and flexible as Windows (NT based Windows) in being a solid consumer level operating system as well as a server one. And jokes about security concerns in Windows need not be said - there are millions of unpatched Linux Apache systems out there that are as exploitable as Windows. Even the Free Software Foundation has been hacked recently.

Of course, Microsoft has the fact that they're Microsoft going for them when IT is considering purchases. And the free/open source alternatives have their price advantages (real and fabricated) over the Mac platform. But does it hurt Apple to have a wide focus? I say no.

There was only one time recently where I believe their focus slipped, and that was with the introduction of the Power Mac G4 Cube. At the time, it just didn't fit into Apple's product matrix, and struggled to find its audience. I know what Apple was trying to do with that system - they were trying to do the equivalent of the higher end bookshelf stereo system - small expensive stereos that sell to a certain audience because they look and sound better than boom boxes (which are hideously ugly these days). I don't think the market really existed at the time to sustain it as a product line. But curiously, the 17 inch iMac of today (as well as the above mentioned Viao W series from Sony) sell in the same range and target a similar aesthetic. Somehow, Apple's product line has rounded itself out enough to allow the high end iMac to sit and sell where it does, while the cube could not. (And - it's been voiced many times that Apple should re-introduce the Cube. Not as a high end machine, but as a low end one - as an iMac or eMac without the built in display. It could sell now for between $400-$500 and target people who already have displays). But at its release, the Cube couldn't decide whether it was a professional or consumer machine. Apple's hardware product matrix at the time had four squares - professional and consumer desktops and portables. The Cube wedged itself between the Pro/Consumer desktops, and for a brief moment Apple lost focus. However, by introducing and maintaining "special edition" higher end versions of its consumer products (from the iMac DV SE, to the iBook SE and then onto the 14 inch iBook), the 17 inch iMac has a much more natural place in the now much wider array of Apple products, especially with the eMac line there to capture the lower end of the market without being substantially different from the iMac line (compared to the G3 iMac and G4 Cube).

An amazing artifact of Apple under the watch of Jobs and Company is that they've been able to bounce back from the Cube and other disappointments (ie - the Switch campaign) without too much damage. Well, maybe it's not amazing. But it's obvious that Apple is different from every other PC company (again - with the possible exception of the Sony Vaio division). Apple continues to put money back into research and development, to open new retail locations, release new products, extend their product offerings, and (generally) remain profitable. Anyone who passes up an XServer and Mac OS X Server (or even LinuxPPC or PowerPC BSD operating system) because Apple also sells iPods, individual music tracks, and sells home movie editors, needs to evaluate their evaluation criteria. Other PC vendors (particularly ones like Gateway, etc), seem to be equally if not more willing to start and stop product lines on a whim (especially companies who sell Linux equipped servers as well as ones with their own Unix variation - and I'm talking as much (or more) about IBM and Sun here as idiots like SCO). Also to be wary of are big umbrella technologies. IBM has gone from their VisualAge/Smalltalk, SOM/DSOM, ComponentBroker ages now to WebSphere - and there are lots of different items offered under that. Microsoft has gone from VBX to OCX to ActiveX "Active...er, Visual Everything!" lines to the new .NET line which they've already scaled back and scaled back and scaled back immensely. How Apple's product line changes (especially now that Mac OS X has settled out) are any worse, I have yet to see.

J. Shell, August 18, 2003 10:52 PM, in Apple / Mac

August 13, 2003

There are some cool and simple applications for doing collaborative editing on Mac OS X: the programmer-focused Hydra (seeking a new name), and the rich text (and seemingly science/math) document focused iStorm. What the world needs now, is for OmniOutliner to bring this same functionality to outlines!

OmniOutliner is my favorite outliner. It has a lot of features, but keeps its focus on being an outliner, instead of being an outliner, a diagram editor (OmniGraffle can open OmniOutliner outlines and make diagrams out of them), and a presentation tool (apparently, Keynote and OmniOutliner can work well together though). OmniOutliner, thus, remains simple, fast, elegant, and surprisingly flexible. Now all I need to do is be able to share outlines with a coworker - either in read only mode ("here's what I'm doing"), or in collaborative editing mode ("here's what we need to do to get this project done"). I'd be happy enough with the first, and overjoyed at the second.

Dave Winer added a feature to Radio Userland a couple of years ago called instant outlining. It sortof combined instant messaging with Radio's outliner. You could subscribe to other outlines and read them in place, and get notified whenever a subscription changed. Since outliners are a great way to both document what you need to do and what you have done, this was a great concept. I'm not sure whatever became of it. I ultimately abandoned Radio. And I really did not like the user interface of its outliner - it was slow and cumbersome and not terribly OS X savvy, compared to OmniOutliner which I was already familiar with. And I must say, I've never been able to get along with Frontier for any major stretch of time - it just didn't work with my brain, even though it's heavily outliner based and I do love outliners. But it was (and still is) a good idea. It satisfies the read-only shared outliner requirement.

I know people who use the outline modes available for Emacs to maintain a personal log that spans many years. I have a similar OmniOutliner outline, DOING!, that I just put back on my Mac OS X dock. Some of it is organized by date (but that ends mid-december 2002), some is organized by project. Either way it's grouped, it's still nice to go back and look at what's been done - and also to see the thought processes applied to certain problems.

Now if only I could share...easily.

J. Shell, August 13, 2003 05:08 PM, in Apple / Mac

Zope is great at serving up dynamic content. But what about static content, such as images or large media files? Zope is great at managing such content (even without the higher level content management tools available), but it will never be as fast as a pure file HTTP server such as Apache.

Enter Zope's cache management system. This is a small framework within Zope to deal with caching and has a simple notification system for when cached objects change. Different cache managers can be plugged in. Two that ship with Zope are a memory cache manager, and an HTTP cache manager. The latter adds headers to the response for cached objects so that downstream cache managers (such as Squid) will cache according to a common policy. But other cache managers may be plugged in as long as they implement the Cache Manager interface (these are early examples of the component style development that dominates Zope 3). One we've been deploying to some success is FSCacheManager.

FSCacheManager does something interesting. It dumps cached objects out to disk where they may be served up by Apache or IIS, without requiring custom objects (such as FSImage, which stores images on disk instead of in Zope's object database) or changes to common Zope URL API's, such as absolute_url(). Using rewrite conditions in Apache, it can be set up so that an image gets fetched out of Zope only when there is no file system representation. When an image or other cached object is updated, the cache management invalidation machinery clears the marked file out of the file system, and it's put back there the next time an HTTP request is made for that object.

We've extended this concept to deal with an application we wrote that - for better or worse - stores its user managed images in MySQL. The images are retrieved through a Python Script that uses traversal_subpath that lets us have a url like data/SOME_UNIQUE_ID. Since these images are not being managed by Zope's machinery, we put in some explicit calls to a similar FS Cache type component that we wrote that also dumps files out to the file system to be served by Apache or other HTTP servers. Again - this speeds things up immensely. If an image seldom changes, why run an SQL query every time for every thumbnail on a page? Especially when you can make the URL uniform and independent of where it's ultimately served out of?

What this means is that most of the time, all that is being retrieved out of Zope is the dynamic page template. It's an effective way of moving a lot of the static data out to where it can be served faster without losing out on the many benefits of Zope.

J. Shell, August 13, 2003 09:38 AM, in Zope

August 11, 2003

TV near its finest, HBO's The Wire has played out on my TV screen twice today (HBO East and Pacific feeds). It's become one of my favorite shows, alongside other famed Baltimore police drama, Homicide, Life on the Street. And now I know why - David Simon was responsible for the book that spawned Homicide (and wrote and produced many of its episodes). He's also responsible for the Baltimore based miniseries The Corner, which I have not seen, but did win an emmy (while The Wire received no nominations this year, even though its a critics favorite - a running theme with these Baltimore dramas).

Cable series, particularly pay cable series, seem to have an advantage over network television in that many of them can play out more like a novel or miniseries. The first season of The Wire was 13 episodes. All of the seasons of The Sopranos are approximately the same length. While Homicide was very character driven, it was also largely episodic. The Wire spans its story lines across an entire season. And, in fact, we continue to see the lives of the characters (dealers and addicts) that were the target of last season in this one.

Pay cable series also have the advantage of being able to fill up as much of their time as they want with no commercial interruption. The Wire fills up its entire hour. Meaning that, in theory, you get as much or more in two episodes as in most theatrical movies. But shows like The Wire, The Sopranos, Six Feet Under, etc, use television to its fullest as a medium. Being presented commercial free is a benefit - you get to sit back and watch. Being presented back to back until completion is a benefit - you're allowed to get drawn into a story, and not a chunk of new episodes followed by a month and a half of reruns followed by new episodes. HBO and their original programming are an established entity. As part of being commercial free, the shows are free of "ratings sweep stunts." And HBO's budget, awareness, and advertising is strong enough to be able to avoid other stunts. I understand that Showtime has an equally excellent crime drama, Street Time, that has had absolutely no promotion and, not surprisingly, few viewers (but again - critical praise). I've read that they are resorting to casting stunts for the new season. This may work out for them - it helped Homicide from time to time.

In any case, the current story of The Wire is nearing its end. I recommend catching it if you haven't. On Saturday, August 23, they'll be running the most recent three episodes prior to the finale.

This weeks episode, by the way, gave some a nice moment to Harper's and The Nation (as well as The New Republic and The Atlantic).

J. Shell, August 11, 2003 12:12 AM, in Review

August 08, 2003

I just got back from San Francisco, where I was manning the Zope.org booth at Linux World Expo.

The first thing I have to say is - San Francisco's climate is ideal, at least as far as temperatures go. It's jacket weather all the time. I enjoyed the city immensely, and finally got to go stand at the Pacific Ocean for the first time since a family reunion trip in Oregon in the early 1990's. Of course, I drank a lot, and had a memorable final drink with a couple of homeless people on the sidewalk at 2:30 in the morning on my last night there.

There were plenty of great shops as well, including a Prada store that I never was able to make it to. It's too bad that the big "Prada Epicenter" store for San Francisco will never be realized. And, fortunately for me, I only found one record store (which was basically next door to my hotel) with Merzbow. I broke my pledge not to buy any more Merzbow during the month of August, and have brought my Merzbow album count up to a still humble 20.

During the days, I found myself at Linux World. The show went well and I had a lot of encouraging conversations. Zope is fun to promote because it's such a unique system, for better or worse. Interestingly, one aisle over was a booth for Open ACS. I wanted to go over and talk to them and see how Open ACS was faring now that Ars Digita is long dead, with the Ars Digita ACS product being folded into Red Hat Enterprise Applications. I never made it over there though.

Linux World is not my personal bag of cats. And I found some things that made me giggle with delight. Namely, the email garden. The email garden is definitely a very nice free service that I believe is sponsored and ran by HP. But, using Red Hat and Mozilla, I found the set up quite interesting:

  • None of the Mozilla installations seemed set up to have the "password manager" feature disabled. Thus, every time you logged into a web mail client, a site like Zope.org, or even online banking, Mozilla would ask if you'd like to save the password. Since it's such a public space, I hope most people said "No" or "Never for this site." I find it interesting that such things weren't disabled automatically by those who knew they were public machines.
  • And here's my favorite - All of the machines had scroll wheel mice. But guess what the scroll wheel would do (at least, in Mozilla)? Nothing! Um... So, Linux is ready for the desktop? I admit - Red Hat's Bluecurve interface is nice and all, and KDE and Gnome are moving along nicely. But still - scroll wheel mice on all the computers that don't work? Someone dropped the ball here.

Finally, I had a chance to play Downhill Domination, a new mountain bike racing game for the PS2, at the Metreon. This is significant because I know a lot of the people involved with the sound and art production of this game. Against better financial judgement, I think I may go out and get this now.

J. Shell, August 8, 2003 04:14 PM, in Joy