William Rivers Pitt of TruthOut.org has an excellent and personal article about Hunter S. Thompson. Regrettably, I have not read any of Thompson's works. But now I'm even more interested. There seem to be so many parallels between his 1972 writing Fear and Loathing on the Campaign Trail and the recent election cycles.
I think I'd like Thompson for the same reason I love William Burroughs - it's balls out, dirty, raw, and true (or just eerily poignant). And they lived their lives this same way.
From Loud Thinking:
One of the clear goals for Rails from the start was to stay at the infrastructure layer. I didn't want Rails to succumb to the lure of high-level components like login systems, forums, content management, and the likes.
[The case against high-level components]
I agree. But I'd like to add that the case is not against component systems / architecture, it's just against this idea that one-bulletin-board-fits-all is how component architectures should work. Components work best when they're small grained and interchangeable. Yes - being able to nicely integrate a complete bulletin board system into a customers site is a nice thing to have. But in my experience, the work often required to do a good integration in a lot of these systems seems to be about equal to writing one from scratch. This is not always the case - I was able to integrate two very different systems with different histories a few months ago, but there were a couple of advantages there: they were both written in house, and both had put some effort towards just enough customization and control-layer separation that it could work.
I've been using a system that a colleague has been building in house, and it's basically a chain of fine-grained components that operate on data as data flows to and from storage. As a result, we have a system that gives us so much more than most generic form display / validation tools for presenting and working with data and user input, more flexibility than most object-relational mapping systems, and tight granular security. There are parts of the system that require a fair amount of work, most of it in configuration (lots of XML sit-ups, as the Rails community might say). But in my experience so far, once the components are wired together they've proven to be very dependable. Once one part of an application is complete, it rarely needs touching. It's difficult for other things to interfere with it. Built around the concept of "no alarms and no surprises," magic is forbidden. Explicit is better than implicit. And the components are given only enough information to do their specific job - whether it's validating input, presenting data on screen, or preparing data for storage.
I first wrote up how such a system might work a couple of years ago in my post "Component-Relational Mapping?" and that particular system has suited me well in a few projects. This other system is simultaneously a much larger (it stands alone with no requirements on Zope or anything else, and doesn't even place any real requirements on how one stores data) and a much simpler system (the core concepts are really simple). Applications built on it are a good example of applying small, tough, and smart components to get jobs done.
I just added Loud Thinking to my subscriptions list, and hope to have time to actually go through and read it soon. Loud Thinking is the David Heinemeier's weblog. Heinemeier is a big dynamic language advocate and is responsible for Ruby On Rails, a Ruby web application toolkit that's getting considerable buzz lately.
I'm not blown away with Rails. I'm personally more interested in toolkits like Seaside and Wee. Many other pieces of Rails I've seen done in Bobo / Zope to varying degrees. I still believe that Bobo, aka the "Python Object Publisher" is a significant piece of web technology and is still a fundamental part of Zope 2 and 3, wherein an HTTP (or FTP, or other) request is translated into an object call. The Python Object Publisher is an ORB [Object Request Broker] in this sense. Beautifully simple and powerful. Don't let the complexities of the systems built on top of it (Zope 2, etc) fool you.
Anyways, Heinemeier seems to have a lot of interesting ideas and is willing to happily talk about them. Contrary to what you may read in trade rags, the web is not dominated by J2EE. Almost all highly popular and/or innovative web sites are implemented in, or were prototyped in, an agile language such as Python, Ruby, Lisp, or Perl. Movable Type? Perl. LiveJournal? Perl. Google? Prototypes in Python. Yahoo Maps? Python (I don't know if the current version still is, but it was a Python one for a long time). There are some very large news organizations running on Zope (Python). There are a growing number of popular Ruby web applications and sites as well.
There are often grumbles in the Python community about there being "too many Python web frameworks." and wondering what can be done to compete with J2EE. Why compete with J2EE? I know of few who are happy with it. How many Java Web Frameworks have sprouted up in recent years as alternatives or add-ons or extreme enhancements of J2EE's web support? Think Java Server Faces. Think Struts. Think Naked Objects. Think Hibernate. And those are just the names I know of as an extreme outsider. I think it's good that there are different web frameworks and toolkits for Python. Part of being agile, I think, is having a good set of tools to choose from to get a job done. It's far better than a rigid specification that is meant to copy the rigid specifications of a technology that no one seems to really enjoy using.
Although - it would be nice if some toolkits or parts of toolkits were made to be more standalone. I'd love to use Nevow's stan in my Zope and FD4 (an in-house data management framework). Stan is an S-expression like system for generating HTML in Python that actually looks good and one isn't scared to maintain (a simple stan example can be found here). Being able to use stan without the rest of Twisted or Nevow would be nice, just like there are many implementations of the wonderful TAL.
Phil Windley (former CTO of Utah!) writes about Running Code and Regular Releases:
Several interactions I’ve had recently with groups building Web applications have led me a renewed appreciation for the power of running code in a development project. Both of these organizations were some time into a large development project and still didn’t have running code and regular, consistent release cycle. This wisdom gets great lip service, so it surprises me to see people who should know better not following it.This blows my mind. I have never been in a situation on the web where I didn't have running code. Release cycles vary from project to project, but I can not think of a situation where you don't have running code.
Yesterday, I moved my code off of the Zope instance I tend to use for development and into a separate instance (a separate running Zope server). I first moved my core application, along with its two supporting frameworks - one for the data management the application is built on, one for the simple workflow engine the application uses - into the new location. I fired off my unit tests and they all passed in the new location, even without having all of the supporting Zope Products in place yet that were used by the web interface. It was gratifying to see that all my tests succeed and know that the application could, at this point, move just about anywhere. And that I had a reasonable assurance of its success by having so much running code in place already.
Perhaps I've lived too long in the world of dynamic languages like Python where the compilation step is absent. Or perhaps it's that I first started programming on a machine with an omnipresent BASIC interpreter which allowed me to see immediate results. I've never been able to fathom a world of blind design-and-hope development. I could see how large systems with scarce resources were scarce could be built this way, but those should be few and far between now.
There's a pretty good article at ONLamp.com about More Test-Driven Development in Python. It's basic unit test work, showcasing writing tests first and code second. I'm never consistent when it comes to unit testing. On some projects, I use them. On others, I don't. Some projects use code that's been around for years and has no tests. Sometimes, it's just really hard to unit test code meant for Zope 2. And when I say that, I mean underlying Python Product based code, not scripts and forms and other UI related elements. My Zope 2 projects have less and less code managed in the ZODB and more from underlying Python objects.
Zope 3, on the other hand, advocates testing heavily. In the developer documentation for writing a new content object, writing unit tests is step #4, just after preparation, initial design, and writing interfaces. Zope 3 also uses Python's doctest system for writing tests inside of prose. The tests are example code that can be executed in the Python interpreter. Functional Tests are also covered in Zope 3, which are tests that can test a whole application and its UI, whereas unittests and doctests are typically used to cover small components and code outside of their environment. The Zope X3 3.0.0 release itself contains nearly 4000 tests in unit or doctest form. It's great to see testing promoted so heavily in its documentation.
On a recent project, I was reminded (again) of how nice it is to have unit tests. I was running behind schedule and had a few simple tests written to cover some early parts of the system, and I was fast approaching the heart of the application - a complex workflow interaction. I knew that testing it by hand would be menial and terrible, because it would require so much repeat typing and multiple web browsers to be open. But since I was behind schedule and had no UI to show progress with, I kept putting off writing tests. The big turnoff was building the test harness - getting a lot of what would have been repeated manual entry into some setup code. I think that unlike most unit tests, testing workflow or any other state-sensitive system can be a bit more difficult because so much work has to be done to get the system to a particular state where you can test a transition or value. It can be daunting, and daunting items can be easy to put off until another day.
But when I got to the point where I started really having to implement this workflow system, I knew without a doubt that I would be sunk if I didn't have tests to automate the process. So I hunkered down and wrote a Python module in my tests directory called support, and filled it with classes, functions, and dummy objects that could be used in unit tests to build up the system. As I tested each piece of workflow, I could add the successful steps of those tests into the support objects so that other tests wouldn't have to duplicate the work. Getting the test harness and initial tests in place took nearly a full day of development time, but it was worth it. I didn't have anything visual to show anybody yet, but it made everything so much more dependable. And I ended up making my basic deadline anyways.
The moral of the story, which is one I have to be reminded of constantly, is that it's worth it to spend the extra time to make a supporting test harness, even if it appears to be a difficult task. Too often I fall victim to the thoughts of "I just need to get this done, I don't have time to start writing tests" and too often I find myself becoming the victim of "man, I wish I had tests for this!" later down the road when it feels like it's too late.