Arc v' September 2002

As a followup to "On .Mac, part 1 - Services", my hopes and wishes for iSync to be released and offer a unique service over .Mac came true - somewhat. iSync is in public beta form, and for some reason or another, much of the .Mac network went down for approximately three hours last night. Myself, and a number of other users, found ourselves locked out of registering machines with our .Mac accounts. Apple fixed the problem earlier today, and registration seemed to go well. Now I have the task in front of me to actually unify my Address Books and core calendars down to the unified and synchronized unit I've been craving.

The story's been going around about Apple extending .Mac memberships. To recap what .Mac is, it's Apple's new umbrella name for their internet services, in some ways reminiscent of .NET Services. It's also a rebranded entirely-for-cost version of Apple's previous iTools services.

This for-cost issue has caused a lot of contention in the online Macintosh community as this feature that was supposed to be "free for life" is now $50-$100 per year. But, as the so-called dot-com era has show us, "Free For Life" often means "free for the lifetime of the provider, which is often very short indeed". The other free disk space services were often slow, had difficult to use web interfaces, and ultimately - they're gone. Aside from Hotmail and Yahoo's free mail services, most of the other major ones are gone - although to my surprise, my Excite account is still alive, but the email account was (unsurprisingly) wiped out.

And even Hotmail's for-free feature set keeps shrinking - and with good cause. It's a huge strain to provide service and uptime for these services, a fact that no one appreciates when things are running well but everyone will raise a ruckus about when services go down. If you take the current list of Apple's iTools accounts and multiply it by the basic size of services offered under the old free banner (20Mb iDisk space, 5Mb Email storage), the resulting amount of disk space involved is immense. Many of these accounts are hardly ever used - until I paid for my .Mac account, I seldom used the iDisk. Many people admit to having multiple accounts for varying reasons too. The reasons include forgotten passwords (so they just created a new account), running multiple sites, or just setting up an account quickly for assorted other reasons and then discarding the account. But on Apple's side, whether any of these accounts are commonly used or not, they're expected to be available instantly.

A friend of mine who previously worked at a company that did messaging services for large corporations told me about their system - they had boxes and systems that theoretically could house thousands and thousands of accounts on a single machine. But the company's biggest selling point was uptime guaruntee, and these boxes needed to be restored from a backup in less than an hour. This was impossible to do if you maxed out a machine's potential - a restore could take 22 hours. So, hard limits (500 accounts) and high prices were put on these services because of the work involved in the service availability guaruntee.

So Apple has many of the same issues to deal with, I'm sure. If the Backup software Apple offers to its .Mac users stores its files to iDisk, my iDisk data had better be there when I need it! And of course, I always expect my mac.com email to be available (and over the years since iTools started, there have been downtimes, but nothing major and nothing recent).

So, a couple of weeks ago, I payed for my subscription. Partially, it was to keep the email address (which I don't use too much any more, except for some mailing lists), but it was also because I was interested in what .Mac (and similarly, whatever Microsoft's .NET services turn out to be, which may be MSN 8) means to the continued integration of the internet and web into more common pieces of our computing life beyond the browser. Apple has done a good (and arguably sneaky) job of integrating .Mac into "Mac OS X":

1. It's very easy to mount your iDisk in the Finder, and since switching iDisk's protocol from AppleTalk to WebDAV (I believe), the connection stays up forever (AppleTalk would time out, and I don't know how well it survived a 'sleep' operation). Speed is pretty good.
2. It's very easy to same to the iDisk from any application. And not just to the iDisk itself, but directly into its top level folders. This can be done without even expanding the navigation browser of any save-file dialog.
3. The integration with the iApps, particularly iPhoto and iCal are nice. Neither app requires a .Mac membership to work, but using .Mac offers some nice conviences for sharing photos and calendars. And iPhoto, like Windows XP, offers nice integration with other web services for ordering prints and photo books directly from the application.
4. If the conspiracy theory I'm sticking with about why Apple's extended the discount .Mac signup period hold true, then we should be seeing iSync soon. And iSync might offer the most compelling reason yet to use .Mac (I hope and hope and hope it does) - doing palm like synchronization of Calendars and Contacts between Macs, using .Mac as the central repository/transport. How this turns out remains to be seen, but it's my not-so-secret wish for the future.

I expect there to be more services in the future, and since I and many others are now paying for those services - Apple now has a bigger incentive to make sure the services keep humming along. And, since I and many others are now paying for those services, the users have more incentive to actually use them. I've been a fairly happy subscriber thus far, and in the next post I'll talk about the impressive Web Applications Apple has running, and some ruminations about the state and use of different web application servers.

Post Rain, a collection of unaltered sunset lomographs (to grow as more get scanned) taken after a significant late summer storm.

Want A Date? We All Cal With ICal (Thanks to Scripting News for the link)

Whats interesting about this (to me) is that Apple went ahead and and said, "lets use a standard format (vCal) for this thing, and just upload it to the server. Lets let each individual machine sort out what to do with the information containted in a calendar."

Contrast this with Microsofts .NET MY Web Services initiative, where calendar sharing was to be one of their for pay features. MS was claiming that federation, or the ability to have more than one host for the information, was part of the plan from the beginning. But because they owned the xml structure and were tightly controlling how the whole business worked, no-one on the outside woud have a leg up on how to implement a federated server. (I suppose you could buy the federated server package from MS?)

In contrast, Apple went with two established standards (vCal & DAV) and didn't try to own the file format.

If there is information that must be shared, the file format should be in the public domain and subjected to peer review. Don't go crazy with that, since you've got to ship sometime and mistakes can be fixed (as long as you always leave one bit in a bit field to mean: more data coming!)

Note that vCal isn't XML. It doesn't need to be. XML and vCal are just file formats. [Steve Zellers' Radio Weblog]

vCal is a decent format. I really like the concept of iCal as a subscriber/client of multiple HTTP served calendars. The day iCal was announced, I threw together a 24Tix calendar of upcoming shows as a Python Script - so it's dynamically updated with new shows as they're put in the system. It took a few minutes, basically of reverse engineering an iCal export in vCal format, and then tracking down the vCal spec to see what else I could/should do with the system. As a result, we have this calendar (subscribe), complete with venue information.

There's still one missing piece - I still want one personal calendar, even across three machines (workstation, home iMac, iBook). I guess this is where calendar servers come in to play, and maybe it is what .NET My Services could offer. At MWNY, Steve Jobs hinted that the upcoming iSync application could sync multiple Macintoshes over .Mac. Just the notion of this one feature is enough for me to pay for .Mac (which I did do, just a couple of days ago). There's part of me that thinks that the Backup software offered as part of .Mac (or usable separately) could be used to get a cheap version of this now. But I'm a little leary. And I'm a little leary of using Backup because, again, I have three machines. Two of which are in active use. And as I learned during my recent iMac clean re-install, Mac OS X is very easy to restore to if you just preserve some aspects of your home directory (the layered architecture of how Mac OS X handles preferences is a delight compared to the overstuffed system folder of the classic Mac OS). Will Backup let me back up both machines to the same account without trampling on each other? I guess I'll find out sooner or later.

So, I still have a dream of being able to easily maintain one calendar, one address book and even one quicken file across multiple machines. I'd love to be able to sync Quicken (even if it just meant "copy the file") to my iBook when traveling (I use Pocket Quicken on my palm, and love it, but sometimes I've wanted to have the full package there (primarily before Pocket Quicken 2.0)) and a couple of OmniOutliner outlines as well. I want to sync up from my iMac while brushing my teeth before heading to the office, and sync down to my G4 Workstation when I get there. And then do the same thing before I leave the office.

This is the service that I want - especially if conduits can be made to do it intelligently, like Pocket Quicken - but between Macs instead of a Mac and a Palm. If iSync can deliver on this, than the $50 I shelled out for .Mac will be more than worth it.

Then, I get home to find a new issue of The New Yorker in my mailbox, as well as keys to one of the big package mailboxes - my copy of Throat's 60" somewhere / 60" somewhere else CD's. EUCCI (myself) appears on 60" somewhere - a collection of one minute unedited field recordings (which are remixed on 60" somewhere else, also in one minute intervals).

What I'm most amazed about in the past week is how well quiet noise (ahem) works such as 60" somewhere else or Nurse With Wound's A Missing Sense (particularly the reissue featuring the Ostranie 1913 version of DADA - in this sense, Merzbild Schwet works equally well) blend into the eclectic Pizzicato 5 album さ・え・ら ジャポン (sa e ra japon). The transition is unbelievable. And delightful.

(
Synth.play({
	var home, fname, file, fname2, file2, dur;
	home = "bliss:bliss:bliss:sweetcorn:Eucci-2:z.New Eucci:0a.sn---ns-:";
	fname = home ++ "hyrum57_01";
	fname2 = home ++ "bassDI_01";
	file = SoundFile.new;
	file2 = SoundFile.new;
	dur = 5.0 * 60;
	file2.readHeader(fname2) and: file2.preloadData;
	file.readHeader(fname) and: file.preloadData;
	a = DiskIn.ar(file, true) * 1.4;
	d = DiskIn.ar(file2, true);
	e = FSinOsc.ar(11800, mul:Crackle.ar(1.8));
	b = KateShuffler.near([d,a], 2.5, 0.7, 0.05)
	  + KateShuffler.near([d,a], 5.0, 1.2, 0.02)
	  + KateShuffler.near([d,a], 5.0, 0.2, 0.12, keep:true)
	  + KateShuffler.near([d,e,d], 1.25, 0.3, 0.08, keep:true);
	c = (RHPF.ar(b, 4180, 0.3) + RLPF.ar(b, 480, 0.13));
	j = KateShuffler.near([b,c], 1.5, 1.4, 0.0021, keep:true)
       + KateShuffler.near([b,c], 3.0, 0.7, 0.001, keep:true);
	j ring1: OnePole.ar(LeakDC.ar(j, 0.35), 0.35) * 0.8;
})
)

It's rough working on a project with little or no software design. I'm a little surprised by this, but after a year of being primarily independant or working with a very small company, it's been made very obvious. We try to do something up front in some quick analysis sessions, and even do some agile modeling (of sorts) to sketch out table/object relationships. But it's not long before the typical ruts happen - even the most agile of models can become difficult to update when you're effectively an army of one, when you have no customer contact, and the real mastermind behind the project is extremely busy with their own projects.

I can come up with a system out of my ass, but there's so much time being spent doing forced thinking - and most of it off the cuff and right into the application server. Remarkably, most of it is turning out well - just slowly.

There should be plenty of upcoming work from Eucci & Co.. I spent much of the weekend gathering and compiling a near final mix of a release that should show up on No Type as part of the Sine Fiction series. New ways of tape rape, or in the words of William S. Burroughs, "Clearly the whole defense must be experiments with two tape recorder mutations."

To add to all of the computer fun I've had recently, yesterday I dropped my Palm m125, sending batteries flying everywhere. I hurried and grabbed them and put them back in, but the Palm had already gone into a semi-reset state. When I turned it back on, I was presented with a screen asking "Erase all data? Up button for Yes, any other button for No.". Not wanting to erase anything (although there was little new data on the device), I pushed any other button numerous times, only to be presented with the same screen. So, my Palm is reset. And, due to a last minute baseball game invitation, it's still at the office.

Nicely, I no longer have to worry about how well a HotSync against a fairly fresh Mac OS X re-installation (clean install) would perform.

Miette is crawling her way back to life. My Web Confidential and Quicken files were saved. And I found a blog of an old friend / coworker today. But, there's still a ways to go before I can really start catching up.

The saving of the iMac could not have happened without the iPod. I was able to turn it into an emergency Mac OS X boot disk loaded with some drive utils. It was almost cute - the scrappy and increasingly multipurpose little iPod trying to save a sisters life. They ran throughout the night, but it was ultimately in vain. The disk utils said "hey, we can help with these things over here, but it will come at the expense of these things over there". After wrapping up some backing up to the iPod and a separate hard drive, the re-install process commenced, and the hard drive's gotten a good cleaning.

Here's hoping the hotsync goes well.

So, the main partition of the iMac was having serious issues, so it was time to do a hardcore clean re-install of Mac OS 9 (going through installing 9.1, 9.2.1, and 9.2.2) and 10.2. whew.

Tonight, the restoration process begins, and new posts will start showing up a bit more regularly.

I had severe problems this morning trying to install Chimera 0.5 (a fully native Mozilla based Mac OS X browser) that took many restarts and some terminal trickery to finally abate. I still don't know the extent of the problem, which is likely unrelated to Chimera itself. But - heads up!

quick explanation of the problems I saw - when trying to copy the new version of Chimera over the old, the Finder would completely freeze up at the beginning of the "Preparing to copy..." process. When trying to throw version 0.4 in the trash, and then emptying the trash, the same problem occured. Then, the Finder would freeze up after login, leaving the system unusable. Even trying to copy the new Chimera in via the Terminal would freeze things up. And all of these freezes spread rapidly to other parts of the UI level (ie - force quit was unavailable, new applications couldn't be launched, etc).

Ugh. Very bad.

Coming in late from a mild evening at dueling piano bar (it's usually quite rowdy, but tonight was surprisingly staid), I caught the latter parts of what appeared to be a terrific PBS Frontline titled Faith and Doubt and Ground Zero. It was a bit late for the show to get my full attention as it deserved, but it looked to be very well balanced and often quite powerful. It seemed to mark on a quite powerful and probably oft-repeated thing: a difference between faith and religion.

I think people who find faith are rather humanist - in my definition, they are people who have found strength in an answer they feel is very right for themselves, and don't necessarily need religion to reinforce it. They believe in the power of the laity over that of the clergy, the noble notion that Man can be genuinely good whether he praises their God, another God, or no God at all. They are often the most effective religious leaders because they preach to the dignity of the individuals rather than to the might of the clergy or traditions or other actions used to hold power over others.

But so often, there are those who just find religion. It's the words, the repititions, and the actions that come to matter more than feeling. They actively recite their gods, prophets, and other clergy leaders words but are often the furthest from any real connection with their God. They can even be dangerous with their view of superiority - they have the right way, we do not.

The story from the little bit of this Frontline special that I watched that stuck with me was the interview with the Lutheran minister who said the prayer at the big inter-faith service held at Yankee Stadium last year for the families of those still missing since the September 11th attacks. He shared the podium with representatives of all the major faiths. And almost immediately after this event, he started getting very hateful messages from people within his tradition, calling him a terrorist. Within two weeks, clergymen from his religion had gathered a petition stating that he should not be allowed to preach, that his collar should be removed, that he should be kicked out of his denomitation, and basically that he wasn't a true Christian. Why? Because "their belief is that the doctrine of the church does not allow a Christian to stand at the same podium with someone of another faith or everybody is going to get the same idea that all religions are equal, and we have made absolute claims, exclusive claims about our faith." (link). He goes on to say:

If religion leads people to make these kinds of accusations at exactly the worse moment in American history, then what's underneath religion? Is religion really part of a lust for power and control in people's lives? Is it a desire for absolute security so strong that people cannot see the need to reach out and help? If that's true, then I've got a lot of wrestling to do with my own religion.

This was sickening to hear, but (sadly) not shocking. Someday, we will learn that no one, but no one!, has an absolute divine superiority granted to them over everybody else, and that any such sense of superiority does not justify violence of any sorts. Here's a man, a good man in his faith, saying a prayer at a very powerful and hopefully helpful prayer service for the genuine living sufferers of a terrible terrible event, and he's being called a heretic and terrorist? And clergymen in his own "faith" are seeking to drive him out?

sigh. Sometimes, I'm an absolute humanist - I see those of strong faith (which may or may not be in any religion - they may just have comfort in themselves, friends, or family, or pure humanism itself) making a positive impact on the world by generally good actions that come as a result of their faith. But a lot of the time, I'm in the misanthropy camp, seeing the despair and ruination brought on the world by the so-called Religious, who will see a brother of their own tradition doing something good, but finding only bad in it because it violates the supposed One True Way.

It's very late, so I'll try to make this quick and write up more later. Today, I finally had enough free time in my coffers to download "Python" from CVS (aka - 2.3 pre-alpha) onto my old iMac, and augment that with "Zope 3" - the ComponentArchitecture project. Then I installed the simple but usable Job Board Example. I soon noticed that said JobBoardExample was missing out on a couple of things - an editing interface, and an XML-RPC interface.

One of the notions of Zope 3 is that it's supposed to be easier for developers to embrace and extend other components by being able to write new "views" for them. So, I decided to write a JobEditView. I made a new Product (esentially, a plain Python package) called JobBoardEditor, and filled it with a couple of files - EditJobView.py, edit.pt (the page template to edit a single job with), and configure.zcml, the configuration file that ties it into the Zope system. It wasn't long until I could go to a Job object and traverse to my new edit form, and then have the submit go to my 'edit' method. But, at this point, I'm running into security issues that are beyond my measure to figure out, especially at 1:00 am with a beer headache. ";->"

My other thing to try out was putting a simple XML-RPC interface on to the JobList. This was particularly interesting to me as I've found that while traditional "Zope" method calls should - in theory - work fine with something like XML-RPC, they seldom do. Sometimes it's because too much HTML is returned from a call, but most of the time lately (for me at least), it's that I want the server to do some processing of the results before sending them out to the client, using the relatively simple data types afforded to XML-RPC. As a result, I've been adding in extra Python Scripts particularly for XML-RPC, and letting them work in the proper places either via acquisition or some other clever tricks. It's not a bad system, but it's not exactly formalized or repeatable, since it's not in a formal Product.

In Zope 3, this is different. I wrote a simple XMLRPCView class. 'View' objects in Zope 3 have two state members - 'request' (the incoming request object), and 'context' - the object the View is applied against. So, for me to add in the new functionality whereby I could just get a list of approved Job titles, I wrote the following Python file in my JobBoardEditor product:

from Zope.Publisher.XMLRPC.XMLRPCView import XMLRPCView
from ZopeProducts.JobBoardEx.IJob import JobState
 
class JobListXMLRPCView(XMLRPCView):
    __implements__ = XMLRPCView.__implements__
 
    def listJobTitles(self):
        joblist = self.context
        job_ids = joblist.query(state=JobState.Approved)
        out = []
        for jid in job_ids:
            out.append(joblist[jid].summary)
        return tuple(out)

Then, I added the following to JobBoardEditor/configure.zcml (after adding xmlns:xmlrpc='http://namespaces.zope.org/xmlrpc' to the head zopeConfigure element):

<xmlrpc:view
    name="joblistmethods"
    for="ZopeProducts.JobBoardEx.IJobList."
    factory=".JobListXMLRPCView." />

With this configuration statement, I've added a new View to be published on the XMLRPC channel (which in Zope 3 runs on a different port than normal HTTP requests, which is probably wise) for objects that implement the IJobList interface. I was able to do all of this outside of the JobBoardEx product. I just had to ensure that the Product was added in after JobBoardEx by using the site's products.zcml file to affect the ordering.

After restarting Zope 3 (which is a very slow restart on this machine), I was able to do this:

Py$ s = Server('http://localhost:8281/zoo/joblistmethods')
Py$ s.listJobTitles()
['test test test']

I actually had created two jobs through the web interface, but had only approved one. So I ducked in to the web interface to approve the other job, and then made the call again:

Py$ s.listJobTitles()
['test test test', 'the other job']

So, what are some benefits of this? One that comes to mind is issue trackers. Earlier in the day, I was writing a Python script to farm through some Tracker data and format it nicely for XML-RPC, with the intent of writing a simple AppleScript Studio application to list pending/accepted Tracker issues. I did this informally, dropping a Python Script into the folder above a particular tracker. Ultimately, it didn't work - and I think a lot of the blame goes on XML-RPC's utter disregard for semi-proper security. Since Zope's always published objects on the web with regards to web security standards (or embarrasments, depending on your view) such as 'Basic Auth', it's had problems with the "security by passing in arguments" design of many XML-RPC API's. But the point I was really going for was packaging. Even if the Tracker Issue finding Python Script had worked over XML-RPC, now I have to find other places to put it by copying and pasting it, ultimately adding another name onto an ever growing namespace.

With Zope 3, however, I could write an XML-RPC view particularly for the application I was designing as a normal Python component, and install and configure it as necessary (and hopefully that configuration work will be better than copying and pasting plain Python Script code). I can name it specially, essentially adding a new namespace for my application. I can distribute it to other users fairly easily.

Another example would be implementing the Blogger API not as a new weblog product for Zope, but as a view/adapter component that publishes on the XML-RPC channel, and communicates with other Zope components, either built in ones or third party.

In summary - my first tangible Zope 3 experience in quite some time has been hopeful and helpful. There's still a long way to go, but the kernel is shaping up nicely. The pattern driven / "extreme programming" sprint driven development has yielded a decent - but still shifting - code base that should be stronger and simpler than Zope 2. Sometimes, there are enough levels of indirection in Zope 3 to make it look more complex, but so far most code has been easier to follow and trace than Zope 2 (although there are a number of empty folders/packages that probably need some brute force removing from CVS) - at least, it's usually easier to tell what's going on (and even this has an at least - at least, once you start learning the new paradigm). Zope's ancester, Bobo (the kernel on which Zope is still based), was all about abstracting the means of publishing an object on the web away from the object itself. Bobo, and then Principia / Zope 1.x, built upon this, but primarily in a single-UI / single protocol type way. It's obvious today that there are many ways of looking at things on the web, including never actually using the web itself. Zope 3 brings us back to the notion of an object publisher being able to serve up objects on different protocols. Zope 2 offers this today, but there is great difficulty in making WebDAV, normal HTTP, FTP, and XML-RPC all live harmoniously in the current design.

And finally, Zope 3 should yield a usable scalable means of adapting the works of other developers into new solutions by providing better control of product/service/view configuration and overrides, such as adding a new XML-RPC API to someone elses bug tracking system without having to alter that bug tracking system, or use secret Python hacks to alter behaviour.