Here’s Andy Baio’s farewell to Upcoming, a service I’ve been involved with for a decade. In a March 2005 blog post I wrote about what I hoped Upcoming would become, in my town and elsewhere, and offered some suggestions to help it along. One was a request for an API which Upcoming then lacked. Andy soon responded with an API. It was one of the pillars of my Elm City project for a long while until, as Andy notes in his farewell post, it degraded and became useless.

Today I pulled the plug and decoupled Upcoming from all the Elm City hubs.

In 2009 Andy and I both spoke at a conference in London. Andy was there to announce a new project that would help people crowdsource funding for creative projects. I was there to announce a project that would help people crowdsource public calendars. Now, of course, Kickstarter is a thing. The Elm City project not so much. But I’m pretty sure I’m on the right track, I’m lucky to be in a position to keep pursuing the idea, and although it’s taking longer than I ever imagined I’m making progress. Success, if it comes, won’t look like Upcoming did in its heyday, but it will be a solution to the same problem that Upcoming addressed — a problem we’ve yet to solve.

That same March 2005 blog post resonates with me for another reason. That was the day I walked around my town photographing event flyers on shop windows and kiosks. When I give presentations about the Elm City project I still show a montage of those images. They’re beautiful, and they’re dense with information that isn’t otherwise accessible.

Event flyers outperform web calendars, to this day, because they empower groups and organizations to be the authoritative sources for information about their public events, and to bring those events to the attention of the public. The web doesn’t meet that need yet but it can, and I’m doing my best to see that it does.

My next community calendar workshop will be at the Peninsula Fine Arts Center in Newport News, on Tuesday April 23 at 6PM. It’s for groups and organizations in the Hampton Roads region of Virginia, including Chesapeake, Hampton, Newport News, Norfolk, Portsmouth, Suffolk, Virginia Beach, Williamsburg, and Yorktown. If you’re someone there who’d like help change the way public calendars work in your region, please sign up on EventBrite so we know you’re coming, or contact me directly.

Here’s the pitch from the workshop’s sponsor and host, the Daily Press:

The Community Calendar Project

It’s about time someone came up with a way to get all community events in one place so everyone, everywhere can find out what’s going on at any given time, on any given day.

It’s about time creators of those events – the people, agencies and organizations who work so hard to bring quality education, support and entertainment to the community – had a way to get their messages out there effortlessly.

It’s about time the public can find out about the happenings and events they really care about and never miss an important event again.

AND it’s “time” – or the lack of it – that makes this community initiative being spearheaded by the Daily Press so valuable to everyone. This community calendar will SAVE time – for the event creators, the event seekers and the websites and platforms that work to make this information available.

The Daily Press is partnering with Jon Udell of Microsoft to bring this project to Hampton Roads and make it among the first communities in the country to have an easily searchable, FREE database of events available to the public. And we want to get all of Hampton Roads involved. The only thing required to participate is to agree to use an iCalendar formatted calendar on your own websites or to create events through Facebook. That’s it. Participation guaranteed.

What is an iCalendar? Simply, iCalendar is a computer file format that allows Internet users to exchange calendars with other Internet users. iCalendar is used and supported by personal calendars such as Google Calendar, Apple Calendar (formerly iCal), Microsoft Outlook and Hotmail, Lotus Notes, Yahoo! Calendar, and others, and by web content management systems including WordPress, Drupal, Joomla, and others.

Many of you may already use one of these applications to publish your calendars online, and that is great! That means you can already participate in the calendar network we are bringing together. The rest of you can easily convert and get on board.We’ll tell you how.

On April 23 you are invited to a presentation of the Community Calendar Project. Jon will be on hand to tell you what it is, why it matters and how to get involved. The gathering will take place at 6 p.m. at the Peninsula Fine Arts Center, 101 Museum Drive (across from The Mariners’ Museum) in Newport News.

Light refreshments will be served. Get your FREE tickets so we know how many are attending.

Hope to see you there.

Bookstores, for all the obvious reasons, are hanging on by their fingernails. What brings people into bookstores nowadays? Some of us still buy and read actual printed books. Some of us enjoy browsing the shelves and tables. Some of us value interaction with friendly and knowledgeable booksellers. And some of us like to see and hear authors when they come to speak and sign books.

There are lots of author events at bookstores. Recently LibraryThing’s Tim Spalding tweeted:

Upcoming bookish events on @LibraryThing Local now over 10,000! http://www.librarything.com/local/helpers

It’s great that LibraryThing “helpers” (individuals, libraries, bookstores) are adding all those events to LibraryThing’s database. But I’d really like to see bookstores help themselves by publishing standard calendar feeds. That way, LibraryThing could ingest those calendars automatically, instead of relying on dedicated helpers to input events one at a time. And the feeds would be available in other contexts as well, syndicating both to our personal calendars (desktop-, phone-, and cloud-based) and to community calendars.

When I saw Tim’s tweet I took a look at how bookstore events are feeding into various elmcity hubs. Here’s a snapshot of what I found:

There are three buckets:

Bright Lights: These are stores whose web calendars are accompanied by standard iCalendar feeds. Events from these stores appear automatically in the Monadnock, Boston, Ann Arbor, and Portland hubs. These stores’ calendars could also be ingested automatically into LibraryThing, and you could subscribe to them directly.

Dim Lights: These are stores whose web calendars are hosted on Facebook. There isn’t a standard iCalendar feed for Facebook calendars, but the elmcity service can synthesize one using the Facebook API. So I say that these stores have “indirect” iCalendar feeds.

Dark Matter: These are stores whose web calendars are available only in HTML format. Some of these calendars are handcrafted web pages, others are served up by content management systems that produce calendar widgets for display but fail to provide corresponding feeds.

There are a few Bright Lights and some Dim Lights, but most bookstore calendars, like most web calendars of all kinds, are Dark Matter. If you’re a bookstore I urge you to become a Bright Light. Making your calendar available to the web of data is as easy as using Google Calendar or Hotmail Calendar. It’s a best practice that bookstores disregard at their peril.

In Book: A Futurist’s Manifesto, which is taglined Essays from the bleeding edge of publishing and is co-edited by Brian O’Leary and Hugh McGuire, there’s a refreshingly forward-thinking chapter on public libraries by the Ann Arbor District Library’s Eli Neiburger. In The End of the Public Library (As We Knew It) Eli describes an intriguing model for libraries as purveyors of digital stuff. If you’re a creator of such stuff, and if you’re willing to sign the AADL Digital Content Agreement, you can license your stuff directly to the library:

The Agreement establishes that the library will pay an agreed-upon sum for a license to distribute to authenticated AADL cardholders, from our servers, an agreed-upon set of files, for an agreed-upon period of time. At the end of the term, we can either negotiate a renewal or remove the content from our servers.

The licenses specifies that no DRM, use controls, or encryption will be used, and no use conditions are presented to the AADL customer. In fact, our stock license also allows AADL users to download the files, use them locally and even create derivative works for personal use.

Pretty radical! Why would you, the creator of said stuff, want to take this crazy leap of faith? Eli explains:

Instead of looking at the license fee as compensation for something like a one-time sale, the pricing works when the rightsholder considers how much revenue they would like to expect during the license term from our 54,000-odd cardholders. For niche creators, it’s not hard for the library to beat that number, and all they have to do to get it is agree to the license and deliver the files to our server.

They’re not releasing their content to the world (especially because it’s already out there). They’re just granting a year or so of downloads to these 54,000 people. They get more revenue than they would likely get from those people up front, and the library gets sustainable, usable digital content for its users.

Eli thinks this won’t work for in-demand mass-market stuff anytime soon, if ever. But as he points out:

When everything is everywhere, libraries need to focus on providing — or producing — things that aren’t available anywhere else, not things that are available everywhere you look.

Of course public libraries have always been producers as well as providers. Things libraries produce include local collections, like the Keene Public Library’s exquisitely-curated historical photos and postcards and the Ann Arbor District Library’s The Making of Ann Arbor.

Libraries are also producers of community events, and here’s one I’m delighted to announce will happen at the Ann Arbor District Library on September 26:

A Seminar on Community Information Management

Everybody lives online now. Knowing how to collect and exchange information is now as important a skill as knowing how to drive, but it’s not enough: in order to make the web really work for you, you have to know how to project yourself online, and how to manage the boundary between what’s private and what’s public.

Cities and towns need to know this too. From the mayor’s office and local schools to the slow-pitch league and the local music scene, communities need to have these same skills if they are to survive and thrive in the 21st Century.

This seminar will explore what those skills are and how we can use them to make our communities stronger. We will use one particular case — sharing and synchronizing event calendars in Ann Arbor — to illustrate ideas, but the basic principles we will discuss can be applied to almost every aspect of community life.

While we’ll be talking about the web, this seminar is not for IT specialists, any more than knowing how to drive is something that only auto mechanics need to know.

A one-hour presentation by Jon Udell of Microsoft will be followed by another hour of Q&A and discussion.

This event is free of charge, and particularly of interest to those working for educational, civic and other not-for-profit organizations. It will be helpful to those who want better ways to get the word out about their own organization’s events and news, as well as those who are searching for such information and not always finding it easy to locate.

We’ll address these questions, among others:

– How can we, as a community, most effectively inform one another about goings-on in the region?

– How can our collective information management skills improve quality of life in the region?

– How can they also help us attract tourism and talent from outside the region?

– How do these same skills apply in other domains of public life such as political discourse and education?

We’re inviting organizations that — like public libraries — are significant producers of community events: public schools, colleges and universities, city governments, hospitals, cultural and environmental nonprofits, sports leagues, providers of social services, and more. We’re specifically looking for the people in such organizations who produce and promote events. If you’re one of those folks in or near Ann Arbor and would like to be invited, please let me know.

In a pair of recent entries, Semantic web 101: Say what you mean and The long tail of the iCalendar ecosystem, I’ve begun to report on what I’m learning about the state of the iCalendar ecosystem as I work in parallel on the elmcity project and on the iCalendar Validator. Today I’ll focus on just one of a number of issues I’ve run into. Consider these two screenshots:

google calendar	hotmail calendar

On the left you see Google Calendar displaying two calendars, each representing a single event — the Brower Youth Awards on October 18 at the Herbst Theater in San Francisco — in a different way. On the right you see Hotmail Calendar displaying the same two calendars. The event will happen at 5:30 Pacific time on the 18th. I found it on the Berkeleyside site whose events page offers a companion iCalendar feed.

If you load that feed into both Google Calendar and Hotmail Calendar, and if your calendars are set to Eastern time, you’ll see what’s shown above. If your calendars are set to another timezone the times will be shifted but the pink ones still won’t match.

The green ones should always match and should always be what you’d expect. For me, looking at a 5:30 Pacific event through the lens of calendars set to Eastern, I’d expect both calendars to display 8:30.

What’s the difference between the pink calendar and the green calendar? Here’s the pink one. It’s just the original calendar reduced to a single event. Like the original it declares its timezone using the nonstandard X-WR-TIMEZONE property:

BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
CALSCALE:GREGORIAN
PRODID:-//Refresh Web Development//Helios Calendar//EN
X-FROM-URL:http://www.berkeleyside.com/BerkeleysideCalendar/events/
X-WR-RELCALID:Berkeleyside
X-WR-CALNAME:Berkeleyside
X-WR-TIMEZONE:America/Los_Angeles
BEGIN:VEVENT
URL;VALUE=URI:http://www.berkeleyside.com/BerkeleysideCalendar/events/index.php?com=detail&eID=6
DTSTART:20111018T173000
DTEND:20111018T210000
SUMMARY:Brower Youth Awards 2011
LOCATION:Herbst Theater - 401 Van Ness \, San Francisco\, CA US 94102
END:VEVENT
END:VCALENDAR

And here’s the green one. Again it’s a derivation of the original calendar that reduces to a single event. But it also declares its timezone in the standard way, using a VTIMEZONE component and then referring to it using the TZID parameter of the DTSTART and DTEND properties:

BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
CALSCALE:GREGORIAN
PRODID:-//Refresh Web Development//Helios Calendar//EN
X-FROM-URL:http://www.berkeleyside.com/BerkeleysideCalendar/events/
X-WR-RELCALID:Berkeleyside
X-WR-CALNAME:Berkeleyside
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
X-LIC-LOCATION:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
URL;VALUE=URI:http://www.berkeleyside.com/BerkeleysideCalendar/events/index.php?com=detail&eID=6
DTSTART;TZID=America/Los_Angeles:20111018T173000
DTEND;TZID=America/Los_Angeles:20111018T210000
SUMMARY:Brower Youth Awards 2011
LOCATION:Herbst Theater - 401 Van Ness \, San Francisco\, CA US 94102
END:VEVENT
END:VCALENDAR

As we see in the picture above, the event time on the green calendar shows up the same way in both Google Calendar and Hotmail Calendar. It should also be the same in any calendar program that supports the iCalendar standard.

The event time on the pink calendar, though, is a up for grabs. Calendar programs that strictly follow the iCalendar standard should ignore X-WR-TIMEZONE and always display the local time, 5:30PM, which will be right for people in the Pacific timezone and wrong for everybody else. Hotmail Calendar does this. Programs that use X-WR-TIMEZONE, on the other hand, can render this calendar just as they would render a standard calendar. Google Calendar does this.

Why do I care? I have to decide whether the elmcity service will or won’t consider X-WR-TIMEZONE to be meaningful. The service is based on DDay.iCal, the same standards-based parser that powers the iCalendar Validator. So when the the service reads the pink calendar, and renders it for users in Berkeley, it will do the wrong thing from their point of view.

To do the right thing for Berkeley it would need to do the wrong thing by iCalendar: transform the nonstandard X-WR-TIMEZONE property into a standard VTIMEZONE component, and then transform all the dates so that they refer to the VTIMEZONE’s TZID. In order to create that VTIMEZONE component, it would interpret X-WR-TIMEZONE value as a TZID (timezone ID) from the Olson database. A Unix-based service would look up the TZID in Olson, find the rule for the timezone — i.e., offsets from GMT for standard time and daylight savings time, and when to appy them — and express the rule using VTIMEZONE syntax. A service running on Windows Azure, like mine, would instead need to map the Olson name to a Windows timezone name, look up the rule using a Windows API, and then express the rule in VTIMEZONE syntax.

Of course this is a slippery slope because, in the end, I’m only guessing what X-WR-TIMEZONE is supposed to mean. Here’s Rick DeNatale engaging in the same kind of guesswork:

Someone pointed me to this icalendar file of Australian holidays for a test case:

http://icalx.com/public/rohanl/Australian32Holidays.ics

This contains NO VTIMEZONE components, but does have the calendar property: X-WR-TIMEZONE:Australia/Sydney

Googling indicates that this is a non-standardized property, but it seems to be used by several calendar apps including Apple’s ical.app
and Google calendar.

I know that it’s non-standard, but it seems to be somewhat important for interoperability. I’m looking for some kind of information about
what it means in general.

It seems to indicate a default tzid for the whole calendar. In the absence of timezone components I’m not sure how to interpret the tzid, though.

Australia/Sydney IS a time zone identifier in the Olsen database, is it standard practice to use olsen tzids in X-WR-TIMEZONE calendar attributes?

Fortunately I can bring some data to bear on this question. Thanks to the iCalendar Validator I can analyze public calendars produced by a variety of iCalendar producers. In The long tail of the iCalendar ecosystem I listed the names of about 600 producers seen recently by the Validator. Of those, about 100 use X-WR-TIMEZONE instead of VTIMEZONE, and 70 of those 100 use local rather that UTC date syntax which implies they are depending on X-WR-TIMEZONE for correct interpretation of those dates.

Note that Google Calendar, the 800-pound gorilla in this space, is not one of those 70 producers. When it writes iCalendar format it uses both X-WR-TIMEZONE and VTIMEZONE; the latter ensures that Google Calendars can be understood properly by standard parsers that don’t support X-WR-TIMEZONE. The 100 producers I’m talking about, though, are using only X-WR-CALENDAR in a way that suggests they expect a nonstandard transformation. The fact that Google Calendar performs that transformation is, of course, a major reason why producers would expect it to happen everywhere.

Should X-WR-TIMEZONE be standard? That’s debatable. It would certainly make life easier for iCalendar producers. They could just mention a timezone rather than having to extract its rule from their operating systems and express the rule in VTIMEZONE syntax. One of the reasons for the success of RSS and Atom, after all, is that it’s always been easy to whip up an RSS or Atom feed which you can then check with the Feed Validator. An analogous simplicity for iCalendar producers would help grow the iCalendar ecosystem.

On the other hand if an iCalendar feed were to only mention a timezone without fully defining it, then the consumer would have to do the work that the producer didn’t. That’s problematic as Doug Day, author of the iCalendar Validator, notes in a recent email exchange:

Unfortunately, without including the actual time zone information in the calendar (i.e. via VTIMEZONE), you can’t be sure that the date/times you’re representing are accurate, even when using X-WR-TIMEZONE. For example, if you’re on a Windows XP machine that hasn’t been updated in 5 or 6 years, your system time zone information will be inaccurate. However, if the VTIMEZONE were included in the calendar, it would remain accurate, even on an older machine with out-of-date time zone definitions. Also, in order to interpret X-WR-TIMEZONE, you’d need to be in an environment where interpreting Olson time zone is realistic (easy on Linux, harder on Windows). I know a global, online time zone registry is in the works, but I don’t think it’s to a point where it’s useful, yet.

You might wonder why all this timezone stuff is even necessary. After all, an iCalendar feed can simply omit VTIMEZONE (and/or X-WR-TIMEZONE), express dates and times in UTC (Coordinated Universal Time), and use UTC syntax for all dates and times. Why not just do that? I asked Doug Day about this a while ago, and here was his reply:

The biggest problem is with recurring events and daylight/standard time transitions. For example, consider the following (all hypothetical):

1. I live in Salt Lake City, Utah

2. I want to schedule a meeting, starting on September 7, 2009 at 9:00 A.M, which recurs every month on the first Monday.

3. Some of the people attending this meeting live outside my current time zone.

So, here are the occurrences you’re ultimately after:

– September 7, 2009 – 9:00 A.M. (3:00 PM UTC)

– October 5, 2009 – 9:00 A.M. (3:00 PM UTC)

– November 2, 2009 – 9:00 A.M. (4:00 PM UTC)

As you can see, once the time changes from daylight back to standard time, so does the UTC representation of that time. So, if you had scheduled your event in UTC time, when the time zone changes, your event time will actually have changed (to 10:00 A.M., rather than 9:00 A.M.)

For this reason, among others, it’s always best to include time zone information whenever available. Traditionally, it’s been pretty difficult to include that information, and it’s more often left out than included.

So what shall I do with X-WR-TIMEZONE? I’ve decided to support it experimentally. If you start a new hub on the elmcity service, the default is to ignore X-WR-TIMEZONE. But if your hub has important sources that depend on it, as Berkeley does, then you can override the default so the times will be as you expect.

Meanwhile we’re going to update the iCalendar Validator to warn producers about this issue. There’s nothing technically invalid about a calendar that uses X-WR-TIMEZONE without VTIMEZONE. To a parser that strictly interprets the RFC5545 standard, that property is just a name that’s “reserved for experimental use.” But as has always been true of the RSS/ATOM Validator, the iCalendar Validator aims to deliver useful real-world guidance. Producers that use X-WR-TIMEZONE alone to declare a timezone should know that while it may often yield expected results, it’s not guaranteed to do so. It would be better to use a standard VTIMEZONE.

A couple of months ago I began saving the iCalendar files that are submitted to the iCalendar Validator. Today I extracted a list of unique names of iCalendar producers along with associated counts of the number of calendars validated for each. Here they are, with Google Calendar at the head and a classic long tail distribution of almost 600 other iCalendar producers. (You can also see them elsewhere by name as well as by count.)

The next step will be to analyze how well these these producers conform to the validator’s interpretation of the iCalendar spec. But the list itself forms an interesting data set. We know intuitively that, after 12 years of evolution, the iCalendar ecosystem must have become broad and diverse. Here’s a nice illustration of that breadth and diversity.

(Note that these counts don’t necessarily reflect the real distribution of iCalendar producers. The iCalendar Validator is closely associated with the elmcity project, so certain producers used heavily there — DDay.iCal, EVDB, Meetup — are overrepresented. On the whole, though, I’d guess this is a reasonable proxy for the distribution of producers.)

Delicious has been part of my life for a long time. I first wrote about it back in August of 2004. I know this because Delicious helped me remember. The link has gone stale because I wrote the article for an online publisher, which turns out to be a good way to get published but a lousy way to stay published. Thankfully Wayback remembers what InfoWorld forgot:

Pub/sub, tags, and human filters

In 2002, InfoWorld gave a Technology of the Year award to “publish/subscribe” technology. In the writeup I mentioned Kenamea, KnowNow, and the Flash Communications Server. The del.icio.us bookmarking system has some of the pub/sub flavor of those systems, as well as some of the blogging flavor.

In the blog network, you publish to a personal identity (your own), and you subscribe to other people’s identities. In systems like KnowNow and Kenamea, people (and also applications) publish to, and subscribe to, topics.

Consider the del.icio.us tag e4x, which I created today to help me keep track of this article on a subject I expect to learn more about soon. At the moment, my e4x page and the systemwide e4x page are the same: mine is the one and only use of that tag.

Even if I’m the only one to collect e4x references by means of that tag, it will have value. I’ll be able to access a set of bookmarks from anywhere, and easily share them. Things could get more interesting if other people’s e4x references start to show up when I visit (or subscribe to) the tag. Whether del.icio.us (or an analogous service) will reach a scale that makes that likely, for specialized as well as common terms, is an interesting question.

Once a tag does reach critical mass, another interesting question arises. Do you monitor the global view or do you rely on one or more user-filtered views? I guess the answer is both, at different times. When a tag is new and receives little traffic, watch the whole thing. If traffic grows too heavy or too noisy, interpose trusted human filters.

Looking back I can see what attracted me to Delicious. It embodies what I’ve come to know as ways of thinking like the web.

I am now working on a service that invites people to learn and apply web thinking in order to systematically inform one another about things. My web service uses Delicious as a partner service. One reason is that I am virtuously lazy. I would much rather use a service than create one. The elmcity project only cares about one thing: calendar syndication. If it can partner with a service like Delicious for other things — managing lists of feeds, configuring hubs — then I can focus on trying to do the one thing that really matters to my project.

But there’s another reason. I believe that people who use Delicious in the way that elmcity curators do are learning to apply some key principles of web thinking. Things like informal contracts, information chemistry, the pub/sub communication pattern, and the structure of information,

I wrote up specific examples recently in Can elmcity and Delicious continue their partnership? Today I realized that I still lack an answer to that question. If the new terms of service are going to require me to swap out Delicious for another service I should get cracking. But first I’ll try again. Is it OK for elmcity to keep using Delicious the way it has been? If anyone reading this can help me get that question answered I will be grateful.

My latest Radar essay makes the modest proposal that Facebook might, in some cases, syndicate my data from elsewhere rather than requiring me to type it in. Most people think that’ll never happen. Paulo Eduardo Neves sums up why not:

I don’t think they have any intention to open gates in their walled garden.

Of course garden gates can work two ways. They can keep things in or out. We can all appreciate why Facebook wants to keep things in. But is it really in Facebook’s interest to keep things out? That would require Facebook to become the home for all of our photos, our calendars, and every other stream of data we create. What a burden! Why not let a decentralized internet carry some of that burden?

What’s matters most to Facebook, I should think, isn’t my photos and my calendars, but the surrounding interaction that it can uniquely enable, capture, and monetize. Couldn’t inbound syndication amplify that interaction? Dunno, just asking.

One of the great pleasures of summer in Keene, NH is our New England Collegiate Baseball League team, the Swamp Bats. I tell people that going to one of the Bats’ home games at the high school’s Alumni Field is like winding the clock back fifty years and bringing a Norman Rockwell painting to life. Fans arrive early and set up lawn chairs along the first base line. Kids run potato sack or frozen T-shirt races between innings. College players from teams around the country, some destined for the big show, remind you how sweet the little show can be.

Of course the Swamp Bats schedule should syndicate through Keene’s calendar hub. And last year it did. The Swamp Bats site was using Google Calendar, which provides an iCalendar feed, so I just added that feed to the hub.

This year there’s a new website based on the Joomla content management system. When I first looked at its rendering of the schedule, my heart sank. The iCalendar feed was gone!

On closer inspection, though, I found that it was still lurking in the background. The sites’ calendar is served up by a Joomla component called GCalendar which wraps an instance of GoogleCalendar. Why? The documentation explains:

No longer are you forced to use an i-frame and settle for a less then ideal look. Now you can integrate a Google Calendar directly into your website – using AJAX. This extension pulls directly from your Google Calendar within Google and displays it directly on your website.

Fair enough. The standard IFrame method for displaying a Google Calendar widget is, indeed, less than ideal. But in the process of wrapping the widget so that webmasters can make pages that look nicer, the iCalendar feed was left on the cutting room floor. That deprives visitors to the Swamp Bats website of the opportunity to add the Swamp Bats calendar to their personal calendar programs. It also prevents the Swamp Bats calendar from syndicating through an elmcity hub or directly to media outlets.

This omission isn’t an exception, it’s more like the rule. Web designers and builders obsess about how sites look, but typically ignore how they connect — or don’t — to the emerging web of data. As a result, I had to ferret out the hidden Google Calendar in order to connect the Swamp Bats schedule to the elmcity hub.

After some poking around I found it by viewing the source of the month view, which looks like this:

It’s not obvious, but if you click the down arrow above June 2011 a list expands:

In the HTML source for the list, two Google Calendar IDs appear:

<tr>
<td>
<input type="checkbox" 
  name="5nmm1e33jj6bfo14jq9jv44p18%40group.calendar.google.com" 
  value="/index.php?option=com_gcalendar&view=jsonfeed&format=raw&gcid=1&Itemid=110" 
  checked="checked" onclick="updateGCalendarFrame(this)"/>
</td>
<td><font color="#6b49b0">Keene Swamp Bats</font></td>
</tr>
<tr>
<td>
<input type="checkbox" 
  name="8s9qdhf61u2q3697vocm6abkg0%40group.calendar.google.com" 
  value="/index.php?option=com_gcalendar&view=jsonfeed&format=raw&gcid=2&Itemid=110" 
  checked="checked" onclick="updateGCalendarFrame(this)"/></td>
<td><font color="#9e8462">Keene Swamp Bats Away</font></td>
</tr>

A Google Calendar ID is an email address at the root of a family of related URLs you can use to embed the calendar’s widget on a page, or subscribe to the calendar’s public (or maybe private) feed. In this case I’m after the public feed. Here’s how you form its URL from a root email address:

http://www.google.com/calendar/ical/ + EmailAddress + /public/basic.ics

Applying the rule to the first item in the list¹ produces this iCalendar feed. I plugged its URL into the Keene hub, and now it knows that the Bats are playing the Holyoke Blue Sox tonight at Alumni Field:

I’ve also subscribed to the feed in Outlook, so I can see the games as an overlay on my personal calendar. Another Swamp Bats fan could do the same in Google Calendar, or Hotmail Calendar, or Apple iCal, or any other standard calendar program.

All’s well that ends well, I suppose, but geez, it really shouldn’t be this hard. Web designers and builders ought to think like the web. If you regard the web is an interactive experience, they do. But the web is also an ecosystem of linked data. That way of thinking has, so far, tragically failed to sink in.

Web pros! Wake up! There’s more to this game than achieving the “ideal look.” When you are working with data, please make it available as data to the people and the systems that need it.

¹ What about the second item? That calendar is empty. It seems the intent was to provide two calendars, one for home games and the other for away games. But in fact, for some reason, the first calendar has both, and the second has none.

My wife Luann showed her new art work at a local venue last Saturday. Here’s how the event looked in Keene’s elmcity hub:

It got there by way of a new technique I just added to elmcity’s repertoire. Until now, the only way to syndicate events from Facebook through an elmcity hub was the one described here. In that scenario a curator asks an elmcity hub to search Facebook for public events in the hub’s location. This hasn’t worked well. If you use Facebook’s API to search for public events using the term Keene, NH you’ll find events, but only ones that include Keene, NH in the event’s title. Events that mention Keene, NH in the location field are oddly missing.

So until now, if you wanted to use Facebook to promote a public event in Keene, you had to use Keene, NH in the title to get it to work. Being married to me, Luann knows to do that, and her event was already appearing in the hub. But for most people this sort of hack will (and should) be a non-starter.

Even if Facebook’s API worked the way I’d expect, and could find public events by location rather than just by name, it’s not really the right thing. The elmcity model puts event owners in charge of data feeds that hubs (and other consumers) access at specified URLs. It’s nice to know that your public events in Facebook can be found via search. But if you want to syndicate those events through a hub you’d rather publish an explicit feed URL.

It turns out that you can, but in an odd way that requires some explaining and raises important questions about the evolving landscape of online identity. The new elmcity technique relies on the Export link at the bottom of Facebook’s Events page. The label, Export, connotes private use of the iCalendar feed behind that link. The feed is primarily for Facebook users who want to sync their Facebook Events pages to their personal calendars. When you click the link, you’re shown an URL like this:

webcal://www.facebook.com/ical/u.php?uid=652….115&key=AQD…qcT

If you change webcal to http you’ve got one of those special sharing URLs that are public, in the sense that they live on the open web, but also private, in the sense that they’re not discoverable. You wouldn’t use this kind of URL for anything really sensitive. But it can be appropriate for calendars, or friends-and-family photo galleries, or other cases where security-by-obscurity is a reasonable tradeoff.

Syndicating Facebook events

When I fetched the URL for Luann’s Facebook events, by way of her Export link, I had an iCalendar feed that was a mixed bag. It included events to which Luann had been invited by friends; those events were marked private. It also included the event that Luann was promoting; that one was marked public. Ideally Facebook would provide a separate URL for just Luann’s (or any Facebook user’s) public events. There would be nothing secret about that URL, it could be shared freely on the open web.

Lacking such an URL from Facebook, elmcity has to synthesize it. To do so, it filters the private feed to include only events that meet two criteria:

1. They belong to the Facebook user, not to a Facebook friend of that user.

2. They are marked public.

Here are the iCalendar properties that enable such filtering:

ORGANIZER;CN=Luann Udell:MAILTO:luann@luannudell.com
CLASS:PUBLIC

In this example, when I excluded everything in Facebook’s iCalendar feed that wasn’t organized by Luann, and wasn’t public, I was left with a feed containing just one event: Luann’s art opening. That’s the feed I wanted to tell the Keene hub to subscribe to.

My first thought was to use the normal elmcity mechanism for subscribing a hub to calendar feeds. The hub’s curator bookmarks the feed in a designated Delicious account. In this case, there would need to be an extra bit of metadata to enable the hub to filter the feed properly. It could easily find events marked PUBLIC. But how would it know to find Luann’s public events? For that it would need the name of a Facebook user, in this case Luann’s name.

At first blush that seemed easy to solve. The elmcity service uses Delicious tags to convey metadata to hubs. I could define a new convention like so:

The combination of the trusted and ics and feed tags is the normal way a curator tells a hub that the bookmarked URL is an iCalendar feed that the hub should try to process. The who=Luann+Udell tag would be extra metadata telling the hub to restrict a Facebook calendar feed to only the events organized by the named Facebook user.

Problem solved? Unfortunately no. Do you see why not? This mechanism leaks private information. Although the elmcity hub isn’t going to publish anything other than Luann’s public event, her Facebook feed also contains a private event from Judy. Anyone who scans the feeds for the Keene hub would see the unfiltered URL and could find Judy’s private event.

If the architecture of elmcity were more typical, this wouldn’t be a problem. Curators would create accounts at elmcity, they’d log into those accounts to add feeds, the lists of feeds subscribed to by hubs would be hidden from the world. But elmcity does things differently. Hubs are transparent conduits through which public information flows. They reveal their sources. Nothing needs to be hidden, and so nothing is hidden. Curators do their work out in the open. Communities served by elmcity hubs can see how those hubs are constituted.

If Facebook provided a Publish link for public events, along with an Export link for all events, then it could participate directly in elmcity-style calendar syndication. Since Facebook doesn’t offer a Publish link, elmcity needs to receive the Export link from curators by way of some private channel of communication.

Syndicating Facebook events privately

If the architecture of elmcity were more typical, curators would have accounts at elmcity and would use those accounts to send private messages to the service. But again, elmcity does things differently. You shouldn’t have to create an account for everything under the sun. You already have plenty of perfectly good accounts. Why not reuse them?

The relationship between elmcity and Delicious is one example of such reuse. A curator creates a new elmcity hub by designating a Delicious account to control it. If the administrator of the elmcity service deems the proposed new hub legitimate, he or she (so far, just me) tells the service to use that Delicious account to specify the hub’s settings and list its feed URLs.

There’s an analogous relationship between elmcity and Twitter. If a hub’s Delicious settings name a curator’s Twitter account, then Twitter becomes a channel for private messages between the curator and the hub. For example, a curator can send a Twitter direct message to the hub that simply says: start. When the hub receives the start message, it immediately refreshes all the hub’s feeds instead of waiting for the next scheduled refresh.

Until recently that was the only control message that a curator could send to a hub. But now there’s another: add_fb_feed. From my Twitter account I just sent a direct message to the elmcity service’s Twitter account. The message said:

add_fb_feed id=652...115 key=AQD...qcT who=Luann+Udell category=art

Which means the following:

1. Make this Facebook iCalendar URL: http://www.facebook.com/ical/u.php?uid=652….115&key=AQD…qcT

2. Restrict it to public events organized by Luann Udell

3. Use art as the tag for events in this feed

The hub read that message and responded:

elmcity received your add_fb_feed command

And then:

facebook feed successfully added

This isn’t ideal. Curators still need to trust the elmcity service not to disclose URLs sent through the private Twitter channel. Such disclosure could happen in any of the following ways:

• The operator of the elmcity service gives up the data on purpose. I wouldn’t, of course, but it’s theoretically possible.

• The elmcity service gives up the data accidentally. It’s just software; mistakes happen.

• A curator gives up the data either accidentally or on purpose. This risk exists because the elmcity service only has trust relationships with curators. They, in turn, have trust relationships with the contributors who provide calendar feed URLs. Contributors who want to use Facebook as a source for public events that flow through an elmcity hub will give their Export URLs to curators. And curators, accidentally or on purpose, could leak those URLs.

This post is already too long, so I’ll say elsewhere why I think you probably shouldn’t use Facebook as the authoritative source for public events that you want to promote. But if you understand the mechanism I’ve explained here, and if you think the risk/reward tradeoff is acceptable, then you’re welcome to use it. If you’re an elmcity curator who has designated a Twitter account for private communication with the service, here’s the new command you can send to add a Facebook feed from one of your contributors:

add_fb_feed id=UID key=KEY who=USER [category=CATEGORY]

Usage is as follows:

add_fb_feed is a required verb

id=UID is a required parameter. Replace UID with the xxx from uid=xxx in your Facebook Export URL

key=KEY is a required parameter. Replace KEY with the yyy from key=yyy in your Facebook Export URL

who=USER is a required parameter. Replace USER with your Facebook name, using + instead of the space character.

category=CATEGORY is an optional parameter. You can use a single term or a comma-separated list of terms. These will appear as tags on each event flowing from the feed through the hub.

If you send a bogus or missing command verb, or id, or key, you’ll get a Twitter message describing what went wrong.