The Memory Leak

Sean’s response to Ed’s mention of “I’d like some cheese.. but I’d rather not know how it’s made” got me thinking about the role of standards.

Suppose I follow Sean’s suggestions and make a batch of cheese. I can’t really call the cheese Stilton unless it’s been blessed by the Stilton Cheesemakers’ Association. Consumers discover a resource using it’s name - in this case a name under control by a standards organization that assures all cheese labeled Stilton follows a strict code and is produced in Derbyshire, Leicestershire, or Nottinghamshire.

Certainly a standards body can overreach itself. I heard somewhere that Monty Python’s Cheese Shop sketch was an allusion to unintended consequences that arose from government policies aimed to regulate retail cheese. There must be some middle ground, where a standard can instill consumer confidence without burdening retailers.

This also applies to geodata. For example, almost all appraisal districts here in Texas use GIS to store parcel data. There is likely a well defined legal definition somewhere for what constitutes a parcel. As far as I know, however, there is no standard (RDF or otherwise) describing how parcels can be published. If such standards were available, appraisal districts could publish their parcels to the GeoWeb where they could be discovered and analyzed by others. But getting consensus from a standards committee would take forever. That leaves us with companies like Google calling the shots.

I bet Google is looking to do with parcels what they’ve already done with the Google Transit feed specification. I don’t think any sort of grass roots movement could have evolved a standard for transit, much less an agency like the Federal Transit Administration.

When Ed says this:

Perhaps a new approach is needed where standards are defined at the same time as new applications and functionality developed, so that the standards process is driven by individuals and organisations implementing new functionality which is standardised once demonstrated to be both stable and useful !

What he really means is this: once it has been demonstrated that Google can monetize a feed, they will support the standard.

Let’s hope the result is better than velveeta.

IBM’s EPA problems seem to have spread, resulting in a GSA suspension on any new Federal work.

Because of the suspension, GSA told its acquisition staff members they can’t solicit offers from the company, award new contracts, place task, delivery, or purchase orders, or consent to subcontracts with IBM.

UPDATE: The glass dome place over IBM has now been removed.

This graph is from a paper on Amazon’s Dynamo. I suspect other months (not during shopping season) would look quite different.

With a name like “Elastic Compute Cloud”, I would expect the price for EC2 to reflect the supply of available compute capacity - isn’t this what elasticity of supply is all about? Currently pricing does not reflect time of use. I wonder how much performance degrades for EC2 users during Christmas shopping season.

Amazon relaxes the Consistency part of the DBMS ACID requirement in order to achieve availability. Maybe another rule could be relaxed, the one saying keys should not have any meaning beyond their use as an ID. If we did this, maybe Peano keys could be used, providing a spatially enabled Dynamo-like system.

America’s Promise Alliance is getting a lot of press for a report they recently published on high school graduation rates. While the report itself contains many analytical maps, none of the news reports I’ve seen include any. I like to see reports recognizing the influence of geography. But I wonder if there may be other explanations. For example, would the rates in different cities still differ that much after adjusting for race/ethnicity?

Too bad GeoCommons is not up and running, otherwise the data could get posted there and we could do further analysis.

Cities in Crisis - A Special Analytic Report on High School Graduation

Something doesn’t make sense here. I often have heard ESRI describe Web2.0 mapping as useful, but not really GIS. To be true GIS, ESRI claims, geoprocessing capability must be provided. But then when I look at ArcGIS Online (Beta), I don’t see any geoprocessing services. I thought I saw this demo’d somewhere, but don’t see anything. I suppose ESRI is working on this - otherwise the name should be changed to ArcData Online.

What I would really like to see is a way for ESRI Business Partners to author geoprocessing services and publish them onto ArcGIS Online. Partners should be allowed to specify a fee that is charged to the user when someone uses their GP service. Of course ESRI would get a percentage of this. I would also like to build geoprocessing services by chaining together other people’s services, maybe even following the chain of resposibility pattern.

A lot of business partners sell desktop (ArcView) for ESRI. At some point ArcGIS Online will be competing with these partners. By allowing partners the ability to author and publish geoprocessing tools on ArcGIS Online, they could continue to add value to ArcGIS.

News.com has a good update on Cloud computing.

John M Willis has posted a helpful summary of cloud computing vendors.

He doesn’t mention Sun Grid. So what’s the difference between a cloud and a grid? Looks like I’m not the only one confused.

This is an elaboration of the MyHouse concept I mentioned in a previous post.

I’d like to have my pantry and refrigerator equiped with RFID so they know what is in stock. I would have a web page where I search for dinner recipes that are presented in a ranked fashion, ordered by how few extra ingredients I need to pick up from the store. In cases when I’m away from home I’d like an option to specify a path I normally travel home on. The web site would rank the recipes based on availability at stores along the path. Since I’m making a buying decision it could be fueled by ad revenue.

It seems like the CMU coke machine should have evolved into this already.

This is an interesting video: Microsoft presenting Dryad research to Google. Dryad is claimed to be a “superset” of MapReduce.

It seems like the complexity of Dryad could be hidden behind a modelbuilder-like interface.

I wonder how valid it is to view this video and think about modelbuilder terminology, replacing the word “vertex” with “tool”, “channel” with “intermediate data”. Note that, like earlier versions of geoprocessing, Dryad cannot handle cycles (loops).

Fantom Planet’s post suggests ESRI is paying more attention to volunteered geographic information VGI. Fantom Planet’s concept of Generation M: Me, MyMaps, MySpace is useful. Once the Generation M becomes home owners, I think we will see MyHouse, a web site that lets me use geography to organize my activities in my neighborhood.

Take flood plains for example. Recently I saw in the newspaper that FEMA was updating the flood plain maps. But wait, Generation M doesn’t read newspapers - MyHouse must get GeoRSS alerts for things like this.

The old maps are inaccurate …

To see the draft updated maps go here I must go here and pull up a very Gen-X ArcIMS site.

We have 90 days to file protests against the new maps. If I had other info I could mashup against the map, it would make it much easier to explain my protest. Protestors should be allowed to mark up a public map where neighbors can review.

FEMA needs to recognize VGI and provide guidelines for the local authorities coordinating the map updates. The guidelines would encourage (RestFul) practices that would allow sites like MyHouse to integrate. The information volunteered by protestors is crucial to allow creation more accurate maps.

On the political side, there are likely other activities in FEMA that could benefit from VGI as well, like for emergency response. So maybe when the next administration cleans house VGI can be introduced comprehensively. (Now when is that California primary?)

On the technical side, once ESRI fixes ElementGraphicsLayer.WriteXml and provides the new mashup capability in 9.3 maybe MyHouse could be built with ArcGIS Server. Until then I must say building MyHouse would be easier with other tools.

On the philosophical side, by selfishly collecting data for MyHouse I would indirectly benefit my neighborhood (my herd).

“All of us have a place in history. Mine is clouds.” -Richard Brautigan

Brautigan was probably the first geospatial Software as a Service (SaaS) evangelist. We need someone alive today who can assume his role. The next big wave is cloud computing. We in the geocommunity are lagging even though it may very well offer more to our industry than any other.

Here is a laundry list of what I’ve seen in my explorations:

Google
Has MapReduce, BigTable and Sawzall. I don’t see any indication they will try to use their searchengines to perform geoprocessing.

Yahoo
Uses Hadoop, an open source implementation of MapReduce, coupled with ZooKeeper for coordination. Not sure how PIG fits into this, but looks like it might do what DryadLinq does.

Microsoft
Is extending Dryad to leverage Linq, in something called DryadLinq looks interesting. Note that Dryad uses SqlServer Data Services (SSDS) on the back end. See the Mix08 presentation. While this looks interesting, it might be a long time. Not sure how Windows Server 2008 HPC fits into this, but worth watching. Ozzie did mention MapReduce in his Mix08 keynote.

Amazon
First kid on the block with AWS, offering S3, EC2 and SimpleDB. See this post showing how Hadoop can be run on EC2 S3.

GISolve
This is the only site I found for spatial parallel processing. Don’t see much recent activity.

HyperTable
This is an open source version of BigTable.

Sun
Sun Redshift is a major grid computing effort. Sun also bought MySql and seem committed, though maybe they don’t see spatial as special.