Archive for March, 2007

RFID for Terrorists

Allpoints mentions how RFID tracking is “coming of age”.  Nowadays seems like a technology hasn’t really come of age until it’s adopted by terrorists, quoting from PhysOrg.com :

“London’s Royal Academy of Engineering suggests that someday a terrorist will be able to read personal details from a distance and set a bomb to go off when a particular person gets within range.”

Please Ship or Get off the Pot

A client has some ArcSDE performance issues that I need to look at.

As I have blogged earlier, the Geodatabase Toolset (gdbt) appears to be what I need. This is a freely downloadable unsupported extension.

There is still no 9.2 version of the gdbt. I don’t understand … why ESRI doesn’t post the source for this tool on arcscripts? I would understand if they were selling the product. But it’s free - what is the business case for not posting the source code?

Work, Play & War

What does it say about us when the latest Playstation has so much more computing power than a Workstation?

Not so long ago it was GIS practioners who had to teach people to spatially navigate digital worlds. Now its the other way around. A new generation of users raised on computer games are demanding game-like experiences from GIS. When I heard people in the Defense User Group meeting demanding 3D dymamic display, I thought OK, ESRI just needs to migrate to the Playstation 3 platform.

However, after looking at this Dr. Dobbs article, I think it will take the deep pockets of a defense project. I remember Reagan looking at some kids playing first person shooter games and commenting that the games were “training tomorrow’s warriors”. The warriors are now trained, now they just need weapons like the ones they’ve been trained upon.

ESRI Dev Summit - Defense SIG

Defense users met during lunch on the second day.  I guess probably 50 attendees.  Show of hands indicated majority are involved in real-time GIS apps.  Relative few doing traditional map production.

A reliable source at ESRI indicated at 9.3 MOLE symbols will implement ISymbol and IPersistStream, becoming more like all the other symbols in ArcGIS.  This will result in allow geodetic geometry for tactical graphics.   This is very good news.

ESRI also asked for input, so here goes …

I really would like to see an enhancement (or fix) to the current display pipeline so that if I call IScreenDisplay.Invalidate on a cacheIndex for a layer that has its own cache (ILayer.Cached == true) then I would not cause a refresh of the overlying caches.  Last I checked, if labels are turned on then the cache for esriViewGraphics gets refreshed.  Regenerating labels is expensive.  I need a way to refresh only my layer’s cache without refreshing the other caches.  IDynamicLayer is overkill when  refreshing only a few times per second.

Speaking of IDynamicLayer, there was interest expressed in having dynamic featurelayers.  An ESRI programmer indicated this is difficult because the underlying data structure is unknown.  I mentioned that the underlying datastructure could be an in-memory featureclass based on a plugin datasource so beefing up the FeatureLayer  to implement IDynamicLayer would be useful.

I guess the hard part would be juggling between IDynamicGlyphs and ISymbols, but seems possible.  Seems like a DynamicRenderer concept would be helpful.

Maybe just use .NET inheritance to inherit from FeatureLayerClass, then also implement IDynamicLayer.  Was told by a helpful user of a way to implement IPluginCursorHelper.NextRecord() without needing to throw a COMException to let the framework know there are no more records.   To me this seems like a bug - why should ESRI even bother calling  NextRecord() when they know there are no more records?  After all, I can see that they just called IsFinished and I returned true.

Another option I will explore is dumping the Plugin altogether and replace it with an in-memory featureclass based on the InMemoryWorkspaceFactory.  But then I’ll need to synch this featureclass with my model instead of just wrapping my model. (speaking in MVC terms).

I also mentioned it would be useful to support transparency at the Symbol level, perhaps something like ISymbol2.Transparency property.  The renderer would also need to be beefed up, to support something similar to IRotationRenderer, maybe call it ITransparencyRenderer, with a TransparencyField property.  That way I could go into arcmap and specify a transparency field for a a particular layer.

ESRI Dev Summit - AGS Geoprocessing

Attended “ArcGIS Server Geoprocessing Deep Dive” this morning. Corey Tucker and Sentha Sivabalan did an excellent job. Room was packed. From show of hands looked like 40% C#, 40% VB.NET and %20 java. Highly recommended.

My only concern about this is the way they are using the term “workflow” in context of Geoprocessing. They defined workflow as a models containing GPTools. What I worry about is that it will confuse people when they talk about workflows in context of SOA. Yesterday’s plenary showed a cool Websphere demo dragging and dropping activities to create a workflow. Hopefully some day we will be able to drag and drop GIS based activities too, along with non-GIS activities. I don’t think ESRI wants to position modelbuilder to compete with WebSphere or Microsofts WF extensions for Visual Studio - even though the IDEs share similar look and feel.

By the same token, IBM doesn’t make life any easier by naming one of its tools Map Designer. The mapping it does is non-spatial.

Workflow is very important to a lot of GIS based activities. Think about all the permit approval processes for building infrastructure. The approval process will be determined based on what jurisdictions you fall within, or what watersheds, or proximity to schools. Cities want to streamline this process so that a real estate developer can quickly check on the status of a building permit he’s applied for. Or perhaps would like to see what the approval process would be to build a particular type of building at a particular location. These workflows span multiple applications. Let’s call them Enterprise Workflows. ModelBuilder workflows, on the other hand do not span multiple applications as far as I can tell.

Maybe somehow geoprocessing will be made to plug into Enterprise workflow? Attachments to the enterprise workflow would include spatial datasets, e.g. a subdivision CAD parcel drawning. Different activities would use spatial processing, overlay the subdivision with watershed boundaries, or existing infrastructure. But there would be a lot of non-spatial activities too, e.g. inspections. I really would like to see how to wrap GIS activities appropriately so they could be dropped into an enterprise workflow.

ESRI Dev Summit Day 1

IBM Breakfast

I’ve sat through more powerpoint presentations than I care to remember stating (with no references) that 80% of all business information has some sort of geographic aspect. Then they follow up with some geocentric diagram with GIS is in the middle and surrounded by smaller applications orbiting around it, in synch with Copernicus rolling over in his grave. That didn’t happen this morning.

Instead, reps from IBM presented a new view based on a graphic similar to this, except with the Enterprise Service Bus as the integration platform.  Diagrams similar to this came up several times, almost as if this is becoming a mandala.

This new way of thinking revolves around a workflow, as enabled by the Enterprise Service Bus. Realization: it’s the workflow, stupid! [Reminds me of the old joke "Census Poller: 'How many people work here?' Respondent: 'About half'"] Geography still matters - but only to the extent that it helps organize workflows, not as an end unto itself.

Julio Olimpio quite forcefully made the point that ESRI is committed to IBM, citing numerous multimillion $ projects they’ve collaborated on. Also mentioned 2 days of IBM’s profits == 1 year of ESRI’s profit. Profits matter. Given IBM’s embrace of opensource, I wonder how they view their relationship with ESRI compared to opensource GIS.

WebSphere looks cool. I’ve been dabbling with the freely downloadable Windows WF extensions. I hope to soon test some workflows with GIS-based activities. I don’t see any place to freely download something from WebSphere (if anyone does, please let me know). This redpaper seems to go into details of what they presented. I wish they would have gone into more detail about what differentiates them from Windows WF. Maybe they realized they’d be perceived as biased so they don’t bother. I look forward to tomorrow’s presentation by Mark Driver of Gartner group, I hope he’ll go into this. I wonder if he’ll talk about open source, as he has in the past.

Plenary Session

Attendance: 1600 (some said 2000) twice what it was in 2006.

Estimated Percent Male: 95%

Estimated Median Age: 40

The mandala for the Framework had 3 circles: Desktop, Server, and ArcOnline - unlike the mandala here, that has 5.

Others have blogged about what was said, so I figure I’ll blog a bit about what was not being said:

COM

Very little mention. It’s still there and its not going away. The web ADF is growing faster than the COM libraries. Service Packs will now contain new functionality. Was implied a lot of the new functionality will be in the web ADF. Interestingly, lots of code was shown, but no OMD’s or any UML that I can recall.

Cost

In the past there was a lot of whining about price. Didn’t hear any of that today. Maybe people realize ease of deployment justify license costs.

Custom Features

I guess Class Extensions are now the recommended approach. Larry Young did a good job presenting Class Extension demo. What ESRI really needs is a new class extension called ClassExtensionContainer. This would follow the composite pattern. Let’s say I wanted to implement the classextension Larry showed us, only to find my featureclass already used the timestamper class extension. Well, if we had something like the ClassExtensionContainer, we could add both as subclass extensions. We would also need a new optional interface with something to establish what order the classextensions are notified for events. Maybe something analogous to IComPropertyPage.Priority.

Moving ArcObjects Code to ArcGIS Server from ArcGIS Desktop

Hosted by Jian Huang and Allan Laframboise. This was excellent. By the afternoon I was so saturated with new material from earlier sessions, it was comforting to see familiar desktop concepts along with a migration path into ArcGIS Server. A lot of good performance statistics for alternative approaches were presented. I look forward to seeing these posted as samples. Would have been nice to hear about debugging Server Object Extensions (SOE’s). I haven’t found a way to step through execution of an SOE using the debugger. I’m wondering if it might be a good practice to develop a class that implements both IServerObjectExtension as well as IExtension so that I can debug it using Arcmap as a test harness.

Flash Disk i/o Performance & Rebasing DLLs

SanDisk has a new 32 GB flash disk coming out that supposedly has 100x faster i/o than magnetic disks. For disk i/o constrained geoprocessing, sure seems like this could improve things.

Still, I wonder how much quicker Arcmap would load. I suspect a lot of the load time is not from disk i/o, but from “rebasing” dlls. I’ve noticed Arcmap startup time seems to increase more than linearly with the number of extension dlls being loaded.

Think about it: The more money you spend on Arcmap extensions, the slower it will load.

This MSDN article describes the costs of rebasing.

Arcmap loads lots of dlls, written independently by different developers. It’s just not practical to collaborate on base addresses. The JIT Extension category improves things, but I still notice slower startup time when more extensions are installed - even if they are in the JIT category. If a toolbar is turned on that references an extension, then that extension will load at startup.

Have you ever noticed how Visual Studio loads slow the first time, but much faster for subsequent loads? This article sheds some light, quote:

“By the way, on the “faster load time” issue, I should mention that when an executable module is unloaded, Windows puts its pages on a “standby” list, a kind of cache from where the module’s pages can be retrieved very efficiently when it is loaded again. So if you load a DLL for a second time, and its pages are still in the standby list, it will load a lot quicker than the first time.”

Seems like ESRI Desktop apps could benefit from a standby list too.

But instead of assigning base addresses by hashing the dll name, it seems like we could rebase them using where they actually end up being loaded in memory, essentially persisting the standby list on disk.

Active Tags: RFID for LBS

Report here shows where RFID might be headed: active tags. Seems like with a wireless grid and active tags, geolocation without GPS should be possible - and at a much lower cost.

Houston has contracted to have 600 square miles of WiFi. This should make it even easier to expand congestion based pricing beyond the toll roads.

What’s this got to do with GIS programming ?

Coming up with an algorithm that updates cost based on congestion levels could be interesting. Assuming pricing is based on time of day, writing a path finding routine could also become more complex.

Seems like taxi services could change too, perhaps allowing trips to be purchased through an eBay-like auction. I guess the unions would not allow this though. That’s too bad, if taxis were cheaper perhaps more people would choose not to own a car.