Latency Must Die: Reducing Latency by Preloading Data

Rich Internet Applications are fast. When you click a button or push a lever, things tend to happen right away. That fast user experience is a primary reason that developers and designers are attracted to RIAs. Latency is reduced (after the initial download) for two reasons
1) Because the UI is inherently preloaded: it is not being shipped back and forth every time the user performs an act of navigation.
2) Because data can potentially be preloaded: the application can download data in the background that it thinks you will use, before you request it.

The real payoff with RIAs happens when the latency just disappears, and you feel like the data is native to your hard drive, even if in reality it’s living on a server thousands of miles away. This requires preloading of both UI and data. Generally, very large data sets cannot be cost-effectively downloaded. So it only makes sense to preload data that has some potential for being requested by the user.
In order to provide an optimal user experience, an RIA architect must decide
a)whether to preload data
a)which data is worth preloading, and
b)what order to preload data in.
Which Data To Preload: it’s an economics problem
Whether you should preload data, and how much data you should preload, depends on the economic value that you attach to reducing latency. Since preloading data is a gamble that the data would be used, it is (like any gamble) effected by the cost of the bet, the odds of a payoff and the value of a payoff. In general, you should preload data if the value provided by the preload
({value of reduced latency}* {odds the data will be requested})
is greater than the extra cost assumed by preloading:
{cost of download}*(1-{odds data will be requested})
For example, if the additional server resources required by the download cost $.01, the zero-latency customer experience is valued at $.10, and the odds of the data being requested are 20%, than the data should be preloaded, since the value of the preload (.2*$.10=$.02) exceeds the cost ($.01*((1-.2)=$.008).
This equation is obviously a cartoon, since preloading data has additional engineering costs (increasing code complexity) and the equation relies on numbers that are impossible to estimate with any degree of precision. Nonetheless, it provides a framework for making the decision of whether or not to preload a given piece of data. The higher the value placed on reduced latency, the more sense it makes to preload data. And the higher the odds that the data will actually be used, the more sense it makes to preload it. As bandwidth and hardware costs decrease, {cost of download} will decrease, which implies that over time Moores law will make it more and more cost-effective to preload more data.
Which Data to Preload: it’s a research problem
What this equation also makes clear is that in order to preload data effectively, we need to be able to estimate the odds that a particular piece of data will be requested by the user. We will also need to have some kind of model of when they will be requesting the data. Data that is likely to be requested first should be preloaded first, to maximize the chance that it will be present at the time the user requests it.
Building up a predictive model that can make these kinds of judgments will be a critical engineering problem for next-generation RIAs. Best guesses at an initial model can be informed by user research with similar types of software. For example, if you were writing an email client for an RIA, spending time watching what uses do immediately after launching Outlook or Eudora would be a good initial step. Tracking the effectiveness of the model (through logging) and tweaking it over time should yield substantial improvements. Predictive models that adapt over time, according to an individuals behavior, may eventually supplant or augment this approach.
Predownloading data is critical to providing low-latency experiences. But blindly downloading data without consideration for how likely the user is to need it is not a scaleable approach. RIA architects will have to consider these issues carefully, and ground their decisions about preloading in user research, in order to create superior user experiences.

4 thoughts on “Latency Must Die: Reducing Latency by Preloading Data

  1. Frederick van Amstel August 27, 2004 / 10:01 am

    Great Article Jon! I know that this wasn´t the point of the article, but I read an inspiring paper written by Bruce Tognazzini witch compares the interface designers with the magicians. He talks about the technics of Manipulation of Time that magicians uses to hide his tricks. The same principles can be used to lure the user that the preloader are more fast than it actually are. You know: games, interesting texts, hipnotic arts.

  2. Jon August 27, 2004 / 10:14 am

    I don’t think you’re off-point at all. That’s the idea … use misdirection, trickery, and technology to fool the user into thinking that everything happens instantly. This applies equally for preloading the UI and for preloading data.
    This is very related to what Cooper writes (in Inmates) about programming culture being dominated by (antiquated) scarcity thinking as well. Grab what you _think_ the user will want, and you have the possibility of delighting him with a lighting-fast interface. Just the way a good parking valet will have your car ready for you at the time you usually go to work!

  3. Fred van Amstel August 27, 2004 / 11:28 am

    If this´nt an off topic, let´s continue with it. So, reading about the story of the mirror for waiting the elevators that Tog tell us, I was convinced that preloaders doesn´t have to make any mention to time. No estimated time, no elapsed time. Let´s show only the illusion. Maybe the size of the archive for the geek one. What do you think about it?

  4. jon August 27, 2004 / 12:27 pm

    I agree, progress bars emphasize the time passing, rather than distracting the user from it. So the question then is what to show? A game or something is a nice start, but not suitable for a “serious” application. Below is an alternative idea:
    Jeff Raskin (co-inventor of the macintosh) designed a small computer called the Canon Cat in the 1980s.
    One of the tricks he used was to have an image of the UI put on the screen the second the computer started up. When the OS was ready, he would take down that image and reveal the actual UI. People take long enough to focus attention and get oriented that they hardly ever clicked on the image, and as a result were under the impression that the computer booted much faster than it actually did.
    Could one use such an approach in a flash preloader? The relevant issues are the size of the final SWF vs. the size of a suitably high-resolution image of that SWF, and positioning / scaling the image so that it looked exactly like the swf.

Comments are closed.