A few scanning tips

Memory problems

Large color images consume a lot of memory, and the memory requirements increase exponentially with the square of the resolution used. The Video Resolution Basics section was about computing image memory requirements.

And that's not nearly all of it either. If your image uses 4 layers, then you have 4 large images in memory. Editors must also retain multiple copies of the image to implement the Undo/Redo History function. Some programs have more efficient ways to store only the changed area of the image instead of the entire previous image, but some only offer one level of Undo/Redo, because Undo is expensive in terms of memory. Some programs can be configured to retain a fewer number of Undo levels, and a smaller number, perhaps 6 or 10 levels instead of 100 Undo levels, may realistically be more appropriate. And descreen filters for moiré can use outrageous amounts of memory too.

Minimal new computers are shipping with 128 MB of system memory now, but a film scanner really wants 256 MB. You will need 256 MB with Windows XP too, and 512 MB is better for large images. Memory is inexpensive now, more can make a noticeable difference in performance, and it is the best upgrade you can do if you use large images.

Why is memory size important?

Handling many megabytes of images can be difficult. The computer needs memory for

1. the image(s)
2. the program(s)
3. the operating system

all in memory at one time. If memory size is insufficient to hold all of this, then the excess bytes that cannot fit in memory (those bytes we don't need at this instant) are temporarily saved in the swap file, called swapping. The swap file is a huge file on hard disk to hold memory bytes that will not fit in memory, until they are needed again (file WIN386.SWP in Win9x, PAGEFILE.SYS in WinXP).

What is swapping?

There are two disk I/O operations when swapping; the "write" to save some unneeded bytes to make room in memory, then the "read" to get the needed bytes back into memory. An active program often needs those bytes back again in only milliseconds, but they still won't fit, so then other bytes must first be saved on disk to make room. Then the first bytes are read from disk back into memory. The entire process may be repeated thousands of times, over and over. This is swapping the unneeded bytes out of memory for those bytes needed back from the disk swap file, back and forth, over and over. But disk access is extremely slow compared to memory access

In moderation, swapping is a wonderful magical feature, allowing operations that we could not otherwise do with the memory we have. But in extreme cases when the disk access LED is always on, our swap file works incessantly, never stops, and our computer is brought to its knees.

A more technical comment about the swapping mechanism: Not every portion of every program is needed at any given moment, technically only one word is accessed at any instant. Windows can move the least-recently-used blocks of memory to the swap file on disk in 4K byte blocks, called "pages" (Windows 3.1 used larger blocks called segments, up to 64K, more an "all or nothing" concept). Swapping is more technically named paging. Paging uses a protected-mode CPU trap mechanism that is similar to a GPF (General Protection Fault). Typically a GPF is a fault when an errant program wildly attempts to access protected memory that it has not reserved, memory that it does not "own" (the program is usually the culprit, but it might be the victim of another program). Similarly, when a program properly accesses valid memory, but which is not currently present in physical RAM, it causes a CPU "page fault" instead. There is a memory driver (fault handler) that recognizes that this memory only needs to be read back in from the swap file, and can correct the condition by swapping, instead of terminating the program and reporting the fault to the user. Most faults are fatal, but page faults are typically correctable. The Windows System Monitor will report a few statistics about page faults, page ins, page outs, etc. A little swapping is very normal.

What is Virtual Memory?

The hard disk is an important part of the memory system, in the form of the Swap File. The swap file on disk is called Virtual Memory. The swap file holds the allocated memory bytes that will not currently fit into physical RAM memory. The disk is very slow in comparison to physical memory, but we are not "out of memory" until the disk is full. When the disk is full, we are out of memory. The swap file is a wonderful invention for occasional use, until we reach the practical limit and start slow incessant swapping.

Windows provides a dynamic swap file size. It's Maximum size setting should be ALL of the room remaining on the disk (the default.). It is pointless to limit it smaller, it does not save anything, because only the Minimum is initially allocated (and that Minimum should be made large enough to satisfy all normal requirements). So the Maximum is not even used until it is needed, but when it is needed, there is no alternative. Limiting the maximum file to less than the available disk space risks crashing sooner if and when that file fills. What's the advantage of that? In earlier days, Windows 3.1 did offer a "permanent" file setting for efficiency (couldn't grow), but this is no longer appropriate for Win9x or XP. A large Minimum also does this for the general case. Windows says, "let us handle it", and it should do just fine in that regard.

The concepts of virtual memory and paging in the Unix core of Macintosh OS X are very much the same. This is simply how modern multi-tasking computers must work.

You must provide ample free disk space for the swap file to grow. When your disk fills so that the swap file cannot grow, you will get "Out of memory" errors. Depending on how smart the software, you may get a system message about the swap file, or you may get a program error about "Out of memory", but either way, your disk is full. It probably has too many huge images on it. Empty your Recycle Bin and Temp folder (see Start - Programs - Accessories - System Tools - Disk Cleanup, in Win98, ME and XP).

If your machine is reduced to a slow crawl while the disk access LED is incessantly active, then the cause is clearly due to swapping, because you are trying to use more memory than you have. Memory unused at this instant in time has to be saved to disk swap file to make room for the bytes needed NOW in real memory. In only a few milliseconds however, your software may need the first bytes again, and so new bytes must be written to disk file to make room to read the first bytes back into memory from disk file again. This may be repeated jillions of times. In excess, this swapping is painfully slow. It's the proverbial 10 pounds of potatoes in the 5 pound sack.

Continued