Caching

As mentioned in a previous article, caching is an excellent way to drastically improve performance with minimal changes/development. Caching accomplishes this goal by shifting resource loads from one type to another. A CD drive is much slower than a mechanical hard drive in general. A mechanical hard drive is much slower than system memory in general. Accessing a remote machine's system memory is much slower than the local machine's system memory. The amount of comparisons between different types of storage mechanisms would be an endless list. Luckily, most developers only deal with a few storage mechanisms when designing/developing a given system.

Just for general information, the following is a list of storage mechanisms and their general quickness. A higher number indicates a faster mechanism, while a lower number indicates a slower mechanism. Again, this is in general. It isn't always the case. The numbers are anywhere from 1 to 100. Also, the general number used is an overall rating for latency and bandwidth. Some devices are better with latency while poor with bandwidth. Some are better with bandwidth and worse with latency. Some are worse with both. Keep in mind that the mechanisms can be used in many different ways. This is simply an indicator on things to look at in order to improve performance. The first section compares local access. The second section compares remote access. Most systems must utilize both types in order to be useful. This means caching becomes more complicated, but nevertheless, the concepts remain the same. Usually preventing remote calls (or keeping them to a minimum) can render the largest performance enhancements.

Local Access

• CPU Register 100
• CPU Cache Memory 90
• System Memory 70
• Database 60
• Hard Drive 40
• DVD Drive 30
• CD Drive 20

Remote Access

• Direct Ethernet Connection 100
• Ethernet LAN 90
• Wireless LAN 70
• T1 Line 50
• Broadband 30
• Dial-up 5

Now that a few storage mechanisms and access mechanisms have been listed, it is now time to look at a specific caching concept and see a real working example. The following Java class file compares the performance of reading a file from the file system (with the aid of the operating system of course) vs reading a file from a cache in system memory. The type of caching mechanism used to utilize the system memory is a HashMap. A HashMap was chosen over a Hashtable because of the removal of synchronization. It would be up to the developer to determine if the code needed to be synchronized. For the example code, it is not required, so a minor performance improvement can be achieved with the HashMap class.

The code was tested against a 73 MB file. The application was executed a total of 2 times, with 5 reads within each execution. A reboot occurred before each execution. The reason for this is because the operating system will utilize its own cache in order to prevent actually fetching the file from the hard drive when it had already been recently read.

The following is a list of the benchmark timings for the executions:

Execution #1

• Read 76699621 bytes from the file system: 18036 ms.
• Write 76699621 bytes into the cache: 0 ms.
• Read 76699621 bytes from the cache: 0 ms.
• Read 76699621 bytes from the file system: 4507 ms.
• Write 76699621 bytes into the cache: 0 ms.
• Read 76699621 bytes from the cache: 0 ms.
• Read 76699621 bytes from the file system: 4727 ms.
• Write 76699621 bytes into the cache: 0 ms.
• Read 76699621 bytes from the cache: 0 ms.
• Read 76699621 bytes from the file system: 5287 ms.
• Write 76699621 bytes into the cache: 0 ms.
• Read 76699621 bytes from the cache: 0 ms.
• Read 76699621 bytes from the file system: 5128 ms.
• Write 76699621 bytes into the cache: 0 ms.
• Read 76699621 bytes from the cache: 0 ms.

Execution #2

• Read 76699621 bytes from the file system: 15973 ms.
• Write 76699621 bytes into the cache: 0 ms.
• Read 76699621 bytes from the cache: 0 ms.
• Read 76699621 bytes from the file system: 6029 ms.
• Write 76699621 bytes into the cache: 0 ms.
• Read 76699621 bytes from the cache: 0 ms.
• Read 76699621 bytes from the file system: 4837 ms.
• Write 76699621 bytes into the cache: 0 ms.
• Read 76699621 bytes from the cache: 0 ms.
• Read 76699621 bytes from the file system: 4717 ms.
• Write 76699621 bytes into the cache: 0 ms.
• Read 76699621 bytes from the cache: 0 ms.
• Read 76699621 bytes from the file system: 4726 ms.
• Write 76699621 bytes into the cache: 0 ms.
• Read 76699621 bytes from the cache: 0 ms.

You will notice the time to read from the cache always says 0. The reason for this is because Java's default system clock is inaccurate on Windows for anything less than approximately 10 milliseconds. In other words, the read from the cache was so fast, it couldn't be registered. Compare those times to the approximate average of 5000 millisecond timings for each fetch from the operating system's internal cache. If the application had many users attempting to read the same files frequently, it is plain to see how much of an improvement that memory cache would make. Notice that the first read from each fresh reboot resulted in a much higher read time. Again, this is because the operating system uses its own internal cache on subsequent reads to avoid accessing the hard disk as much as possible.

The example source code can be obtained here:

http://www.everlastsoftware.com/examples/source/java/CacheExample.zip

In order to execute, run 'java -Xmx512M CacheExample %FILE% %TIMES_TO_LOOP%'

• %FILE% = The file to do the read test again.
• %TIMES_TO_LOOP% = Number of times to read the file.

A batch file was supplied for execution on Windows. Simply modify the filename and time to loop in the batch file before execution.

Now it must be mentioned, of course, that system memory is a limited resource. Some machines may already be starved for free memory, so creating a memory cache would not help performance much. In fact, it may even hurt performance. So it is always important to consider the environment that the application will be in as well as how it will be used. The concepts discussed in this article simply demonstrate potential speed improvements through the use of caching.