Please note, new blog at http://www.acheron.org/darryl/

Tuesday, July 12, 2005New Internet Explorer window -- new session?

Has anyone noticed when you open a new instance of Internet Explorer (not by using CTRL-N, but clicking on the shortcut), that a completely new session is created? Now, I'm pretty sure this didn't happen with older versions of IE... SP2 change? A change for the better?

Saturday, July 02, 2005Java vs. ColdFusion: String Concatenation and CSV Generation

For a little while now I've been playing around with Java, and trying to replicate some common ColdFusion concepts in the language. Recently I came across a problem where generating a CSV file was taking a very long time, often making the server unresponsive. The code in question built a large string using concatenation, and then wrote the string to a file. It occurred to me that in-line Java might be a better tool for the job. A work colleague suggested using the StringBuffer/StringBuilder (5.0) class, as it is much more efficient than the String class. As you will see below, I ultimately ended up using the BufferedWriter class, and the execution time went from 5 minutes to 40 seconds!

Testing the theory

I decided to run some tests, to see how much quicker (or slower) in-line Java is than ColdFusion. The two tests are: I created a test script that appended a string to an existing string 1000 or 10,000 (or more) times. I also created a set of test scripts that built a string and/or appended a string to a CSV file.

ColdFusion vs. Java: String concatenation

I concluded that Java is a lot better at string concatenation than ColdFusion. This is especially true when dealing with very large strings. I deliberately did not test the CFSAVECONTENT tag as it not a string concatenation method. When concatenating a string 1000 times, there wasn't too much between them. ColdFusion took 172ms, whereas the Java class took 78ms. However, when increasing to 10,000 interations, ColdFusion took 17547ms (17s), and Java only took 406ms! I didn't dare test ColdFusion any further, and I know from experience that it would have taken quite some time.

ColdFusion vs. Java: CSV generation

I pitted Java's BufferedWriter class against ColdFusion's CFSAVECONTENT and CFFILE tags. I used the CFSAVECONTENT tag, because it is a lot quicker than concatenation. The Java test simply appended data to a file as it went, whereas the ColdFusion test built the string using CFSAVECONTENT and then wrote it to a file. Ultimately I concluded that the BufferedWriter class is a lot more efficient than the ColdFusion method. Building a 1000 row CSV file took Java 15ms, whereas ColdFusion took 47ms. When building a 10,000 row CSV file, the gap increased -- Java 64ms, ColdFusion 328ms. A 1,000,000 row file 49.6MB took Java 6547ms, whereas ColdFusion took 10860ms. In the 1,000,000 row test, the amount of memory used by CFSAVECONTENT was quite high. The server became sluggish after this, and memory did not seem to be freed. Using the BufferedWriter class however, you could barely notice any memory usage (as you would expect). The good thing about using the BufferedWriter class is that it uses very little memory. As soon as the initial buffer size (8192 characters) is reached, it is written to the file, and the buffer (memory) is then flushed/released. It is a fair statement that if you're generating a CSV file, then you should use Java. It is a much more scalable solution.

Results

Code Examples

Generating CSV with CFSAVECONTENT Note: I have moved some code to its own line for the sake of readibility. <cfscript> variables.iNrTimesToLoop = url.nr; variables.sStringToConcat = "The quick brown fox jumped over the fence: "; variables.sFileName = "#getcurrenttemplatepath()#_test_#gettickcount()#.txt"; start = gettickcount(); </cfscript> <cfset variables.sstring = ""> <cfoutput> <cfsavecontent variable="variables.sString"> <cfloop from="1" to="#variables.iNrTimesToLoop#" index="variables.x"> #variables.sStringToConcat##variables.x##chr(13)##chr(10)# </cfloop> </cfsavecontent> </cfoutput> <cffile action="append" file="#variables.sFileName#" output="#variables.sString#"> <cfscript> end = gettickcount(); total = end-start; </cfscript> <cfoutput> Total time: #total#ms </cfoutput> Java StringBuffer class oStringBuffer = CreateObject("java", "java.lang.StringBuffer").init(JavaCast("int",initSize)); // Append a string oStringBuffer.append("A string"); // Get string back from buffer sString = oStringBuffer.toString(); Java BufferedWriter class oFileWriter = CreateObject("java", "java.io.FileWriter").init("filename",JavaCast("boolean","true")); oBufferedWriter = CreateObject("java", "java.io.BufferedWriter").init(oFileWriter); // Write a string to buffer oBufferedWriter.write("A string");