Wednesday, March 12, 2014

Using FIFO with cacheSetProperties

Although it is not explicitly stated in the CF10 docs, some testing today verified that a "FIFO" algorithm does function when provided as part of a cache property configuration.

FIFO is a "first-in, first out" algorithm, which means the oldest object in my cache will be the first one to be flushed out once my cache reaches its maximum size.

First let's create a cache with FIFO.

<cfscript>


CacheRegionNew(
"testfifo",
{
DISKPERSISTENT=false,
MAXELEMENTSINMEMORY=7,
STATISTICS=true,
OBJECTTYPE='object',
MEMORYEVICTIONPOLICY='fifo'
}
);

</cfscript>

Note that we have set our cache up with a policy of FIFO, and given it a maximum of 7 elements.

Next, add some objects to our cache. Here, test2 is an old test object I happen to have laying around. :)

<cfscript>
for (i=1;i LTE 7; i=i+1) {

objname="obj_#i#";
cacheput(
objname,
new test2(),
1,
1,
'testfifo',
true
);
sleep(100);
}
</cfscript>

Let's dump the cache. 

<cfscript>
if (CacheRegionExists("testfifo"))
{
writedump(
CacheGetAllIds('testfifo')
);
}
</cfscript>

We should see this.





We now have 7 objects in our cache. Just to prove that our cache is not running with a different algorithm, do a cacheGet on the oldest object. ('OBJ_1'). 

<cfscript>

x = cacheGet(
'obj_1',
'testfifo'
);

</cfscript>

If you wish, you can dump our first object and verify it has received a cache hit.

<cfscript>
writeDump(
cacheGetMetaData(
'obj_1',
'object',
'testfifo'
)
);

</cfscript>



We are now ready to begin implicitly evicting items from the cache. We do this by adding a new object to the cache.

<cfscript>
objname="cacheitem_#getTickCount()#";
cacheput(
objname,
new test2(),
1,
1,
'testfifo',
true
);

</cfscript>

If our FIFO cache is working correctly, our first object "OBJ_1" should now be gone from the cache since the cache is full and it was our oldest object.

Dump the region again.

<cfscript>
if (CacheRegionExists("testfifo"))
{
writedump(
CacheGetAllIds('testfifo')
);
}
</cfscript>




That's it. Our oldest object has been evicted from the cache. If our cache had been running with LRU (Least Recently Used) or LFU (Less Frequently Used), a different object would have been flushed. In both of those scenarios "OBJ_1" would have been maintained in the cache because of the cache hit it received.

Friday, January 10, 2014

cfprocessingdirective

For years we have been using the tag cfprocessingdirective Tag at the top of our templates like this.

<cfprocessingdirective suppresswhitespace="yes" pageencoding="utf-8">

Then one of our colleagues found this in the Adobe Docs

"The cfprocssingdirective tag has limitations that depend on the attribute you use. For this reason, Adobe recommends that you include either the pageencoding or suppresswhitespace attribute in a cfprocessingdirective tag, not both." 

Ah hah. Afterwards we began to modify the templates.

<cfprocessingdirective pageEncoding="utf-8" />
<cfprocessingdirective suppressWhitespace="true">
...[some code]
</cfprocessingdirective>

It struck me however after playing around with the tag that the suppressWhitespace attribute did not seem to have an effect on the white space emitted. The white space was being stripped out regardless whether the tag was present or not. A quick check showed me that the "Enable Whitespace Management" Option on the server was active.

Thus it appears this setting essentially replicates at the page level the global setting for Whitespace Management.

Friday, January 25, 2013

cfhttp in java

We had some problems with CF8 not recognizing SANs SSL certificates. The certificates were valid and properly configured, but cfhttp constantly delivered an "I/O Exception:".

The only way I could find around this was to make the calls in Java. In case it helps someone, the Java I used to make a post to a soap web service was basically as follows:

<cfsavecontent variable="requestXML"><?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Header>

  </soap:Header>
  <soap:Body>

  </soap:Body>
</soap:Envelope>
</cfsavecontent>


<cfscript>
        myUrl = "xxx"; // web service url
        objUrl = createobject("java","java.net.URL").init(myUrl);
        conn = objUrl.openConnection();

        //configure the request
        conn.setDoOutput(true);
        conn.setUseCaches(false);
         conn.setRequestMethod("POST");
        conn.setRequestProperty("SOAPAction", "xxx my soap action"); 
       
        //output stream actions
        ostream = conn.getOutputStream();
        ostream.write(Javacast("String",requestxml).toString().getBytes());
        ostream.flush();
        ostream.Close();
        // set input
        inS =createobject("java","java.io.InputStreamReader").init(conn.getInputStream());
        inVar = createObject("java","java.io.BufferedReader").init(inS);

        builder = createObject("java","java.lang.StringBuilder").init(javacast("int",1000));
        line    = "";



        do
        {
           line = inVar.readLine();
           lineCheck = isDefined("line");
           if(lineCheck)
           {
              builder.append(line);
           }
        } while(lineCheck);


        retvar = builder.toString();

</cfscript>
 <cfoutput>#retvar#</cfoutput>



The problems seems to be fixed in newer versions of CF.  In CF 10 I was able to make the requests to the servers with SANs certificates without a problem.

Wednesday, November 28, 2012

Using cfproperty to reduce java class files

I haven't seen much blogged about this, so I thought it would be worth bringing up.

As part of our jump from CF8 to CF10 I've been experimenting with cfproperty and the concept of implicit getters/setters. Due to the way ColdFusion compiles these implicit functions it seems to me you can significantly reduce the number of java class files that ColdFusion creates. Obviously the effect is very noticeable in OO applications with lots of beans and hundreds of getter/setter functions.

To prove this let's take the following CFC with the old school explicit declarations for all of our getters & setters. I'll call it "explicit.cfc"

<cfcomponent output="false">
       
    <cffunction name="init" output="false" returntype="any" hint="our formal constructor block.">
            <cfset variables.instance.id    =1>
            <cfset variables.instance.name="">
            <cfset variables.instance.title="">
    </cffunction>

    <cffunction name="getID" output="false" returntype="string">
        <cfreturn variables.instance.id />
    </cffunction>
    <cffunction name="setID" output="false">
        <cfargument name="ID" type="string" required="true">
        <cfset variables.instance.id=arguments.id>
    </cffunction>

    <cffunction name="getName" output="false" returntype="string">
        <cfreturn variables.instance.name />
    </cffunction>
    <cffunction name="setName" output="false">
        <cfargument name="Name" type="string" required="true">
        <cfset variables.instance.Name=arguments.Name>
    </cffunction>

    <cffunction name="getTitle" output="false" returntype="string">
        <cfreturn variables.instance.title />
    </cffunction>
    <cffunction name="setTitle" output="false">
        <cfargument name="Title" type="string" required="true">
        <cfset variables.instance.Title=arguments.Title>
    </cffunction>

</cfcomponent>


It is a bean-like object with accessors for three variables: id, name, and title.

Now let's write a file that uses our object.

<cfprocessingdirective pageEncoding="utf-8" suppressWhitespace="true">
<cfscript>
    ex = new explicit();
    ex.init();
    ex.setName('Tester');  
    writeOutput("<p>  ID:#ex.getID()#  Name:#ex.getName()#  </p>" );
</cfscript>
</cfprocessingdirective>


Finally, go into the CF administrator and check " " in the Caching menu.

Now call up that page we created in your browser, then check in the .class files directory for CF and see what the server compiled. It will look like this:


 As we can see, CF has created a class file for our CFC itself, and every function inside of it. It also created a class file for our calling page.

Now let's take a CFC with the new style implicit cfproperty declarations for our getters & setters. Let's call it "implicit.cfc"

<cfcomponent output="false" accessors="true">

    <cfproperty name="id" />
    <cfproperty name="name" />
    <cfproperty name="title" />
      
        <cffunction name="init" output="false" returntype="any" hint="our formal constructor block.">
            <cfset setID(1)>
            <cfset setName('Tester')>           
        </cffunction>

</cfcomponent>

Go back and edit our file. We now create an implicit.cfc object.

<cfprocessingdirective pageEncoding="utf-8" suppressWhitespace="true">
<cfscript>
    ex = new implicit();
    ex.init();
    ex.setName('Tester');  
    writeOutput("<p>  ID:#ex.getID()#  Name:#ex.getName()#  </p>" );
</cfscript>
</cfprocessingdirective>


And what gets created now for java class files?


Just three files: one for the CFC itself, one for the Init() function inside of it, and one for the calling page.

Obviously the effect of this in a large application is that you're going to save yourself from creating a lot of 3kb class files for every getter/setter.

You may be asking if all if this matters, and it most cases it probably is not of great concern. In my experience it only really matters if you are seeing java.lang.OutOfMemoryError: PermGen space errors on your server. This is the space in the java heap where your class definitions are stored and it is possible to fill it up if you're running a lot of big applications on the server. Fewer classes should result in a reduced PermGen usage.

I've seen the PermGen error a number of times on a 32 bit machine with CF8. We got around it in the end by increasing the size of our perm-gen space in the JVM to 256mb, but even then we kept bumping up on the limit in some cases. On a 64 bit machine this should be even less of a problem though due to the ability to run with a much larger JVM.

The fact that each ColdFusion function results in a Java class file has always bugged me, and I'm glad this is no longer always the case. In theory fewer classes should also result in a performance gain, although, to be fair, I suspect in most cases the practical effect will be marginal.


Tuesday, November 20, 2012

Using MAT with Coldfusion

I spend a lot of time doing optimization and performance analysis on our sites, and over the years I've developed or used a lot of tools to help with this.

The simplest is probably a simple jstat reporting tool. We take snapshots of our servers with jstat about once a minute, and save the results in the database. With the results I can generate a lot of useful information about our JVM. To do this I use a few basic cfchart commands and get some basic results like the following, detailing heap usage in the server.

This little peak into the JVM is useful, and with it we get some basic statistics as to the health of our runtime environment. The key figures for us have always been how much memory is occupied by our tenured generation objects. If we have  a memory leak somewhere in our application or server scope, then that yellow line in the middle will make a relentless march upward, eventually flatlining when no more memory can be reclaimed through garbage collection.

The chart is useful, but it doesn't tell you exactly WHAT is contributing to instability with the Heap.
This brings us to a fun little tool: the Eclipse Memory Analyzer (MAT).

MAT is a tool for analyzing Heap Dumps. What's a heap dump? It's a snapshop of everything in your heap at the point in time the heap dump was made.

Heap Dumps are made with the jmap utility, which comes in the bin directory of every JDK as of Java 5. To do a heap dump just on a Windows machine with a Coldfusion Server running, do as follows:

  1. Go to the task managerr and find the processID (pid) of the coldfusion server.
  2. Execute "C:\mypathtojdk\bin\jmap" -dump:format=b,file=C:\MyPathToHeapDumps\heap.bin xxxx, replacing the paths where noted and the xxx will need to be replaced with the pid.
That's it. Now let's fire up MAT. You'll want to go to File / Open Heap Dump and pick out that  binary file you've just generated. The very first time you open the heap dump it may take awhile depending on the size of your file -- MAT parses your file and caches the results so future views will proceed more quickly.


The default option is a report that automatically checks for "Leak Suspects". This is sometimes useful, but not always. With Coldfusion you'll find in any case that you need to drill down through all of the java objects that are part of the CF library before you get to the true offenders.

Here's the leak report; and yeah I know there's not much in that JVM right now.  :)




There are a number of reports, and you'll just have to play around to see which one tells you the most. It varies, and depends to some extent on preference. I generally go into the Histogram, sort my objects by retained Heap, look for possible suspects, and begin following the paths to garbage collection.




One tip: at least as of CF 8 you will find that Coldfusion Applications are represented as FastHashTables in the JVM.  In the following screenshot I've opened one of these FastHashTables. Inside of it is an object of coldfusion.runtime.ApplicationScope, and we can see which application it is by looking at the attributes on the left.

Obviously if one of these is consuming a lot of memory in your "retained Heap" column, then you know where the culprit lies.

Friday, September 21, 2012

Undefined elements in structs returned from web service

I was accessing some functions remotely today in CF10 when I came across some weird behavior. Namely, the values returned by StructkeyList, StructkeyExists, and a Dump of a Struct are consistent with each other.

Take the following code and place it somewhere in a CFC to be accessed remotely.

    <cffunction name="getTest" output="no" returntype="struct" access="remote">

        <cfset result=structnew()>
                <cfset result.testvar = "some var">
                <cfset result.fuseactions = ArrayNew(1)>
   
        <cfreturn result>
   
    </cffunction>

Now access it from a calling page. You'll need to put in your own server and CFC name.

<cfinvoke
    webservice="http://xxx/mycfc.cfc?wsdl"
    returnvariable = "response"
    refreshWSDL = "true"
    method="gettest"
/>

This is where it gets interesting. Let's do a dump of our return variable.

Notice that the "FUSEACTIONS" key in our struct seems interesting. Here we see "undefined" where we would have expected an empty array. I'm guessing the empty array goes missing during the serialization.

Ok, but let's go further. Let's do #structKeyList(response)# on the struct. We see, as expected, our two elements.


Now it gets strange. Try to do a StructKeyExists on the FUSEACTIONS key.

#structkeyExists(response,'FUSEACTIONS')#

It delivers "NO". Trying to dump the struct element results in an error.



Now I know that the concept of NULL values was introduced in CF9. (Something I missed, still toddling along with CF8). So let's try:

#isnull(response.FUSEACTIONS)#

Sure enough, it delivers YES.

Still, this seems a little weird to me. Intuitively I would expect to see an empty array inside the Struct, since that was how the variable was declared. In any case, the inconsistency between StructKeyExists and StructKeyList is a change from CF8 and it broke some old code of ours.

 I've filed this as bug 3334756 by Adobe. Let's see what they say.



Wednesday, September 12, 2012

filexists() with a url equals a hung request

I had a bug in one spot where a url was being given to the fileexists() function instead of an absolute path.
According to the documentation this should not work. Sure enough it really doesn't, but instead of returning NO the thread seems to hang. (CF 10 , version 10,282462). Not exactly sure why, more investigation is needed here.

I've never worked on version 9 before, but according to this thread, it was actually possible to do a fileexists on a url as of 9.0.1

http://forums.adobe.com/thread/765614