StinkyLittleFriend

ColdFusion; extracting substrings and multibyte chars

Tue, 13 Mar 2012 07:01:00 +0100

We have a service that returns xml. The xml is not how you'd expect i.e. where a parent node represents a record and the child nodes represent fields relevant to the current record. So to be clear, it isn't like this:



	
		mylinkhere
		mylink text here
	
	
		myotherlinkhere
		myotherlink text here
	
...etc

Instead it concatenates the contents of fields so it would concatenate the two "link" fields in one node and concatentate the two "linkText" nodes in another node. Then it would add a "length" node to show how long each entry would be - it ends up looking looking like this:



	
		9,15
		mylinkheremyotherlinkhere
	
	
		16,21
		mylink text heremyotherlink text here

I assumed the the length represented the character count and based on that assumption I had used the ColdFusion Mid() function. It turns out the length actually represented the byte length, so if there were any multibyte characters then my mid function would extract too much. Here is a code sample using the first technique (I have represented the xml as a struct):














	Extract the url strings and the link text strings assuming each character is a single byte character
	
		
		#elem#

		(length of string = #len(str[elem].text)#)

		(sum of reported lengths = #ArraySum(ListToArray(str[elem].lengths))#)

		
			
			
			#u#

The question marks '?' are supposed to represent multibyte chars, unfortunately something on this environment means it is not retaining these chars. Anyway I found that converting the string to bytes using getBytes("UTF-8") then using Java's "java.util.Arrays" and the "copyOfRange()" method meant I could extract the bytes I required and the using CF's CharsetEncode("UTF-8") convert the bytes back into a string. See below for the revised version.




	Extract the url strings and the link text strings
	
	
		
		
		#elem#

		
		(length of byte array = #len(b)#)

		(sum of reported lengths = #ArraySum(ListToArray(str[elem].lengths))#)

		
			
			
			
			
		#u#

Here is a link to the sample.

ColdFusion, .Net (dotnet) and byte arrays

Thu, 27 Oct 2011 02:39:00 +0100

Problem: Using ColdFusion's integration with .Net you can use 'System.IO.File' to read in a file and get back the byte array of the file, however it is not a a byte array that you'd expect i.e. a binary object. It is actually translated to an CF Array. This makes it difficult to use CF to deal with .Net byte arrays. Solution: However you can use .Net's 'System.Convert' to convert it to a Base64 string, something CF can deal with. See code below:












 



CF binary length: #ArrayLen(binaryFileCF)#

.Net binary length: #ArrayLen(binaryFileDN)#

The result looks something like this: Of course this is not limited to just reading binary files, anything that .Net returns as a byte array (e.g. 'System.Net.WebClient' DownloadData()) and consequently gets translated by Java/ColdFusion as an 'Array' can be treated the same way.

ColdFusion, .Net (dotnet) and NTLM

Tue, 18 Oct 2011 08:05:00 +0100

Problem: Trying to make an HTTP request against a web site that has Intergated Windows Authentication (IWA) aka NTLM - sometimes CFHTTP just doesn't cut it.Solution: Since IWA is associated to Microsoft products I thought I could use ColdFusion's .Net integration and actually use .Net to do the NTLM handshake that CFHTTP doesn't do. In fact it didn't take long to piece together a very 'procedural heavy' proof of concept. Using this as my primary reference. I am in the process, right now, of rewriting my 'procedural heavy' proof of concept so that it is more reusable and slightly fuller featured. Of course there are other HTTP-NTLM options that can be found on RIAForge

Convert epoch date to human readable date

Wed, 04 May 2011 05:31:00 +0100

I have a function that takes two arguments an epoch date and a date mask and it returns the formatted date. The line that does all the work is:

This works ... for epoch dates less than 19 January 2038. My application recently started throwing an error when trying to convert 2177456400 to a date. So I needed a more robust way of converting an epoch date to an actual date.

2177456400 is 01 Jan 2039.

Google getting into the wedding spirit

Fri, 29 Apr 2011 13:42:00 +0100

Just seen that Google have added Mr and Mrs Pegman to Google StreetView for Wills and Kate.

SQL to remove duplicates, keep most recent record

Fri, 29 Apr 2011 09:57:00 +0100

First off, I am not a DBA, although as a developer I have to write a lot of SQL. I have found, when writing SQL, especially when I am after a complicated dataset/recordset, that I have to get into the right mind set. Anyway I was faced with a problem where an existing database table had duplicate records. I wanted to delete the duplicates but keep the most recent record.Before I start deleting stuff I make sure I get the right dataset by creating the SELECT statement. The table looked something like:

TABLE_WITH_DUPLICATES
	ID	The unique ID (primary key) for each record calculated using a SEQUENCE (Oracle) - number incremented by 1
	GUID	A GUID that identifies the record non-unique column
	A bunch of other columns that aren't relevant

First thing I did was visualise the dataset I wanted i.e. using a subset of the existing data in the table I manually work out the result I am expecting, nothing too in-depth though. This helps me keep focus on the end result while writing the SQL. So the next thing I did was write a normal select query that would help me find out the number of duplicates or occurrences, this will become the result set to base everything on:


SELECT 		guid, count(guid) as numoccurrences
FROM 		table_with_duplicates
GROUP BY 	guid

Then using the resultset above I get the IDs and GUIDs of those records that have duplicates i.e. 'numoccurrences' greater than 1.


SELECT 		twd.id, occ.numoccurrences, twd.guid
FROM 		table_with_duplicates twd,
			(SELECT 		guid, count(guid) as numoccurrences
			 FROM 		table_with_duplicates
			 GROUP BY 	guid) occ
WHERE 			occ.numoccurrences > 1
  AND			occ.guid = twd.guid

So now I end up with all records (IDs, GUIDs, numoccurrences) that are duplicated. As the IDs are unique and incremented I know for each duplicated set of records the highest ID is the most recent, if I had a timestamp column then I would probably have used that.


SELECT 		max(id) maxid, numoccurrences, guid
FROM 		(SELECT 		twd.id, occ.numoccurrences, twd.guid
			 FROM 		table_with_duplicates twd,
						(SELECT 	guid, count(guid) as numoccurrences
						 FROM 		table_with_duplicates
						 GROUP BY 	guid) occ
			 WHERE 			occ.numoccurrences > 1
			   AND			occ.guid = twd.guid)
GROUP BY 	numoccurrences, guid

So now I know what the highest ID value is for each duplicated set of results. I can now say get me all duplicate sets excluding the record with the highest ID in each duplicate set.


SELECT 		twd2.id, twd2.guid
FROM 		table_with_duplicates twd2,
		(SELECT 	max(id) maxid, numoccurrences, guid
		 FROM 		(SELECT 		twd.id, occ.numoccurrences, twd.guid
				 FROM 			table_with_duplicates twd,
							(SELECT 	guid, count(guid) as numoccurrences
							 FROM 		table_with_duplicates
							 GROUP BY 	guid) occ
				 WHERE 			occ.numoccurrences > 1
				   AND			occ.guid = twd.guid)
		 GROUP BY 	numoccurrences, guid) findmaxid
WHERE 		twd2.guid = findmaxid.guid
  AND		twd2.id <> findmaxid.maxid
ORDER BY 	twd2.guid

Then it is just a matter of plumbing this into a delete statement


DELETE FROM table_with_duplicates
WHERE ID IN (
SELECT 		twd2.id
FROM 		table_with_duplicates twd2,
		(SELECT 	max(id) maxid, numoccurrences, guid
		 FROM 		(SELECT 		twd.id, occ.numoccurrences, twd.guid
				 FROM 			table_with_duplicates twd,
							(SELECT 	guid, count(guid) as numoccurrences
							 FROM 		table_with_duplicates
							 GROUP BY 	guid) occ
				 WHERE 			occ.numoccurrences > 1
				   AND			occ.guid = twd.guid)
		 GROUP BY 	numoccurrences, guid) findmaxid
WHERE 		twd2.guid = findmaxid.guid
  AND		twd2.id <> findmaxid.maxid
ORDER BY 	twd2.guid)

And that is how complicated it is.

Recurse through folder structure to a specific depth

Mon, 27 Jul 2009 02:43:00 +0100

Problem: A user at work wanted to copy an existing folder structure 3 levels deep and exclude all files. It would have been time consuming to create the folders manually and we couldn't copy 'n paste the folder structure because there was about 120GB of data and about 63,000 folders!Solution: I did a bit of googling (as you do) and found Raymond Camden's blog entry "Dumping a Recursive Directory List" which helped. Although cfdirectory's recurse="yes" would've taken forever going through 63,000 folders and it also seemed like overkill especially if all the user wanted was the first few levels of the folder structure. So using Ray's example as a starting point I made a couple of minor tweaks and made it recurse through the top three levels. Initially I wanted to list out the directories like Ray had done. Once I was happy it was going to work as expected I then created the folder structure.















#getFolders(sDirPath=sInitialDir, iDepth=iDepth, fCreate=true)#


	
	
	
	
	
	
	
	
	
	
	
	
	
	
		SELECT 		*
		FROM 		qryFiles
		WHERE 		type = 'Dir'
	
	
	
		
		
			
			
			Copy #directory##separator##name# to #sTempPath##separator##name#
				 
				
					(Warning! directory already exists.)
				
					(Ready!)
				
				
				
					
						Creating ...
						
						SUCCESS!
						
							FAILED! - #cfcatch.message#
						
					
				
				#getFolders(directory & separator & name, iDecrement, arguments.fCreate)#

How to get local drive information programmatically

Fri, 19 Jun 2009 05:38:00 +0100

Problem: You want to find out how much free space you have on your server's local drives. You also want to find out how much space you have used on your local drives. How can you do this programmatically?Assumptions: You are using a Windows machine. You are using ColdFusion 8. Solution: If you have .Net version 2 or greater AND the ColdFusion 8 .Net Service installed you can try using Ben Forta's 'GetDriveInfo() UDF Powered By .NET' Alternatively you can use this :











	
		Drive: #arrRoots[i]#

		Available Space: #arrRoots[i].getFreeSpace()#

		Total Space: #arrRoots[i].getTotalSpace()#

FireFox 2 Default Theme

Sat, 17 Feb 2007 04:03:00 +0100

Just in case you, like me, aren't really a fan of the default FireFox 2 'Theme' it is easy enough to change. Go to https://addons.mozilla.org/firefox/3479/ and download the "Winestripe" theme this is the theme that FireFox 1.5 used. Also you may not like the placing of the 'Close' tab buttons. To change this go to 'about:config' in your FF browser address bar scroll down to 'browser.tabs.closeButtons' and change the value. The options are: 0: Close appears on current open tab
1: Close appears on the right of each tab (default)
2: Close dissappears (middle click to close)
3: Close appears on the right of the browser Happy customising FireFox :)