IWDatacapture.callServer call fails / caveat

problem:

when i build teamsite DCTs i always find a new and wonderful way to break something.

Today i was building a DCT that used IWDatacapture.callServer to call a cgi script that would get some stuff from the database and then out put it into a hidden frame and do a javascript callback to the parent window.

everything was going well until i made the mistake of making 2 calls to different IWDatacapture.callServer cgi scripts one right after the other. one of them would always fail.

solution:
The problem stems from the fact that CGI scripts that output html make use of a single blank.html file to write out their contents. If you call 2 of them "almost at the same time" one of them will overwrite the other's blank.html file and cause it to fail. The only reasonable solution is to combine both CGIs into one file and let it write to one blank.html file, you can also try to make sure that CGIs dont get called at the same time, but that would be difficult to control.

So it seems that its probably a reasonable requirement to only allow ONE IWDatacapture.callServer call per DCR, otherwise things might break. How sad.

Java: Understanding Weak References

The other day i was reading programming at reddit and stumbled on this article:
Understanding Weak References which was a good read but i still had some questions, so i turned to Java Developer Forums for help. I didnt get flamed! (much). What i got was a very good response from a user named kdgregory which is note-to-dogself worthy.

My questions are in blue boxes:

If weak references are great at stopping leaks, why not use them all the time?

They're great at stopping leaks in the situation where you're attaching data to an object that (should) have a limited lifetime. If you used a regular map (most likely an IdentityHashMap), you'd be creating a strong reference to the object via the map, so it would never be collected, even when there are no other references to it (I call this a leak, other people don't).

For example, ThreadLocal uses weak references, because they'll be cleared when the thread ends.

ObjectOutputStream should use weak references (and a changed protocol) so that it can be used in long-running conversations without constantly calling reset(). But it doesn't.

"WeakHashMap works exactly like HashMap, except that the keys (not the values!) are referred to using weak references."

• does this imply that hashmap stores values in weak refs, if so, why?

• should i use WeakHashMap for tasks where i will have another reference to objects i put into the map. ie i dont put my only ref into this map?

No, HashMap doesn't use weak references for either key or value. The author's wording (and for that matter, the JDK JavaDoc wording) is a little misleading.

One typical use for weak references is in a canonicalizing map to reduce memory consumption with duplicated data. With such a map, you would store an object as both key and value, and retrieve the canonical value when presented with an arbitrary key -- think String.intern(). However, you can't use WeakHashMap to do this without wrapping the value in a weak reference also.

When do i not use weak references ?

Most of the time. Any reference object means that you have to access your original object in two steps (have to call get() on the reference) rather than just one. And normally, you'd know the lifetime of your objects.

music for november

Using the google mp3 search i find some good stuff, im going to put all the november music into this post.

this place has a player built in, very nice:
http://media.soundfixrecords.com/player/xspf_player.swf

lots of full albums here - seems like dance / dnb:
http://danav.org/audio/

parse xml with JDOM and Xpath

every now and again its good to have your config in an xml file, but i am usually to lazy to remember to how to do it properly and just use java.util.Properties. Well no more of that!

This is a short self contained code sample on how to parse some xml using jdom and jaxen

you will need:
jdom lib
jaxen lib (which may be old now)

<br /> public void example() throws JDOMException, IOException {<br /> // import org.jdom.Document;<br /> // import org.jdom.Element;<br /> // import org.jdom.JDOMException;<br /> // import org.jdom.input.SAXBuilder;<br /> // import org.jdom.xpath.XPath;<br /> String fn = "/path/to/some.xml";<br /> FileInputStream in = new FileInputStream(fn);<br /> SAXBuilder builder = new SAXBuilder();<br /> Document doc = builder.build(in);<br /> in.close();<br /> XPath xpath = XPath.newInstance("//some/xpath[1]/expression");<br /> List list = xpath.selectNodes(doc);<br /> for (Iterator iter = list.iterator(); iter.hasNext();) {<br /> Element elem = (Element) iter.next();<br /> <br /> String name = elem.getAttributeValue("name");<br /> String childText = elem.getChild("child").getTextTrim();<br /> String text = elem.getTextTrim();<br /> }<br /> }<br />

wait in windows bat script - good way

i recently wrote about using ping to pause for a number of seconds in a batch script. It was brought to my attention that you can use /wait to actually wait for a process to complete like so:

start /wait /i "unimportant title put whatever here" "C:\path\to\some\executable.exe"

or if you have an msi installer you can do this:

msiexec /i "C:\path\to\msiinstaller.msi" /qr

extract links using htmlparser

i have come to realize that even though i love to do everything in regex, extracting links with regex is not really the best idea.

the best idea is to use htmlparser to extract the links.

This code sniplet will extract links from a page body, and given the link where the body was found it will resolve the relative links. This is tailored to be used by a crawler.

<br />public static Collection<String> retrieveLinks(String url, byte[] body, String charSet) {<br /><br /> Collection<String> result = new ArrayList<String>();<br /> <br /> URI uriLink;<br /> try {<br /> uriLink = new URI(url);//url of the body<br /> String st = new String(body, charSet);<br /> Parser parser = new Parser();<br /> parser.setInputHTML(st);<br /> NodeList list = parser.extractAllNodesThatMatch(new NodeClassFilter (LinkTag.class));<br /> for (int i = 0; i < list.size (); i++){<br /> LinkTag extracted = (LinkTag)list.elementAt(i);<br /> <br /> if(!extracted.isHTTPLikeLink()) continue; //ignore mailto / javascript and other weird protocol links<br /> <br /> <br /> String extractedLink = extracted.extractLink().replaceAll("&", "&");//we need to unescape these<br /> <br /> extractedLink = extractedLink.replaceAll(" ","%20");//URI class doesnt like spaces in URLs, but content creators dont care.. :P<br /> extractedLink = extractedLink.trim();//URI will barf on a link that look like "foo.com "<br /> if(extractedLink.length() == 0) continue; // i am betting that a link that looks like "" is user error. with our url scheme, it most certainly is.. <br /> if(extractedLink.startsWith("#")) continue; //skip all anchors, they are useless to a crawler<br /> if(extractedLink.matches("(?i)^javascript:.*"))continue; //HTMLParser thinks anything but lower case 'javascript:' is a non-js link<br /> <br /> URI uriRelative = new URI(extractedLink);<br /> //System.out.println("i:"+i);<br /> URI resolved = uriLink.resolve(uriRelative);<br /> //linkfilter will return true if crawling from <br /> //url --> resolvedUrl<br /> //is allowed by some rules<br /> String link = resolved.toString();<br /> //here we should check if the link is acceptable.<br /> result.add(link);<br /> <br /> }<br /> }<br /> } catch (URISyntaxException e) {<br /> // TODO: this is a bad link, log this!<br /> log.info("Bad Link Syntax on page:["+url+"] BAD LINK:"+e.getMessage());<br /> } catch (UnsupportedEncodingException e) {<br /> // TODO Auto-generated catch block<br /> e.printStackTrace();<br /> } catch (ParserException e) {<br /> // TODO Auto-generated catch block<br /> e.printStackTrace();<br /> } <br /> <br /> return result;<br /> }<br />

wait or pause in windows batch script

problem:
i am told that windows .bat files do not have a sleep or wait command.
this is pretty sucky for when you want to wait or sleep, isnt it?

solution:
do this terrible hack
ping 1.0.0.0 -n 1 -w 5000 >NUL

that will block for 5 seconds. yey!

also, if you need to wait for a process to finish, i blogged about that here

also, if you have the ability to install something on the box where you will run your script, you should get the Windows Server 2003 Resource Kit tools from Microsoft: which works with XP. This gives you a real sleep command! (Thanks to Tricky for pointing this out in the comments)