Java: Understanding Weak References

The other day i was reading programming at reddit and stumbled on this article:
Understanding Weak References which was a good read but i still had some questions, so i turned to Java Developer Forums for help. I didnt get flamed! (much). What i got was a very good response from a user named kdgregory which is note-to-dogself worthy.

My questions are in blue boxes:

If weak references are great at stopping leaks, why not use them all the time?

They're great at stopping leaks in the situation where you're attaching data to an object that (should) have a limited lifetime. If you used a regular map (most likely an IdentityHashMap), you'd be creating a strong reference to the object via the map, so it would never be collected, even when there are no other references to it (I call this a leak, other people don't).

For example, ThreadLocal uses weak references, because they'll be cleared when the thread ends.

ObjectOutputStream should use weak references (and a changed protocol) so that it can be used in long-running conversations without constantly calling reset(). But it doesn't.

"WeakHashMap works exactly like HashMap, except that the keys (not the values!) are referred to using weak references."

• does this imply that hashmap stores values in weak refs, if so, why?

• should i use WeakHashMap for tasks where i will have another reference to objects i put into the map. ie i dont put my only ref into this map?

No, HashMap doesn't use weak references for either key or value. The author's wording (and for that matter, the JDK JavaDoc wording) is a little misleading.

One typical use for weak references is in a canonicalizing map to reduce memory consumption with duplicated data. With such a map, you would store an object as both key and value, and retrieve the canonical value when presented with an arbitrary key -- think String.intern(). However, you can't use WeakHashMap to do this without wrapping the value in a weak reference also.

When do i not use weak references ?

Most of the time. Any reference object means that you have to access your original object in two steps (have to call get() on the reference) rather than just one. And normally, you'd know the lifetime of your objects.

music for november

Using the google mp3 search i find some good stuff, im going to put all the november music into this post.

this place has a player built in, very nice:
http://media.soundfixrecords.com/player/xspf_player.swf

lots of full albums here - seems like dance / dnb:
http://danav.org/audio/

parse xml with JDOM and Xpath

every now and again its good to have your config in an xml file, but i am usually to lazy to remember to how to do it properly and just use java.util.Properties. Well no more of that!

This is a short self contained code sample on how to parse some xml using jdom and jaxen

you will need:
jdom lib
jaxen lib (which may be old now)

<br /> public void example() throws JDOMException, IOException {<br /> // import org.jdom.Document;<br /> // import org.jdom.Element;<br /> // import org.jdom.JDOMException;<br /> // import org.jdom.input.SAXBuilder;<br /> // import org.jdom.xpath.XPath;<br /> String fn = "/path/to/some.xml";<br /> FileInputStream in = new FileInputStream(fn);<br /> SAXBuilder builder = new SAXBuilder();<br /> Document doc = builder.build(in);<br /> in.close();<br /> XPath xpath = XPath.newInstance("//some/xpath[1]/expression");<br /> List list = xpath.selectNodes(doc);<br /> for (Iterator iter = list.iterator(); iter.hasNext();) {<br /> Element elem = (Element) iter.next();<br /> <br /> String name = elem.getAttributeValue("name");<br /> String childText = elem.getChild("child").getTextTrim();<br /> String text = elem.getTextTrim();<br /> }<br /> }<br />

wait in windows bat script - good way

i recently wrote about using ping to pause for a number of seconds in a batch script. It was brought to my attention that you can use /wait to actually wait for a process to complete like so:

start /wait /i "unimportant title put whatever here" "C:\path\to\some\executable.exe"

or if you have an msi installer you can do this:

msiexec /i "C:\path\to\msiinstaller.msi" /qr

extract links using htmlparser

i have come to realize that even though i love to do everything in regex, extracting links with regex is not really the best idea.

the best idea is to use htmlparser to extract the links.

This code sniplet will extract links from a page body, and given the link where the body was found it will resolve the relative links. This is tailored to be used by a crawler.

<br />public static Collection<String> retrieveLinks(String url, byte[] body, String charSet) {<br /><br /> Collection<String> result = new ArrayList<String>();<br /> <br /> URI uriLink;<br /> try {<br /> uriLink = new URI(url);//url of the body<br /> String st = new String(body, charSet);<br /> Parser parser = new Parser();<br /> parser.setInputHTML(st);<br /> NodeList list = parser.extractAllNodesThatMatch(new NodeClassFilter (LinkTag.class));<br /> for (int i = 0; i < list.size (); i++){<br /> LinkTag extracted = (LinkTag)list.elementAt(i);<br /> <br /> if(!extracted.isHTTPLikeLink()) continue; //ignore mailto / javascript and other weird protocol links<br /> <br /> <br /> String extractedLink = extracted.extractLink().replaceAll("&", "&");//we need to unescape these<br /> <br /> extractedLink = extractedLink.replaceAll(" ","%20");//URI class doesnt like spaces in URLs, but content creators dont care.. :P<br /> extractedLink = extractedLink.trim();//URI will barf on a link that look like "foo.com "<br /> if(extractedLink.length() == 0) continue; // i am betting that a link that looks like "" is user error. with our url scheme, it most certainly is.. <br /> if(extractedLink.startsWith("#")) continue; //skip all anchors, they are useless to a crawler<br /> if(extractedLink.matches("(?i)^javascript:.*"))continue; //HTMLParser thinks anything but lower case 'javascript:' is a non-js link<br /> <br /> URI uriRelative = new URI(extractedLink);<br /> //System.out.println("i:"+i);<br /> URI resolved = uriLink.resolve(uriRelative);<br /> //linkfilter will return true if crawling from <br /> //url --> resolvedUrl<br /> //is allowed by some rules<br /> String link = resolved.toString();<br /> //here we should check if the link is acceptable.<br /> result.add(link);<br /> <br /> }<br /> }<br /> } catch (URISyntaxException e) {<br /> // TODO: this is a bad link, log this!<br /> log.info("Bad Link Syntax on page:["+url+"] BAD LINK:"+e.getMessage());<br /> } catch (UnsupportedEncodingException e) {<br /> // TODO Auto-generated catch block<br /> e.printStackTrace();<br /> } catch (ParserException e) {<br /> // TODO Auto-generated catch block<br /> e.printStackTrace();<br /> } <br /> <br /> return result;<br /> }<br />

wait or pause in windows batch script

problem:
i am told that windows .bat files do not have a sleep or wait command.
this is pretty sucky for when you want to wait or sleep, isnt it?

solution:
do this terrible hack
ping 1.0.0.0 -n 1 -w 5000 >NUL

that will block for 5 seconds. yey!

also, if you need to wait for a process to finish, i blogged about that here

also, if you have the ability to install something on the box where you will run your script, you should get the Windows Server 2003 Resource Kit tools from Microsoft: which works with XP. This gives you a real sleep command! (Thanks to Tricky for pointing this out in the comments)

extract links from a webpage

this is useful if you decide to make a web crawler, and dont want to bother with an html parser. You have to read the body of the page into a string, and then use this regex to extract all the links.

supported link types: <img src=...<a href=...
<br /><br />String body = "..."; //body of the page<br />Matcher m = Pattern.compile("(?im)(?:(?:(?:href)|(?:src))[ ]*?=[ ]*?[\"'])(((?:http|https)(?::\\/{2}[\\w]+)(?:[\\/|\\.]?)(?:[^\\s\"]*))|((?:\\/{0,1}[\\w\\.]+)+))[\"']").matcher(body);<br />while(m.find()){<br />String absolute = m.group(2);<br />String relative = m.group(3);<br />}<br />

an absolute url is in the form of "http://something.com/blah"
a relative url is in the form of "/something/path.blee"

now you can figure out what to do with these...

SQL: change column datatype

in this note to self, i need have a table where a column (COL2) is an NVARCHAR, and it need to become an NCLOB:

CREATE table TABLENAME_TEMP as select * from TABLENAME;
ALTER table TABLENAME drop COLUMN COL2;
DELETE from TABLENAME;
ALTER table TABLENAME add (COL2 NCLOB NOT NULL);
-- at this point the columns may not line up, so specify order...
INSERT into TABLENAME TN (TN.COL1, TN.COL3, TN.COL2) select * from TABLENAME_TEMP;
COMMIT;
DROP table TABLENAME_TEMP;

now the table TABLENAME has a column which is an NCLOB, yey!s

Remove HTML tags regex

when using an HTML parser is too much work, you may want to use a small regex to remove all the html tags

this is java:
<br />String output = input.replaceAll("</?\\w++[^>]*+>", "");<br />

Teamsite: default OpenDeploy URL

http://TS-server:8081/iw/opendeploy/login

Opendeploy deployment fails with "REQUEST_DENIED"

Problem:
I am trying to manually kick off a teamsite deployment with opendeploy 6.1, and i get this error:

Executing(ant): /app/OpenDeployNG/bin/iwodcmd start ddconfig.xml -k iwdd=deployment1

Locating OpenDeploy service.
Got OpenDeploy service
>>>>>-- Start deployment ddconfig.
iwodstart running in default synchronous mode.
Need to wait for deployment to complete.
***ERROR - Starting deployment.
Caught exception. Details: REQUEST_DENIED

Solution:
in [OD_HOME]\bin run the following command as root:
./iwodauthorize -r add od-admin [user you tried to deploy as]
so in my case:
./iwodauthorize -r add od-admin ant

then list all the users who can run deployments, just to check if it worked:
./iwodauthorize -r list od-admin

Good place for EBM and Industrial?

a bunch of different stuff:
http://www.vysniukas.com/download/-%20music/

http://www.stormwish.com/music/?N=A

a bunch of goth stuff within

http://www.gwydion.net/~helen/

some music which is good, and also some which is not so good

this has a ton of stuff:
http://maelwys.psyche7.net/

check out the music folder

SQL: using rownum to make new primary keys for insert

INSERT INTO DEVICE

SELECT 1002403148+ROWNUM, DEVICE_TYPE_ID, 11, DISPLAY_NAME, SORT_ORDER FROM DEVICE;

the old uids stopped at 1002403148 and i needed to reinsert everything from the same table but changing one column value.

java.lang.NoSuchMethodError: org.hibernate.hql.antlr.HqlBaseParser.recover

Problem:
After a deployment to our prod server, some pages do not display and we see this error in the logs:

java.lang.NoSuchMethodError: org.hibernate.hql.antlr.HqlBaseParser.recover(Lantlr/RecognitionException;Lantlr/collections/impl/BitSet;)V

It smells like a lib issue, but its not, its actually an HQL syntax issue

the offending HQL was this - can you spot the problem:
'select dattrDesc, dattrDesc.deviceLocalContent.displayName, dattrDesc.deviceLocalContent.deviceContentId, dattrDesc.attributeName, dattrDesc.attributeValue, dattrDesc.link, '', '', '' from DeviceAttributes dattrDesc'
this thing was commited into a database table.

Solution:
This part , '', '', '' created an issue, it should have been , '''', '''', '''' to escape the quotes for SQL.

teamsite does not support NCLOBs, deployments fail

We were recently faced with a problem where some teamsite "submit" deployments would fail if they contained NCLOBs. Teamsite support got back to us and told us that hey dont acctually support NCLOB datatype?!!!!!!? yes that doesnt make sense since they claim to be this great content management system. We need NCLOBs to support large strings of international chars.

Funny part was that some deployments would succeed while others would fail.

Solution:
It turned out that opendeploy comes with its own oracle driver, which was old, and that submit deployments ran as one user who had the old driver in the path, and the save deployments ran as another user with the good driver in the path.

the old driver had some problems with NCLOB, using the latest driver solved this problem

lots of music

i wonder how long this one is going to last.

http://jaymatadee.aster.net.pl/Music/

the speed isnt that great but the selection is awesome

cannot find HD installing windows on R60e

problem:
tried to install a fresh copy of windows on a thinkpad R60e, setup would eventually give me this error:
setup did not find any hard disk drives installed on your computer

It turns out that this is a problem with windows setup not recognizing any SATA drives during install.

Solution:
While poeple on teh nets all talk about using a floppy and installing SATA drivers from this floppy, i found this solution unacceptable.

Fortunately the thinkpad R60 (and perhaps many other computers) have an option in the BIOS that enables SATA compatibility mode. Enabling this mode solved my problem. Yey 2 hours wasted!

reverse an integer (in a crazy way)

<br />public static int reverse(int i) {<br /> i = (i & 0x55555555) <<>>> 1) & 0x55555555;<br /> i = (i & 0x33333333) <<>>> 2) & 0x33333333;<br /> i = (i & 0x0f0f0f0f) <<>>> 4) & 0x0f0f0f0f;<br /> i = (i <<>>> 8) & 0xff00) | (i >>> 24);<br /> return i;<br />}<br />


compare this with my really old post about the same problem

passwordless ssh/scp setup

I need to set up cron jobs which scp stuff all over the place. scp works best when you have rsa keys setup to allow you to ssh in without a password.

here are the steps to set this up (on solaris 8, possibly linux):

1) setup the public key and send it to the server
ssh-keygen -t rsa
[do not enter password when asked]
scp ~/.ssh/id_rsa.pub [user]@[server]:~/

2) open a terminal on [server] and
mkdir ~/.ssh
touch ~/.ssh/authorized_keys
cat ~/id_rsa.pub >> ~/.ssh/authorized_keys
rm ~/id_rsa.pub

Done! That was easy!

Note: if the above doesnt work for some reason, make sure that you do this to the .ssh dirs on the SERVER you are trying to connect to:
chmod go-rxw .ssh

download some mp3s

the mp3 firefox quicksearch is very handly for finding sites with a ton of mp3s:

http://www.sasessiontc.com/music/

more to come under the new label 'music'