Music for December

EBM, industrial, and some other random stuff:
http://www.pimpworks.org/~killgreedy/

some trance and EBM:
http://fawx.net/jag/music/

mostly rock and pop, but there is a motherload here, possibly 40-50 gb:

http://duesti209.dyndns.org/root/Musik/

Unix: find largest files / folder to free up space

Unix boxes like to fill up with log files, and when they get really full, stuff starts to break. These commands will tell you where the big log files sit so you can delete them or gzip them.

first, in root run
df -k
/dev/md/dsk/d3 4129290 4020939 67059 99% /var

then go to /var and find what it taking up all the space:
du -sk * | sort -m | head
2855134 mail
..
193102 tmp

so mail is taking up space, now lets look for files bigger then 100MB:
find . -size +100000000c -ls
22615 483984 -rw-r--r-- 1 root other 495339455 Dec 14 15:09 ./mail/access_log
113593 209784 -rw-r--r-- 1 root other 214696853 Dec 14 15:08 ./mail/servletd.log
208492 354768 -rw-rw-rw- 1 admin mail 363091751 Dec 14 14:29 ./mail/admin

now you can see what is steal your megabytes.

the most useful program in the world

clear&&perl -e'$|=1;@l=qw{| / -  \\};while(){print 
qq{$l[$_++%@l]\x0d};select($z,$z,$z,.2*(rand(2)**2))}'

i can think of like 9 uses for this already!

Websphere: starting up and shutting down

I have a websphere 6 clustered enviornment that i sometimes need to start up and shut down without using the web console.

there are 2 boxes in the cluster, one of them has the deployment manager on it.

each box runs 2 JVMs, which are called are called App1 and App1.1

The box that has the deployment manager needs to start/stop that, while the other does not.

FOR SHUTDOWN:
[WAS_HOME]/profiles/AppSrv01/bin/stopServer.sh App1 -username [user] -password [pass]
[WAS_HOME]/profiles/AppSrv01/bin/stopServer.sh App1.1 -username [user] -password [pass]
[WAS_HOME]/profiles/AppSrv01/bin/stopNode.sh -username [user] -password [pass]
[WAS_HOME]/profiles/Dmgr01/bin/stopManager.sh -username [user] -password [pass]

FOR STARTUP:
[WAS_HOME]/profiles/Dmgr01/bin/startManager.sh -username [user] -password [pass]
[WAS_HOME]/profiles/AppSrv01/bin/startNode.sh -username [user] -password [pass]
[WAS_HOME]/profiles/AppSrv01/bin/startServer.sh App1 -username [user] -password [pass]
[WAS_HOME]/profiles/AppSrv01/bin/startServer.sh App1.1 -username [user] -password [pass]

regex to split a document into words

Say you wanted to split a document into words, but works like are'nt shouldnt be split on the ' and numbers like 10,004,333 should remain intact, but other punctuation should be removed from the resulting word array.

A good way to do this is using Scanner's findWithinHorizon and a regular expression. This way you dont need to read the entire document into memory before processing it.

<br />Pattern p = Pattern.compile("[A-Za-z]+(?:['-][A-Za-z]+)?|\\d+(?:[,.]\\d+)?");<br />Scanner sc = new Scanner(new File("/path/to/the/file.txt"));<br />String word = null;<br />while ((word = sc.findWithinHorizon(p, 0)) != null)<br />{<br /> // etc.<br />}<br />

Teamsite DCT design considerations

I dont claim to be an expert on Teamsite, but i have put much blood sweat and tears into it. The following is my understanding of how teamsite tracks database changes, and what that means for anyone who designs datacapture templates and uses OpenDeploy.

The IWDELTRACKER tables uses the following info to track records: path of the xml, all its primary keys as defined in the dd files, and all its foreign keys. This works really well if you have 1 primary key, which is paired up with many foreign keys, but not so well if you have many primary keys, which are paired up with many(or one) foreign keys. This is the reason why one of my DCTs is flawed. In this DCT you make a resource, and assign it to many areas, so the primary key in that DCT comes from a replicant, and the foreign key comes from the main body. The problem we are facing with resource DCT right now is that if the user maps a resource to many areas, and then they delete some areas from the DCT, the changes are not deployed to the database.

Rules of thumb are:

1. Only create top-down relationships
2. NEVER use a replicant for a UID
3. Always test for relationship deletions.

Oracle: convert nclob to nvarchar2 (part 2)

I needed to convert a column in a database from NCLOB to NVARCHAR2(2000) which is the max you can store in NVARCHAR2. The problem was that some columns in the table where larger then 2000 chars. To solve this i needed to use the oracle length function to select only rows who's lenth was less then 2000. In this example sql, the table that has the NCLOB is called THETABLE, and the column that needs to become NVARCHAR2 is called THECOLUMN.

<br />CREATE TABLE THETABLE_TEMP as select * from THETABLE;<br />alter table THETABLE drop COLUMN THECOLUMN;<br />alter table THETABLE add (THECOLUMN NVARCHAR2(2000));<br />UPDATE THETABLE p SET p.THECOLUMN = <br />(SELECT t.THECOLUMN FROM THETABLE_TEMP t <br />WHERE t.PRIMEKEY = p.PRIMEKEY <br />and dbms_lob.getlength(t.THECOLUMN) <= 2000) <br />

dbms_lob.getlength(column) is the key to success here

IWDatacapture.callServer call fails / caveat

problem:

when i build teamsite DCTs i always find a new and wonderful way to break something.

Today i was building a DCT that used IWDatacapture.callServer to call a cgi script that would get some stuff from the database and then out put it into a hidden frame and do a javascript callback to the parent window.

everything was going well until i made the mistake of making 2 calls to different IWDatacapture.callServer cgi scripts one right after the other. one of them would always fail.

solution:
The problem stems from the fact that CGI scripts that output html make use of a single blank.html file to write out their contents. If you call 2 of them "almost at the same time" one of them will overwrite the other's blank.html file and cause it to fail. The only reasonable solution is to combine both CGIs into one file and let it write to one blank.html file, you can also try to make sure that CGIs dont get called at the same time, but that would be difficult to control.

So it seems that its probably a reasonable requirement to only allow ONE IWDatacapture.callServer call per DCR, otherwise things might break. How sad.

Java: Understanding Weak References

The other day i was reading programming at reddit and stumbled on this article:
Understanding Weak References which was a good read but i still had some questions, so i turned to Java Developer Forums for help. I didnt get flamed! (much). What i got was a very good response from a user named kdgregory which is note-to-dogself worthy.

My questions are in blue boxes:

If weak references are great at stopping leaks, why not use them all the time?

They're great at stopping leaks in the situation where you're attaching data to an object that (should) have a limited lifetime. If you used a regular map (most likely an IdentityHashMap), you'd be creating a strong reference to the object via the map, so it would never be collected, even when there are no other references to it (I call this a leak, other people don't).

For example, ThreadLocal uses weak references, because they'll be cleared when the thread ends.

ObjectOutputStream should use weak references (and a changed protocol) so that it can be used in long-running conversations without constantly calling reset(). But it doesn't.

"WeakHashMap works exactly like HashMap, except that the keys (not the values!) are referred to using weak references."

• does this imply that hashmap stores values in weak refs, if so, why?

• should i use WeakHashMap for tasks where i will have another reference to objects i put into the map. ie i dont put my only ref into this map?

No, HashMap doesn't use weak references for either key or value. The author's wording (and for that matter, the JDK JavaDoc wording) is a little misleading.

One typical use for weak references is in a canonicalizing map to reduce memory consumption with duplicated data. With such a map, you would store an object as both key and value, and retrieve the canonical value when presented with an arbitrary key -- think String.intern(). However, you can't use WeakHashMap to do this without wrapping the value in a weak reference also.

When do i not use weak references ?

Most of the time. Any reference object means that you have to access your original object in two steps (have to call get() on the reference) rather than just one. And normally, you'd know the lifetime of your objects.

music for november

Using the google mp3 search i find some good stuff, im going to put all the november music into this post.

this place has a player built in, very nice:
http://media.soundfixrecords.com/player/xspf_player.swf

lots of full albums here - seems like dance / dnb:
http://danav.org/audio/

parse xml with JDOM and Xpath

every now and again its good to have your config in an xml file, but i am usually to lazy to remember to how to do it properly and just use java.util.Properties. Well no more of that!

This is a short self contained code sample on how to parse some xml using jdom and jaxen

you will need:
jdom lib
jaxen lib (which may be old now)

<br /> public void example() throws JDOMException, IOException {<br /> // import org.jdom.Document;<br /> // import org.jdom.Element;<br /> // import org.jdom.JDOMException;<br /> // import org.jdom.input.SAXBuilder;<br /> // import org.jdom.xpath.XPath;<br /> String fn = "/path/to/some.xml";<br /> FileInputStream in = new FileInputStream(fn);<br /> SAXBuilder builder = new SAXBuilder();<br /> Document doc = builder.build(in);<br /> in.close();<br /> XPath xpath = XPath.newInstance("//some/xpath[1]/expression");<br /> List list = xpath.selectNodes(doc);<br /> for (Iterator iter = list.iterator(); iter.hasNext();) {<br /> Element elem = (Element) iter.next();<br /> <br /> String name = elem.getAttributeValue("name");<br /> String childText = elem.getChild("child").getTextTrim();<br /> String text = elem.getTextTrim();<br /> }<br /> }<br />

wait in windows bat script - good way

i recently wrote about using ping to pause for a number of seconds in a batch script. It was brought to my attention that you can use /wait to actually wait for a process to complete like so:

start /wait /i "unimportant title put whatever here" "C:\path\to\some\executable.exe"

or if you have an msi installer you can do this:

msiexec /i "C:\path\to\msiinstaller.msi" /qr

extract links using htmlparser

i have come to realize that even though i love to do everything in regex, extracting links with regex is not really the best idea.

the best idea is to use htmlparser to extract the links.

This code sniplet will extract links from a page body, and given the link where the body was found it will resolve the relative links. This is tailored to be used by a crawler.

<br />public static Collection<String> retrieveLinks(String url, byte[] body, String charSet) {<br /><br /> Collection<String> result = new ArrayList<String>();<br /> <br /> URI uriLink;<br /> try {<br /> uriLink = new URI(url);//url of the body<br /> String st = new String(body, charSet);<br /> Parser parser = new Parser();<br /> parser.setInputHTML(st);<br /> NodeList list = parser.extractAllNodesThatMatch(new NodeClassFilter (LinkTag.class));<br /> for (int i = 0; i < list.size (); i++){<br /> LinkTag extracted = (LinkTag)list.elementAt(i);<br /> <br /> if(!extracted.isHTTPLikeLink()) continue; //ignore mailto / javascript and other weird protocol links<br /> <br /> <br /> String extractedLink = extracted.extractLink().replaceAll("&", "&");//we need to unescape these<br /> <br /> extractedLink = extractedLink.replaceAll(" ","%20");//URI class doesnt like spaces in URLs, but content creators dont care.. :P<br /> extractedLink = extractedLink.trim();//URI will barf on a link that look like "foo.com "<br /> if(extractedLink.length() == 0) continue; // i am betting that a link that looks like "" is user error. with our url scheme, it most certainly is.. <br /> if(extractedLink.startsWith("#")) continue; //skip all anchors, they are useless to a crawler<br /> if(extractedLink.matches("(?i)^javascript:.*"))continue; //HTMLParser thinks anything but lower case 'javascript:' is a non-js link<br /> <br /> URI uriRelative = new URI(extractedLink);<br /> //System.out.println("i:"+i);<br /> URI resolved = uriLink.resolve(uriRelative);<br /> //linkfilter will return true if crawling from <br /> //url --> resolvedUrl<br /> //is allowed by some rules<br /> String link = resolved.toString();<br /> //here we should check if the link is acceptable.<br /> result.add(link);<br /> <br /> }<br /> }<br /> } catch (URISyntaxException e) {<br /> // TODO: this is a bad link, log this!<br /> log.info("Bad Link Syntax on page:["+url+"] BAD LINK:"+e.getMessage());<br /> } catch (UnsupportedEncodingException e) {<br /> // TODO Auto-generated catch block<br /> e.printStackTrace();<br /> } catch (ParserException e) {<br /> // TODO Auto-generated catch block<br /> e.printStackTrace();<br /> } <br /> <br /> return result;<br /> }<br />

wait or pause in windows batch script

problem:
i am told that windows .bat files do not have a sleep or wait command.
this is pretty sucky for when you want to wait or sleep, isnt it?

solution:
do this terrible hack
ping 1.0.0.0 -n 1 -w 5000 >NUL

that will block for 5 seconds. yey!

also, if you need to wait for a process to finish, i blogged about that here

also, if you have the ability to install something on the box where you will run your script, you should get the Windows Server 2003 Resource Kit tools from Microsoft: which works with XP. This gives you a real sleep command! (Thanks to Tricky for pointing this out in the comments)

extract links from a webpage

this is useful if you decide to make a web crawler, and dont want to bother with an html parser. You have to read the body of the page into a string, and then use this regex to extract all the links.

supported link types: <img src=...<a href=...
<br /><br />String body = "..."; //body of the page<br />Matcher m = Pattern.compile("(?im)(?:(?:(?:href)|(?:src))[ ]*?=[ ]*?[\"'])(((?:http|https)(?::\\/{2}[\\w]+)(?:[\\/|\\.]?)(?:[^\\s\"]*))|((?:\\/{0,1}[\\w\\.]+)+))[\"']").matcher(body);<br />while(m.find()){<br />String absolute = m.group(2);<br />String relative = m.group(3);<br />}<br />

an absolute url is in the form of "http://something.com/blah"
a relative url is in the form of "/something/path.blee"

now you can figure out what to do with these...

SQL: change column datatype

in this note to self, i need have a table where a column (COL2) is an NVARCHAR, and it need to become an NCLOB:

CREATE table TABLENAME_TEMP as select * from TABLENAME;
ALTER table TABLENAME drop COLUMN COL2;
DELETE from TABLENAME;
ALTER table TABLENAME add (COL2 NCLOB NOT NULL);
-- at this point the columns may not line up, so specify order...
INSERT into TABLENAME TN (TN.COL1, TN.COL3, TN.COL2) select * from TABLENAME_TEMP;
COMMIT;
DROP table TABLENAME_TEMP;

now the table TABLENAME has a column which is an NCLOB, yey!s

Remove HTML tags regex

when using an HTML parser is too much work, you may want to use a small regex to remove all the html tags

this is java:
<br />String output = input.replaceAll("</?\\w++[^>]*+>", "");<br />

Teamsite: default OpenDeploy URL

http://TS-server:8081/iw/opendeploy/login

Opendeploy deployment fails with "REQUEST_DENIED"

Problem:
I am trying to manually kick off a teamsite deployment with opendeploy 6.1, and i get this error:

Executing(ant): /app/OpenDeployNG/bin/iwodcmd start ddconfig.xml -k iwdd=deployment1

Locating OpenDeploy service.
Got OpenDeploy service
>>>>>-- Start deployment ddconfig.
iwodstart running in default synchronous mode.
Need to wait for deployment to complete.
***ERROR - Starting deployment.
Caught exception. Details: REQUEST_DENIED

Solution:
in [OD_HOME]\bin run the following command as root:
./iwodauthorize -r add od-admin [user you tried to deploy as]
so in my case:
./iwodauthorize -r add od-admin ant

then list all the users who can run deployments, just to check if it worked:
./iwodauthorize -r list od-admin

Good place for EBM and Industrial?

a bunch of different stuff:
http://www.vysniukas.com/download/-%20music/

http://www.stormwish.com/music/?N=A

a bunch of goth stuff within

http://www.gwydion.net/~helen/

some music which is good, and also some which is not so good

this has a ton of stuff:
http://maelwys.psyche7.net/

check out the music folder

SQL: using rownum to make new primary keys for insert

INSERT INTO DEVICE

SELECT 1002403148+ROWNUM, DEVICE_TYPE_ID, 11, DISPLAY_NAME, SORT_ORDER FROM DEVICE;

the old uids stopped at 1002403148 and i needed to reinsert everything from the same table but changing one column value.

java.lang.NoSuchMethodError: org.hibernate.hql.antlr.HqlBaseParser.recover

Problem:
After a deployment to our prod server, some pages do not display and we see this error in the logs:

java.lang.NoSuchMethodError: org.hibernate.hql.antlr.HqlBaseParser.recover(Lantlr/RecognitionException;Lantlr/collections/impl/BitSet;)V

It smells like a lib issue, but its not, its actually an HQL syntax issue

the offending HQL was this - can you spot the problem:
'select dattrDesc, dattrDesc.deviceLocalContent.displayName, dattrDesc.deviceLocalContent.deviceContentId, dattrDesc.attributeName, dattrDesc.attributeValue, dattrDesc.link, '', '', '' from DeviceAttributes dattrDesc'
this thing was commited into a database table.

Solution:
This part , '', '', '' created an issue, it should have been , '''', '''', '''' to escape the quotes for SQL.

teamsite does not support NCLOBs, deployments fail

We were recently faced with a problem where some teamsite "submit" deployments would fail if they contained NCLOBs. Teamsite support got back to us and told us that hey dont acctually support NCLOB datatype?!!!!!!? yes that doesnt make sense since they claim to be this great content management system. We need NCLOBs to support large strings of international chars.

Funny part was that some deployments would succeed while others would fail.

Solution:
It turned out that opendeploy comes with its own oracle driver, which was old, and that submit deployments ran as one user who had the old driver in the path, and the save deployments ran as another user with the good driver in the path.

the old driver had some problems with NCLOB, using the latest driver solved this problem

lots of music

i wonder how long this one is going to last.

http://jaymatadee.aster.net.pl/Music/

the speed isnt that great but the selection is awesome

cannot find HD installing windows on R60e

problem:
tried to install a fresh copy of windows on a thinkpad R60e, setup would eventually give me this error:
setup did not find any hard disk drives installed on your computer

It turns out that this is a problem with windows setup not recognizing any SATA drives during install.

Solution:
While poeple on teh nets all talk about using a floppy and installing SATA drivers from this floppy, i found this solution unacceptable.

Fortunately the thinkpad R60 (and perhaps many other computers) have an option in the BIOS that enables SATA compatibility mode. Enabling this mode solved my problem. Yey 2 hours wasted!

reverse an integer (in a crazy way)

<br />public static int reverse(int i) {<br /> i = (i & 0x55555555) <<>>> 1) & 0x55555555;<br /> i = (i & 0x33333333) <<>>> 2) & 0x33333333;<br /> i = (i & 0x0f0f0f0f) <<>>> 4) & 0x0f0f0f0f;<br /> i = (i <<>>> 8) & 0xff00) | (i >>> 24);<br /> return i;<br />}<br />


compare this with my really old post about the same problem

passwordless ssh/scp setup

I need to set up cron jobs which scp stuff all over the place. scp works best when you have rsa keys setup to allow you to ssh in without a password.

here are the steps to set this up (on solaris 8, possibly linux):

1) setup the public key and send it to the server
ssh-keygen -t rsa
[do not enter password when asked]
scp ~/.ssh/id_rsa.pub [user]@[server]:~/

2) open a terminal on [server] and
mkdir ~/.ssh
touch ~/.ssh/authorized_keys
cat ~/id_rsa.pub >> ~/.ssh/authorized_keys
rm ~/id_rsa.pub

Done! That was easy!

Note: if the above doesnt work for some reason, make sure that you do this to the .ssh dirs on the SERVER you are trying to connect to:
chmod go-rxw .ssh

download some mp3s

the mp3 firefox quicksearch is very handly for finding sites with a ton of mp3s:

http://www.sasessiontc.com/music/

more to come under the new label 'music'

ant SCP task hangs

problem:
i am running ant 1.6.5 and using the scp task. When the task starts, it stops responding, breaking the rest of my ant build in the process. Sometimes the task transfers one file (out of a few 100) and hangs.

solution:
From doing a bit of research i found out that some versions of the library required by this task - jsch - cause ant build to hang. The trick is finding a version of this lib which works with your version of ant.

I know that jsch version 0.1.29 works with ant 1.6.5.

get jsch-0.1.29.jar here

css problems

Ive gotta write this stuff down before i forget it all:

problem:
z-index does not work. div with a higher z-index is getting overlapped by a div with lower z-index

solution:
one of the divs need to have this set -> position:absolute;



problem:
i need IE and Firefox styles on the same style sheet using a hack

solution:
#someId {
/* IE can only see this rule, IE uses this rule */
position: relative;
top: -15px;
}
html>body #someId {
/* Firefox can see this rule, rule overrides previous rule */
position: relative;
top: 25px;
}



problem:
i want to change the position of a div, lift it up by 100 pixels

solution:
there are 2 ways to do this,
margin-top:-100px;
or
position:relative;
top:-100px;

these have slightly different results, and i should try both.

UNIX: get the last modified file in a dir

problem:
i need a command that will tell me the last modified file or dir

solution:
ls /the/dir/ -1 -t | head -1

teamsite: passing parameters from javascript to perl

Teamsite (the evil abomination called software it is) allows the unfortunate souls using it to call perl scripts for javascript and pass parameters to the perl script.

the syntax looks something like this:

var params = new Object();
params.val = "some string";
var server = window.location.hostname;
IWDatacapture.callServer("http://"+server+"/iw-bin/somescript.ipl",params);

problem:
sometimes the params dont carry over correctly from javascript to the perl script, ie, all params have values in the javascript, but on the perl script side, they have no values - ""

the problem manifests if you do this:
when you assign the params in this way:
var params = new Object();
params.val = SomeFunctionWhichReturnsAString();
now chances are that params.val = "" in the perl script.

solution:
this problem stinks of pointers. params.val is set given a reference to something in the javascript heap, when perl takes over, that references points to nothing. Either way, this is how you get around it:
var params = new Object();
params.val = SomeFunctionWhichReturnsAString()+"";

now everything works. why? well... because teamsite is evil.

links i enjoy lately

bunch of links which i have been finding useful:

bunch of music
addictive game
hit counter for this blog
cool pictures/art?

lastly, google mp3 quick search link for firefox:
location:
http://www.google.com/search?q={-inurl:(htm|html|php) intitle:%22index of%22 %22last modified%22 %22parent directory%22 description size (wma|mp3) %22%s%22}
keyword:
m

Teamsite: IWDataDeploy FAILED TDbSchemaSynchronizer create failure

Interwoven Teamsite is the worst piece of software i have ever used, and googling its errors gives you 0 hits. I am going to do the world a favor and change that!

Problem: I needed to deploy DCT data to the table which teamsite uses for its own record keeping. The file which takes care of this deployment is called AREA_dd.cfg where area is the name of the DCT. After making sure that teamsite is using the correct db info, and that all fields i was deploying existed i was getting the error: IWDataDeploy FAILED, where the root cause might have been:
(from the log in OD_HOME/OpenDeployNG/log/iwddd_something.log)

DBD: SELECT * FROM USER_TABLES WHERE TABLE_NAME='IWTRACKER'
DBD: Table [iwtracker] exists.
DBD: DEFAULT__DEVICE__MAIN_STAGING not registered
DBD: Error occured in TDbSchemaSynchronizer
DBD: ERROR:TDbSchemaSynchronizer create failure.


(DEVICE is the name of my DCT)
Solution:
This solution is a hack, but it works. Open the database you are trying to commit to from teamsite, and run the following SQL query:
INSERT INTO IWTRACKER (NAME) VALUES('DEFAULT__DEVICE__MAIN_STAGING');
COMMIT;

where instead of 'DEFAULT__DEVICE__MAIN_STAGING' put the thing which the previous error claims is not registered. Save the DCT again and tail the log, the deployment should succeed. (if it fails, you are probably screwed because the support is terrible and the documentation is teh suck)

edit 11/6/2009: I no longer use teamshite, so please dont ask me any questions about this evil thing, i wont know the answers.

Find and replace with VI

this will replace all occurrences of match_pattern in the file

:g/match_pattern/s//replace_string/g
alternatively, you can use ' : ' instead of ' / ' to make path (/) slashes easier to manage (no escape (\) character needed)
:g:match_pattern:s::replace_string:g

Using EL in custom tags as attributes

Problem:
I was working on a custom tag and i needed to pass it a bean from a JSP page kind of like this:
<customtag var="${somebean.something.else}">
I realized that what i was getting inside my custom tag was a String, not my bean.

Solution:
You need to use the ExpressionEvaluator to evaluate the expression, and get the bean, here is the code i used in my SimpleTag: (i was getting a Date object by the way)

<br /><br />ExpressionEvaluator eval = getJspContext().getExpressionEvaluator();<br />Date date = null;<br />//dateVar this is the EL expression<br />try { //this thing here resolves the date expression into a java object!<br /> date = (Date)eval.evaluate(dateVar, Date.class, getJspContext().getVariableResolver(), new FunctionMapper(){<br />public Method resolveFunction(String arg0, String arg1) {<br /> return null; //i couldnt figure out how to get a FunctionMapper from somewhere...<br />}<br /> });<br />} catch (ELException e1) {<br /> throw new JspException(e1);<br />}<br />


Edit Aug 7, 2008:
It has come to my attention that in jsp 2.0 the EL expressions are evaluated before being passed to the custom tags, so you dont need to do any of this crap anymore.

how to make .so files load in unix

Ive had this problem too many times and so i should blog the solution. The problem usually looks something like this: You try to run something and you get an error screaming about failing to load some file ending in .so

example:

failed: Can't load '/app/teamsite/iw-perl/site/lib/auto/DBD/Oracle/
Oracle.so' for module DBD::Oracle: ld.so.1: perl: fatal: libnnz10.so: open failed: No such
file or directory at /app/teamsite/iw-perl/lib/DynaLoader.pm line 229.

solution:

All these .so problems seem to be have a common solution. You need tell unix where to look for your libraries. This is how you do it.

1. Figure out where your application has its .so files. Usually this is in a directory called lib, and it usually contains more then one .so file.
2. You need to become root, and run this command:
3. crle
4. write down everything which is output, because if you break things you will need to go back to this. On solaris 10 this is the important piece which has to always be in the output: Default Library Path (ELF): /usr/lib
5. Now we need to add our library locations to the Default Library Path, run this as root:
   
6. crle -c /var/ld/ld.config -l /usr/lib:/your/directory/where/the/so/files/are
7. exit root

Now try to run your application and see if it runs. If it does not run then it may complain about another .so file in another directory which you missed. You will need to add it to the end of the path in step 5 like this:
crle -c /var/ld/ld.config -l /usr/lib:/your/directory/where/the/so/files/are:/another/lib/path

Important: if you remove the /usr/lib part from the Default Library Path, shit will hit the fan.
Use this at your own risk.

Flash in IE6 xml compression problem

Problem:

There seems to be a problem with flash player running on IE6 loading XML files from a web server which has compression enabled. It seems like the XML is never decompressed correctly, so whatever flash that is using it breaks. Everything seems to work fine if you use firefox or IE7 to access the same flash swf. I found one page talking about this problem, right here:

http://blog.mediacatalyst.com/pivot/entry.php?id=281

Solution:

First let me say, that on our servers the XML was generated by a JSP, so i had a bit more control over html headers and such. This is what i did to solve this problem:

1. Set Cache-Control HTML header to nothing ("")
   
2. Set Pragma HTML header to nothing
   
3. Make sure that the XML file had the XML declaration as the very first thing in the file
4. Set the JSP page content-type to text/xml

How to backup firefox plugins

I needed to back up my plugins to install the DOM inspector, which requires a firefox reinstall. Here is how to back up the firefox plugins:

C:\Documents & Settings\USERNAME\Application Data\Mozilla\Firefox\Profiles\
there are directories in there
copy those out to save them
then.... after you reinstall, copy them back and run "firefox -profilemanager"
and create a "NEW" profile using the directory you just copied

It works cross platform also... put the stuff on a jumpdrive and have the same profile in Linux and Windows

source: www.jacobsylvia.com

How To: Implement RMI from scratch

The java.lang.reflect.Proxy class will let you provide an arbitrary implementation for an interface.

So, start with that on the client. Figure out how to produce a factory for interface objects. Understand how the proxy invocation works.

Then, create a socket connection to the server. Wrap it with an ObjectInputStream/ObjectOutputStream.

Now you need to define a protocol between client and server. For example, let's say that the client sends an integer object ID, followed by a method name, followed by an array of parameters. The server needs to find the object, and invoke the appropriate method. Then it needs to send the result back to the client.

Then you can get into distributed GC ...

source: java dev forums

JSP: custom tag to modify its body

This is a skeleton of what is needed to build a custom jsp tag which does something to its body. For example a tag could change the urls of all the links found in its body.

<br />package com.mycompany.package.of.the.tag;<br /><br />import javax.servlet.jsp.JspException;<br />import javax.servlet.jsp.JspWriter;<br />import javax.servlet.jsp.tagext.BodyContent;<br />import javax.servlet.jsp.tagext.BodyTagSupport;<br /><br />public class CustomTag extends BodyTagSupport {<br />private static final long serialVersionUID = 1L;<br /><br />public int doAfterBody() throws JspException {<br /> try {<br /> BodyContent bc = getBodyContent();<br /> String body = bc.getString();<br /> // do something to the body here.<br /> JspWriter out = bc.getEnclosingWriter();<br /> if(body != null) {<br /> out.print(buff.toString());<br /> }<br /> } catch(IOException ioe) {<br /> throw new JspException("Error: "+ioe.getMessage());<br /> }<br /> return EVAL_BODY_INCLUDE;<br /> }<br />}<br />

make a file called CustomTag.tld and put it into WEB-INF:
<br /><?xml version = "1.0" encoding = "ISO-8859-1" ?><br /><!DOCTYPE taglib<br />PUBLIC "-//Sun Microsystems, Inc.//DTD JSP Tag Library 1.1//EN"<br />"http://java.sun.com/j2ee/dtds/web-jsptaglibrary_1_1,dtd"><br /><taglib><br /><tlibversion>1.0</tlibversion><br /><jspversion>1.1</jspversion><br /><uri>com.mycompany.CustomTag</uri><br /><shortname>Custom Tag</shortname><br /><tag><br /><name>ct</name><br /><tagclass>com.mycompany.package.of.the.tag.CustomTag</tagclass><br /><bodycontent>JSP</bodycontent><br /></tag><br /></taglib><br /><br />

use it like this on your JSP:
<%@ taglib uri="com.mycompany.CustomTag" prefix="x"%>

<x:ct>
here is some stuff inside the tag!
<c:out value="${someparam}"/>
</x:ct>

How to make a redirect servlet

make a servlet which calls the following method on doPost() and on doGet()

<br />private void doRedirect(HttpServletRequest request, HttpServletResponse response) throws ServletException {<br /> String servletPath = request.getServletPath();<br /> //do something with the path<br /> String newServletPath = servletPath.toLowerCase();<br /> try {<br /> getServletContext().getRequestDispatcher(newServletPath).forward(request, response);<br /> } catch(IOException ioE) {<br /> throw new ServletException("Unable to forward request: " + servletPath + " to path: " + newServletPath, ioE);<br /> }<br />}<br />

improve eclipse performance in windows

this eclipse plugin forces windows keep eclipse in memory / reduces thrashing

http://suif.stanford.edu/pub/keepresident/

UNIX: list what sudo permissions you have

> sudo -l
Password:
You may run the following commands on this host:
(root) /usr/local/apache/bin/
(root) NOPASSWD: /app/teamsite/iw-home/opendeploy/bin/iwsyncdb.ipl
(root) NOPASSWD: /etc/init.d/smb
(root) NOPASSWD: /usr/local/samba/bin
(root) NOPASSWD: /bin/vi /etc/group
(root) NOPASSWD: /usr/local/bin/top
(root) NOPASSWD: /bin/vi /etc/group
(root) NOPASSWD: /usr/bin/ls
(root) NOPASSWD: /usr/ucb/vipw
(root) NOPASSWD: /usr/local/bin/top
(root) /bin/tail

Javascript: matching with regex

There seem to be 3 ways to use regular expressions in javascript:
1.
var val = document.getElementById("some_id").value;
val.replace(/^\s+|\s+$/g, ""); //trim using regex
2.
var alpha = /^[A-Za-z]+$/ ;
var val = document.getElementById("some_id").value;
if(alpha.test(val)){
//val matches alpha
}
3.

var val = document.getElementById("some_id").value;
var alpha ='((?:[a-z][a-z]+))'; // Word 1
var p = new RegExp(alpha,["i"]);
var m = p.exec(val);
if (m.length>0)
{
   var word=m[1];
   //word is the capture group 1
}

if anyone knows a better way, feel free to share.

NoClassDefFoundError: com/sun/mail/util/SharedByteArrayInputStream

problem:
when trying to send mail i get error: NoClassDefFoundError: com/sun/mail/util/SharedByteArrayInputStream

solution:
get mail.jar and put it on your classpath

http://java.sun.com/products/javamail/downloads/index.html

Java: find nested balanced tags

Problem:
I needed to parse out some divs from a html page, and java's regex cant do it. atleast when you have arbitrary number of nested balanced elements.

Solution:
<br /><br />private String removeDivContents(String id, String clazz,<br /> String text){<br />String div = "<div class="\"><br /> +clazz.toLowerCase()+"\" id=\""+id.toLowerCase()+"\">";<br />String textl = text.toLowerCase();<br />int index = textl.indexOf(div);<br />if(index == -1){<br /> return text;<br />}<br />int loc = index + div.length();<br />int end = findBalancedClosingTag(loc, textl,<br /> "<div","</div>", 1);<br />return text.substring(0,index) + text.substring(end);<br />}<br /><br />private int findBalancedClosingTag(int start,<br />String textl, String startTag, String endTag, int unClosedStarts){<br />int index = textl.indexOf(endTag,start);<br />if(index == -1){<br /> return start;<br />}<br />Pattern p = Pattern.compile(startTag);<br />Matcher m = p.matcher(textl.substring(start, index));<br />unClosedStarts --; // we are in this function so we closed ONE start tag<br />while(m.find()){<br /> unClosedStarts ++;<br />}<br />if(unClosedStarts == 0){<br /> return index + endTag.length();<br />} else {<br /> return findBalancedClosingTag(index + endTag.length(), textl, startTag, endTag, unClosedStarts);<br />}<br />}<br />
Use at your own risk.

UNIX: dd trick to speed up disk access times

This was stolen from the Lucene mailing list as a strategy to warm up an IndexSearcher:
Something like dd if=/path/to/index/foo.cfs of=/dev/null
Basically, force the data through the kernel preemptively, so FS caches it.
Run vmstat while doing it, and if the index hasn't been cached by the FS, you should see a spike in IO activity while dd is running.

source: (something in the middle of the thread)
http://www.gossamer-threads.com/lists/lucene/java-user/43418

java.lang.NoSuchFieldError:prohibited Lucene Highligher

Problem:

java.lang.NoSuchFieldError: prohibited
    at
org.apache.lucene.search.highlight.QueryTermExtractor
.getTermsFromBooleanQuery(QueryTermExtractor.java:91)

Solution:

You are using an old (pre 1.9) highlighter lib on a
1.9+ lucene. You need to pull the latest highlighter
sources from the svn repos, make a jar out of them and
use that inst

also, 'prohibited' referes to actual method param inside
of highlighter. 

nice regex util class, replaceAll

elliotth.blogspot.com/java-implementation-of-rubys-gsub.html

Here is an example of what this does:

String result = new Rewriter("([0-9]+) US cents") {
public String replacement() {
  long dollars = Long.parseLong(group(1))/100;
     return "$" + dollars;
  }
}.rewrite("5000 US cents");
System.out.println(result);

prints>> $50

nice!

java.io.IOException: read past EOF on Lucene

Problem:
This is the error:
java.io.IOException: read past EOF

Lucene throws that error when you try to read an index.

Solution:
The lucene index was most likely corrupted at some point. In my case, it broke during the ant's copy in the deployment.

Other pages say that it may be because the index uses a weird char set where chars may be bigger or smaller then what you think they are, so if you think they are 8 bits, and they are actually 7, you will read past EOF.