extract links from a webpage

this is useful if you decide to make a web crawler, and dont want to bother with an html parser. You have to read the body of the page into a string, and then use this regex to extract all the links.

supported link types: <img src=...<a href=...
<br /><br />String body = "..."; //body of the page<br />Matcher m = Pattern.compile("(?im)(?:(?:(?:href)|(?:src))[ ]*?=[ ]*?[\"'])(((?:http|https)(?::\\/{2}[\\w]+)(?:[\\/|\\.]?)(?:[^\\s\"]*))|((?:\\/{0,1}[\\w\\.]+)+))[\"']").matcher(body);<br />while(m.find()){<br />String absolute = m.group(2);<br />String relative = m.group(3);<br />}<br />

an absolute url is in the form of "http://something.com/blah"
a relative url is in the form of "/something/path.blee"

now you can figure out what to do with these...

SQL: change column datatype

in this note to self, i need have a table where a column (COL2) is an NVARCHAR, and it need to become an NCLOB:

CREATE table TABLENAME_TEMP as select * from TABLENAME;
ALTER table TABLENAME drop COLUMN COL2;
DELETE from TABLENAME;
ALTER table TABLENAME add (COL2 NCLOB NOT NULL);
-- at this point the columns may not line up, so specify order...
INSERT into TABLENAME TN (TN.COL1, TN.COL3, TN.COL2) select * from TABLENAME_TEMP;
COMMIT;
DROP table TABLENAME_TEMP;

now the table TABLENAME has a column which is an NCLOB, yey!s

Remove HTML tags regex

when using an HTML parser is too much work, you may want to use a small regex to remove all the html tags

this is java:
<br />String output = input.replaceAll("</?\\w++[^>]*+>", "");<br />

Teamsite: default OpenDeploy URL

http://TS-server:8081/iw/opendeploy/login

Opendeploy deployment fails with "REQUEST_DENIED"

Problem:
I am trying to manually kick off a teamsite deployment with opendeploy 6.1, and i get this error:

Executing(ant): /app/OpenDeployNG/bin/iwodcmd start ddconfig.xml -k iwdd=deployment1

Locating OpenDeploy service.
Got OpenDeploy service
>>>>>-- Start deployment ddconfig.
iwodstart running in default synchronous mode.
Need to wait for deployment to complete.
***ERROR - Starting deployment.
Caught exception. Details: REQUEST_DENIED

Solution:
in [OD_HOME]\bin run the following command as root:
./iwodauthorize -r add od-admin [user you tried to deploy as]
so in my case:
./iwodauthorize -r add od-admin ant

then list all the users who can run deployments, just to check if it worked:
./iwodauthorize -r list od-admin

Good place for EBM and Industrial?

a bunch of different stuff:
http://www.vysniukas.com/download/-%20music/

http://www.stormwish.com/music/?N=A

a bunch of goth stuff within

http://www.gwydion.net/~helen/

some music which is good, and also some which is not so good

this has a ton of stuff:
http://maelwys.psyche7.net/

check out the music folder

SQL: using rownum to make new primary keys for insert

INSERT INTO DEVICE

SELECT 1002403148+ROWNUM, DEVICE_TYPE_ID, 11, DISPLAY_NAME, SORT_ORDER FROM DEVICE;

the old uids stopped at 1002403148 and i needed to reinsert everything from the same table but changing one column value.

java.lang.NoSuchMethodError: org.hibernate.hql.antlr.HqlBaseParser.recover

Problem:
After a deployment to our prod server, some pages do not display and we see this error in the logs:

java.lang.NoSuchMethodError: org.hibernate.hql.antlr.HqlBaseParser.recover(Lantlr/RecognitionException;Lantlr/collections/impl/BitSet;)V

It smells like a lib issue, but its not, its actually an HQL syntax issue

the offending HQL was this - can you spot the problem:
'select dattrDesc, dattrDesc.deviceLocalContent.displayName, dattrDesc.deviceLocalContent.deviceContentId, dattrDesc.attributeName, dattrDesc.attributeValue, dattrDesc.link, '', '', '' from DeviceAttributes dattrDesc'
this thing was commited into a database table.

Solution:
This part , '', '', '' created an issue, it should have been , '''', '''', '''' to escape the quotes for SQL.

teamsite does not support NCLOBs, deployments fail

We were recently faced with a problem where some teamsite "submit" deployments would fail if they contained NCLOBs. Teamsite support got back to us and told us that hey dont acctually support NCLOB datatype?!!!!!!? yes that doesnt make sense since they claim to be this great content management system. We need NCLOBs to support large strings of international chars.

Funny part was that some deployments would succeed while others would fail.

Solution:
It turned out that opendeploy comes with its own oracle driver, which was old, and that submit deployments ran as one user who had the old driver in the path, and the save deployments ran as another user with the good driver in the path.

the old driver had some problems with NCLOB, using the latest driver solved this problem

lots of music

i wonder how long this one is going to last.

http://jaymatadee.aster.net.pl/Music/

the speed isnt that great but the selection is awesome

cannot find HD installing windows on R60e

problem:
tried to install a fresh copy of windows on a thinkpad R60e, setup would eventually give me this error:
setup did not find any hard disk drives installed on your computer

It turns out that this is a problem with windows setup not recognizing any SATA drives during install.

Solution:
While poeple on teh nets all talk about using a floppy and installing SATA drivers from this floppy, i found this solution unacceptable.

Fortunately the thinkpad R60 (and perhaps many other computers) have an option in the BIOS that enables SATA compatibility mode. Enabling this mode solved my problem. Yey 2 hours wasted!

reverse an integer (in a crazy way)

<br />public static int reverse(int i) {<br /> i = (i & 0x55555555) <<>>> 1) & 0x55555555;<br /> i = (i & 0x33333333) <<>>> 2) & 0x33333333;<br /> i = (i & 0x0f0f0f0f) <<>>> 4) & 0x0f0f0f0f;<br /> i = (i <<>>> 8) & 0xff00) | (i >>> 24);<br /> return i;<br />}<br />


compare this with my really old post about the same problem

passwordless ssh/scp setup

I need to set up cron jobs which scp stuff all over the place. scp works best when you have rsa keys setup to allow you to ssh in without a password.

here are the steps to set this up (on solaris 8, possibly linux):

1) setup the public key and send it to the server
ssh-keygen -t rsa
[do not enter password when asked]
scp ~/.ssh/id_rsa.pub [user]@[server]:~/

2) open a terminal on [server] and
mkdir ~/.ssh
touch ~/.ssh/authorized_keys
cat ~/id_rsa.pub >> ~/.ssh/authorized_keys
rm ~/id_rsa.pub

Done! That was easy!

Note: if the above doesnt work for some reason, make sure that you do this to the .ssh dirs on the SERVER you are trying to connect to:
chmod go-rxw .ssh

download some mp3s

the mp3 firefox quicksearch is very handly for finding sites with a ton of mp3s:

http://www.sasessiontc.com/music/

more to come under the new label 'music'

ant SCP task hangs

problem:
i am running ant 1.6.5 and using the scp task. When the task starts, it stops responding, breaking the rest of my ant build in the process. Sometimes the task transfers one file (out of a few 100) and hangs.

solution:
From doing a bit of research i found out that some versions of the library required by this task - jsch - cause ant build to hang. The trick is finding a version of this lib which works with your version of ant.

I know that jsch version 0.1.29 works with ant 1.6.5.

get jsch-0.1.29.jar here

css problems

Ive gotta write this stuff down before i forget it all:

problem:
z-index does not work. div with a higher z-index is getting overlapped by a div with lower z-index

solution:
one of the divs need to have this set -> position:absolute;



problem:
i need IE and Firefox styles on the same style sheet using a hack

solution:
#someId {
/* IE can only see this rule, IE uses this rule */
position: relative;
top: -15px;
}
html>body #someId {
/* Firefox can see this rule, rule overrides previous rule */
position: relative;
top: 25px;
}



problem:
i want to change the position of a div, lift it up by 100 pixels

solution:
there are 2 ways to do this,
margin-top:-100px;
or
position:relative;
top:-100px;

these have slightly different results, and i should try both.

UNIX: get the last modified file in a dir

problem:
i need a command that will tell me the last modified file or dir

solution:
ls /the/dir/ -1 -t | head -1

teamsite: passing parameters from javascript to perl

Teamsite (the evil abomination called software it is) allows the unfortunate souls using it to call perl scripts for javascript and pass parameters to the perl script.

the syntax looks something like this:

var params = new Object();
params.val = "some string";
var server = window.location.hostname;
IWDatacapture.callServer("http://"+server+"/iw-bin/somescript.ipl",params);

problem:
sometimes the params dont carry over correctly from javascript to the perl script, ie, all params have values in the javascript, but on the perl script side, they have no values - ""

the problem manifests if you do this:
when you assign the params in this way:
var params = new Object();
params.val = SomeFunctionWhichReturnsAString();
now chances are that params.val = "" in the perl script.

solution:
this problem stinks of pointers. params.val is set given a reference to something in the javascript heap, when perl takes over, that references points to nothing. Either way, this is how you get around it:
var params = new Object();
params.val = SomeFunctionWhichReturnsAString()+"";

now everything works. why? well... because teamsite is evil.

links i enjoy lately

bunch of links which i have been finding useful:

bunch of music
addictive game
hit counter for this blog
cool pictures/art?

lastly, google mp3 quick search link for firefox:
location:
http://www.google.com/search?q={-inurl:(htm|html|php) intitle:%22index of%22 %22last modified%22 %22parent directory%22 description size (wma|mp3) %22%s%22}
keyword:
m

Teamsite: IWDataDeploy FAILED TDbSchemaSynchronizer create failure

Interwoven Teamsite is the worst piece of software i have ever used, and googling its errors gives you 0 hits. I am going to do the world a favor and change that!

Problem: I needed to deploy DCT data to the table which teamsite uses for its own record keeping. The file which takes care of this deployment is called AREA_dd.cfg where area is the name of the DCT. After making sure that teamsite is using the correct db info, and that all fields i was deploying existed i was getting the error: IWDataDeploy FAILED, where the root cause might have been:
(from the log in OD_HOME/OpenDeployNG/log/iwddd_something.log)

DBD: SELECT * FROM USER_TABLES WHERE TABLE_NAME='IWTRACKER'
DBD: Table [iwtracker] exists.
DBD: DEFAULT__DEVICE__MAIN_STAGING not registered
DBD: Error occured in TDbSchemaSynchronizer
DBD: ERROR:TDbSchemaSynchronizer create failure.


(DEVICE is the name of my DCT)
Solution:
This solution is a hack, but it works. Open the database you are trying to commit to from teamsite, and run the following SQL query:
INSERT INTO IWTRACKER (NAME) VALUES('DEFAULT__DEVICE__MAIN_STAGING');
COMMIT;

where instead of 'DEFAULT__DEVICE__MAIN_STAGING' put the thing which the previous error claims is not registered. Save the DCT again and tail the log, the deployment should succeed. (if it fails, you are probably screwed because the support is terrible and the documentation is teh suck)

edit 11/6/2009: I no longer use teamshite, so please dont ask me any questions about this evil thing, i wont know the answers.