check if a website is responding script

recently i needed to check if a website was responding, and if not, page myself (by sending an email to my pager). Here is the unix shell script i used to make this happen.

Required programs:
wget
working mail (i use xmail on solaris 8 in this script)

script:

#!/bin/sh
## Script will check if the "host" is up, if the host is down, send an email
## You should cron it every 5 mins or so.

#uncomment to debug:
#set -x

## change these:
host="http://dogself.com"
email="my-pager-number@skytel.com,my-real-email@gmail.com"

## locations of stuff:
mailx="/usr/bin/mailx"
wget="/usr/bin/wget"
log="/path/to/a/writable/log/file.log"

now=`date '+%m/%d/%Y %H:%M:%S'`
rm ${log}
#when checking connection, do 2 tries, and time out in 7 seconds
${wget} -O /dev/null -t 2 -T 7 ${host} -o ${log}
grep "saved \[" ${log}
if [ $? = "1" ];
then
 echo "site:[${host}] is down"
 ${mailx} -s "PRODUCTION is DOWN at ${now}" ${email} < ${log}
else
 echo "site:[${host}] is up"
fi

appfuse 1.8.1 ehcache NoSuchMethodError

Problem:
I installed appfuse-light-all 1.8.1, installed acegi security, and ajax. After doing a build i get the following error when Spring starts up:

Error creating bean with name 'userManager' defined in ServletContext resource [/WEB-INF/applicationContext.xml]
.... bunch of other errors ...
Invocation of init method failed; nested exception is java.lang.NoSuchMethodError: net.sf.ehcache.Cache.<init>

Solution:
The problem here is that i am using an older version of ehcache for some reason. Doing a bit more digging shows that i have the following versions of ehcache in my M2 repository:
[M2_REPO]/net/sf/ehcache/ehcache/1.2.3/
[M2_REPO]/net/sf/ehcache/ehcache/1.3.0/

version 1.2.3 is what is messing things up for me, it gets loaded by Spring first and causes the error. Deleting that directory does not work because "ant deploy" does the following if the dependency is missing:

Buildfile: build.xml
[artifact:dependencies] Downloading: net/sf/ehcache/ehcache/1.2.3/ehcache-1.2.3.jar
[artifact:dependencies] Downloading: net/sf/ehcache/ehcache/1.2.3/ehcache-1.2.3.jar

Now, since i dont know how to use MAVEN, i cant remove the code that fetches the old version, so i do the evil hack to prevent it from being created:

remove the directory "1.2.3", make a copy of "1.3.0" directory, call it "1.2.3". rename all the files inside to say 1.3.0 instead of 1.2.3.

Also, if you are deploying to tomcat as a directory, you might need to go into the webapps directory in tomcat, locate your app, and its WEB-INF and delete ehcache-1.2.3.jar thats is probably there now.

Now you can use appfuse.

if anyone knows of a better way to solve this, leave a comment.

firefox: tab and enter do not work

I have been effected by the evil firefox bug which causes tab and enter to stop working. As of this writing there is no fix for this.

Here is what i have found out about living with this bug and other info:

Workarounds:
1. shift-enter works instead of enter. No replacement for tab exists.
2. opening a new windows or restarting firefox fixes this problem for a while
- people suggested using session fix addon to remember your tabs while you restart firefox

other info:
1. people claim its a firebug issue or a web developer issue. i have both of those installed, by the way
2. there is a firefox bug report on this here
3. some posts on message boards which might eventually find a fix for this problem are:
http://www.nabble.com/Enter-and-Tab-keys-stop-working-td14302706i20.html
http://support.mozilla.com/tiki-view_forum_thread.php?comments_parentId=38626&forumId=1


at this point the best solution seems to use that session fix plugin, or maybe do a reinstall.

shell scripting with /bin/sh

I found this half decent tutorial / how-to on shell scripting with /bin/sh which i do lots but suck at.

here it is:
http://ooblick.com/text/sh/

also, to check if a file DOES NOT exist:

if [ ! -e /path/to/file ];
then
# do something
fi

ok i am done!

common regex cookbook

I see nice regexes on the java forums from time to time, i am going to add them all here for use!

<br />//1. replace leading white space in a line:<br />str = str.replaceAll("(?m)(?<=^ *) ", ".");<br /><br />

Music for April

finally found some nice open directory:

http://www.artofassassin.com/Music/

Escape cp-1252 chars in HTML

This is for when people paste crap from MS Word into html and add all those funky characters that looks horrible.

My friend gave me this perl one liner (ok, you can use it as a one liner if you wanted to) to escape the the evil CP-1252 characters in your HTML.
<br />sub htmlEscape1252 {<br /> my $str = shift;<br /><br /> # remove control characters and nbsp<br /> $str =~ s{([\x00-\x08\x11\x12\x14-\x19\x7f\xA0])}{ }g;<br /><br /> # escape the rest with &#123; syntax<br /> $str =~ s{([\x80-\x9f\xA1-\xff])}{'&#'.ord($1).';'}ge;<br /><br /> return $str;<br />}<br />
Notes for porting:
ord($1) converts capture group 1 to ASCII.
assumes $str has your html

Bonus!
the above regex replaces control chars!

Teamsite DCR java parser

Working with teamsite DCRs is painful. This is my attempt to make it a little bit less painful.

I made this program in order to make it easier to edit and parse teamsite data capture records. I should have done this a long time ago, and actually this is the first step towards freeing myself from OpenDeploy.

Anyway, here is the code for the parser:

<br />package com.dogself.teamsite.parser;<br /><br />import java.io.File;<br />import java.io.FileInputStream;<br />import java.io.FileOutputStream;<br />import java.io.IOException;<br />import java.util.ArrayList;<br />import java.util.List;<br /><br />import org.jdom.Document;<br />import org.jdom.Element;<br />import org.jdom.JDOMException;<br />import org.jdom.input.SAXBuilder;<br />import org.jdom.output.XMLOutputter;<br />import org.jdom.xpath.XPath;<br /><br />/**<br /> * Utility to open interwoven teamsite DCRs, access the values of all the items, and update item values.<br /> * <br /> * Note: replicant support is not yet implemented!<br /> * @author koryakm<br /> *<br /> */<br />public class DCRParser {<br /> private String path;<br /> private Document dom;<br /> <br /> public DCRParser(String path){<br /> this.path = path;<br /> try {<br /> parse();<br /> } catch (JDOMException e) {<br /> e.printStackTrace();<br /> } catch (IOException e) {<br /> e.printStackTrace();<br /> }<br /> }<br /> /**<br /> * return all the values of the given item as text<br /> * @param name = name of the item to get<br /> * @return<br /> * @throws JDOMException <br /> */<br /> @SuppressWarnings("unchecked")<br /> public ArrayList<string> getItemText(String name) throws JDOMException{<br /> XPath xpath = XPath.newInstance("//item[@name='"+name+"']/value"); <br /> ArrayList<string> arr = new ArrayList<string>(); <br /> List<element> nodes = xpath.selectNodes(dom);<br /> for(Element node: nodes){<br /> arr.add(node.getText());<br /> }<br /> return arr;<br /> }<br /> <br /> /**<br /> * Get all of the items named 'name' - returns a collection of DOM elements, you can change their values<br /> * in order to modify the original xml<br /> * @param name<br /> * @return<br /> * @throws JDOMException<br /> */<br /> @SuppressWarnings("unchecked")<br /> public ArrayList<element> getItem(String name) throws JDOMException{<br /> XPath xpath = XPath.newInstance("//item[@name='"+name+"']/value"); <br /> return new ArrayList<element>(xpath.selectNodes(dom));<br /> }<br /> /**<br /> * <br /> * @param xpath<br /> * @return<br /> * @throws JDOMException<br /> */<br /> @SuppressWarnings("unchecked")<br /> public ArrayList<element> getByXPath(String xpath) throws JDOMException{<br /> XPath xp = XPath.newInstance(xpath); <br /> return new ArrayList<element>(xp.selectNodes(dom));<br /> }<br /> <br /> /**<br /> * get an item using a path like this /main/item1/ (sorta like in formAPI)<br /> * @param path<br /> * @return<br /> * @throws JDOMException<br /> */<br /> @SuppressWarnings("unchecked")<br /> public ArrayList<element> getItemByPath(String path) throws JDOMException{<br /> String[] parts = path.split("/");<br /> String exp = "record";<br /> for(String p: parts){<br /> if(p.length()==0) continue;<br /> exp += "/item[@name='"+p+"']/value";<br /> }<br /> XPath xpath = XPath.newInstance(exp); <br /> return new ArrayList<element>(xpath.selectNodes(dom));<br /> }<br /> <br /> /**<br /> * proprietery: get uid of the DCR<br /> * @return<br /> * @throws JDOMException<br /> */<br /> public String getUID() throws JDOMException{<br /> return getItemByPath("/main/uid").get(0).getText();<br /> }<br /> /**<br /> * save this DCR<br /> * @throws IOException<br /> */<br /> public void saveDCR() throws IOException{<br /> saveDCR(path);<br /> }<br /> /**<br /> * save this DCR "as"<br /> * @param path<br /> * @throws IOException<br /> */<br /> public void saveDCR(String path) throws IOException{<br /> File f = new File(path);<br /> if(!f.exists()) f.createNewFile();<br /> FileOutputStream out = new FileOutputStream(f);<br /> XMLOutputter outputter = new XMLOutputter();<br /> try {<br /> outputter.output(dom, out); <br /> }<br /> catch (IOException e) {<br /> e.printStackTrace();<br /> }<br /> }<br /> private void parse() throws JDOMException, IOException{<br /> FileInputStream in = new FileInputStream(path); <br /> SAXBuilder builder = new SAXBuilder(); <br /> dom = builder.build(in); <br /> } <br />}<br />

Note: The code above does not yet truly support replicants, as in, you cant say 'give me the 3rd text field from the 2nd replicant..' unless you can figure out the xpath for it, which somewhat defeats the purpose of having a DCR parser.


Here is a driver class that uses my parser in order to do something silly.

<br />package com.dogself.teamsite.parser;<br /><br />import java.io.File;<br />import java.io.IOException;<br />import java.util.ArrayList;<br />import java.util.Arrays;<br /><br />import org.jdom.Element;<br />import org.jdom.JDOMException;<br /><br />public class Driver {<br /><br /> public Driver(String root) throws JDOMException, IOException{<br /> File[] files = getAllFilesUnderDir(root, true);<br /> for(File f: files){<br /> DCRParser parser = new DCRParser(f.getAbsolutePath());<br /> //get the title element (i know there is only 1)<br /> Element title = parser.getItem("txtTitle").get(0);<br /> String text = title.getText();<br /> //remove all html tags from the title<br /> text = text.replaceAll("<[^>]+>", "");<br /> //fix stupidness<br /> text = text.replaceAll("®", "&reg;");<br /> title.setText(text);<br /> parser.saveDCR();//save changes<br /> System.out.println("fixed title in:"+f.getAbsolutePath());<br /> }<br /> }<br /> public static void main(String[] args) {<br /> try {<br /> Driver d = new Driver("z:/test");<br /> } catch (JDOMException e) {<br /> e.printStackTrace();<br /> } catch (IOException e) {<br /> // TODO Auto-generated catch block<br /> e.printStackTrace();<br /> }<br /> }<br /> <br /> @SuppressWarnings("unchecked")<br /> public static File[] getAllFilesUnderDir(String fullpath, boolean recursive) {<br /> ArrayList queue = new ArrayList(10);<br /> ArrayList matched = new ArrayList();<br /> File root = new File(fullpath);<br /> File[] files = root.listFiles();<br /> if (files == null || files.length == 0) {<br /> return new File[] {};<br /> }<br /> queue.addAll(Arrays.asList(files));<br /> for (int j = 0; j < child =" (File)" files =" child.listFiles();"> 0)?((File[]) matched.toArray(new File[] {})):(new File[] {});<br /> }<br />}<br /><br />

in order to use this parser you will need the following libs:
- JDOM
- JAXEN

email regex that doesnt suck

why is it so hard to find a good email java regex on google? They all suck.

Here are a few regexes that i would use in a production system to check if an email is valid:

"([\\w-]+(?:\\.[\\w-]+)*@(?:[\\w-]+\\.)+\\w{2,7})\\b"

Something a little more light weight:

"[^\\s]+@[^\\s]+([.]\\w{2,3})+"

and if all else fails, this might be useful:
Apache Commons EmailValidator class API

Path does not start with a "/" character

Problem:
I get this error when using the tiles plugin when building out some tiles:

java.lang.IllegalArgumentException: Path .some.tiles.path does not start with a "/" character

first, lets pretend that the tile in question looks like this:
<definition name=".some.tiles.path" extends=".some.other.tiles.path"> ..some stuff.. </definition>

Solution:
ive noticed that the reason for this error can be one of the following:

1. if you look at the tile that this tile extends, and what that tile extends, until you find the tile that extends nothing, that tile should have a definition like this:
<definition name=".some.base.tile" path="/a/real/path/to/some/jsp/somejsp.jsp"> ..some stuff.. </definition>
the key thing, is that the path of the parent tile starts with a "/" character

2. if #1 is not your problem, then you might have a duplicate definition of the tile name which the error talks about.

3. Another thing that can happen is that your tiles path that you are referencing in struts-config.xml does not exist at all, because maybe you misspelled it, in which case that same error is thrown.

B. Fleischmann - Gain Lyrics

I have looked everywhere for the lyrics of the song "Gain" by B. Fleischmann which is found here on you tube by the way. They dont seem to exist on teh intarwebs.

Anyway, i have transcribed them as best as i can, if anyone thinks i got something wrong, please leave a comment.


B. Fleischmann - Gain Lyrics

lets face another day
lets see another fence
lets have another crash
lets get another chance

where all reason ends, i put my heart
where foresight ends, i want to start
as long as i haven't tried to gain
i will not stop, will not complain

lets give it one more try
don't want to make you cry
how else can we survive
unless we dry our eyes try our lives

where all reason ends, i put my heart
where foresight ends, i want to start
as long as i haven't tried to gain
i will not stop, will not complain

JSP syntax: empty checks, params, misc

This stuff is so elementary that i am always annoyed to forget it every time i dont use jsp for a few months.

1. Get params from the url request, ie http://dogself.com?param1=test&something=poop
<c:out value="param1 is: ${param.param1} something is:${param.something}"/>

also there is ${params} for getting lists, but i would need to read up on that

2. checking if a request attribute is null or empty:
<c:if test="${param.page != null && not empty param.page}">
<c:import url="${base_html_dir}${param.page}"></c:import>
</c:if>


3. setting variables:
<c:set var="thingToSet" value="${1+ 3}">
this by default has the scope of 'page', also can be 'session' defined by attribute 'scope'

ill add more as i go

Document base /path/to/ROOT does not exist or is not a readable directory

I recently installed tomcat 5.5 on my slicehost, and it was somewhat painful, probably because i deviated from the basic setup so much.
Anyway, one last error (in /var/log/tomcat5.5/catalina.out) that i was getting was this:

SEVERE: Error starting static Resources
java.lang.IllegalArgumentException: Document base /home/mk/public_html/ROOT does not exist or is not a readable directory

I was setting up my app context path using CATALINA_HOME/conf/Catalina/localhost/ROOT.xml and so it was strange that tomcat was looking for a directory called ROOT.

Solution:

I was overlooking this warning before the error:
WARNING: A docBase /home/mk/public_html/dogself.com inside the host appBase has been specified, and will be ignored

the problem was that i set my appBase inside of /etc/tomcat5.5/conf/server.xml to be /home/mk/public_html/ and this was a parent directory of app's docBase.

Changing the appBase to /home/mk/public_html/webapps solved this problem.

Music for february

this place has lots of rock from back when i used to listen to rock:
http://saintsmythe.illemonati.com/tsx/

Recently i've been interested in a progressive metal band called "Sonata Arctica", this place as a decent selection of them:

http://www.rockandroll-server.com.ar/GOD/mp3/Sonata/

a ton of DnB:
http://www.riotnerd.com/nmp3s/

thats all until i find more betterer stuff

find duplicate files on your machine

ive had need for this before. A program to find duplicate files on your computer. i have been told by a very reliable source that this is good:

http://www.beautylabs.net/software/dupseek.html

so i am going to write it down so that next time i need this, i will have it

dupes are evil

Java: reading / writing international characters

It is actually very easy to support international characters in java. The only trick is knowing the charset that the text is encoded in. Most of the time is a safe bet to try UTF-8 if you dont know. If you use Apache's HTTPClient it has methods to get the charset of the stream you are reading from.


<br />//to open the file for reading:<br />BufferedReader read = new BufferedReader(new InputStreamReader(new FileInputStream(file), Charset.forName("UTF8")));<br />//to open file for writing:<br />BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file), Charset.forName("UTF8")));<br />

Music for January

this place has lots of kraftwerk and other interesting stuff:

http://elf.art.pl/

this place has a BUNCH of music, lots of normal music with weird Japanese bands i've never heard of:

http://side7.homeunix.net/s/mp3/

Unix: Compress or delete files older then X days

Problem:
I have some logs in some directories that i need to back up if the file is at least 12 days old, if the file is older then 24 days, i need to delete it

Solution:

to compress:
find <path within you're looking> -xdev -mtime +<number of days> -exec /usr/bin/compress {} \;

to delete:
find <path within you're looking> -xdev -mtime +<number of days> -exec /usr/bin/rm -f {} \;

Note: This was tested on Solaris 8

Examples:

find /usr/local/apache/logs/ -xdev -name "sar.*" -mtime +24 -exec /usr/bin/rm -f {} \;
find /usr/local/apache/logs/ -xdev -name "sar.*" -mtime +12 -exec /usr/bin/compress {} \;

Cooking: How to make sushi rice

Making perfect sushi rice is hard, it has to be cooked though, soft and sticky. Ive always had trouble doing it because i would end up burning the rice on the bottom of the pot (i dont have a rice cooker). My rice wouldnt be sticky enough to stick to the seaweed. My friend Sam always emphasized to me how important it was to make the rice "just right" when making sushi.

Enough, babble! i found a blog entry that makes cooking sushi rice easy! it actually came out great the first time i tried the oven method.

http://beyondsalmon.blogspot.com/2006/02/technique-of-week-how-to-make-sushi.html

how to: use cron

ok, now that i really know how to use the cron i should blog it incase i forget

this assumes you have nothing in your cron / or your crontab is in sync with hostname.cron because you always use this method to modify the crontab.

0. run crontab -l to make sure your crontab is empty, if its not, paste what it has to the bottom of the file you make in step 1

1. make a file called hostname. cron and add this into the top of it for later:
#########################################
# #
# Cron file for [username] #
# #
#########################################
# min hour dom month dow usr cmd
# use '*' as skipped element

2. add your stuff to it, here is a better explanation on what to add:
01 * * * * echo "This command is run at one min past every hour"
17 8 * * * echo "This command is run daily at 8:17 am"
17 20 * * * echo "This command is run daily at 8:17 pm"
00 4 * * 0 echo "This command is run at 4 am every Sunday"
* 4 * * Sun echo "So is this"
42 4 1 * * echo "This command is run 4:42 am every 1st of the month"
01 * 19 07 * echo "This command is run hourly on the 19th of July"

3. save your file.

4. run this:
crontab < hostname.cron

done!

useful info happily stolen from here: http://www.unixgeeks.org/security/newbie/unix/cron-1.html