iwlocal event fails - teamsite

Problem:
Today we had a horrible problem with teamsite. It appeared that no submit deployments were running since the box was rebooted for some patching.

Our submit deployments are configured in [TS_HOME]/local/iwlocal.cfg
where on 'iwatsub' event, a custom script kicks off the deployment.
This custom script logs to a file, and tailing that file showed that the script never ran, ever!

Solution:
after a bit of digging, i realized that the event subsystem was probably down. This is how you restart it, to fix this problem:

sudo /etc/init.d/iw.local stop
sudo /etc/init.d/iw.local start

after restarting iw.local the submit deployments started running again. It may be the case, that iw.local needs to run after a reboot, and does not do so automatically.

DBD: Exception Message: ORA-01407: cannot update to NULL

Here is another evil teamsite / opendeploy error that makes no sense:

DBD: INSERT INTO TABLE_NAME(row_id) VALUES (?)
DBD: Column: row_id, field: main/0/uid, Index: 1,Converting '100' to DECIMAL
DBD: DELETE FROM TABLE_NAME row_id = ?
DBD: Column: row_id, field: null, Index: 1,Converting '200' to DECIMAL
DBD:
DBD: *******************************************************
DBD: SQLException occured in TDbSchemaGroupCfg
DBD: Exception Message: ORA-01407: cannot update ("PRDLIVE"."TABLE_NAME"."ROW_ID") to NULL

At first glance it seems like you are updating a row, and setting a column to a null value that is configured as "not null"

but really you are trying to delete a row that does not exist! You can verify this:
in your db run the following queries:
<br />select * from TABLE_NAME where ROW_ID = 200;<br />-- this should return 0 rows<br />select * from IWDELTRACKER where keycolvalue = 200 and path = 'path/to/the/DCT/that/broke/the/deployment.xml';<br />-- this should return 1 row, and this is the evil suck that is breaking everything!<br />delete from IWDELTRACKER where keycolvalue = 200 and path = 'path/to/the/DCT/that/broke/the/deployment.xml';<br />-- now you got rid of the evil record<br />commit;<br />

so, now you have deleted the record of the row from your IWDELTRACKER that does not really exist in your database. re-run the deployment and it should not fail this time

Teamsite DCR java parser

Working with teamsite DCRs is painful. This is my attempt to make it a little bit less painful.

I made this program in order to make it easier to edit and parse teamsite data capture records. I should have done this a long time ago, and actually this is the first step towards freeing myself from OpenDeploy.

Anyway, here is the code for the parser:

<br />package com.dogself.teamsite.parser;<br /><br />import java.io.File;<br />import java.io.FileInputStream;<br />import java.io.FileOutputStream;<br />import java.io.IOException;<br />import java.util.ArrayList;<br />import java.util.List;<br /><br />import org.jdom.Document;<br />import org.jdom.Element;<br />import org.jdom.JDOMException;<br />import org.jdom.input.SAXBuilder;<br />import org.jdom.output.XMLOutputter;<br />import org.jdom.xpath.XPath;<br /><br />/**<br /> * Utility to open interwoven teamsite DCRs, access the values of all the items, and update item values.<br /> * <br /> * Note: replicant support is not yet implemented!<br /> * @author koryakm<br /> *<br /> */<br />public class DCRParser {<br /> private String path;<br /> private Document dom;<br /> <br /> public DCRParser(String path){<br /> this.path = path;<br /> try {<br /> parse();<br /> } catch (JDOMException e) {<br /> e.printStackTrace();<br /> } catch (IOException e) {<br /> e.printStackTrace();<br /> }<br /> }<br /> /**<br /> * return all the values of the given item as text<br /> * @param name = name of the item to get<br /> * @return<br /> * @throws JDOMException <br /> */<br /> @SuppressWarnings("unchecked")<br /> public ArrayList<string> getItemText(String name) throws JDOMException{<br /> XPath xpath = XPath.newInstance("//item[@name='"+name+"']/value"); <br /> ArrayList<string> arr = new ArrayList<string>(); <br /> List<element> nodes = xpath.selectNodes(dom);<br /> for(Element node: nodes){<br /> arr.add(node.getText());<br /> }<br /> return arr;<br /> }<br /> <br /> /**<br /> * Get all of the items named 'name' - returns a collection of DOM elements, you can change their values<br /> * in order to modify the original xml<br /> * @param name<br /> * @return<br /> * @throws JDOMException<br /> */<br /> @SuppressWarnings("unchecked")<br /> public ArrayList<element> getItem(String name) throws JDOMException{<br /> XPath xpath = XPath.newInstance("//item[@name='"+name+"']/value"); <br /> return new ArrayList<element>(xpath.selectNodes(dom));<br /> }<br /> /**<br /> * <br /> * @param xpath<br /> * @return<br /> * @throws JDOMException<br /> */<br /> @SuppressWarnings("unchecked")<br /> public ArrayList<element> getByXPath(String xpath) throws JDOMException{<br /> XPath xp = XPath.newInstance(xpath); <br /> return new ArrayList<element>(xp.selectNodes(dom));<br /> }<br /> <br /> /**<br /> * get an item using a path like this /main/item1/ (sorta like in formAPI)<br /> * @param path<br /> * @return<br /> * @throws JDOMException<br /> */<br /> @SuppressWarnings("unchecked")<br /> public ArrayList<element> getItemByPath(String path) throws JDOMException{<br /> String[] parts = path.split("/");<br /> String exp = "record";<br /> for(String p: parts){<br /> if(p.length()==0) continue;<br /> exp += "/item[@name='"+p+"']/value";<br /> }<br /> XPath xpath = XPath.newInstance(exp); <br /> return new ArrayList<element>(xpath.selectNodes(dom));<br /> }<br /> <br /> /**<br /> * proprietery: get uid of the DCR<br /> * @return<br /> * @throws JDOMException<br /> */<br /> public String getUID() throws JDOMException{<br /> return getItemByPath("/main/uid").get(0).getText();<br /> }<br /> /**<br /> * save this DCR<br /> * @throws IOException<br /> */<br /> public void saveDCR() throws IOException{<br /> saveDCR(path);<br /> }<br /> /**<br /> * save this DCR "as"<br /> * @param path<br /> * @throws IOException<br /> */<br /> public void saveDCR(String path) throws IOException{<br /> File f = new File(path);<br /> if(!f.exists()) f.createNewFile();<br /> FileOutputStream out = new FileOutputStream(f);<br /> XMLOutputter outputter = new XMLOutputter();<br /> try {<br /> outputter.output(dom, out); <br /> }<br /> catch (IOException e) {<br /> e.printStackTrace();<br /> }<br /> }<br /> private void parse() throws JDOMException, IOException{<br /> FileInputStream in = new FileInputStream(path); <br /> SAXBuilder builder = new SAXBuilder(); <br /> dom = builder.build(in); <br /> } <br />}<br />

Note: The code above does not yet truly support replicants, as in, you cant say 'give me the 3rd text field from the 2nd replicant..' unless you can figure out the xpath for it, which somewhat defeats the purpose of having a DCR parser.


Here is a driver class that uses my parser in order to do something silly.

<br />package com.dogself.teamsite.parser;<br /><br />import java.io.File;<br />import java.io.IOException;<br />import java.util.ArrayList;<br />import java.util.Arrays;<br /><br />import org.jdom.Element;<br />import org.jdom.JDOMException;<br /><br />public class Driver {<br /><br /> public Driver(String root) throws JDOMException, IOException{<br /> File[] files = getAllFilesUnderDir(root, true);<br /> for(File f: files){<br /> DCRParser parser = new DCRParser(f.getAbsolutePath());<br /> //get the title element (i know there is only 1)<br /> Element title = parser.getItem("txtTitle").get(0);<br /> String text = title.getText();<br /> //remove all html tags from the title<br /> text = text.replaceAll("<[^>]+>", "");<br /> //fix stupidness<br /> text = text.replaceAll("®", "&reg;");<br /> title.setText(text);<br /> parser.saveDCR();//save changes<br /> System.out.println("fixed title in:"+f.getAbsolutePath());<br /> }<br /> }<br /> public static void main(String[] args) {<br /> try {<br /> Driver d = new Driver("z:/test");<br /> } catch (JDOMException e) {<br /> e.printStackTrace();<br /> } catch (IOException e) {<br /> // TODO Auto-generated catch block<br /> e.printStackTrace();<br /> }<br /> }<br /> <br /> @SuppressWarnings("unchecked")<br /> public static File[] getAllFilesUnderDir(String fullpath, boolean recursive) {<br /> ArrayList queue = new ArrayList(10);<br /> ArrayList matched = new ArrayList();<br /> File root = new File(fullpath);<br /> File[] files = root.listFiles();<br /> if (files == null || files.length == 0) {<br /> return new File[] {};<br /> }<br /> queue.addAll(Arrays.asList(files));<br /> for (int j = 0; j < child =" (File)" files =" child.listFiles();"> 0)?((File[]) matched.toArray(new File[] {})):(new File[] {});<br /> }<br />}<br /><br />

in order to use this parser you will need the following libs:
- JDOM
- JAXEN

Teamsite DCT design considerations

I dont claim to be an expert on Teamsite, but i have put much blood sweat and tears into it. The following is my understanding of how teamsite tracks database changes, and what that means for anyone who designs datacapture templates and uses OpenDeploy.

The IWDELTRACKER tables uses the following info to track records: path of the xml, all its primary keys as defined in the dd files, and all its foreign keys. This works really well if you have 1 primary key, which is paired up with many foreign keys, but not so well if you have many primary keys, which are paired up with many(or one) foreign keys. This is the reason why one of my DCTs is flawed. In this DCT you make a resource, and assign it to many areas, so the primary key in that DCT comes from a replicant, and the foreign key comes from the main body. The problem we are facing with resource DCT right now is that if the user maps a resource to many areas, and then they delete some areas from the DCT, the changes are not deployed to the database.

Rules of thumb are:

1. Only create top-down relationships
2. NEVER use a replicant for a UID
3. Always test for relationship deletions.

IWDatacapture.callServer call fails / caveat

problem:

when i build teamsite DCTs i always find a new and wonderful way to break something.

Today i was building a DCT that used IWDatacapture.callServer to call a cgi script that would get some stuff from the database and then out put it into a hidden frame and do a javascript callback to the parent window.

everything was going well until i made the mistake of making 2 calls to different IWDatacapture.callServer cgi scripts one right after the other. one of them would always fail.

solution:
The problem stems from the fact that CGI scripts that output html make use of a single blank.html file to write out their contents. If you call 2 of them "almost at the same time" one of them will overwrite the other's blank.html file and cause it to fail. The only reasonable solution is to combine both CGIs into one file and let it write to one blank.html file, you can also try to make sure that CGIs dont get called at the same time, but that would be difficult to control.

So it seems that its probably a reasonable requirement to only allow ONE IWDatacapture.callServer call per DCR, otherwise things might break. How sad.

Teamsite: default OpenDeploy URL

http://TS-server:8081/iw/opendeploy/login

Opendeploy deployment fails with "REQUEST_DENIED"

Problem:
I am trying to manually kick off a teamsite deployment with opendeploy 6.1, and i get this error:

Executing(ant): /app/OpenDeployNG/bin/iwodcmd start ddconfig.xml -k iwdd=deployment1

Locating OpenDeploy service.
Got OpenDeploy service
>>>>>-- Start deployment ddconfig.
iwodstart running in default synchronous mode.
Need to wait for deployment to complete.
***ERROR - Starting deployment.
Caught exception. Details: REQUEST_DENIED

Solution:
in [OD_HOME]\bin run the following command as root:
./iwodauthorize -r add od-admin [user you tried to deploy as]
so in my case:
./iwodauthorize -r add od-admin ant

then list all the users who can run deployments, just to check if it worked:
./iwodauthorize -r list od-admin

Teamsite: IWDataDeploy FAILED TDbSchemaSynchronizer create failure

Interwoven Teamsite is the worst piece of software i have ever used, and googling its errors gives you 0 hits. I am going to do the world a favor and change that!

Problem: I needed to deploy DCT data to the table which teamsite uses for its own record keeping. The file which takes care of this deployment is called AREA_dd.cfg where area is the name of the DCT. After making sure that teamsite is using the correct db info, and that all fields i was deploying existed i was getting the error: IWDataDeploy FAILED, where the root cause might have been:
(from the log in OD_HOME/OpenDeployNG/log/iwddd_something.log)

DBD: SELECT * FROM USER_TABLES WHERE TABLE_NAME='IWTRACKER'
DBD: Table [iwtracker] exists.
DBD: DEFAULT__DEVICE__MAIN_STAGING not registered
DBD: Error occured in TDbSchemaSynchronizer
DBD: ERROR:TDbSchemaSynchronizer create failure.


(DEVICE is the name of my DCT)
Solution:
This solution is a hack, but it works. Open the database you are trying to commit to from teamsite, and run the following SQL query:
INSERT INTO IWTRACKER (NAME) VALUES('DEFAULT__DEVICE__MAIN_STAGING');
COMMIT;

where instead of 'DEFAULT__DEVICE__MAIN_STAGING' put the thing which the previous error claims is not registered. Save the DCT again and tail the log, the deployment should succeed. (if it fails, you are probably screwed because the support is terrible and the documentation is teh suck)

edit 11/6/2009: I no longer use teamshite, so please dont ask me any questions about this evil thing, i wont know the answers.