The open-jdk package itself does is not distributed with any shared objects/libraries (.so files) on Hardy, although open-jre does.
Normally you need not set LD_LIBRARY_PATH manually, as the java binaries should automatically seek any standard open-jre libraries at their well-known locations. If, however, you are encountering issues with library dependencies (see how to use the ldd command to identify such unmet dependencies, below), you may need to point LD_LIBRARY_PATH to either /usr/lib/jvm/java-6-openjdk/jre/lib/amd64/ (IIRC Slicehost's architecture is AMD 64bit) or /usr/lib/jvm/java-6-openjdk/jre/lib/i386/.
Also export JAVA_HOME as /usr/lib/jvm/java-6-openjdk
You may confirm that java satisfies all shared objects/libraries dependencies by running:
ldd $JAVA_HOME/bin/java
For a complete list of the files distributed as part of open-jdk and open-jre, see the open-jdk (amd64) file list and the open-jre (amd64) file list, or run:
dpkg -L openjdk-6-jdk
dpkg -L openjdk-6-jre
...on your system.