Mostly Harmless

We have a lot of data files that needs to be copied to each of the machine we want to run our code on. Further when these data files are updated, they need to be updated on all the machines. This means that the developer has to spend a lot of time just copying these data files around. I wrote a simple solution to this where latest versions of all the data files are maintained at a central server accessible via ssh. These remote date files are copied to the local machine when required in an on-demand fashion transparently by the Java program (after comparing last modification times of the local and remote file).

For accessing files over SFTP, we are using Apache Commons VFS along with Jsch. These libraries (especially commons VFS) is not well documented. I am therefore posting some code snippets from our code documenting the API

The first code snippet demonstrates the API for copying a file from remote location to the local machine:

/**
 * Copies a remote file to local filesystem.
 */
public static void copyRemoteFile(String host, String user,
    String password, String remotePath, String localPath) throws IOException {
    // we first set strict key checking off
    FileSystemOptions fsOptions = new FileSystemOptions();
    SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(
            fsOptions, "no");
    // now we create a new filesystem manager
    DefaultFileSystemManager fsManager = (DefaultFileSystemManager) VFS
            .getManager();
    // the url is of form sftp://user:pass@host/remotepath/
    String uri = "sftp://" + user + ":" + password + "@" + host
            + "/" + remotePath;
    // get file object representing the local file
    FileObject fo = fsManager.resolveFile(uri, fsOptions);
    // open input stream from the remote file
    BufferedInputStream is = new BufferedInputStream(fo.getContent()
            .getInputStream());
    // open output stream to local file
    OutputStream os = new BufferedOutputStream(new FileOutputStream(
            localPath));
    int c;
    // do copying
    while ((c = is.read()) != -1) {
        os.write(c);
    }
    os.close();
    is.close();
    // close the file object
    fo.close();
    // NOTE: if you close the file system manager, you won't be able to 
    // use VFS again in the same VM. If you wish to copy multiple files,
    // make the fsManager static, initialize it once, and close just
    // before exiting the process.
    fsManager.close();
    System.out.println("Finished copying the file");
}

Unfortunately the Commons VFS api does not provide a way to check last modification time of a remote file. I had to write that code using the Jsch API. Below is a code snippet that returns last modification time in seconds:

/**
 * Returns a Sftp session conncted using the Jsch library.
 */
public static Session connectSFTP(final String host, final String user,
        final String pass) throws JSchException {
    JSch jsch = new JSch();
    Session session = jsch.getSession(user, host, 22);
    session.setUserInfo(new UserInfo() {
        public String getPassphrase() {
            return null;
        }
        public String getPassword() {
            return null;
        }
        public boolean promptPassphrase(String string) {
            return false;
        }
        public boolean promptPassword(String string) {
            return false;
        }
        public boolean promptYesNo(String string) {
            return true;
        }
        public void showMessage(String string) {
        }
    });
    session.setPassword(pass);
    session.connect();
    return session;
}

/**
 * Returns last modification time of a remote file in seconds.
 */
public static int getLastModificationTime(String host, String user,
        String password, String remotePath) throws IOException,
        JSchException, SftpException {
    Session session = connectSFTP(host, user, password);
    ChannelSftp chan = (ChannelSftp) session.openChannel("sftp");
    chan.connect();
    SftpATTRS attrs = chan.lstat(remotePath);
    int time = attrs.getMTime();
    chan.disconnect();
    session.disconnect();
    return time;
}

I hope that this code is useful to others. Please leave a comment if you see any error of have a suggestion.

Labels: java, sftp

# posted by nileshbansal @ 11:48 pm - -   -bookmark at -del.icio.us   -