Argonne National Laboratory


Remote Data Backups with FDT

Department of Energy Office of Science
GM/CA @ APS Sponsors:
National Institute of General Medical Sciences (NIGMS) and National Cancer Institute (NCI) of the National Institutes of Health (NIH)



Backing up data from GM/CA @ APS with FDT

FDT is a Java implementation of fast data transfer developed at CERN (now the development is hosted by Github). Using this utility one can achieve substantial increase of data transfer rates over WAN compared to traditional SCP/SFTP. When comparing FDT vs bbcp, perhaps the later is more flexible, but since FDT is written in Java, it works for any operating system including MS Windows. You should be aware that FDT does not encrypt data. Only authentication is encrypted, although there are external plugins to implement encryption. If this is a concern, revert to Globus or SCP/SFTP.

Step-by-step instructions

  1. Download the latest fdt.jar from Github to a computer at your institution. This file is a Java-executable, i.e. you also need Java Runtime Environment (JRE) on the operating system.
  2. Copy fdt.jar to your home directory at the GM/CA @ APS computer (yes, you need the same JAR file at both ends):
    scp -p ftd.jar
    where X=1 for 23ID-D and X=2 for 23ID-B.
  3. Read the FDT documentation explaining the program options. You will be using FDT in the SCP mode.
  4. Because of firewall considerations, FDT needs to be started at a computer running in your institution and instructed to connect to (X=1 for 23ID-D and X=2 for 23ID-B).
  5. Use the following command to copy data:
    java -jar fdt.jar -p 50000 -pull -r \
    Here '-p 50000' is the TCP port to be used (this option is mandatory and the port number must not be changed), '-pull' instructs FDT to pull data from, and '-r' means recursive directory scanning.

GM/CA @ APS is an Office of Science User Facility operated for the U.S. Department of Energy by Argonne National Laboratory

UChicago Argonne LLC | Privacy & Security Notice | Contact Us | A-Z Index | Search