Backing up data from GM/CA @ APS with FDT
FDT is a
Java implementation of fast data transfer developed at CERN (now the development
is hosted by Github). Using this utility one can achieve substantial increase of
data transfer rates over WAN compared to traditional SCP/SFTP. When comparing FDT
vs bbcp, perhaps the later is more flexible, but since FDT is written in
Java, it works for any operating system including MS Windows. You should be aware
that FDT does not encrypt data. Only authentication is encrypted, although there are
plugins to implement encryption. If this is a concern, revert to Globus or SCP/SFTP.
- Download the latest fdt.jar
from Github to a computer at your institution. This file is a Java-executable, i.e. you also need
Java Runtime Environment (JRE) on the operating
- Copy fdt.jar to your home directory at the GM/CA @ APS computer (yes, you need the same JAR file
at both ends):
scp -p ftd.jar username@blXws5.gmca.aps.anl.gov
where X=1 for 23ID-D and X=2 for 23ID-B.
- Read the FDT
documentation explaining the program options. You will be using FDT in the SCP mode.
- Because of firewall considerations, FDT needs to be started at a computer running in your
institution and instructed to connect to blXws5.gmca.aps.anl.gov (X=1 for 23ID-D and X=2
- Use the following command to copy data:
java -jar fdt.jar -p 50000 -pull -r \
Here '-p 50000' is the TCP port to be used (this option is mandatory and the port number must
not be changed), '-pull' instructs FDT to pull data from blXws5.gmca.aps.anl.gov, and '-r'
means recursive directory scanning.