User Program - While at the Beamline - Data Management and Backup
GM/CA @ APS recommends and supports external USB
disks as inexpensive and fast way to backup users data. We strongly recommend
using USB3 drives due to much higher speeds compared to USB2.
The table below provides a speed comparison of different connector types for
typical 5400rpm HDD external drives (the advantages of USB3 for SSD drives are
even higher as the speeds may reach 200-300 MB/s):
| Connector || Bus Speed || Real Speed || 200GB backup time || Restricted by |
| USB3 || 500 MB/s || 40-50 MB/s || 1 hour || disk spindle |
| USB2 || 40 MB/s || 20 MB/s || 3 hours || bus |
Not all computers may have USB-3 ports. Please ask your host if in trouble finding
the right hub.
Some users have problems distinguishing between the USB-3 and USB-2
ports. Please look at the labeling and the color. The USB-3 ports typically have
blue connectors and cables and/or the SS (Super-Speed) marking.
Support for DVD disks is not provided because at the rate of 200GB of data
per experiment one would have to burn more than 45 disks.
Supported disk formats
The supported formats on external disks are FAT32,
NTFS, exFAT, ext2/ext3/ext4,
xfs, and HFS+.
We have only Linux computers at the beamlines. Therefore some Windows-specific
and MacOSX-specific extensions, for example encrypted drives like WD Passport
that require some Windows or MacOSX utility to unlock the encrypted partition,
may not work. When choosing format for your hard drive, keep in mind that FAT32
has the best record in terms of portability between different operating systems
(Windows, Linux and MacOSX), while HFS+ has the shortest history of support
under Linux. Please, try to avoid bringing disks with multiple partitions and
disks with already stored data as in some problematic cases your disk may need
to be re-partitioned or reformatted.
The HFS+ format is supported on the second-day
workstations ws4, ws5, and ws7 only. You will have to
temporarily disable HFS+ journaling in order to allow writing to HFS+
drives. Please consult this HFS+ manual for the
USB disks can be connected to any beamline workstation
except for ws2 and ws3 as they may be controling the expertiment . In the second-day
area USB-3 disks can be connected to most workstations with the SS port label
(see the marking examples above). Usually disks are mounted automatically upon connecting
provided user has logged in before connecting the disk. If the disk is not auto-mounted,
go to Applications -> System Tools > Disk Utility, then find your disk in the
left-pane list, choose the right partition and click the "Mount Volume" button:
Additional details of mounting external disks under Linux operating system
are described elsewhere.
GM/CA @ APS runs 10Gb network and users are welcome to SFTP or FTP out
their data to the servers at their home institutions. External access the
GM/CA SFTP and Globus GridFTP servers is
also possible, but needs to be requested in advance. The access can be allocated
for the time of experiment plus one day. That policy is enforced in order to
avoid conflict of resources between different user groups. The real rate of
remote data transfer will depend on the network connectivity to the users
institution. Please check the Network Speed
Testing page. Unless there is a bottleneck on user's side, one can expect
the SFTP rates of 5-8MB/s and GridFTP
rates of 20-25MB/s. For a 200GB data set it may take about 10 and 3 hours
respectively. For further up to 5x improvements of Globus speeds, we offer
Globus over ScienceDMZ. In order to fully utilize the
advantages of ScienceDMZ, a respective node may need to be setup at your
Institution. Please request information from your host.
Please be adviced to never using Wi-Fi connnections
for network data transfers as it drastically reduces speeds, especially at long
For the latest changes with the backup procedures and detailed technical
instructions please refer the Backup Manual