Remote access: Speed tests

Doing remote data collection and transferring images to your home institution obviously requires fast network connection. But how fast is the 'fast' and is my connection speed sufficient? You can use Speedtests by OOKLA to benchmark your connection. Point your web browser to:

and choose a server close to Argonne, for example "Chicago, IL - Frontier", or "Chicago, IL - Comcast", or "Chicago, IL - Speedtest.net". You should see the download upload speeds no less than 600Mbps and 400Mbps respectively. These are the numbers we observe for connections from the University of Michigan and which are just about appropriate. To observe these speeds you should have a computer with at least a gigabit wired connection.

Wi-Fi should not be used under any circumstances, even if you have 802.11ac claiming 1.7 Gbps or 3.4 Gbps speeds. The matter is that Wi-Fi connections are prone to packet loss with subsequent re-sending. Wi-Fi may work fast when transferring data between two local computers, but the speeds of long-distance connections deteriorate drastically even when only a short arm of the connection (e.g. within a room) is over Wi-Fi. This is because corrupted packets have to travel all the long-distance way to the endpoint which then discovers corruption and requests re-sending.

If your numbers are significantly lower, that may become a show stopper for remote access. It is important to understand where the problem may occur. Such task is not trivial because in most cases there is no a single bottleneck, but the delays add up non-linearly. In other words, you may have a descent speed between A and B and between B and C, but slow between A and C in spite of the fact that the connection between A and C may be routed via B. It is a good idea to work with your institution's network admins and bring them into contact with the GMCA staff.

Wget tests

The test below is to verify downloading speeds of a large 88MB file from GM/CA. Compare your results with those for the University of Michigan (you can also use a web browser instead of wget):

wget https://www.gmca.aps.anl.gov/libreoffice-core-5.0.6.2-3.el7.x86_64.rpm
........................................................
2019-06-11 16:09:41 (64.5 MB/s) - libreoffice-core-5.0.6.2-3.el7.x86_64.rpm saved

SCP tests

Finally, if you already have an SSH access to the GM/CA computers, you may test the real data transfer speeds. Make a directory, e.g. TEST with twenty or so data frames produced by the detector and copy them to your institution. Below are typical speeds for the University of Michigan obtained with 32MB Rayonix-300 CCD frames:

$ scp -rp [email protected]/TEST_IMG .
series_1_master.h5 100%	297MB  75.1MB/s	00:03
series_2_master.h5 100%	297MB 101.2MB/s 	00:02
series_3_master.h5 100%	297MB 129.7MB/s 	00:02
series_4_master.h5 100%	297MB 125.0MB/s 	00:02
series_5_master.h5 100%	297MB 122.7MB/s 	00:02
c3_1_01_master.h5  100%	 89MB  96.7MB/s	00:00
c3_1_01_data_1.h5  100%	 31MB  72.3MB/s	00:00
c3_0_01_00003.cbf  100%	 17MB  61.6MB/s	00:00
series_00001_4.cbf 100%	 17MB  56.8MB/s	00:00
series_00001_5.cbf 100%	 17MB  45.9MB/s	00:00

To estimate required speeds, you should consider that when data collection is running permanently, the Rayonix-300 detector frames are typically acquired at the rate of 22fpm (frames per minute), which corresponds to 32MB x 22/60 = 11.73MB/s. With Pilatus3-6m typical rates are much higher: 50fps (frames per second), which corresponds to 6MB x 50 = 300MB/s. With Eiger-16M they may be even higher: up to 130fps, which corresponds to 18MB x 130 = 2.4GB/s. However, given typical overhead of crystal mounting, centering and screening, average daily data rates are ~200GB/day with Rayonix-300, ~1TB/day with Pilatus3-6m and ~3TB/day with Eiger-16m. This amounts to required average speeds of 2MB/s, 10MB/s and 30MB/s with Rayonix, Pilatus and Eiger respectively. With some experiments the amounts of data may be up to several times higher than average. We strongly recommend using Globus for transferring Eiger and Pilatus data.

Improving the speeds

It may be possible to improve network speeds by tuning the TCP parameters on your computer, although the administrative access to your computer and some computer administrating skills may be required. Please read the fasterdata.es.net guide.