GM/CA @ APS computing environment has similar structure for all three beamlines: 23-ID-D, 23-ID-B and 23-BM. The 23-ID-D and 23-ID-B beamlines which are equipped with fast Eiger2-16M CdTe and Eiger-16M detectors respectively share two storage arrays: an ultrafast 150 TB buffer storage based on NVMe flash drives and 1.5 PB main storage consisting of 216 x 8 TB hard drives. Both storage arrays utilize BeeGFS distributed file system. The BM beamline equipped with the Pilatus3-6M detector provides a 0.5 PB shared storage with BeeGFS file system. The computing infrastructure of the ID beamlines clusters is shown on the picture below.
Users who log on the beamline workstations have their home directories on the storage array and thus all the workstations access the same home directory for a given user account. The storage capacity allows us to keep users data for three months after the experiment.
Computers at the ID beamlines are connected to internal 100 Gbps fiber network and the workstations accessible from outside the lab (ws2, ws5, ws6, and ws8) are connected to 100 Gbps uplink. The BM subnet has 40Gbps fiber network and 10 Gbps uplink.
Users are provided with two groups of workstations. One group is allocated for collecting and processing data on the day of experiment (day-1 workstations). These are blXws2, blXws6, and blXkeithley (inside end station), where "X" stands for the beamline number ('1' for IDD, '2' for IDB, and '3' for BM). The other group (day-2 workstations) consisting of ws5 and ws8 is offered for those users who wish to continue processing or backing up their data after the experiment is over. The other workstations ws9 to ws12 of IDD combined with ws9 to ws12 of IDB) are used for automatic data processing from BluIce. They do not have monitors and keyboards for user access, but can be ssh-ed to.
All computers operate under Linux Mint, a refined freeware clone of Ubuntu LTS Linux operating system, and MATE graphical desktop environment. All of them have most common crystallographic data processing software packages installed including HKL3000/HKL2000, Phenix, PyMol, and etc.
The following computing policies are implemented: