QP2 Launcher Guide

Beamline tools for diffraction data, processing, and analysis

Start Here

Image Viewer (iv) Live diffraction review, dataset browsing, heat maps, plugins, and processing launch.

Data Viewer (dv) View datasets, processing job status, and results in Datasets, Processing, and Strategy tabs.

Processing Launcher (dp) Manual processing submission for one or more master files.

Dose Planner Dose planning, RADDOSE-3D, and strategy export.

Use Image Viewer (iv); run iv at the command line, or launch iv for a particular dataset from the context menu in Data Viewer (dv).

Use Data Viewer (dv) and inspect the Processing tab, or use Image Viewer (iv), load the dataset, then select the plugin to view results from the pipeline.

Use Data Viewer (dv), then Processing -> right-click -> Re-process Selected....

Use Image Viewer (iv), then dataset tree -> right-click -> Serial Processing.

Use Beam Center, preferably through iv -> Tools -> Update Master File (to correct geometry).

Use HDF5 Combiner, typically from iv -> Serial Processing -> Hits Combiner.

Use Image Viewer (iv), then choose a processing plugin such as XDS, nXDS, or Crystfel, or use Analysis -> Index/Strategy for indexing-related workflows.

Use Image Viewer (iv), then adjust display behavior with the right-hand color bar and View -> Lock Contrast/Colormap or View -> Auto-Contrast on Zoom/Pan/Frame change.

Use Image Viewer (iv); zoom in on the image for precise inspection, and use the Viewer Controls area to see the live pixel-information readout while you inspect the image.

Use Data Processing Launcher (dp).

Use Dose Planner.

Quick Start

See screenshot and diagram ideas
Launch the main image viewer with qp2/bin/iv; use it for live diffraction review, dataset browsing, processing launches, heat maps, and 3D volume tools.
Launch qp2/bin/dv to view datasets in Datasets, and processing job status and results in Processing and Strategy.
Launch the manual processing dialog with qp2/bin/dp; add one or more master files, choose a pipeline, then click Submit Jobs to Cluster.
Launch the dose workflow with qp2/bin/dose_planner; load beamline defaults, optionally pull strategy results, then run Find Best Parameters.
Reprocess one dataset quickly in dv: Processing -> select one row -> right-click -> Re-process Selected....
Reprocess several datasets quickly in dv: Processing -> multi-select rows -> right-click -> Re-process Selected....
Process several raw datasets quickly in dv: Datasets -> multi-select rows -> right-click -> Process Selected Datasets....
Run serial workflows quickly in iv: dataset tree -> right-click -> Serial Processing -> choose nXDS, CrystFEL, or xia2.ssx.
Make a 2D heat map quickly in iv: select one raster run -> right-click -> Show 2D Grid Heatmap....
Build a 3D reconstruction quickly in iv: select two orthogonal raster runs -> right-click -> Construct 3D Volume....

Overview

QP2 ships several launcher scripts under qp2/bin/. A few of them are everyday user tools, while others are beamline utilities, batch helpers, or administrative scripts.

The main user-facing launchers are:

iv: diffraction image viewer and live analysis workspace
dv: view datasets in Datasets, and processing job status and results in Processing and Strategy
dp: manual processing job launcher
dose_planner: dose and exposure planning tool
beam_center: rewrite HDF5 geometry metadata and estimate beam center
batch_beam_center: batch beam-center correction for many master files
mock_collect / mock_streamer: replay HDF5 datasets as synthetic Redis detector streams
hdf5combiner: combine selected frames from many datasets into one HDF5 output
h5_to_cbf: HDF5-to-CBF conversion helper
xtallife: crystal lifetime calculator
strategy: scriptable strategy CLI for XDS or MOSFLM
spreadsheet_editor: puck spreadsheet editor and upload utility

Common Tasks

Reprocess a dataset

When to use this:

You already have a processing entry in dv and want to run it again with different settings.

Where to do it:

qp2/bin/dv

Steps:

Open dv.
Go to the Processing tab.
Select one result row.
Right-click and choose Re-process Selected....
Review the preloaded dataset list and job settings in the processing dialog.
Adjust the pipeline or parameters if needed.
Click Submit Jobs to Cluster.

What to expect:

QP2 attempts to recover the original master file from the processing record.
If the source file can be found, the dp-style dialog opens preloaded with that dataset.
If QP2 cannot recover or locate the original master file, reprocessing stops with a warning.

Process several datasets

When to use this:

You want to process several related runs, or you want to merge multiple compatible datasets.

Where to do it:

qp2/bin/dv
qp2/bin/iv
qp2/bin/dp

Steps from dv:

Open dv.
Go to Datasets if you are starting from raw data, or Processing if you are rerunning previous jobs.
Multi-select the rows you want.
Right-click and choose Process Selected Datasets... or Re-process Selected....
In the launcher dialog, confirm the dataset list.
Enable merge behavior if the selected pipeline supports it.
Submit the jobs.

Steps from dp:

Open dp.
Use Add Files..., Add Directories..., or Add from List....
Confirm the dataset table.
Choose the pipeline.
Enable Merge all datasets into ONE job if appropriate.
Submit.

What to expect:

Merge-friendly pipelines include xia2, autoPROC, xia2_ssx, XDS, and nXDS.
The output root is auto-derived from the first dataset unless you override it.

Run serial data

When to use this:

You are working with serial crystallography, hit finding, or multi-dataset serial processing.

Where to do it:

qp2/bin/iv

Steps:

Open iv and load the relevant datasets.
In the dataset tree, select one serial dataset or a group of related datasets.
Right-click and open Serial Processing.
Choose one of the serial workflows:
- Run xia2.ssx...
- Run CrystFEL Batch ...
- result-driven analysis actions for nXDS or CrystFEL
Review the settings dialog if one appears.
Submit the job.

What to expect:

xia2.ssx is the direct route for many serial jobs.
CrystFEL and nXDS are also exposed through the same context menu.
Some serial analysis actions only become useful after results already exist.

Image Viewer (iv)

Make a 2D heat map

When to use this:

You want a raster-style 2D view of a grid scan in the image viewer.

Where to do it:

qp2/bin/iv

Steps:

Open iv.
In the dataset tree, select a single raster run.
Right-click and choose Show 2D Grid Heatmap....
Inspect the heat map and change the active plugin or metric as needed.

What to expect:

The action appears only for runs QP2 recognizes as raster-like.
The viewer uses run naming and dataset-path hints to determine when this option is valid.

Image Viewer (iv)

Build a 3D reconstruction

When to use this:

You have two compatible raster runs collected at roughly orthogonal orientations and want a 3D volume view.

Where to do it:

qp2/bin/iv

Steps:

Open iv.
In the dataset tree, select exactly two raster runs.
Right-click and choose Construct 3D Volume....
Review the generated volume view.

What to expect:

The action appears only when QP2 detects two runs with compatible geometry, typically close to 90 degrees apart.
If the run pair does not match those conditions, the option does not appear.

Image Viewer (iv)

Simulate a live collection

When to use this:

You want to test live-mode behavior, Redis-triggered processing, or beamline workflows without a real detector stream.

Where to do it:

qp2/bin/mock_collect
qp2/bin/mock_streamer

Quick GUI steps:

Open mock_collect.
Set Redis host, port, and stream name.
Add one or more *_master.h5 files or a directory containing them.
Optionally set Collect Mode, Artificial Lag, or File Arrival Delay.
Click Start Streaming.

Quick CLI example:

qp2/bin/mock_streamer /data/test_collection --rate 50 --mode RASTER --loop

What to expect:

QP2 replays existing HDF5 data as synthetic Eiger-style Redis events.
The default Redis stream is eiger.
--reset deletes the chosen stream before starting, so use it carefully.

Reprocess from a plugin in `iv`

When to use this:

You are already in iv and want to re-run processing or analysis from the selected plugin instead of going back to the dataset-tree context menu.

Where to do it:

qp2/bin/iv

Steps:

Open iv and load the dataset.
Select the processing plugin you want in the analysis pane, such as XDS, nXDS, Crystfel, autoPROC, xia2, or xia2 SSX.
Open the plugin Actions menu.
Choose Clear and Re-run.
Wait for the plugin to clear stored result state and submit the processing again.

What to expect:

In live mode, a selected processing plugin can auto-run if QP2 does not detect a previous run for that dataset.
In review mode, Clear and Re-run is the usual plugin-driven reprocessing path.
Refresh only reloads displayed results; it does not submit a new job.

Image Viewer (iv)

How plugin auto-run works in live mode

When to use this:

You want to understand why a selected processing plugin sometimes starts work automatically in iv during live data collection.

Where to do it:

qp2/bin/iv

What to do:

Open iv in live mode.
Load or follow a dataset through Redis.
Select a processing plugin such as XDS, nXDS, Crystfel, autoPROC, xia2, or xia2 SSX.
Let the dataset finish collecting.

What to expect:

If the selected plugin does not detect a previous run for that dataset, it can auto-submit the first processing run.
This behavior is tied to live mode and dataset-completion events.
If prior results already exist, the plugin is more likely to switch into result-display or refresh behavior instead of launching a fresh run.

Image Viewer (iv)

How to tell refresh from rerun

When to use this:

You are in a plugin pane in iv and want to avoid confusing a result refresh with real reprocessing.

Where to do it:

qp2/bin/iv

Quick rule:

Refresh reloads current results.
Clear and Re-run triggers new processing or analysis.

What to expect:

Refresh reads existing files or Redis-backed results and redraws the plugin display.
Refresh does not submit a new job.
Clear and Re-run clears the stored plugin result state first, then submits analysis or processing again.
In review mode, Clear and Re-run is usually the main plugin-driven reprocessing path.

Image Viewer (iv)

Launch QP2 programs

When to use this:

You want the shortest possible reminder of which launcher to use.

Common launch commands:

qp2/bin/iv: open the image viewer
qp2/bin/dv: open the dataset and processing-results viewer
qp2/bin/dp: open the processing launcher
qp2/bin/dose_planner: open the dose planning tool

Notes:

These wrappers set up the runtime environment before calling the Python module.
qp2/bin/iv may force --nolive for users who are not in an authorized group.
qp2/bin/dose_planner also loads CCP4 before launching.

Install QP2

When to use this:

You need a local editable install for development, testing, or workstation use.

Steps:

Open a shell in qp2/.
Install the package:

pip install -e .

If you need development tools, install the optional extras:

pip install -e .[dev]

What to expect:

QP2 requires Python >=3.9.
The package depends on GUI, Redis, database, and beamline-adjacent tooling, so a plain Python install may still need local system libraries.
Some workflows also expect facility services such as Redis, MySQL/PostgreSQL access, CCP4, or cluster submission tools.

Main Applications

For the four most-used tools, start with these detailed pages:

Image Viewer (iv): live diffraction review, dataset browsing, plugins, heat maps, and plugin-driven processing
Data Viewer (dv): datasets in Datasets, and processing job status and results in Processing and Strategy
Data Processing Launcher (dp): manual processing submission for one or more master files
Dose Planner: dose planning, RADDOSE-3D, and strategy export

Launcher Catalog

User-facing applications

Launcher	Purpose	Typical use	Detailed page
`iv`	Diffraction image viewer	Live viewing, processing launch, plugins, heat maps	Yes
`dv`	Dataset and results viewer	View datasets, processing job status, and results	Yes
`dp`	Processing launcher dialog	Manual job submission	Yes
`dose_planner`	Dose planning GUI	Exposure planning and RADDOSE-3D	Yes
`beam_center`	Beam-center utility	Rewrite HDF5 geometry metadata and estimate beam center	Yes
`batch_beam_center`	Batch beam-center utility	Apply one correction across many master files	Yes
`mock_collect` / `mock_streamer`	Mock collection workflow	Replay HDF5 datasets as synthetic Redis detector streams	Yes
`hdf5combiner`	HDF5 combiner workflow	Combine selected frames into one HDF5 dataset	Yes
`h5_to_cbf`	Conversion helper	Convert master files into numbered CBF images	Yes
`xtallife`	Crystal lifetime calculator	Lightweight lifetime and wedge estimation	Yes
`strategy`	Strategy CLI	Scripted XDS or MOSFLM strategy runs	Yes
`spreadsheet_editor`	Spreadsheet metadata editor	Edit puck spreadsheets and upload to pyBluice	Yes
`chat`	Chat-style interface	Separate chat-style helper workflow	No

Processing and analysis launchers

Launcher	Purpose	Typical use
`strategy`	Strategy CLI	Run XDS or MOSFLM strategy jobs
`beam_center`	Metadata rewrite utility	Rewrite HDF5 geometry metadata or estimate beam center
`batch_beam_center`	Batch metadata rewrite utility	Apply one beam-center correction across many master files
`h5_to_cbf`	HDF5-to-CBF converter	Convert detector master files for downstream tools
`serial_xds`	Serial processing helper	Run serial or SSX-oriented XDS workflows
`index_crystfel`	CrystFEL indexing helper	Run CrystFEL-oriented indexing jobs
`raster3d`	Raster-volume helper	Build raster-derived 3D workflows
`crystfel_geometry`	Geometry helper	Generate or inspect CrystFEL geometry
`dps`	Processing shortcut	Open a secondary processing entrypoint

qp2-pipeline | Pipeline runner | Execute a pipeline directly |

Utility and support launchers

Launcher	Purpose	Typical use
`gmca_search`	Search helper	Search GMCA-related data
`backup_tool`	Backup helper	Run backup or archive tasks
`slurm_check`	Cluster-status helper	Check queue or Slurm state
`show_analysis.sh`	Shell helper	Display current analysis state quickly
`dose_server`	Dose service helper	Run the dose-related service process
`alcf`	ALCF helper	Run ALCF integration tasks
`mock_collect`	Collection-simulator GUI	Simulate live collection through the GUI
`mock_streamer`	Collection-simulator CLI	Simulate live collection from the command line

Admin and environment scripts

Launcher	Purpose	Typical use
`qp2_env.sh`	Environment wrapper	Set up the shared launcher environment
`apply_migration_v5.sh`	Migration script	Apply schema or data migration
`update_all_installs.sh`	Update helper	Refresh multiple QP2 installs

Search Keywords

Use these terms when searching the docs or site:

image viewer
diffraction viewer
live mode
review mode
Redis
heat map
2D grid heat map
3D volume
reprocess dataset
process selected datasets
serial processing
nXDS
CrystFEL
xia2
xia2.ssx
XDS
autoPROC
Dozor
strategy
dose planner
RADDOSE-3D
ESAF

Start Here

Quick Start

Overview

Common Tasks

Reprocess a dataset

Process several datasets

Run serial data

Make a 2D heat map

Build a 3D reconstruction

Simulate a live collection

Reprocess from a plugin in iv

How plugin auto-run works in live mode

How to tell refresh from rerun

Launch QP2 programs

Install QP2

Main Applications

Launcher Catalog

User-facing applications

Processing and analysis launchers

Utility and support launchers

Admin and environment scripts

Search Keywords

Related Pages

Reprocess from a plugin in `iv`