Data Processing

1 Visualising live results
2 Viewing detailed results of data processing
3 Retriggering - PX
4 Retriggering - CX
5 XDS
6 xdsme
7 CXASAP

Data analysis and processing - CX

The AutoDataset system relies on xdsme (a python wrapper) to run XDS and reduce the data. The system is triggered automatically with any dataset and the results are presented in two places.

Selection PX or CX in the CAP user changer screen changes the way the data is auto-processed.

Visualising live results

The processing of your datasets can be viewed by the processing webpage. It can be accessed by:

Open the quick access toolbar on any beamline computer, select “All processing results (dataset and indexing)”. Or you can open “Autodataset results (with retriggering and merging)” but it will only include datasets, and not crystal screening.
Opening a browser on a beamline computer and navigating to http://processing/processing


Data processing web page

The data processing web page will show every crystal screening and data collection run, with the initial processing of the frames to show the resolution, space group, unit cell, run number and directory. During the data processing the coloured status label on the right will change through the steps of XDS. When it is finished it will change to Complete. If the data process fails, it will change to Failed.

Viewing detailed results of data processing

To view more information about your dataset, clicking on the dataset in the processing page.

Under processing summary, take note of the number of Overloads. If you have any overloads at all you need to drastically increase your attenuation, by up to an order of magnitude to prevent count rate correction issues.

50% attenuation (50% transmission) → 95% attenuation (5% transmission)

The “Collection parameters”, “Retrigger parameters” and “Processing results” may be expanded or collapsed as desired.

Under “Processing results” you will find information about your data quality, such as resolution limits and completeness, and summary graphs of your data.

Dataset Processing Information	Processing results and Graphs

Retriggering - PX

If your data fails to process you can set it up to reprocess under different parameters. To do this click on the row of that dataset in the processing page, scroll to the bottom and click on the blue “Setup retriggering” button.

The best option for retriggering will depend on the reason for the dataset processing failure:

Radiation damage or crystal alignment: Cut down the number of frames used for processing, using the “Start Frame” and “End Frame” boxes in the “New” column
- Rcp vs Batch will get worse, increasing in value.
If you do not know the expected unit cell and space group, click the “Weak” and “Brute” boxes
If your diffraction looks weak, click the “Weak” box
If you want better data and are happy for processing to run slower, click "Slow". Nb. You cannot combine “Slow” and “Brute”
If you know the unit cell and space group, enter them in the “New” column and the retriggering will try to use them. You must provide both the unit cell and space group, and enter the space group in the form [a, b, c, alpha, beta, gamma]
If your data is better or worse than the Low Res and High Res cut offs, enter them in the “New” column.
If there are ice rings, click “Ice”

After you have selected the options or entered information, click on the blue “Retrigger” button.


Retriggering options

Retriggering - CX

If your data fails to process you can set it up to reprocess under different parameters. To do this click on the row of that dataset in the processing page, scroll to the bottom and click on the blue “Setup retriggering” button.

The most common failure to process is due to a failure to index the lattice. Either because there very few reflections per frame, or there are issues with the diffraction pattern (multi-crystal, plates, anisotropy, streaky reflections, etc)

The best option for retriggering will depend on the reason for the dataset processing failure:

Radiation damage or crystal alignment: Cut down the number of frames used for processing, using the “Start Frame” and “End Frame” boxes in the “New” column
- Rcp vs Batch will get worse, increasing in value.
Failure to index. As CX data often has few reflections per frame, uncheck WEAK, and cheak BRUTE. this will use more frames to collect reflections and will not include noise tin the peak table.
If you know the unit cell and space group, enter them in the “New” column and the retriggering will try to use them. You must provide both the unit cell and space group, and enter the space group in the form [a, b, c, alpha, beta, gamma].
If your data is better or worse than the Low Res and High Res cut offs, enter them in the “New” column.