xdsme at home

To process MX1 Eiger data at your home institute, you will need to follow these instructions.

You MUST use the xdsme from https://github.com/JunAishima/xdsme to process your data from this detector - the metadata necessary for it to be recognized by Pierre Legrand's xdsme is not present in the current (as of 2019/06/24) Eiger2 master file
xdsgui apparently has the same problem - no guarantees, but the following script will add the description field to your master file, as well as omega_start that is also not in the files currently produced by the detector:

1 Linux instructions
- 1.1 If using Neggia - potentially 2x as fast as using H5Toxds (see below)
2 Mac instructions
3 Mac instructions with SBgrid
4 Generally useful pages
5 Old H5ToXDS directions
- 5.2 Alternative method to convert HDF5 files to CBF

import h5py

import numpy

mod_file = h5py.File("original_master.h5")

mod_file.create_dataset("/entry/instrument/detector/description", data = "Dectris Eiger2 9M")

omega = mod_file.get('/entry/sample/goniometer/omega')

mod_file.create_dataset('/entry/sample/goniometer/omega_start', data=omega[0])

mod_file.close

Your Python will need to have h5py added (pip install h5py) and you will also need the HDF5 library installed (for CentOS, “yum install hdf5”)

Linux instructions

Requirements

python 2.7 - type "python" at the commandline and it will tell you what version
- If you don't have python installed, particularly on MacOS, our recommendation is to use download and install Anaconda python (https://www.anaconda.com/download/ )
- xdsme requires pyfive to read metadata from the HDF5 master file - install it into your python environment with pip install git+https://github.com/jjhelmus/pyfive.git
xdsme - download from https://github.com/JunAishima/xdsme . (git clone or download directly)
- Needs to be unzipped to a folder that is in the path or the folder where it goes needs to be added to the default PATH
xds - http://xds.mpimf-heidelberg.mpg.de/html_doc/downloading.html for academic users. Note that for commercial users, a license is required ($$)
- Also needs to be on a folder that is in the default PATH
Neggia library binaries - https://www.dectris.com/neggia.html - Requires free account - download the one that is appropriate for your operating system
Neggia library from source - https://github.com/dectris/neggia

Set up your environment correctly

export PATH=/path/to/xdsme/bin/Linux_i586:/path/to/xds/bin:$PATH

If using Neggia - potentially 2x as fast as using H5Toxds (see below)

The location of the Neggia library does not need to be provided if you set an environment variable. (Optionally, it can be provided when running xdsme.) Edit to the location you have the library. Also,

export XDS_LIB_HDF5DEC=/usr/local/lib/dectris-neggia.so

xdsme --eiger my_master_1_master.h5

OR

Mac instructions

Thanks to Stephanie Gras (Monash) for helping us with this procedure!

get xds from the same location as shown above
get MacOS Neggia library to read HDF5 files into XDS directly - register, then login to the Dectris website (http://www.dectris.com ) first - https://www.dectris.com/neggia.html?file=tl_files/tl_files_restricted/download/software/dectris-neggia-macos.so.gz
gunzip the neggia file to get a dectris-neggia-macos.so file
get the XDS.INP files from your visit. add the line "LIB=/neggia/location/dectris-neggia.macos.so" somewhere in the XDS.INP file (put in your actual file location in place of /neggia/location). you should now be able to run xds_par on the directories

To reprocess from scratch, follow the instructions for the Linux section above to install and run xdsme.

Mac instructions with SBgrid

Thanks to Bryce for letting me help him set things up!

There are a couple of weird things - first is that xdsme is an old version, the second is that while pyfive is available in the sbgrid python environment, it is not compatible with more recent xdsme due to bugs that are only fixed in the master

xdsme:

Neggia is located at: /programs/i386-mac/xds/<date>/lib/dectris-neggia.so

pyfive: pip install git+https://github.com/jjhelmus/pyfive

Then you can run xdsme!

Generally useful pages

http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Eiger
http://dectris.com/
The original xdsme - https://github.com/legrandp/xdsme
XDS - http://xds.mpimf-heidelberg.mpg.de/

Old H5ToXDS directions

The H5ToXds directions are deprecated as neggia seems to be faster, good on computing resources, and is stable.

Where to get it (Linux only)

available at the bottom of:

https://www.dectris.com/EIGER_X_Features.html ---> Neggia is preferred if you can use it (see line below)

How to add H5ToXds to the PATH

export PATH=/path/to/H5ToXds:$PATH

How to run xdsme by giving it the location of your master file

xdsme --eiger my_master_1_master.h5
xdsme --help to see all of the options available

If you have enough memory and have /dev/shm setup, try the H5ToXds script at http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Eiger - be sure to rename your H5ToXds binary and link it in the script

Alternative method to convert HDF5 files to CBF

get and install eiger2cbf - http://www.mrc-lmb.cam.ac.uk/harry/imosflm/ver721/downloads/
get the XDS.INP files from your visit. you should now be able to run xds_par on the directories