xpdan package

Submodules

xpdan.999-load module

xpdan.callbacks module

class xpdan.callbacks.SaveBaseClass(template, handler_reg, root_map=None, base_folders=None, **kwargs)[source]

Bases: xpdan.vend.callbacks.core.Retrieve

Base class for saving files with human friendly file names.

For each document this class applys a format to the string based off the document which has been seen. When the time comes for the file to be written any templating which has not been formatted will be removed from the filename.

Parameters

  • template (str) – The templated filename

  • handler_reg (dict) – The registry of file handlers for loading files from disk

  • root_map (dict) – Mapping between the old file root and a new root, used for loading files from disk

  • kwargs (dict) – All extra kwargs are passed to the filename formatter when the start document is received

Notes

Every instance of __independent_vars__ will be replaced with {name}_{data}_{units}_ for each independent variable of the experiment in the template.

descriptor(doc)[source]
event(doc)[source]
start(doc)[source]
class xpdan.callbacks.SaveCalib(template, handler_reg, root_map=None, base_folders=None, **kwargs)[source]

Bases: xpdan.callbacks.SaveBaseClass

Callback for saving pyFAI calibrations as .poni files

event(doc)[source]
class xpdan.callbacks.SaveIntensity(template, handler_reg, root_map=None, base_folders=None, **kwargs)[source]

Bases: xpdan.callbacks.SaveBaseClass

Callback for saving Q and tth .chi files

event(doc)[source]
class xpdan.callbacks.SaveMask(template, handler_reg, root_map=None, base_folders=None, **kwargs)[source]

Bases: xpdan.callbacks.SaveBaseClass

Callback for saving masks as .msk and .npy files

event(doc)[source]
class xpdan.callbacks.SaveMeta(template, handler_reg, root_map=None, base_folders=None, **kwargs)[source]

Bases: xpdan.callbacks.SaveBaseClass

Callback for saving metadata files

event(doc)[source]
start(doc)[source]
class xpdan.callbacks.SavePDFgetx3(template, handler_reg, root_map=None, base_folders=None, **kwargs)[source]

Bases: xpdan.callbacks.SaveBaseClass

Callback for saving PDF, F(Q), S(Q) files

event(doc)[source]
class xpdan.callbacks.SaveTiff(template, handler_reg, root_map=None, base_folders=None, **kwargs)[source]

Bases: xpdan.callbacks.SaveBaseClass

Callback for saving Tiff files

event(doc)[source]
class xpdan.callbacks.StartStopCallback[source]

Bases: bluesky.callbacks.core.CallbackBase

Print the time for analysis

event(doc)[source]
start(doc)[source]
stop(doc)[source]

xpdan.data_reduction module

xpdan.data_reduction.integrate_and_save(headers, *, db=<databroker._core.Broker object>, save_dir='/home/travis/acqsim/xpdUser/tiff_base', visualize=False, polarization_factor=0.99, mask_setting='default', mask_kwargs={'alpha': 3.0, 'bs_width': 13, 'edge': 30, 'lower_thresh': 0.0, 'tri_offset': 13, 'v_asym': 0}, image_data_key='pe1_image', pdf_config=None)

Integrate and save dark subtracted images for given list of headers

Parameters

  • headers (list) – a list of databroker.header objects

  • db (databroker.broker.Broker instance) – The databroker holding the data, this must be specified as a db= in the function call (keyword only argument)

  • save_dir (str) – The folder in which to save the data, this must be specified as a save_dir= in the function call (keyword only argument)

  • visualize (bool, optional) – If True visualize the data. Defaults to False

  • polarization_factor (float, optional) – polarization correction factor, ranged from -1(vertical) to +1 (horizontal). default is 0.99. set to None for no correction.

  • mask_setting (str optional) – If ‘default’ reuse mask created for first image, otherwise mask all images. Defaults to ‘default’

  • mask_kwargs (dict, optional) – dictionary stores options for automasking functionality. default is defined by an_glbl.auto_mask_dict. Please refer to documentation for more details

  • image_data_key (str, optional) – The key for the image data, defaults to pe1_image

  • pdf_config (dict, optional) – Configuration for making PDFs, see pdfgetx3 docs. Defaults to dict(dataformat='QA', qmaxinst=28, qmax=22)

Note

complete docstring of masking functionality could be find in mask_img

See also

xpdan.tools.mask_img()

xpdan.data_reduction.integrate_and_save_last(**kwargs)

Integrate and save dark subtracted images for the latest header

Parameters

  • headers (list) – a list of databroker.header objects

  • db (databroker.broker.Broker instance) – The databroker holding the data, this must be specified as a db= in the function call (keyword only argument)

  • save_dir (str) – The folder in which to save the data, this must be specified as a save_dir= in the function call (keyword only argument)

  • visualize (bool, optional) – If True visualize the data. Defaults to False

  • polarization_factor (float, optional) – polarization correction factor, ranged from -1(vertical) to +1 (horizontal). default is 0.99. set to None for no correction.

  • mask_setting (str optional) – If ‘default’ reuse mask created for first image, otherwise mask all images. Defaults to ‘default’

  • mask_kwargs (dict, optional) – dictionary stores options for automasking functionality. default is defined by an_glbl.auto_mask_dict. Please refer to documentation for more details

  • image_data_key (str, optional) – The key for the image data, defaults to pe1_image

  • pdf_config (dict, optional) – Configuration for making PDFs, see pdfgetx3 docs. Defaults to dict(dataformat='QA', qmaxinst=28, qmax=22)

Note

complete docstring of masking functionality could be find in mask_img

See also

xpdan.tools.mask_img()

xpdan.data_reduction.save_last_tiff(**kwargs)

Save images obtained from dataBroker as tiff format files.

Parameters

  • headers (list) – a list of databroker.header objects

  • db (databroker.broker.Broker instance) – The databroker holding the data, this must be specified as a db= in the function call (keyword only argument)

  • save_dir (str) – The folder in which to save the data, this must be specified as a save_dir= in the function call (keyword only argument)

  • visualize (bool, optional) – If True visualize the data. Defaults to False

  • image_data_key (str, optional) – The key for the image data, defaults to pe1_image

xpdan.data_reduction.save_tiff(headers, *, db=<databroker._core.Broker object>, save_dir='/home/travis/acqsim/xpdUser/tiff_base', visualize=False, image_data_key='pe1_image')

Save images obtained from dataBroker as tiff format files.

Parameters

  • headers (list) – a list of databroker.header objects

  • db (databroker.broker.Broker instance) – The databroker holding the data, this must be specified as a db= in the function call (keyword only argument)

  • save_dir (str) – The folder in which to save the data, this must be specified as a save_dir= in the function call (keyword only argument)

  • visualize (bool, optional) – If True visualize the data. Defaults to False

  • image_data_key (str, optional) – The key for the image data, defaults to pe1_image

xpdan.data_reduction_core module

xpdan.data_reduction_core.integrate_and_save(headers, *, db, save_dir, visualize=False, polarization_factor=0.99, mask_setting='default', mask_kwargs=None, image_data_key='pe1_image', pdf_config=None)[source]

Integrate and save dark subtracted images for given list of headers

Parameters

  • headers (list) – a list of databroker.header objects

  • db (databroker.broker.Broker instance) – The databroker holding the data, this must be specified as a db= in the function call (keyword only argument)

  • save_dir (str) – The folder in which to save the data, this must be specified as a save_dir= in the function call (keyword only argument)

  • visualize (bool, optional) – If True visualize the data. Defaults to False

  • polarization_factor (float, optional) – polarization correction factor, ranged from -1(vertical) to +1 (horizontal). default is 0.99. set to None for no correction.

  • mask_setting (str optional) – If ‘default’ reuse mask created for first image, otherwise mask all images. Defaults to ‘default’

  • mask_kwargs (dict, optional) – dictionary stores options for automasking functionality. default is defined by an_glbl.auto_mask_dict. Please refer to documentation for more details

  • image_data_key (str, optional) – The key for the image data, defaults to pe1_image

  • pdf_config (dict, optional) – Configuration for making PDFs, see pdfgetx3 docs. Defaults to dict(dataformat='QA', qmaxinst=28, qmax=22)

Note

complete docstring of masking functionality could be find in mask_img

See also

xpdan.tools.mask_img()

xpdan.data_reduction_core.integrate_and_save_last(**kwargs)[source]

Integrate and save dark subtracted images for the latest header

Parameters

  • headers (list) – a list of databroker.header objects

  • db (databroker.broker.Broker instance) – The databroker holding the data, this must be specified as a db= in the function call (keyword only argument)

  • save_dir (str) – The folder in which to save the data, this must be specified as a save_dir= in the function call (keyword only argument)

  • visualize (bool, optional) – If True visualize the data. Defaults to False

  • polarization_factor (float, optional) – polarization correction factor, ranged from -1(vertical) to +1 (horizontal). default is 0.99. set to None for no correction.

  • mask_setting (str optional) – If ‘default’ reuse mask created for first image, otherwise mask all images. Defaults to ‘default’

  • mask_kwargs (dict, optional) – dictionary stores options for automasking functionality. default is defined by an_glbl.auto_mask_dict. Please refer to documentation for more details

  • image_data_key (str, optional) – The key for the image data, defaults to pe1_image

  • pdf_config (dict, optional) – Configuration for making PDFs, see pdfgetx3 docs. Defaults to dict(dataformat='QA', qmaxinst=28, qmax=22)

Note

complete docstring of masking functionality could be find in mask_img

See also

xpdan.tools.mask_img()

xpdan.data_reduction_core.save_last_tiff(**kwargs)[source]

Save images obtained from dataBroker as tiff format files.

Parameters

  • headers (list) – a list of databroker.header objects

  • db (databroker.broker.Broker instance) – The databroker holding the data, this must be specified as a db= in the function call (keyword only argument)

  • save_dir (str) – The folder in which to save the data, this must be specified as a save_dir= in the function call (keyword only argument)

  • visualize (bool, optional) – If True visualize the data. Defaults to False

  • image_data_key (str, optional) – The key for the image data, defaults to pe1_image

xpdan.data_reduction_core.save_tiff(headers, *, db, save_dir, visualize=False, image_data_key='pe1_image')[source]

Save images obtained from dataBroker as tiff format files.

Parameters

  • headers (list) – a list of databroker.header objects

  • db (databroker.broker.Broker instance) – The databroker holding the data, this must be specified as a db= in the function call (keyword only argument)

  • save_dir (str) – The folder in which to save the data, this must be specified as a save_dir= in the function call (keyword only argument)

  • visualize (bool, optional) – If True visualize the data. Defaults to False

  • image_data_key (str, optional) – The key for the image data, defaults to pe1_image

xpdan.db_utils module

xpdan.db_utils.query_background(docs, db, schema=1)[source]

Get background data from databroker

Parameters

  • db (Broker instance)

  • docs (tuple of dict)

  • schema (int) – Schema version

Returns

The list of headers which meet the criteria

Return type

list of Header

xpdan.db_utils.query_dark(docs, db, schema=1)[source]

Get dark data from databroker

Parameters

  • db (Broker instance)

  • docs (tuple of dict)

  • schema (int) – Schema version

Returns

The list of headers which meet the criteria

Return type

list of Header

xpdan.db_utils.query_flat_field(docs, db, schema=1)[source]

Get flat_field data from databroker

Parameters

  • db (Broker instance)

  • docs (tuple of dict)

  • schema (int) – Schema version

Returns

The list of headers which meet the criteria

Return type

list of Header

xpdan.db_utils.scan_diff(hdrs, verbose=True, blacklist=None)[source]

Get the metadata differences between scans in hdrs list

Parameters

  • hdrs (list of Header objects) – The headers to be diffed

  • verbose (bool, optional) – If true prints the results. Defaults to True

  • blacklist (list of str, optional) – List of keys to not be included in diff. If None, defaults to uid

Returns

The dictionary of keys with at least one different value across the scans. The values are the results for each header.

Return type

dict

xpdan.db_utils.scan_summary(hdrs, fields=None, verbose=True)[source]

Provide one line summaries of headers

Parameters

  • hdrs (list of headers) – The headers from the databroker

  • fields (list of str, optional) – ‘Specify a list of fields to summarize. If None, the following will be returned ['sample_name', 'sp_type', 'sp_startingT', 'sp_endingT'] defaults to None

  • verbose (bool, optional) – If True print the summary

Returns

List of summary strings

Return type

list

xpdan.db_utils.sort_scans_by_hdr_key(hdrs, key, verbose=True)[source]

In a list of hdrs, group the scans by header-key.

Use this function to find all the scans in the list of headers that have a particular key value, such as ‘sample_name’=’Ni’

Function returns a list of indices for the position in the list, associated with the metadata key values.

Parameters

  • hdrs (list of Header objects) – The dictionary containing {‘key-value’:[list of scan indices]}. For example a search over ‘sample_name’ might return {‘Ni’:[0,1,2,3,4,9,10],’gold nanoparticles’:[5,6,7,8]}

  • key (str) – The scans will be sorted by the values of this metadata key, e.g., ‘sample_name’

  • verbose (bool, optional) – If true prints the results. Defaults to True

Returns

The dictionary containing {‘key-value’:[list of scan indices]}. For example a search over ‘sample_name’ might return {‘Ni’:[0,1,2,3,4,9,10],’gold nanoparticles’:[5,6,7,8]}

Return type

dict

xpdan.db_utils.temporal_prox(res, docs)[source]

xpdan.dev_utils module

xpdan.fuzzybroker module

Enhanced databroker with fuzzy search options

class xpdan.fuzzybroker.FuzzyBroker(mds, reg=None, plugins=None, filters=None, auto_register=True, external_fetchers=None, name=None, event_sources=None)[source]

Bases: databroker._core.Broker

beamtime_dates(**kwargs)[source]

Get info for each beamtime

Parameters

  • keys (iterable of str) – The keys to be included in the return

  • beamtime_key (str) – The key for the unique beamtime key

  • print_results (bool) – If true prints the information

Returns

The list of beamtimes and their associated information

Return type

list of dicts

Fuzzy search a databroker for given keys

Parameters

  • keys (list of str) – The list of strings to be accessed

  • search_string (str) – The string to be searched for

  • size (int or ‘all’, optional) –

    The number of results to be returned, if ‘all’ all are returned.

    Defaults to 100 results

Returns

A list

Return type

list

Notes

This search can take a long time as they turn over the entire databroker searching for the search_string please make a point to filter the databroker to shorten the total search time

Return the most similar set of values to the search string.

Parameters

  • key (list of str) – The list of strings to be accessed

  • search_string (str) – The string to be searched for

  • size (int, optional) –

    The number of results to be returned.

    Defaults to 100 results

Returns

A list of headers which are close to the query. The queried value will be unique in the list so a search for piLast=’Alice’ will return only one header with Alice as the PI.

Return type

list

Examples

>>> db = Broker(...) # Contains runs from Bob, Alice, Bob, and Eve
>>> fuzzy_set_search(db, 'bt_piLast', 'Bob')
['Bob', 'Alice', 'Eve']

Fuzzy search a databroker

Parameters

  • search_string (str) – The string to be searched for

  • size (int, optional) –

    The number of results to be returned.

    Defaults to 100 results

Returns

A list of headers which contain close matches

Return type

list

Notes

This search can take a long time as they turn over the entire databroker (and all of the dictionaries inside) searching for the search_string please make a point to filter the databroker to shorten the total search time.

xpdan.fuzzybroker.beamtime_dates(db, keys=('beamtime_uid', 'bt_safN', 'facility', 'beamline'), beamtime_key='beamtime_uid', print_results=True)[source]

Get info for each beamtime

Parameters

  • db (databroker instance) – The databroker to be searched

  • keys (iterable of str) – The keys to be included in the return

  • beamtime_key (str) – The key for the unique beamtime key

  • print_results (bool) – If true prints the information

Returns

The list of beamtimes and their associated information

Return type

list of dicts

Fuzzy search a databroker for given keys

Parameters

  • db (databroker.DataBroker instance) – The databroker to be searched

  • keys (list of str) – The list of strings to be accessed

  • search_string (str) – The string to be searched for

  • size (int or ‘all’, optional) –

    The number of results to be returned, if ‘all’ all are returned.

    Defaults to 100 results

Returns

A list

Return type

list

Return the most similar set of values to the search string.

Parameters

  • db (databroker.DataBroker instance) – The databroker to be searched

  • key (list of str) – The list of strings to be accessed

  • search_string (str) – The string to be searched for

  • size (int, optional) –

    The number of results to be returned.

    Defaults to 100 results

Returns

A list

Return type

list

Examples

>>> db = Broker(...) # Contains runs from Bob, Alice, Bob, and Eve
>>> fuzzy_set_search(db, 'bt_piLast', 'Bob')
['Bob', 'Alice', 'Eve']

Fuzzy search a databroker

Parameters

  • db (databroker.DataBroker instance) – The databroker to be searched

  • search_string (str) – The string to be searched for

  • size (int, optional) –

    The number of results to be returned.

    Defaults to 100 results

Returns

A list

Return type

list

xpdan.glbl module

Shim for loading the glbl_dict from xpdconf

xpdan.glbl_gen module

xpdan.tools module

xpdan.utils module

Module contents