Astromon API

db

astromon.db.add_regions(regions, dbfile=None)

Add exclusion regions to the astromon_regions table.

A unique region ID is automatically generated. Raises an exception if the astromon_regions table is not in the dbfile.

Parameters

• regions (astropy.table.Table-compatible) – This parameter gets converted to an astropy.table.Table.

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

astromon.db.get_cross_matches(name='astromon_21', dbfile=None, **kwargs)

Get a standard cross-match of observations, x-ray sources and catalog counterparts in dbfile.

A standard cross-match is a pre-computed cross-match between x-ray sources and catalog counterparts. The name of the standard cross-match specifies the algorithm and the set of parameters that have been used to cross-match.

If you want a non-standard cross-match, with your own set of parameters, refer to the compute_cross_matches function.

This function returns a Table indexed by OBSID and adds a few columns on the fly:

• time

• c_loc

• x_loc

Parameters

• name (str) – name of the standard cross-match.

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

• kwargs (dict) –

  All extra arguments are passed directly to cross_match.filter_matches. These are some of the allowed arguments:

  • snr

  • dr

  • r_angle

  • start

  • stop

  • r_angle_grating

  • near_neighbor_dist

  • sim_z

  • exclude_regions

  • exclude_bad_targets

  • exclude_categories

  Other extra arguments can be passed. In these cases, the argument name determines on which column to filter. Each argument is interpreted according to the type of value passed:

  • list values cause a row to be selected if the value at that row and column is in the list.

  • other values cause a row to be selected if the value at that row and column is equal to the argument value

Returns

Return type

astropy.table.Table

astromon.db.get_table(table_name, dbfile=None)

Get an entire table from the DB file.

Known table names:

• astromon_xcorr

• astromon_xray_src

• astromon_cat_src

• astromon_obs

• astromon_regions

• astromon_meta

Parameters

• table_name (str) – Name of the table to retrieve. If the requested table is not in the file, an exception will be raised.

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

Returns

Return type

astropy.table.Table

astromon.db.remove_regions(regions, dbfile=None)

Remove exclusion regions (by ID) from the astromon_regions table.

Raises an exception if the astromon_regions table is not in the dbfile.

Parameters

• regions (list) – list of integer region ID.

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

astromon.db.save(table_name, data, dbfile)

Insert data into a table, deleting previous entries for the same OBSID.

If the table does not exist, it is created using pre-existing table definitions.

Parameters

• table_name (str) – The name of the table.

• data (astropy.table.Table) –

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

cross_match

astromon.cross_match.CROSS_MATCHES = ['astromon_21', 'astromon_22', 'default']

Available cross-matching algorithms to be used in compute_cross_matches.

astromon.cross_match.CROSS_MATCHES_ARGS = {'astromon_21': {'catalogs': ['ICRS', 'Tycho2'], 'dr': 3.0, 'method': 'simple', 'name': 'astromon_21', 'near_neighbor_dist': 6.0, 'r_angle': 120.0, 'r_angle_grating': 120.0, 'snr': 3}, 'astromon_22': {'catalogs': ['ICRS', 'Tycho2'], 'dr': 3.0, 'method': 'simple', 'name': 'astromon_22', 'near_neighbor_dist': 6.0, 'r_angle': 120.0, 'r_angle_grating': 24.0, 'snr': 3}, 'default': {'catalogs': ['ICRS', 'Tycho2'], 'dr': 3.0, 'method': 'simple', 'name': 'astromon_21', 'near_neighbor_dist': 6.0, 'r_angle': 120.0, 'r_angle_grating': 120.0, 'snr': 3}}

Standard cross-match arguments.

astromon.cross_match.compute_cross_matches(name=None, astromon_obs=None, astromon_xray_src=None, astromon_cat_src=None, dbfile=None, exclude_regions=False, exclude_bad_targets=False, logging_tag='', **kwargs)

Cross-match x-ray sources with catalog counterparts.

The arguments to this function are relayed to the actual implementation. The default algorithm is the name=”simple” cross-match (see the documentation of simple_cross_match for details).

Parameters

• name (str) – Can be the name one of the standard cross-matches or the name of the algorithm to use (only ‘simple’ is implemented). Default: ‘simple’.

• astromon_obs (astropy.table.Table) – Table of astromon observations as returned by db.get_table("astromon_obs"). The default is to get it from dbfile.

• astromon_xray_src (astropy.table.Table) – Table of x-ray sources as returned by db.get_table("astromon_xray_src") or observation.Observation.get_sources. The default is to get it from dbfile.

• astromon_cat_src (astropy.table.Table) – Table of catalog counterparts as returned by db.get_table("astromon_cat_src") or rough_match. The default is to get it from dbfile.

• dbfile (str) – HDF5 or SQLite file where the tables are. The default is to let db.get_table decide.

• exclude_bad_targets (bool) – Default is False.

• exclude_regions (bool) – Default is False.

• logging_tag (str) – A string to prepend to the logging messages.

• kwargs (dict) –

  Arguments passed to the algorithm implementation. Ignored if name is one of the standard cross-matches. kwargs is implementation dependant, but the arguments of the ‘simple’ algorithm are:

  • catalogs: A list of catalog names. The order matters. Default is [‘ICRS’, ‘Tycho2’]

  • snr: Minimum signal-to-noise ratio of x-ray sources to consider. Default is 3.

  • r_angle: Maximum r_angle (in arcsec). Default is 120 arcsec (2 arcmin).

  • dr: Maximum separation between x-ray source and catalog counterpart (in arcsec). Default is 3 arcsec.

  • r_angle_grating: Maximum r_angle in the case of grating observations (in arcsec). Default is 24 arcsec (0.4 arcmin).

  • near_neighbor_dist: Only consider x-ray sources with no other x-ray source within this radial distance (arcsec). Default is 6 arcsec.

  • start: Only consider observations after this CxoTime. Default is to consider all.

astromon.cross_match.filter_matches(matches, snr=None, dr=None, r_angle=None, start=None, stop=None, r_angle_grating=None, near_neighbor_dist=None, sim_z=None, exclude_regions=False, exclude_bad_targets=False, exclude_categories=(), **kwargs)

Return a mask to filter out cross-matches based on some common criteria.

Parameters

• matches (astropy.table.Table) – Table of cross-matches as returned by db.get_cross_matches or cross_match

• snr (float) – Filter matches based on signal-to-noise. Selects matches[‘snr’] <= snr.

• dr (float) – Filter matches based on angular offset. Selects matches[‘dr’] <= dr.

• r_angle (float) – Filter matches based on distance to optical axis. Selects matches[‘r_angle’] <= r_angle (apply to non-grating observations only).

• start (CxoTime) – Filter matches based on time. Selects matches[‘tstart’] >= start.

• stop (CxoTime) – Filter matches based on time. Selects matches[‘tstart’] <= stop.

• r_angle_grating (float) – Filter matches based on distance to optical axis (apply to grating observations only). Selects matches[‘r_angle_grating’] <= r_angle_grating.

• near_neighbor_dist (float) – Filter matches based on distance to closest neighbor. Selects matches[‘near_neighbor_dist’] <= near_neighbor_dist.

• sim_z (float) – Maximum allowed SIM-Z in mm.

• exclude_bad_targets (bool) – Default is False.

• exclude_regions (bool) – Default is False.

• exclude_categories (tuple) – A list of observation categories to exclude. Default is empty.

• kwargs (dict) – The keys in kwargs determine on which columns to filter. The values of kwargs are used according to their type: - list types cause a row to be selected if the column value is in the list. - otherwise rows are selected if the column value is equal to the kwargs value

astromon.cross_match.get_bad_target_mask(matches)

Returns a mask to remove targets that are known to give cross-matches with large residuals.

This is not used in standard processing and is here for convenience.

Parameters

matches (astropy.table.Table) – Table of astromon cros-matches as returned by cross_match and db.get_cross_matches. Required.

astromon.cross_match.get_excluded_regions_mask(matches, regions=None)

Returns a mask to remove x-ray sources that are close to excluded regions.

Parameters

matches (astropy.table.Table) – Table of astromon cros-matches as returned by cross_match and db.get_cross_matches. Required.

astromon.cross_match.rough_match(sources, time, radius=<Quantity 3. arcsec>, catalogs=('Tycho2', 'ICRS', 'USNO-B1.0', '2MASS', 'SDSS'))

Find sources in a set of catalogs around the x-ray sources given in sources, within an angular separation of at most radius (in astropy units).

This function uses the ‘ra’, ‘dec’, ‘id’ and ‘obsid’ columns of the sources table. The obsid column is optional and is used only to check that the query corresponds to a single OBSID.

Note

The current implementation of this function creates a temporary table with the cartesian product of the x-ray sources and the catalog sources tables (length ~ n_x_ray_sources^2). This is intended to be used with few x-ray sources at a time.

The alternative to this decision would have been to use a custom join function, which can lead to wrong results if the sources happen to be closer than 2*radius. See warning on Custom Join functions. Using a cartesian join allows one to get rough matches without a requirement in near_neighbor_dist.

Parameters

• sources (astropy.table.Table) – Table of x-ray sources as returned by db.get_table("astromon_xray_src") or observation.Observation.get_sources. Required.

• time (CxoTime-compatible) – Time of the observation. Required to account for proper motion.

• radius (astropy.units.Quantity) – Radius around the x-ray sources to look for counterparts.

• catalogs (list) – A list of catalog names. Default is (‘Tycho2’, ‘ICRS’, ‘USNO-B1.0’, ‘2MASS’, ‘SDSS’)

astromon.cross_match.simple_cross_match(astromon_obs, astromon_xray_src, astromon_cat_src, name='', catalogs=('ICRS', 'Tycho2'), snr=3, r_angle=120.0, dr=3, r_angle_grating=24.0, near_neighbor_dist=6.0, start=None, stop=None, logging_tag='')

The simplest cross-match of x-ray sources with catalog counterparts.

This algorithm selects pairs of x-ray sources and catalog counterparts according to these criteria:

• Observations done after start.

• X-ray sources with signal-over-noise ratio > snr.

• X-ray sources at most r_angle off-axis (grating observations) or r_angle_grating arcsec (non-grating observations).

• Angular separation between X-ray and catalog counterpart less than dr arcsec.

• X-ray sources that are at most near_neighbor_dist arcsec from the closest x-ray source.

• Counterparts from catalogs included in catalog.

The selected pairs are sorted according to catalog and angular separation. The order of precedence for the catalogs is the order in the catalogs argument. The first pair for each x-ray source is selected as the catalog match.

Warning

This algorithm does not check whether two or more sources are paired with the same counterpart within a radius dr. To prevent this from happening, near_neighbor_dist should be at least twice dr.

Note

Remember that this cross-match function is applied over a set of rough matches resulting from applying rough_match. In consequence, dr should be smaller than the one used in rough_match.

Parameters

• astromon_obs (astropy.table.Table) – Table of astromon observations as returned by db.get_table("astromon_obs"). Required.

• astromon_xray_src (astropy.table.Table) – Table of x-ray sources as returned by db.get_table("astromon_xray_src") or observation.Observation.get_sources. Required.

• astromon_cat_src (astropy.table.Table) – Table of catalog counterparts as returned by db.get_table("astromon_cat_src") or rough_match. Required.

• name (str) – The name of the algorithm to use or the name of a standard set of arguments. Default is ‘simple’

• catalogs (list) – A list of catalog names. The order matters. Default is [‘ICRS’, ‘Tycho2’]

• snr (float) – Minimum signal-to-noise ratio of x-ray sources to consider. Default is 3.

• r_angle (float) – Maximum r_angle (in arcsec). Default is 120 arcsec (2 arcmin).

• dr (float) – Maximum separation between x-ray source and catalog counterpart (in arcsec). Default is 3 arcsec.

• r_angle_grating (float) – Maximum r_angle in the case of grating observations (in arcsec). Default is 24 arcsec (0.4 arcmin).

• near_neighbor_dist (float) – Only consider x-ray sources with no other x-ray source within this radial distance (arcsec). Default is 6 arcsec.

• start (CxoTime-compatible timestamp) – Only consider observations after this time. Default is to consider all.

• stop (CxoTime-compatible timestamp) – Only consider observations before this time. Default is to consider all.

• logging_tag (str) – A string to prepend to the logging messages.

observation

class astromon.observation.Observation(obsid, workdir=None, source='arc5gl', ciao_prefix=None, archive_dir=None, archive_regex=None, use_ciao=True)

Class to encapsulate calls to CIAO and arc5gl.

Parameters

• obsid (int) –

• workdir (pathlib.Path or str.) – Top-level working directory. Used to set PFILES and ASCDS_WORK_PATH and to download files. The actual working directory will be {workdir}/obs{int(obsid)//1000:02d}/{obsid}. If not given, the working directory will be a temporary directory.

• source (str) – One of ‘arc5gl’ (to get data using arc5gl) or ‘archive’ (to get data from chandra public archive using CIAO’s download_chandra_obsid).

• archive_dir (pathlib.Path or str.) – Top-level archive directory. Used to permanently store the result files. The actual archive directory will be {archive_dir}/obs{int(obsid)//1000:02d}/{obsid}.

• ciao_prefix (str) – The location of CIAO.

• logger (logging.Logger.) – If not provided, the root logger is used.

• archive_regex (list of str.) – Files matching any regex in the list are archived.

• use_ciao (bool) – If this is False, there will be no calls to CIAO tools (and some methods will just fail).

archive(*regex, destination=None)

Move observation files to an archive location.

Parameters

• regex (list of str) – Optional. If not given, self.archive_regex is used. Files matching any of the strings are arcived in a long-term location.

• destination (pathlib.Path or str.) – Optional. If not given, self.archive_dir is used. Long-term location where to store files.

calculate_centroids()

Re-compute centroids.

download(ftypes=None)

Download observation files using the Chandra archive or arc5gl.

Parameters

ftypes (list of str) – Each must be a key in self.archive_file_locations.

filter_events(radius=180, psfratio=1)

Filter x-ray events outside a radius around the optical axis.

filter_sources(radius=180, psfratio=1)

Filter detected sources outside a radius around the optical axis.

get_info()

Get observation info.

Observation info is a combination of OBSPAR (from the obs0 file) and the sequence summary from https://icxc.harvard.edu/cgi-bin/mp/target.cgi.

get_obspar()

Get the contents of the obs0 file as a dictionary.

get_sources(version='celldetect')

Returns a table of sources formatted for the astromon_xray_source SQL table

make_images(evt=None)

Create image. Also creates the exposure map and psfmap.

process()

Main routine to process a single obsid with “standard” steps.

This function skips:

• observations with obs_mode other than “POINTING”.

• ACIS observations with readmode other than “TIMED” and dtycycle different than 0.

repro()

Reprocess data.

run_celldetect(snr=3)

Run celldetect.

run_wavdetect(edition, skip_exist=False, scales='1.4 2 4 8 16 32')

Run wavdetect.

utils

class astromon.utils.Ciao(prefix=None, workdir=None, logger=None)

Encapsulate calls to CIAO tools.

This class does two things:

• Set the CIAO context (PFILES and ASCDS_WORK_PATH).

• Handle calls to subprocesses.

Parameters

• prefix (str) – The location of CIAO.

• workdir (pathlib.Path or str) – Working directory. Used to set PFILES and ASCDS_WORK_PATH.

• logger (logging.Logger) – If not provided, the root logger is used.

pget(command, param='value', logging_tag='')

Call CIAO’s pget and return the standard output.

exception astromon.utils.FlowException

Exception class to interrupt the execution flow.

This exception class is used by logging_call_decorator and is silently ignored unless instructed otherwise.

astromon.utils.calalign_from_files(calalign_dir=None)

Get data from calalign files in calalign_dir.

Parameters

calalign_dir (pathlib.Path) – Directory where to find the calalign files.

Returns

Return type

astropy.table.Table

class astromon.utils.chdir(directory)

Context manager to temporarily change to a directory.

astromon.utils.ciao_context

Context manager to set CIAO context (PFILES and ASCDS_WORK_PATH)

Parameters

• directory (pathlib.Path or str) – Working directory. Used to set PFILES and ASCDS_WORK_PATH.

• logger (logging.Logger) – If not provided, the root logger is used.

astromon.utils.communicate(process, logger=None, level='WARNING', text=False, logging_tag='')

Real-time reading of a subprocess stdout.

Parameters

• process – process returned by subprocess.Popen

• logger (logging.Logger) –

• level (str or int) – the logging level

• text (bool) – whether the process output is text or binary

astromon.utils.get_calalign_offsets(all_matches, ref_calalign=None, calalign_dir=None)

Get a table with the yag/zag offsets subtracted by the aca_misalign matrix.

The table includes two sets of offsets: the offsets used when the observation was processed, and the offsets one would get when using a reference CALALIGN.

Parameters

all_matches (astropy.table.Table) – The table of x-ray sources. The only required columns are ‘obsid’ and ‘x_id’

Returns

A table with keys:

• calalign_dy

• calalign_dz

• aca_misalign

• fts_misalign

• calalign_version

• ref_calalign_dy

• ref_calalign_dz

• ref_aca_misalign

• ref_fts_misalign

• ref_calalign_version

Return type

astropy.table.Table

astromon.utils.logging_call_decorator = <astromon.utils.LoggingCallDecorator object>

Decorator to add logging messages at the start/end of the decorated function.