Astromon API

db

astromon.db.add_regions(regions, dbfile=None)

Add exclusion regions to the astromon_regions table.

A unique region ID is automatically generated. Raises an exception if the astromon_regions table is not in the dbfile.

Parameters:

• regions (astropy.table.Table-compatible) – This parameter gets converted to an astropy.table.Table.

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

astromon.db.get_cross_matches(name='astromon_21', dbfile=None, **kwargs)

Get a standard cross-match of observations, x-ray sources and catalog counterparts in dbfile.

A standard cross-match is a pre-computed cross-match between x-ray sources and catalog counterparts. The name of the standard cross-match specifies the algorithm and the set of parameters that have been used to cross-match.

If you want a non-standard cross-match, with your own set of parameters, refer to the compute_cross_matches function.

This function returns a Table indexed by OBSID and adds a few columns on the fly:

• time

• c_loc

• x_loc

Parameters:

• name (str) – name of the standard cross-match.

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

• kwargs (dict) –

  All extra arguments are passed directly to cross_match.filter_matches. These are some of the allowed arguments:

  • snr

  • dr

  • r_angle

  • start

  • stop

  • r_angle_grating

  • near_neighbor_dist

  • sim_z

  • exclude_regions

  • exclude_bad_targets

  • exclude_categories

  Other extra arguments can be passed. In these cases, the argument name determines on which column to filter. Each argument is interpreted according to the type of value passed:

  • list values cause a row to be selected if the value at that row and column is in the list.

  • other values cause a row to be selected if the value at that row and column is equal to the argument value

Return type:

astropy.table.Table

astromon.db.get_regions(obsid=None, dbfile=None, radius=<Quantity 5. arcmin>)

Get exclusion regions from the astromon_regions table.

Parameters:

• obsid (int) – If provided, only return regions with obsid=0 or centered around the nominal pointing of the given obsid, as stored in the obspar file.

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

• radius (astropy.units.Quantity) – Maximum distance from the nominal pointing of the observation to include a region. Only used if obsid is provided. Default is 5 arcmin.

Return type:

astropy.table.Table

astromon.db.get_table(table_name, dbfile=None)

Get an entire table from the DB file.

Known table names:

• astromon_xcorr

• astromon_xray_src

• astromon_cat_src

• astromon_obs

• astromon_regions

• astromon_meta

Parameters:

• table_name (str) – Name of the table to retrieve. If the requested table is not in the file, an exception will be raised.

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

Return type:

astropy.table.Table

astromon.db.is_in_excluded_region(position, obsid=None, regions=None, dbfile=None)

Returns a mask to remove x-ray sources that are close to excluded regions.

Parameters:

• position (astropy.coordinates.SkyCoord) – Array of astropy.coordinates.SkyCoord objects.

• obsid (array-like) – Array of OBSIDs corresponding to each position.

astromon.db.remove_regions(regions, dbfile=None)

Remove exclusion regions (by ID) from the astromon_regions table.

Raises an exception if the astromon_regions table is not in the dbfile.

Parameters:

• regions (list) – list of integer region ID.

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

astromon.db.save(table_name, data, dbfile, ignore_obsid=False)

Insert data into a table, deleting previous entries for the same OBSID.

If the table does not exist, it is created using pre-existing table definitions.

If ignore_obsid is False, and data has an “obsid” column, this function replaces all rows in the table whose OBSID is in data[“obsid”]. It does not replace the whole table. If data has all rows of a given obsid removed, those entries are NOT removed from the table. For example, the following code has no effect:

data = db.get_table(“astromon_xray_src”, dbfile) data = data[data[‘obsid’] != 12345] db.save(“astromon_xray_src”, data, dbfile)

To remove all entries for a given obsid, set ignore_obsid=True. If you do that, the entire table is replaced by data. The following removes all entries for obsid 12345:

data = db.get_table(“astromon_xray_src”, dbfile) data = data[data[‘obsid’] != 12345] db.save(“astromon_xray_src”, data, dbfile, ignore_obsid=True)

Parameters:

• table_name (str) – The name of the table.

• data (astropy.table.Table)

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

• ignore_obsid (bool) – If True, do not consider obsid to decide which rows to keep in the table.

astromon.db.sync_regions(from_file, to_file, remove=False)

Sync exclusion regions from one astromon DB file to another.

Parameters:

• from_file (pathlib.Path) – Source DB file.

• to_file (pathlib.Path) – Destination DB file.

astromon.db.update_regions(regions, dbfile=None)

Update exclusion regions in the astromon_regions table.

Things to keep in mind:

• last_modified is updated automatically.

• Changes region_id_str are ignored.

• Entries that do not differ from the values in the table are not updated (last_modified is not set).

• This does not remove regions. Use remove_regions for that.

Raises an exception if:

• the astromon_regions table is not in the dbfile.

• any of the region_id in regions does not exist in the table.

Parameters:

• regions (astropy.table.Table-compatible) – This parameter gets converted to an astropy.table.Table.

• dbfile (pathlib.Path) – File where tables are stored. The default is $ASTROMON_FILE or $SKA/data/astromon/astromon.h5

cross_match

astromon.cross_match.CROSS_MATCHES = ['astromon_21', 'astromon_22', 'default']

Available cross-matching algorithms to be used in compute_cross_matches.

astromon.cross_match.CROSS_MATCHES_ARGS = {'astromon_21': {'catalogs': ['ICRS', 'Tycho2'], 'dr': 3.0, 'method': 'simple', 'name': 'astromon_21', 'near_neighbor_dist': 6.0, 'r_angle': 120.0, 'r_angle_grating': 120.0, 'snr': 3}, 'astromon_22': {'catalogs': ['ICRS', 'Tycho2'], 'dr': 3.0, 'method': 'simple', 'name': 'astromon_22', 'near_neighbor_dist': 6.0, 'r_angle': 120.0, 'r_angle_grating': 24.0, 'snr': 3}, 'default': {'catalogs': ['ICRS', 'Tycho2'], 'dr': 3.0, 'method': 'simple', 'name': 'astromon_21', 'near_neighbor_dist': 6.0, 'r_angle': 120.0, 'r_angle_grating': 120.0, 'snr': 3}}

Standard cross-match arguments.

astromon.cross_match.compute_cross_matches(name=None, astromon_obs=None, astromon_xray_src=None, astromon_cat_src=None, dbfile=None, exclude_regions=False, exclude_bad_targets=False, logging_tag='', **kwargs)

Cross-match x-ray sources with catalog counterparts.

The arguments to this function are relayed to the actual implementation. The default algorithm is the name=”simple” cross-match (see the documentation of simple_cross_match for details).

Parameters:

• name (str) – Can be the name one of the standard cross-matches or the name of the algorithm to use (only ‘simple’ is implemented). Default: ‘simple’.

• astromon_obs (astropy.table.Table) – Table of astromon observations as returned by db.get_table("astromon_obs"). The default is to get it from dbfile.

• astromon_xray_src (astropy.table.Table) – Table of x-ray sources as returned by db.get_table("astromon_xray_src") or observation.Observation.get_sources. The default is to get it from dbfile.

• astromon_cat_src (astropy.table.Table) – Table of catalog counterparts as returned by db.get_table("astromon_cat_src") or rough_match. The default is to get it from dbfile.

• dbfile (str) – HDF5 or SQLite file where the tables are. The default is to let db.get_table decide.

• exclude_bad_targets (bool) – Default is False.

• exclude_regions (bool) – Default is False.

• logging_tag (str) – A string to prepend to the logging messages.

• kwargs (dict) –

  Arguments passed to the algorithm implementation. Ignored if name is one of the standard cross-matches. kwargs is implementation dependant, but the arguments of the ‘simple’ algorithm are:

  • catalogs: A list of catalog names. The order matters. Default is [‘ICRS’, ‘Tycho2’]

  • snr: Minimum signal-to-noise ratio of x-ray sources to consider. Default is 3.

  • r_angle: Maximum r_angle (in arcsec). Default is 120 arcsec (2 arcmin).

  • dr: Maximum separation between x-ray source and catalog counterpart (in arcsec). Default is 3 arcsec.

  • r_angle_grating: Maximum r_angle in the case of grating observations (in arcsec). Default is 24 arcsec (0.4 arcmin).

  • near_neighbor_dist: Only consider x-ray sources with no other x-ray source within this radial distance (arcsec). Default is 6 arcsec.

  • start: Only consider observations after this CxoTime. Default is to consider all.

astromon.cross_match.filter_matches(matches, snr=None, dr=None, r_angle=None, start=None, stop=None, r_angle_grating=None, near_neighbor_dist=None, sim_z=None, exclude_regions=False, exclude_bad_targets=False, exclude_categories=(), **kwargs)

Return a mask to filter out cross-matches based on some common criteria.

Parameters:

• matches (astropy.table.Table) – Table of cross-matches as returned by db.get_cross_matches or cross_match

• snr (float) – Filter matches based on signal-to-noise. Selects matches[‘snr’] <= snr.

• dr (float) – Filter matches based on angular offset. Selects matches[‘dr’] <= dr.

• r_angle (float) – Filter matches based on distance to optical axis. Selects matches[‘r_angle’] <= r_angle (apply to non-grating observations only).

• start (CxoTime) – Filter matches based on time. Selects matches[‘tstart’] >= start.

• stop (CxoTime) – Filter matches based on time. Selects matches[‘tstart’] <= stop.

• r_angle_grating (float) – Filter matches based on distance to optical axis (apply to grating observations only). Selects matches[‘r_angle_grating’] <= r_angle_grating.

• near_neighbor_dist (float) – Filter matches based on distance to closest neighbor. Selects matches[‘near_neighbor_dist’] <= near_neighbor_dist.

• sim_z (float) – Maximum allowed SIM-Z in mm.

• exclude_bad_targets (bool) – Default is False.

• exclude_regions (bool) – Default is False.

• exclude_categories (tuple) – A list of observation categories to exclude. Default is empty.

• kwargs (dict) – The keys in kwargs determine on which columns to filter. The values of kwargs are used according to their type: - list types cause a row to be selected if the column value is in the list. - otherwise rows are selected if the column value is equal to the kwargs value

astromon.cross_match.get_bad_target_mask(matches)

Returns a mask to remove targets that are known to give cross-matches with large residuals.

This is not used in standard processing and is here for convenience.

Parameters:

matches (astropy.table.Table) – Table of astromon cros-matches as returned by cross_match and db.get_cross_matches. Required.

astromon.cross_match.get_excluded_regions_mask(matches, regions=None)

Returns a mask to remove x-ray sources that are close to excluded regions.

Parameters:

matches (astropy.table.Table) – Table of astromon cros-matches as returned by cross_match and db.get_cross_matches. Required.

astromon.cross_match.rough_match(sources, time, radius=<Quantity 3. arcsec>, catalogs=('Tycho2', 'ICRS', 'USNO-B1.0', '2MASS', 'SDSS'))

Find sources in a set of catalogs around the x-ray sources given.

Find sources in a set of catalogs around the x-ray sources given in sources, within an angular separation of at most radius (in astropy units).

This function uses the ‘ra’, ‘dec’, ‘id’ and ‘obsid’ columns of the sources table. The obsid column is optional and is used only to check that the query corresponds to a single OBSID.

Note

The current implementation of this function creates a temporary table with the cartesian product of the x-ray sources and the catalog sources tables (length ~ n_x_ray_sources^2). This is intended to be used with few x-ray sources at a time.

The alternative to this decision would have been to use a custom join function, which can lead to wrong results if the sources happen to be closer than 2*radius. See warning on Custom Join functions. Using a cartesian join allows one to get rough matches without a requirement in near_neighbor_dist.

Parameters:

• sources (astropy.table.Table) – Table of x-ray sources as returned by db.get_table("astromon_xray_src") or observation.Observation.get_sources. Required.

• time (CxoTime-compatible) – Time of the observation. Required to account for proper motion.

• radius (astropy.units.Quantity) – Radius around the x-ray sources to look for counterparts.

• catalogs (list) – A list of catalog names. Default is (‘Tycho2’, ‘ICRS’, ‘USNO-B1.0’, ‘2MASS’, ‘SDSS’)

astromon.cross_match.simple_cross_match(astromon_obs, astromon_xray_src, astromon_cat_src, name='', catalogs=('ICRS', 'Tycho2'), snr=3, r_angle=120.0, dr=3, r_angle_grating=24.0, near_neighbor_dist=6.0, start=None, stop=None, logging_tag='')

The simplest cross-match of x-ray sources with catalog counterparts.

This algorithm selects pairs of x-ray sources and catalog counterparts according to these criteria:

• Observations done after start.

• X-ray sources with signal-over-noise ratio > snr.

• X-ray sources at most r_angle off-axis (grating observations) or r_angle_grating arcsec (non-grating observations).

• Angular separation between X-ray and catalog counterpart less than dr arcsec.

• X-ray sources that are at most near_neighbor_dist arcsec from the closest x-ray source.

• Counterparts from catalogs included in catalog.

The selected pairs are sorted according to catalog and angular separation. The order of precedence for the catalogs is the order in the catalogs argument. The first pair for each x-ray source is selected as the catalog match.

Warning

This algorithm does not check whether two or more sources are paired with the same counterpart within a radius dr. To prevent this from happening, near_neighbor_dist should be at least twice dr.

Note

Remember that this cross-match function is applied over a set of rough matches resulting from applying rough_match. In consequence, dr should be smaller than the one used in rough_match.

Parameters:

• astromon_obs (astropy.table.Table) – Table of astromon observations as returned by db.get_table("astromon_obs"). Required.

• astromon_xray_src (astropy.table.Table) – Table of x-ray sources as returned by db.get_table("astromon_xray_src") or observation.Observation.get_sources. Required.

• astromon_cat_src (astropy.table.Table) – Table of catalog counterparts as returned by db.get_table("astromon_cat_src") or rough_match. Required.

• name (str) – The name of the algorithm to use or the name of a standard set of arguments. Default is ‘simple’

• catalogs (list) – A list of catalog names. The order matters. Default is [‘ICRS’, ‘Tycho2’]

• snr (float) – Minimum signal-to-noise ratio of x-ray sources to consider. Default is 3.

• r_angle (float) – Maximum r_angle (in arcsec). Default is 120 arcsec (2 arcmin).

• dr (float) – Maximum separation between x-ray source and catalog counterpart (in arcsec). Default is 3 arcsec.

• r_angle_grating (float) – Maximum r_angle in the case of grating observations (in arcsec). Default is 24 arcsec (0.4 arcmin).

• near_neighbor_dist (float) – Only consider x-ray sources with no other x-ray source within this radial distance (arcsec). Default is 6 arcsec.

• start (CxoTime-compatible timestamp) – Only consider observations after this time. Default is to consider all.

• stop (CxoTime-compatible timestamp) – Only consider observations before this time. Default is to consider all.

• logging_tag (str) – A string to prepend to the logging messages.

observation

class astromon.observation.Observation(obsid, workdir=None, source='arc5gl', ciao_prefix=None, archive_dir=None, archive_regex=None, use_ciao=True)

Class to encapsulate calls to CIAO and arc5gl.

Parameters:

• obsid (int)

• workdir (pathlib.Path or str.) – Top-level working directory. Used to set PFILES and ASCDS_WORK_PATH and to download files. The actual working directory will be {workdir}/obs{int(obsid)//1000:02d}/{obsid}. If not given, the working directory will be a temporary directory.

• source (str) – One of ‘arc5gl’ (to get data using arc5gl) or ‘archive’ (to get data from chandra public archive using CIAO’s download_chandra_obsid).

• archive_dir (pathlib.Path or str.) – Top-level archive directory. Used to permanently store the result files. The actual archive directory will be {archive_dir}/obs{int(obsid)//1000:02d}/{obsid}.

• ciao_prefix (str) – The location of CIAO.

• logger (logging.Logger.) – If not provided, the root logger is used.

• archive_regex (list of str.) – Files matching any regex in the list are archived.

• use_ciao (bool) – If this is False, there will be no calls to CIAO tools (and some methods will just fail).

archive(*regex, destination=None)

Move observation files to an archive location.

Parameters:

• regex (list of str) – Optional. If not given, self.archive_regex is used. Files matching any of the strings are arcived in a long-term location.

• destination (pathlib.Path or str.) – Optional. If not given, self.archive_dir is used. Long-term location where to store files.

calculate_centroids = <astromon.task.Dependent object>

create_archive_symlinks()

Create symlinks from working directory to the archive.

Normally this should not be needed, but it might be convenient so one works only in the working directory. It is a bit slow (about 0.07 seconds, which translates to 5 minutes when creating instances for 5000 observations).

dmcoords(name, **kwargs)

Call dmcoords with the given arguments.

Examples

Get the off-axis angle, given (x, y) “sky” coordinates:

obs.dmcoords(“theta”, option=”sky”, x=4069.94266994267, y=4076.716625716626)

Get the off-axis angle, given (ra, dec) in celestial (“cel”) coordinates:

obs.dmcoords(“theta”, option=”cel”, celfmt=”deg”, ra=20.46451186, dec=-28.34952557)

Parameters:

• name (str) – Name of the output parameter.

• kwargs (dict) – Arguments to pass to dmcoords.

download(ftypes=None, revision=None, force=False)

Download observation files using the Chandra archive or arc5gl.

Parameters:

ftypes (list of str) – Each must be a key in self.archive_file_locations.

filter_events()

Filter x-ray events outside a radius around the optical axis.

filter_sources()

Filter detected sources outside a radius around the optical axis.

get_info()

Get observation info.

Observation info is a combination of OBSPAR (from the obs0 file) and the sequence summary from https://icxc.harvard.edu/cgi-bin/mp/target.cgi.

get_off_axis_angle(ra=None, dec=None, x=None, y=None)

Get APPROXIMATE off-axis angle in arcmin for given (ra, dec) or sky (x, y) coordinates.

This is not the most accurate calculation of the off-axis angle. It uses the coordinates in the sky coordinate system, which is a tangent plane at the nominal pointing. It is still within 1 arcsec of the true off-axis angle for all cases we care about.

Parameters:

• ra (np.array) – Right ascension in degrees.

• dec (np.array) – Declination in degrees.

• x (np.array) – Sky X coordinate in pixels.

• y (np.array) – Sky Y coordinate in pixels.

Returns:

Off-axis angle in arcmin. Same shape as input arrays.

Return type:

np.array

property is_selected

Check if the observation fulfills the requirements for astromon processing.

This function skips:

• observations with obs_mode other than “POINTING”.

• ACIS observations with readmode other than “TIMED” and dtycycle different than 0.

Returns:

True if the observation is suitable for astromon processing, False otherwise.

Return type:

bool

make_images()

Create image. Also creates the exposure map and psfmap.

process()

Main routine to process a single obsid with “standard” steps.

repro()

Reprocess data.

run_celldetect()

Run celldetect.

run_wavdetect()

Run wavdetect.

utils

class astromon.utils.Ciao(prefix=None, workdir=None, logger=None)

Encapsulate calls to CIAO tools.

This class does two things:

• Set the CIAO context (PFILES and ASCDS_WORK_PATH).

• Handle calls to subprocesses.

Parameters:

• prefix (str) – The location of CIAO.

• workdir (pathlib.Path or str) – Working directory. Used to set PFILES and ASCDS_WORK_PATH.

• logger (logging.Logger) – If not provided, the root logger is used.

pget(command, param='value', logging_tag='')

Call CIAO’s pget and return the standard output.

astromon.utils.calalign_from_files(calalign_dir=None)

Get data from calalign files in calalign_dir.

Parameters:

calalign_dir (pathlib.Path) – Directory where to find the calalign files.

Return type:

astropy.table.Table

class astromon.utils.chdir(directory)

Context manager to temporarily change to a directory.

astromon.utils.ciao_context

Context manager to set CIAO context (PFILES and ASCDS_WORK_PATH)

Parameters:

• directory (pathlib.Path or str) – Working directory. Used to set PFILES and ASCDS_WORK_PATH.

• logger (logging.Logger) – If not provided, the root logger is used.

astromon.utils.communicate(process, logger=None, level='WARNING', text=False, logging_tag='')

Real-time reading of a subprocess stdout.

Parameters:

• process – process returned by subprocess.Popen

• logger (logging.Logger)

• level (str or int) – the logging level

• text (bool) – whether the process output is text or binary

astromon.utils.get_calalign_offsets(all_matches, ref_calalign=None, calalign_dir=None)

Get a table with the yag/zag offsets subtracted by the aca_misalign matrix.

The table includes two sets of offsets: the offsets used when the observation was processed, and the offsets one would get when using a reference CALALIGN.

Parameters:

all_matches (astropy.table.Table) – The table of x-ray sources. The only required columns are ‘obsid’ and ‘x_id’

Returns:

A table with keys:

• calalign_dy

• calalign_dz

• aca_misalign

• fts_misalign

• calalign_version

• ref_calalign_dy

• ref_calalign_dz

• ref_aca_misalign

• ref_fts_misalign

• ref_calalign_version

Return type:

astropy.table.Table

astromon.utils.get_near_neighbor_dist(sources, all_sources=None, id_col='COMPONENT')

Calculate the distance from each source in sources to the nearest source in all_sources.

The distance is the euclidean distance in the y_angle/z_angle plane.

If all_sources is None, sources is used for both.

Parameters:

• sources (astropy.table.Table) – The sources to calculate the nearest neighbor distance for.

• all_sources (astropy.table.Table) – The sources to use as potential neighbors.

• id_col (str) – The column to use as the unique identifier for each source.

astromon.utils.logging_call_decorator = <astromon.utils.LoggingCallDecorator object>

Decorator to add logging messages at the start/end of the decorated function.