gwcelery.tasks.detchar module

Data quality and detector characterization tasks.

These tasks are mostly focused on checking interferometer state vectors. By design, the [LIGO] and [Virgo] state vectors share the same definitions for the first 8 fields.

LIGO also has a [DMT] DQ vector that provides some additional instrumental checks.

References

[LIGO]https://wiki.ligo.org/Calibration/TDCalibReview
[Virgo]https://dcc.ligo.org/G1801125/
[DMT]https://wiki.ligo.org/DetChar/DmtDqVector
gwcelery.tasks.detchar.dmt_dq_vector_bits

DMT DQ vector bits (LIGO only).

gwcelery.tasks.detchar.state_vector_bits

State vector bitfield definitions for LIGO and Virgo.

gwcelery.tasks.detchar.no_dq_veto_pycbc_bits

No DQ veto stream bitfield definitions for Virgo. NOTE: Since the results for these bits will be NOT()ed, the bit definitions are the NO_* versions of what the bit * actually is. This is an inelegant but the simplest solution since the logic used in these channels are opposite to those in all the other checked channels.

gwcelery.tasks.detchar.create_cache(ifo)[source]

Find .gwf files and create cache.

Parameters:ifo (str) – Interferometer name (e.g. H1).
Returns:
Return type:glue.lal.Cache

Example

>>> create_cache('H1')
[<glue.lal.CacheEntry at 0x7fbae6b71278>,
  <glue.lal.CacheEntry at 0x7fbae6ae5b38>,
  <glue.lal.CacheEntry at 0x7fbae6ae5c50>,
 ...
  <glue.lal.CacheEntry at 0x7fbae6b15080>,
  <glue.lal.CacheEntry at 0x7fbae6b15828>]

Note that running this example will return an I/O error, since /dev/shm gets overwritten every 300 seconds.

Notes

There are two main ways which this function can fail, which need to be accounted for in the future. The first is that the directory (typically /dev/shm/llhoft) is found, but the files in question corresponding to the timestamp are not in place. This can happen if the function is late to the game, and hence the data have been deleted from memory and are no longer stored in /dev/shm/llhoft. It can also happen if through some asynchronous processes, the call is early, and the data files have not yet been written to /dev/shm/llhoft. The second way is if /dev/shm/llhoft is not found and hence data never shows up.

In these cases, the desired behaviour will be for the function to wait a period of ~5 seconds and try again. If it still returns an I/O error of this type, then the function will return a flag and stop trying (this can happen by setting a maximum number of retries to 1).

This is important for if gwcelery is run locally (and not on a cluster), where /dev/shm is inaccessible.

gwcelery.tasks.detchar.dqr_json(state, summary)[source]

Generate DQR-compatible json-ready dictionary from process results, as described in data-quality-report.design.

Parameters:
  • state ({'pass', 'fail'}) – State of the detchar checks.
  • summary (str) – Summary of results from the process.
Returns:

Ready to be converted into json.
Return type:

dict
gwcelery.tasks.detchar.check_idq(cache, channel, start, end)[source]

Looks for iDQ frame and reads them.

Parameters:
  • cache (glue.lal.Cache) – Cache from which to check.
  • channel (str) – which idq channel (pglitch)
  • end (start,) – GPS start and end times desired.
Returns:

Tuple mapping iDQ channel to its maximum P(glitch).
Return type:

tuple

Example

>>> check_idq(cache, 'H1:IDQ-PGLITCH-OVL-100-1000',
              1216496260, 1216496262)
('H1:IDQ-PGLITCH-OVL-100-1000', 0.87)
gwcelery.tasks.detchar.check_vector(cache, channel, start, end, bits, logic_type='all')[source]

Check timeseries of decimals against a bitmask. This is inclusive of the start time and exclusive of the end time, i.e. [start, …, end).

Parameters:
  • cache (glue.lal.Cache) – Cache from which to check.
  • channel (str) – Channel to look at, e.g. H1:DMT-DQ_VECTOR.
  • end (start,) – GPS start and end times desired.
  • bits (gwpy.TimeSeries.Bits) – Definitions of the bits in the channel.
  • logic_type (str, optional) – Type of logic to apply for vetoing. If all, then all samples in the window must pass the bitmask. If any, then one or more samples in the window must pass.
Returns:

Maps each bit in channel to its state.
Return type:

dict

Example

>>> check_vector(cache, 'H1:GDS-CALIB_STATE_VECTOR', 1216496260,
                 1216496262, state_vector_bits)
{'H1:HOFT_OK': True,
 'H1:OBSERVATION_INTENT': True,
 'H1:NO_STOCH_HW_INJ': True,
 'H1:NO_CBC_HW_INJ': True,
 'H1:NO_BURST_HW_INJ': True,
 'H1:NO_DETCHAR_HW_INJ': True}
(task)gwcelery.tasks.detchar.check_vectors(event, graceid, start, end)[source]

Perform data quality checks for an event and labels/logs results to GraceDb.

Depending on the pipeline, a certain amount of time (specified in check_vector_prepost) is appended to either side of the superevent start and end time. This is to catch DQ issues slightly before and after the event, such as that appearing in L1 just before GW170817.

A cache is then created for H1, L1, and V1, regardless of the detectors involved in the event. Then, the bits and channels specified in the configuration file (llhoft_channels) are checked. If an injection is found in the active detectors, ‘INJ’ is labeled to GraceDb. If an injection is found in any detector, a message with the injection found is logged to GraceDb. If no injections are found across all detectors, this is logged to GraceDb.

A similar task is performed for the DQ states described in the DMT-DQ_VECTOR, LIGO GDS-CALIB_STATE_VECTOR, and Virgo DQ_ANALYSIS_STATE_VECTOR. If no DQ issues are found in active detectors, ‘DQOK’ is labeled to GraceDb. Otherwise, ‘DQV’ is labeled. In all cases, the DQ states of all the state vectors checked are logged to GraceDb.

This skips MDC events.

Parameters:
  • event (dict) – Details of event.
  • graceid (str) – GraceID of event to which to log.
  • end (start,) – GPS start and end times desired.