gwcelery.conf module

GWCelery application configuration.

This module defines configuration variables and default values, including both generic options for Celery as well as options that control the behavior of specific GWCelery tasks.

To override the configuration, define the CELERY_CONFIG_MODULE environment variable to the fully qualified name of any Python module that can be located in sys.path, including any of the following presets:

gwcelery.conf.worker_proc_alive_timeout = 8

The timeout when waiting for a new worker process to start up.

gwcelery.conf.worker_log_format = '[%(asctime)s: %(levelname)s/%(processName)s/%(threadName)s] %(message)s'

Custom worker log format that includes the thread name.

gwcelery.conf.condor_accounting_group = 'ligo.dev.o4.cbc.pe.bayestar'

HTCondor accounting group for Celery workers launched with condor_submit.

gwcelery.conf.expose_to_public = False

Set to True if events meeting the public alert threshold really should be exposed to the public.

gwcelery.conf.igwn_alert_server = None

None == DEFAULT_SERVER

Type:

IGWN alert server

gwcelery.conf.igwn_alert_noauth = False

IGWN alert server no-authetication

gwcelery.conf.igwn_alert_group = 'gracedb-playground'

IGWN alert group.

gwcelery.conf.gracedb_host = 'gracedb-playground.ligo.org'

GraceDB host.

gwcelery.conf.voevent_broadcaster_address = ':5342'

The VOEvent broker will bind to this address to send GCNs. This should be a string of the form host:port. If host is empty, then listen on all available interfaces.

gwcelery.conf.voevent_broadcaster_whitelist = []

List of hosts from which the broker will accept connections. If empty, then completely disable the broker’s broadcast capability.

gwcelery.conf.voevent_receiver_address = '68.169.57.253:8099'

The VOEvent listener will connect to this address to receive GCNs. For options, see GCN’s list of available VOEvent servers. If this is an empty string, then completely disable the GCN listener.

gwcelery.conf.email_host = 'imap.gmail.com'

IMAP hostname to receive the GCN e-mail notice formats.

gwcelery.conf.superevent_d_t_start = {'gstlal': 1.0, 'mbta': 1.0, 'pycbc': 1.0, 'spiir': 1.0}

Pipeline based lower extent of superevent segments. For cwb and lib this is decided from extra attributes.

gwcelery.conf.superevent_d_t_end = {'gstlal': 1.0, 'mbta': 1.0, 'pycbc': 1.0, 'spiir': 1.0}

Pipeline based upper extent of superevent segments For cwb and lib this is decided from extra attributes.

gwcelery.conf.superevent_query_d_t_start = 100.0

Lower extent of superevents query

gwcelery.conf.superevent_query_d_t_end = 100.0

Upper extent of superevents query

gwcelery.conf.superevent_default_d_t_start = 1.0

Default lower extent of superevent segments

gwcelery.conf.superevent_default_d_t_end = 1.0

Default upper extent for superevent segments

gwcelery.conf.superevent_far_threshold = 0.0002777777777777778

Maximum false alarm rate to consider events superevents.

gwcelery.conf.significant_alert_far_threshold = {'burst': 3.1709791983764586e-08, 'cbc': 3.8580246913580245e-07, 'test': 3.8580246913580245e-07}

Group specific maximum false alarm rate to consider sending significant alerts.

gwcelery.conf.preliminary_alert_far_threshold = {'burst': 2.3148148148148147e-05, 'cbc': 2.3148148148148147e-05, 'test': 2.3148148148148147e-05}

Group specific maximum false alarm rate to consider sending less significant alerts.

gwcelery.conf.early_warning_alert_trials_factor = 4.0

Trials factor for early warning alerts. There are two pipelines that are producing early warning events: gstlal and spiir.

gwcelery.conf.early_warning_alert_far_threshold = 3.8580246913580245e-07

False alarm rate threshold for significant early warning alerts.

gwcelery.conf.significant_alert_trials_factor = {'burst': 4.0, 'cbc': 5.0}

Trials factor corresponding to trigger categories. For CBC and Burst, trials factor is the number of pipelines. CBC pipelines are gstlal, pycbc, mbta and spiir. Burst searches are cwb.allsky, cwb.bbh and cwb.imbh.

gwcelery.conf.preliminary_alert_trials_factor = {'burst': 8.0, 'cbc': 8.0}

Trials factor for less significant alerts.

gwcelery.conf.snews_gw_far_threshold = 1.1574074074074073e-05

Maximum false alarm rate for a superevent to send out a coincidence alert between an external SNEWS alert and the superevent.

gwcelery.conf.sog_paper_far_threshold = {'gw': 3.1709791983764586e-11, 'joint': 3.1709791983764588e-12}

False alarm rate thresholds for producing a manuscript of speed of gravity measurement in low-latency.

gwcelery.conf.superevent_clean_up_timeout = 270.0

The orchestrator will wait this many seconds from the time of the application of the GCN_PRELIM_SENT label to revise the preferred event out of the accumulated events.

gwcelery.conf.pe_timeout = 345.0

The orchestrator will wait this many seconds from the time of the creation of a new superevent to the time that parameter estimation begins, in case the preferred event is updated with high latency.

gwcelery.conf.check_vector_prepost = {'AGILE': [2, 2], 'CWB': [1.5, 1.5], 'Fermi': [2, 2], 'HardwareInjection': [2, 2], 'INTEGRAL': [2, 2], 'LIB': [1.5, 1.5], 'MBTA': [2, 2], 'SNEWS': [10, 10], 'Swift': [2, 2], 'gstlal': [2, 2], 'oLIB': [1.5, 1.5], 'pycbc': [2, 2], 'spiir': [2, 2]}

Seconds before and after the superevent start and end times which the DQ vector check will include in its check. Pipeline dependent.

gwcelery.conf.uses_gatedhoft = {'AGILE': False, 'CWB': True, 'Fermi': False, 'HardwareInjection': False, 'INTEGRAL': False, 'LIB': False, 'MBTA': True, 'SNEWS': False, 'Swift': False, 'gstlal': True, 'oLIB': False, 'pycbc': True, 'spiir': True}

Whether or not a pipeline uses gated h(t). Determines whether or not the DMT-DQ_VECTOR will be analyzed for data quality.

gwcelery.conf.llhoft_glob = '/dev/shm/kafka/{detector}_O3ReplayMDC/*.gwf'

File glob for playground low-latency h(t) frames. Currently points to O3 MDC Mock Data Challange data. See https://git.ligo.org/emfollow/mock-data-challenge

gwcelery.conf.llhoft_channels = {'H1:DMT-DQ_VECTOR': 'dmt_dq_vector_bits', 'H1:GDS-CALIB_STATE_VECTOR': 'ligo_state_vector_bits', 'L1:DMT-DQ_VECTOR': 'dmt_dq_vector_bits', 'L1:GDS-CALIB_STATE_VECTOR': 'ligo_state_vector_bits', 'V1:DQ_ANALYSIS_STATE_VECTOR': 'virgo_state_vector_bits'}

Low-latency h(t) state vector configuration. This is a dictionary consisting of a channel and its bitmask, as defined in gwcelery.tasks.detchar.

gwcelery.conf.idq_channels = ['H1:IDQ-FAP_OVL_16_4096', 'L1:IDQ-FAP_OVL_16_4096']

Low-latency iDQ false alarm probability channel names from O3 replay.

gwcelery.conf.idq_fap_thresh = 0.01

If FAP is below this threshold, and idq_veto for the pipeline is true, DQV will be labeled for the event.

gwcelery.conf.idq_veto = {'AGILE': False, 'CWB': False, 'Fermi': False, 'HardwareInjection': False, 'INTEGRAL': False, 'LIB': False, 'MBTA': False, 'SNEWS': False, 'Swift': False, 'gstlal': False, 'oLIB': False, 'pycbc': False, 'spiir': False}

If true for a pipeline, iDQ values below the threshold defined in idq_fap_thresh will cause DQV to be labeled. Currently all False, pending iDQ review (should be done before O3).

gwcelery.conf.low_latency_frame_types = {'H1': 'H1_O3ReplayMDC_llhoft', 'L1': 'L1_O3ReplayMDC_llhoft', 'V1': 'V1_O3ReplayMDC_llhoft'}

Types of low latency frames used in Parameter Estimation (see gwcelery.tasks.inference) and in cache creation for detchar checks (see gwcelery.tasks.detchar).

gwcelery.conf.high_latency_frame_types = {'H1': None, 'L1': None, 'V1': None}

Types of high latency frames used in Parameter Estimation and in cache creation for detchar checks. They do not exist for O3Replay data. (see gwcelery.tasks.inference and gwcelery.tasks.detchar)

gwcelery.conf.strain_channel_names = {'H1': 'H1:GDS-CALIB_STRAIN_INJ1_O3Replay', 'L1': 'L1:GDS-CALIB_STRAIN_INJ1_O3Replay', 'V1': 'V1:Hrec_hoft_16384Hz_INJ1_O3Replay'}

Names of h(t) channels used in Parameter Estimation (see gwcelery.tasks.inference) and in detchar omegascan creation (see gwcelery.tasks.detchar).

gwcelery.conf.state_vector_channel_names = {'H1': 'H1:GDS-CALIB_STATE_VECTOR', 'L1': 'L1:GDS-CALIB_STATE_VECTOR', 'V1': 'V1:DQ_ANALYSIS_STATE_VECTOR'}

Names of state vector channels used in Parameter Estimation (see gwcelery.tasks.inference)

gwcelery.conf.detchar_bit_definitions = {'dmt_dq_vector_bits': {'bits': {1: 'NO_OMC_DCPD_ADC_OVERFLOW', 2: 'NO_DMT-ETMY_ESD_DAC_OVERFLOW'}, 'channel': 'DMT-DQ_VECTOR', 'description': {'NO_DMT-ETMY_ESD_DAC_OVERFLOW': 'ETMY ESD DAC not overflowing', 'NO_OMC_DCPD_ADC_OVERFLOW': 'OMC DCPC ADC not overflowing'}}, 'ligo_state_vector_bits': {'bits': {0: 'HOFT_OK', 1: 'OBSERVATION_INTENT', 5: 'NO_STOCH_HW_INJ', 6: 'NO_CBC_HW_INJ', 7: 'NO_BURST_HW_INJ', 8: 'NO_DETCHAR_HW_INJ'}, 'channel': 'GDS-CALIB_STATE_VECTOR', 'description': {'HOFT_OK': 'h(t) was successfully computed', 'NO_BURST_HW_INJ': 'No burst HW injection', 'NO_CBC_HW_INJ': 'No CBC HW injection', 'NO_DETCHAR_HW_INJ': 'No HW injections for detector characterization', 'NO_STOCH_HW_INJ': 'No stochastic HW injection', 'OBSERVATION_INTENT': '"observation intent" button is pushed'}}, 'virgo_state_vector_bits': {'bits': {0: 'HOFT_OK', 1: 'OBSERVATION_INTENT', 5: 'NO_STOCH_HW_INJ', 6: 'NO_CBC_HW_INJ', 7: 'NO_BURST_HW_INJ', 8: 'NO_DETCHAR_HW_INJ', 10: 'GOOD_DATA_QUALITY_CAT1'}, 'channel': 'DQ_ANALYSIS_STATE_VECTOR', 'description': {'GOOD_DATA_QUALITY_CAT1': 'Good data quality (CAT1 type)', 'HOFT_OK': 'h(t) was successfully computed', 'NO_BURST_HW_INJ': 'No burst HW injection', 'NO_CBC_HW_INJ': 'No CBC HW injection', 'NO_DETCHAR_HW_INJ': 'No HW injections for detector characterization', 'NO_STOCH_HW_INJ': 'No stochastic HW injection', 'OBSERVATION_INTENT': '"observation intent" button is pushed'}}}

Bit definitions for detchar checks

gwcelery.conf.omegascan_durations = [(0.75, 0.25), (1.5, 0.5), (7.5, 2.5)]

Durations for omegascans, before and after t0

gwcelery.conf.pe_results_path = '/home/docs/public_html/online_pe'

Path to the results of Parameter Estimation (see gwcelery.tasks.inference)

gwcelery.conf.pe_results_url = 'https://ldas-jobs.ligo.caltech.edu/~docs/online_pe/'

URL of page where all the results of Parameter Estimation are outputted (see gwcelery.tasks.inference)

gwcelery.conf.raven_coincidence_windows = {'GRB_Burst': [-600, 60], 'GRB_CBC': [-5, 1], 'GRB_CBC_SubFermi': [-11, 1], 'GRB_CBC_SubSwift': [-20, 10], 'SNEWS': [-10, 10]}

Time coincidence windows passed to ligo-raven. External events and superevents of the appropriate type are considered to be coincident if within time window of each other.

gwcelery.conf.mock_events_simulate_multiple_uploads = False

If True, then upload each mock event several times in rapid succession with random jitter in order to simulate multiple pipeline uploads.

gwcelery.conf.only_alert_for_mdc = False

If True, then only sends alerts for MDC events. Useful for times outside of observing runs.

gwcelery.conf.joint_mdc_freq = 2

Determines how often an external MDC event will be created near an MDC superevent to test the RAVEN alert pipeline, i.e for every x MDC superevents an external MDC event is created.

gwcelery.conf.joint_O3_replay_freq = 10

Determines how often an external replay event will be created near an superevent to test the RAVEN alert pipeline, i.e for every x O3 replay superevents an external MDC event is created.

gwcelery.conf.bilby_default_mode = 'fast_test'

Sampling mode of bilby

gwcelery.conf.condor_retry_kwargs = {'max_retries': 45, 'retry_backoff': True, 'retry_backoff_max': 600, 'retry_jitter': False}

Retry settings of condor.submit task. With these settings, a condor job is no longer tracked ~6 hours after it starts.

gwcelery.conf.development module

Application configuration for gracedb-dev1.ligo.org.

Inherits all settings from gwcelery.conf.test, with the exceptions below.

gwcelery.conf.development.gracedb_host = 'gracedb-dev1.ligo.org'

GraceDB host.

gwcelery.conf.development.sentry_environment = 'development'

Record this environment tag in Sentry log messages.

gwcelery.conf.playground module

Application configuration for gracedb-playground.ligo.org.

gwcelery.conf.playground.sentry_environment = 'playground'

Record this environment tag in Sentry log messages.

gwcelery.conf.playground.mock_events_simulate_multiple_uploads = True

If True, then upload each mock event several times in rapid succession with random jitter in order to simulate multiple pipeline uploads.

gwcelery.conf.playground.kafka_alert_config = {'gcn': {'skymap_encoder': <function <lambda>>, 'suffix': 'json', 'url': 'kafka://kafka.test.gcn.nasa.gov/igwn.gwalert'}, 'scimma': {'skymap_encoder': <function <lambda>>, 'suffix': 'avro', 'url': 'kafka://kafka.scimma.org/igwn.gwalert-playground'}}

Kafka broker configuration details

gwcelery.conf.playground.voevent_broadcaster_address = ':5341'

The VOEvent broker will bind to this address to send GCNs. This should be a string of the form host:port. If host is empty, then listen on all available interfaces.

gwcelery.conf.playground.voevent_broadcaster_whitelist = ['capella2.gsfc.nasa.gov']

List of hosts from which the broker will accept connections. If empty, then completely disable the broker’s broadcast capability.

gwcelery.conf.playground.voevent_receiver_address = '50.116.49.68:8094'

The VOEvent listener will connect to this address to receive GCNs. For options, see GCN’s list of available VOEvent servers. If this is an empty string, then completely disable the GCN listener.

gwcelery.conf.production module

Application configuration for gracedb.ligo.org.

Inherits all settings from gwcelery.conf.playground, with the exceptions below.

gwcelery.conf.production.condor_accounting_group = 'ligo.prod.o4.cbc.pe.bayestar'

HTCondor accounting group for Celery workers launched with condor_submit.

gwcelery.conf.production.expose_to_public = True

Set to True if events meeting the public alert threshold really should be exposed to the public.

gwcelery.conf.production.igwn_alert_group = 'gracedb'

IGWN alert group.

gwcelery.conf.production.gracedb_host = 'gracedb.ligo.org'

GraceDB host.

gwcelery.conf.production.kafka_alert_config = {'gcn': {'skymap_encoder': <function <lambda>>, 'suffix': 'json', 'url': 'kafka://kafka.gcn.nasa.gov/igwn.gwalert'}, 'scimma': {'skymap_encoder': <function <lambda>>, 'suffix': 'avro', 'url': 'kafka://kafka.scimma.org/igwn.gwalert'}}

Kafka broker configuration details

gwcelery.conf.production.voevent_broadcaster_address = ':5341'

The VOEvent broker will bind to this address to send GCNs. This should be a string of the form host:port. If host is empty, then listen on all available interfaces.

gwcelery.conf.production.voevent_broadcaster_whitelist = ['capella2.gsfc.nasa.gov']

List of hosts from which the broker will accept connections. If empty, then completely disable the broker’s broadcast capability.

gwcelery.conf.production.llhoft_glob = '/dev/shm/kafka/{detector}/*.gwf'

File glob for low-latency h(t) frames.

gwcelery.conf.production.low_latency_frame_types = {'H1': 'H1_llhoft', 'L1': 'L1_llhoft', 'V1': 'V1_llhoft'}

Types of frames used in Parameter Estimation (see gwcelery.tasks.inference) and in cache creation for detchar checks (see gwcelery.tasks.detchar).

gwcelery.conf.production.high_latency_frame_types = {'H1': 'H1_HOFT_C00', 'L1': 'L1_HOFT_C00', 'V1': 'V1Online'}

Types of high latency frames used in Parameter Estimation (see gwcelery.tasks.inference) and in cache creation for detchar checks (see gwcelery.tasks.detchar).

gwcelery.conf.production.idq_channels = ['H1:IDQ-FAP_OVL_16_4096', 'L1:IDQ-FAP_OVL_16_4096']

Low-latency iDQ false alarm probability channel names from live O3 frames

gwcelery.conf.production.strain_channel_names = {'H1': 'H1:GDS-CALIB_STRAIN_CLEAN', 'L1': 'L1:GDS-CALIB_STRAIN_CLEAN', 'V1': 'V1:Hrec_hoft_16384Hz'}

Names of h(t) channels used in Parameter Estimation (see gwcelery.tasks.inference)

gwcelery.conf.production.sentry_environment = 'production'

Record this environment tag in Sentry log messages.

gwcelery.conf.production.only_alert_for_mdc = True

If True, then only sends alerts for MDC events. Useful for times outside of observing runs.

gwcelery.conf.production.bilby_default_mode = 'production'

Sampling mode of bilby

gwcelery.conf.production.condor_retry_kwargs = {'max_retries': None, 'retry_backoff': True, 'retry_backoff_max': 600, 'retry_jitter': True}

Retry settings of condor.submit task.

gwcelery.conf.test module

Application configuration for gracedb-test.ligo.org.

Inherits all settings from gwcelery.conf.playground, with the exceptions below.

gwcelery.conf.test.igwn_alert_group = 'gracedb-test'

IGWN alert group.

gwcelery.conf.test.gracedb_host = 'gracedb-test.ligo.org'

GraceDB host.

gwcelery.conf.test.kafka_alert_config = {'gcn': {'skymap_encoder': <function <lambda>>, 'suffix': 'json', 'url': 'kafka://kafka.dev.gcn.nasa.gov/igwn.gwalert'}, 'scimma': {'skymap_encoder': <function <lambda>>, 'suffix': 'avro', 'url': 'kafka://kafka.scimma.org/igwn.gwalert-test'}}

Kafka broker configuration details

gwcelery.conf.test.sentry_environment = 'test'

Record this environment tag in Sentry log messages.

gwcelery.conf.test.mock_events_simulate_multiple_uploads = True

If True, then upload each mock event several times in rapid succession with random jitter in order to simulate multiple pipeline uploads.