gwcelery.tasks.orchestrator module¶
This module implements the alert orchestrator, which responsible for the vetting and annotation workflow to produce preliminary, initial, and update alerts for gravitational-wave event candidates.
The orchestrator consists of two IGWN Alert message handlers:
handle_superevent()
is called for each superevent. It waits for a short duration oforchestrator_timeout
seconds for the selection of the superevent by thesuperevent manager
to stabilize, then performs data quality checks. If the data quality checks pass, then it callsearlywarning_preliminary_alert()
to copy annotations from the preferred event and send the preliminary notice.handle_cbc_event()
is called for each CBC event. It performs some CBC-specific annotations that depend closely on the CBC matched-filter parameters estimates and that might influence selection of the preferred event: rapid sky localization with BAYESTAR and rapid source classification.Note that there is no equivalent of this task for burst events because both burst searches (cWB, LIB) have integrated source localization and have no other annotations.
Preliminary Alerts¶
The flow chart below illustrates the operation of these two tasks.
Initial and Update Alerts¶
The initial_alert()
and
update_alert()
tasks create Initial and Update alerts
respectively. At the moment, there is no handler or user interface to trigger
these tasks, and they must be invoked manually (see
Command-Line Tools). A flow chart for the initial alerts is
shown below; the flow chart for update alerts is the same.
Retraction Alerts¶
Likewise, the retraction_alert()
task creates Retraction
alerts, and at the moment must be invoked manually. A flow chart is shown below.
Tasks¶
Tasks that comprise the alert orchestrator.
The orchestrator is responsible for the vetting and annotation workflow to produce preliminary, initial, and update alerts for gravitational-wave event candidates.
- (task)gwcelery.tasks.orchestrator.handle_superevent(alert)[source]¶
Schedule annotations for new superevents.
After waiting for a time specified by the
orchestrator_timeout
configuration variable for the choice of preferred event to settle down, this task performs data quality checks withgwcelery.tasks.detchar.check_vectors()
and callsearlywarning_preliminary_alert()
to send a preliminary notice.
- (task)gwcelery.tasks.orchestrator.handle_cbc_event(alert)[source]¶
Perform annotations for CBC events that depend on pipeline-specific matched-filter parameter estimates.
Notes
This IGWN alert message handler is triggered by a new upload or by updates that include the file
pipeline.p_astro.json
. If also generates pipeline.p_astro.json information for pipelines that do not provide such information.The table below lists which files are created as a result of a new upload, and which tasks generate them.
File
Task
bayestar.multiorder.fits
em_bright.json
pipeline.p_astro.json
- (task)gwcelery.tasks.orchestrator.handle_burst_event(alert)[source]¶
Perform annotations for burst events that depend on pipeline-specific
- (task)gwcelery.tasks.orchestrator.handle_posterior_samples(alert)[source]¶
Generate multi-resolution and flat-resolution FITS files and skymaps from an uploaded HDF5 file containing posterior samples.
- (task)gwcelery.tasks.orchestrator._create_mattermost_channel(superevent_id)[source]¶
Creates a mattermost channel when ADVREQ label is applied and posts a cooresponding gracedb link of that event in the channel
Channel name : O4 RRT {superevent_id}
Parameters:¶
- superevent_id: str
The superevent id
- (task)gwcelery.tasks.orchestrator._set_pipeline_preferred_events(pipeline_event, superevent_id)[source]¶
Return group for setting pipeline preferred event using
gracedb.add_pipeline_preferred_event()
.
- (task)gwcelery.tasks.orchestrator._update_if_dqok(event, superevent_id)[source]¶
Update preferred_event of superevent_id to event_id if DQOK label has been applied.
- (task)gwcelery.tasks.orchestrator._create_voevent(classification, *args, **kwargs)[source]¶
Create a VOEvent record from an EM bright JSON file.
- Parameters:
classification (tuple, None) – A collection of JSON strings, generated by
gwcelery.tasks.em_bright.source_properties()
andgwcelery.tasks.p_astro.compute_p_astro()
or content of{gstlal,mbta}.p_astro.json
uploaded by {gstlal,mbta} respectively; or None*args – Additional positional arguments passed to
gwcelery.tasks.gracedb.create_voevent()
.**kwargs – Additional keyword arguments passed to
gwcelery.tasks.gracedb.create_voevent()
.
- Returns:
The filename of the newly created VOEvent.
- Return type:
- (task)gwcelery.tasks.orchestrator._create_label_and_return_filename(filename, label, graceid)[source]¶
Proxy that evaluates object once.
Proxy
will evaluate the object each time, while the promise will only evaluate it once.
- (task)gwcelery.tasks.orchestrator._leave_log_message_and_return_event_dict(event, superevent_id, message, **kwargs)[source]¶
Wrapper around
gracedb.upload()
that returns the event dictionary.
- (task)gwcelery.tasks.orchestrator._update_superevent_and_return_event_dict(event, superevent_id)[source]¶
Wrapper around
gracedb.update_superevent()
that returns the event dictionary.
- (task)gwcelery.tasks.orchestrator._proceed_if_not_blocked_by(files, superevent_id, block_by)[source]¶
Return files in case the superevent does not have labels block_by
- (task)gwcelery.tasks.orchestrator._revise_and_send_second_less_significant_alert(alert, query, superevent_id)[source]¶
Proxy that evaluates object once.
Proxy
will evaluate the object each time, while the promise will only evaluate it once.
- (task)gwcelery.tasks.orchestrator._annotate_fits_and_return_input(input_list, superevent_id)[source]¶
Unpack the output of the skymap, embright, p-astro download group in the beginning of the
earlywarning_preliminary_alert()
canvas and callannotate_fits()
.- Parameters:
input_list (list) – The output of the group that downloads the skymap, embright, and p-astro files. This list is in the form [skymap, skymap_filename], [em_bright, em_bright_filename], [p_astro_dict, p_astro_filename], though the em-bright and p-astro lists can be populated by Nones
superevent_id (str) – A list of the sky map, em_bright, and p_astro filenames.
- (task)gwcelery.tasks.orchestrator._unpack_args_and_send_earlywarning_preliminary_alert(input_list, alert, alert_type)[source]¶
Unpack the output of the skymap, embright, p-astro download group in the beginning of the
earlywarning_preliminary_alert()
canvas and callgwcelery.tasks.orchestrator.earlywarning_preliminary_initial_update_alert()
.- Parameters:
input_list (list) – The output of the group that downloads the skymap, embright, and p-astro files. This list is in the form [skymap, skymap_filename], [em_bright, em_bright_filename], [p_astro_dict, p_astro_filename], though the em-bright and p-astro lists can be populated by Nones
alert (dict) – IGWN-Alert dictionary
alert_type (str) – alert_type passed to
earlywarning_preliminary_initial_update_alert()
- (task)gwcelery.tasks.orchestrator.earlywarning_preliminary_alert(event, alert, alert_type='preliminary', initiate_voevent=True)[source]¶
Produce a preliminary alert by copying any sky maps.
This consists of the following steps:
Copy any sky maps and source classification from the preferred event to the superevent.
Create standard annotations for sky maps including all-sky plots by calling
gwcelery.tasks.skymaps.annotate_fits()
.Create a preliminary VOEvent.
Send the VOEvent to GCN and notices to SCiMMA and GCN.
Apply the GCN_PRELIM_SENT or LOW_SIGNIF_PRELIM_SENT depending on the significant or less-significant alert respectively.
Create and upload a GCN Circular draft.
- (task)gwcelery.tasks.orchestrator._get_pe_far_and_event(superevent)[source]¶
Return FAR and event input to PE workflow.
The input FAR is the lowest FAR among CBC and Burst-BBH triggers. The input event is the preferred event if it is a CBC trigger, otherwise the CBC trigger with the lowest FAR is returned.
- (task)gwcelery.tasks.orchestrator.parameter_estimation(far_event, superevent_id, pe_pipeline)[source]¶
Parameter Estimation with Bilby and RapidPE-RIFT. Parameter estimation runs are triggered for CBC triggers which pass the FAR threshold and are not mock uploads. For those which do not pass these criteria, this task uploads messages explaining why parameter estimation is not started.
- (task)gwcelery.tasks.orchestrator.earlywarning_preliminary_initial_update_alert(filenames, superevent, alert_type, filecontents=None)[source]¶
Create a canvas that sends an earlywarning, preliminary, initial, or update notice.
- Parameters:
filenames (tuple) – A list of the sky map, em_bright, and p_astro filenames.
superevent (dict) – The superevent dictionary, typically obtained from an IGWN Alert or from querying GraceDB.
alert_type ({'less-significant', 'earlywarning', 'preliminary', 'initial', 'update'} # noqa: E501) – The alert type.
Notes
Tasks that call this function should be decorated with
gwcelery.tasks.gracedb.task
rather thangwcelery.app.task
so that a synchronous call togwcelery.tasks.gracedb.get_log()
is retried in the event of GraceDB API failures. If EM_COINC is in labels will create a RAVEN circular.
- (task)gwcelery.tasks.orchestrator._update_rapidpe_pastro(input_list, em_bright=None, pipeline_pastro=None)[source]¶
If p_terr from rapidpe is different from the p_terr from the most recent preferred event, replaces rapidpe’s p_terr with pipeline p_terr. Returns a tuple of em_bright, rapidpe pastro and a boolean(rapidpe_pastro_updated) indicating if rapidpe pastro has been updated. If p_terr in rapidpe has been updated, the return list contains the updated pastro and the rapidpe_pastro_updated is True. Else, the return list contains the rapidpe pastro from the input_list and rapidpe_pastro_updated is False.
- (task)gwcelery.tasks.orchestrator._update_rapidpe_pastro_shouldnt_run()[source]¶
Proxy that evaluates object once.
Proxy
will evaluate the object each time, while the promise will only evaluate it once.
- (task)gwcelery.tasks.orchestrator._upload_rapidpe_pastro_json(input_list, superevent_id, rapidpe_pastro_filename)[source]¶
Add public tag to RapidPE_RIFT.p_astro.json if p_terr from the preferred event is same as the p_terr in RapidPE_RIFT.p_astro.json. Else, uploads an updated version of RapidPE_RIFT.p_astro.json with file content from the task update_rapidpe_pastro.
- (task)gwcelery.tasks.orchestrator._check_pastro_and_send_alert(input_classification, skymap, em_bright, superevent, alert_type, raven_coinc=False)[source]¶
Wrapper for
send()
meant to take a potentially new p-astro as input from the preceding task.
- (task)gwcelery.tasks.orchestrator.initial_alert(filenames, alert)[source]¶
Produce an initial alert.
This does nothing more than call
earlywarning_preliminary_initial_update_alert()
withalert_type='initial'
.- Parameters:
Notes
This function is decorated with
gwcelery.tasks.gracedb.task
rather thangwcelery.app.task
so that a synchronous call togwcelery.tasks.gracedb.get_log()
is retried in the event of GraceDB API failures.
- (task)gwcelery.tasks.orchestrator.update_alert(filenames, superevent_id)[source]¶
Produce an update alert.
This does nothing more than call
earlywarning_preliminary_initial_update_alert()
withalert_type='update'
.- Parameters:
Notes
This function is decorated with
gwcelery.tasks.gracedb.task
rather thangwcelery.app.task
so that a synchronous call togwcelery.tasks.gracedb.get_log()
is retried in the event of GraceDB API failures.