gwcelery.tasks.external_triggers module

This module listens to the GCN VOevent XML notices from SNEWS and the Fermi, Swift, and SVOM missions, as well as Kafka JSON alerts from CHIME and the targeted searches from Fermi and Swift. It is also responsible for carrying out tasks related to external trigger-gravitational wave coincidences, including looking for temporal coincidences, creating combined GRB-GW sky localization probability maps, and computing their joint temporal and spatio-temporal false alarm rates.

There are two GCN, one Kafka, and three IGWN Alert message handlers in the gwcelery.tasks.external_triggers module:

Swift, CHIME_, and IceCube.

Flow charts

GCN VOEvent Ingestion

digraph exttrig { compound = true nodesep = 0.1 ranksep = 0.1 node [ fillcolor = white shape = box style = filled target = "_top" ] graph [ labeljust = "left" style = filled target = "_top" ] SNEWS_GCN [ style="rounded" label="SNEWS GCN recieved" ] GRB_GCN [ style="rounded" label="GRB\nGCN recieved" ] subgraph cluster_gcn_handle { href = "../gwcelery.tasks.external_triggers.html#gwcelery.tasks.external_triggers.handle_grb_gcn" label = <<B><FONT face="monospace">handle_gcn</FONT></B>> Ignore_gcn [ label="Ignore" ] Likely_noise [ shape=diamond label="Is the event\nlikely non-astrophysical?" ] Event_exists_in_Gracedb [ shape=diamond label="Does the event already\nexist in GraceDB" ] Update_existing_event_in_gracedb [ label="Update the existing\nevent in GraceDB" ] Create_new_event_in_gracedb [ label="Create a new event\nin GraceDB" ] Grab_create_skymap [ label="Grab and/or\ncreate external sky map" ] Launch_detchar_tasks [ label="Launch detector\ncharacterization checks\naround data" ] } SNEWS_GCN -> Likely_noise [ lhead = cluster_gcn_handle ] GRB_GCN -> Likely_noise [ lhead = cluster_gcn_handle ] Likely_noise -> Event_exists_in_Gracedb[label="no"] Likely_noise -> Ignore_gcn[label="yes"] Event_exists_in_Gracedb -> Update_existing_event_in_gracedb[label="yes"] Event_exists_in_Gracedb -> Create_new_event_in_gracedb[label="no"] Update_existing_event_in_gracedb -> Grab_create_skymap Create_new_event_in_gracedb -> Grab_create_skymap Create_new_event_in_gracedb -> Launch_detchar_tasks }

Kafka Alert Ingestion

digraph exttrig { compound = true nodesep = 0.1 ranksep = 0.1 node [ fillcolor = white shape = box style = filled target = "_top" ] graph [ labeljust = "left" style = filled target = "_top" ] KAFKA_FERMI [ style="rounded" label="Fermi Kafka\nalert recieved" ] KAFKA_SWIFT [ style="rounded" label="Swift Kafka\nalert recieved" ] KAFKA_CHIME [ style="rounded" label="CHIME Kafka\nalert recieved" ] subgraph cluster_kafka_handle { href = "../gwcelery.tasks.external_triggers.html#gwcelery.tasks.external_triggers.handle_kafka_alert" label = <<B><FONT face="monospace">handle_kafka_alert</FONT></B>> Ignore_gcn [ label="Mark with NOT_GRB label\nto prevent publication" ] Likely_noise [ shape=diamond label="Is the FAR too high\nor a retraction notice?" ] Event_exists_in_Gracedb [ shape=diamond label="Does the superevent or\nexternal event already\nexist in GraceDB?" ] Update_existing_event_in_gracedb [ label="Update the existing\nevent in GraceDB" ] Create_new_event_in_gracedb [ label="Create a new event\nin GraceDB" ] Grab_create_skymap [ label="Use provided and/or\ncreate external sky map" ] Launch_detchar_tasks [ label="Launch detector\ncharacterization checks\naround data" ] } KAFKA_FERMI -> Likely_noise [ lhead = cluster_kafka_handle ] KAFKA_SWIFT -> Likely_noise [ lhead = cluster_kafka_handle ] KAFKA_CHIME -> Likely_noise [ lhead = cluster_kafka_handle ] Likely_noise -> Event_exists_in_Gracedb[label="no"] Likely_noise -> Ignore_gcn[label="yes"] Ignore_gcn -> Event_exists_in_Gracedb Event_exists_in_Gracedb -> Update_existing_event_in_gracedb[label="yes"] Event_exists_in_Gracedb -> Create_new_event_in_gracedb[label="no"] Update_existing_event_in_gracedb -> Grab_create_skymap Create_new_event_in_gracedb -> Grab_create_skymap Create_new_event_in_gracedb -> Launch_detchar_tasks }

IGWN Alert Handling

digraph exttrig { compound = true nodesep = 0.1 ranksep = 0.1 node [ fillcolor = white shape = box style = filled target = "_top" ] graph [ labeljust = "left" style = filled target = "_top" ] GRB_External_Trigger_or_Superevent_IGWN_Alert [ style="rounded" label="GRB external trigger or\nSuperevent IGWN Alert received" ] subgraph cluster_grb_igwn_alert_handle { href = "../gwcelery.tasks.external_triggers.html#gwcelery.tasks.external_triggers.handle_grb_igwn_alert" label = <<B><FONT face="monospace">handle_grb_igwn_alert</FONT></B>> Is_New_IGWN_Alert [ shape=diamond label="Is there\na new superevent or\nexternal event?" ] Is_Label_Exttrig_IGWN_Alert [ shape=diamond label="Is there a new label\nin the external event?" ] Are_Labels_Exttrig_Complete [ shape=diamond label=" Does this label\ncomplete a set indicating\nboth sky maps are available?" ] Is_File_Exttrig_IGWN_Alert [ shape=diamond label="Is there a new file\n in the external event,\nupdating a sky map?" ] Perform_Raven_Search [ label="Perform Raven\ncoincidence search(es)" ] Does_Label_Launch_Pipeline [ shape=diamond label="Are the labels a\ncomplete set, indicating a\ncoincidence and both sky maps\nare available?" ] Launch_Raven_Pipeline [ label="Relaunch Raven\nPipeline" ] Create_Combined_Skymap [ label="Create combined GW-GRB\nsky map" ] } GRB_External_Trigger_or_Superevent_IGWN_Alert -> Is_New_IGWN_Alert [ lhead = cluster_grb_igwn_alert_handle ] Is_New_IGWN_Alert -> Perform_Raven_Search[label="yes"] Is_New_IGWN_Alert -> Is_Label_Exttrig_IGWN_Alert[label="no"] Is_Label_Exttrig_IGWN_Alert -> Are_Labels_Exttrig_Complete[label="yes"] Are_Labels_Exttrig_Complete -> Launch_Raven_Pipeline[label="yes"] Is_Label_Exttrig_IGWN_Alert -> Is_File_Exttrig_IGWN_Alert[label="no"] Is_File_Exttrig_IGWN_Alert -> Does_Label_Launch_Pipeline[label="yes"] Does_Label_Launch_Pipeline -> Launch_Raven_Pipeline[label="yes"] Launch_Raven_Pipeline -> Create_Combined_Skymap }

digraph exttrig { compound = true nodesep = 0.1 ranksep = 0.1 node [ fillcolor = white shape = box style = filled target = "_top" ] graph [ labeljust = "left" style = filled target = "_top" ] SNEWS_External_Trigger_or_Superevent_IGWN_Alert [ style="rounded" label="SNEWS external trigger or\nSuperevent IGWN Alert received" ] subgraph cluster_snews_igwn_alert_handle { href = "../gwcelery.tasks.external_triggers.html#gwcelery.tasks.external_triggers.handle_snews_igwn_alert" label = <<B><FONT face="monospace">handle_snews_igwn_alert</FONT></B>> ignore [ label="Ignore" ] is_new_exttrig_igwn_alert [ shape=diamond label="Is this a new type SNEWS\nexternal trigger IGWN Alert?" ] is_new_superevent_igwn_alert [ shape=diamond label="Is this a new type\nsuperevent IGWN Alert?" ] perform_raven_search [ label="Perform Raven\ncoincidence search" ] } SNEWS_External_Trigger_or_Superevent_IGWN_Alert -> is_new_exttrig_igwn_alert [ lhead = cluster_snews_igwn_alert_handle ] is_new_exttrig_igwn_alert -> perform_raven_search[label="yes"] is_new_exttrig_igwn_alert -> is_new_superevent_igwn_alert[label="no"] is_new_superevent_igwn_alert -> perform_raven_search[label="yes"] is_new_superevent_igwn_alert -> ignore[label="no"] }

Tasks

gwcelery.tasks.external_triggers.REQUIRED_LABELS_BY_TASK = {'SoG': {'ADVOK', 'RAVEN_ALERT', 'SKYMAP_READY'}, 'compare': {'EM_COINC', 'EM_READY', 'EXT_SKYMAP_READY'}}

These labels should be present on an external event to consider it to be ready for sky map comparison or for post-alert analysis, such as a measurment of the speed of gravity (SoG).

gwcelery.tasks.external_triggers.FERMI_GRB_CLASS_VALUE = 4

This is the index that denote GRBs within Fermi’s Flight Position classification.

gwcelery.tasks.external_triggers.P_ASTRO_THRESH = 0.5

This value denotes the threshold of the equivalent p_astro estimate, above which we will consider the external event. This is currently used for Fermi Flight notices, CHIME FRB events, and IceCube events

gwcelery.tasks.external_triggers.NOTICE_TYPE_DICT = {'110': 'FERMI_GBM_INITIAL_ALERT', '111': 'FERMI_GBM_FLIGHT_POSITION', '112': 'FERMI_GBM_GROUND_POSITION', '115': 'FERMI_GBM_FINAL_POSITION', '131': 'FERMI_GBM_SUBTHRESHOLD', '202': 'SVOM_ECLAIRS_WAKEUP', '60': 'SWIFT_BAT_GRB_ALERT', '61': 'SWIFT_BAT_GRB_POSITION'}

These map known GCN notice type IDs to their rough string equivalent. This is only needed while we still support VOEvent XML format alerts. See https://gcn.gsfc.nasa.gov/filtering.html for this list.

(task)gwcelery.tasks.external_triggers.handle_snews_gcn(payload)[source]

Handles the GCN notice payload from SNEWS alerts.

Prepares the alert to be sent to graceDB as external events, updating the info if it already exists.

Parameters:

payload (bytes) – XML GCN notice alert packet in bytes string format

(task)gwcelery.tasks.external_triggers.handle_grb_gcn(payload)[source]

Handles the payload from Fermi, Swift, and SVOM GCN notices.

Filters out candidates likely to be noise. Creates external events from the notice if new notice, otherwise updates existing event. Then creates and/or grabs external sky map to be uploaded to the external event.

More info for these notices can be found at: Fermi-GBM: https://gcn.gsfc.nasa.gov/fermi_grbs.html Fermi-GBM sub: https://gcn.gsfc.nasa.gov/fermi_gbm_subthresh_archive.html Swift: https://gcn.gsfc.nasa.gov/swift.html SVOM-ECLAIRS: https://gcn.nasa.gov/missions/svom

Parameters:

payload (bytes) – XML GCN notice alert packet in bytes string format

(task)gwcelery.tasks.external_triggers.handle_kafka_alert(alert)[source]

Handles the payload from Kafka JSON notices. Currently this only includes events a from a MOU partner in our joint subthreshold targeted search, as well as FRBs from CHIME.

Filters out candidates likely to be noise. Creates external events from the notice if new notice, otherwise updates existing event. Then creates and/or grabs external sky map to be uploaded to the external event.

More info for these notices can be found at: Fermi-GBM: https://gcn.gsfc.nasa.gov/fermi_grbs.html Swift: https://gcn.gsfc.nasa.gov/swift.html SVOM-ECLAIRS: https://gcn.nasa.gov/missions/svom

Parameters:

alert (dict) – Kafka alert packet

(task)gwcelery.tasks.external_triggers.handle_superevent_igwn_alert(alert)[source]

Parse an IGWN alert message related to superevent triggers and dispatch it to other tasks.

Notes

This IGWN alert message handler is triggered by creating a new superevent, a label associated with completeness of skymaps or change in state, or if a sky map file is uploaded:

  • New superevent triggers a coincidence search with gwcelery.tasks.raven.coincidence_search().

  • If other type of IGWN alert, pass to _handle_skymaps to decide whether to re-run RAVEN pipeline based on labels or whether to add labels that could start this process.

Parameters:

alert (dict) – IGWN alert packet

(task)gwcelery.tasks.external_triggers.handle_grb_igwn_alert(alert)[source]

Parse an IGWN alert message related to GRB external triggers and dispatch it to other tasks.

Notes

This IGWN alert message handler is triggered by creating a new GRB external trigger event, a label associated with completeness of skymaps or change in state, or if a sky map file is uploaded:

  • New event triggers a coincidence search with gwcelery.tasks.raven.coincidence_search().

  • If other type of IGWN alert, pass to _handle_skymaps to decide whether to re-run RAVEN pipeline based on labels or whether to add labels that could start this process.

Parameters:

alert (dict) – IGWN alert packet

(task)gwcelery.tasks.external_triggers.handle_frb_igwn_alert(alert)[source]

Parse an IGWN alert message related to FRB external triggers and dispatch it to other tasks.

Notes

This IGWN alert message handler is triggered by creating a new FRB external trigger event, a label associated with completeness of skymaps or change in state, or if a sky map file is uploaded:

  • New event triggers a coincidence search with gwcelery.tasks.raven.coincidence_search().

  • If other type of IGWN alert, pass to _handle_skymaps to decide whether to re-run RAVEN pipeline based on labels or whether to add labels that could start this process.

Parameters:

alert (dict) – IGWN alert packet

(task)gwcelery.tasks.external_triggers.handle_hen_igwn_alert(alert)[source]

Parse an IGWN alert message related to HEN external triggers and dispatch it to other tasks.

Notes

This IGWN alert message handler is triggered by creating a new HEN external trigger event, a label associated with completeness of skymaps or change in state, or if a sky map file is uploaded:

  • New event triggers a coincidence search with gwcelery.tasks.raven.coincidence_search().

  • If other type of IGWN alert, pass to _handle_skymaps to decide whether to re-run RAVEN pipeline based on labels or whether to add labels that could start this process.

Parameters:

alert (dict) – IGWN alert packet

(task)gwcelery.tasks.external_triggers.handle_snews_igwn_alert(alert)[source]

Parse an IGWN alert message related to Supernovae external triggers and dispatch it to other tasks.

Notes

This igwn_alert message handler is triggered whenever a new Supernovae external event is created:

Parameters:

alert (dict) – IGWN alert packet

(task)gwcelery.tasks.external_triggers._launch_external_detchar(event)[source]

Launch detchar tasks for an external event.

Parameters:

event (dict) – External event dictionary

Returns:

event – External event dictionary

Return type:

dict

(task)gwcelery.tasks.external_triggers._create_replace_external_event_and_skymap(events, payload, search, pipeline, label=None, ext_group='External', notice_date=None, notice_type=None, skymap=None, skymap_link=None, use_radec=False)[source]

Either create a new external event or replace an old one if applicable Then either uploads a given sky map, try to download one given a link, or create one given coordinates.

Parameters:

  • events (list) – List of external events sharing the same trigger ID

  • payload (bytes) – VOEvent or JSON of event being considered, in bytes string format

  • search (str) – Search of external event

  • pipeline (str) – Pipeline of external evevent

  • label (list) – Label to be uploaded along with external event. If None, removes ‘NOT_GRB’ label from event

  • ext_group (str) – Group of external event, ‘External’ or ‘Test’

  • notice_date (str) – External event trigger time in ISO format

  • notice_type (str) – GCN notice type

  • skymap (str) – Base64 encoded sky map

  • skymap_link (str) – Link to external sky map to be downloaded

  • use_radec (bool) – If True, try to create sky map using given coordinates