{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# TemplateBank\n", "\n", "A TemplateBank is a thin wrapper on an [obsplus EventBank](https://niosh-mining.github.io/obsplus/notebooks/interfaces/eventbank.html). TemplateBanks handle database tasks for storing, selecting and reading in catalogues of templates from/on disk. The idea is to have a suite of templates for all possible events of interest; when something interesting happens this database can be queried to provide the relevant templates for that trigger. Because `TemplateBank`s subclass `EventBank`, all the look-up methods for `EventBank` are available.\n", "\n", "Lets have a look at how we might generate a `TemplateBank` from scratch for a day of the New Zealand national catalogue." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "7 Event(s) in Catalog:\n", "2019-06-21T03:46:25.054642Z | -38.824, +175.865 | 2.178845358 M | manual\n", "2019-06-21T03:52:00.142063Z | -38.826, +175.855 | 1.230595585 M | manual\n", "2019-06-21T08:28:16.143864Z | -38.827, +175.866 | 1.364494767 M | manual\n", "2019-06-21T08:29:23.083982Z | -38.827, +175.878 | 2.066853896 M | manual\n", "2019-06-21T09:07:54.730731Z | -38.819, +175.904 | 1.420446524 M | manual\n", "2019-06-21T15:34:49.878441Z | -38.764, +175.615 | 1.530488164 M | manual\n", "2019-06-22T09:35:27.682975Z | -38.819, +175.866 | 1.880393466 M | manual\n" ] } ], "source": [ "from rt_eqcorrscan.database import TemplateBank\n", "from obspy.clients.fdsn import Client\n", "from obspy import UTCDateTime\n", "\n", "bank = TemplateBank(base_path=\"./template_bank\")\n", "client = Client(\"GEONET\")\n", "catalog = client.get_events(\n", " starttime=UTCDateTime(2019, 6, 21),\n", " endtime=UTCDateTime(2019, 6, 23),\n", " latitude=-38.8, longitude=175.8, maxradius=0.2)\n", "print(catalog)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First we will put the events into the database." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timelatitudelongitudedepthmagnitudeevent_descriptionassociated_phase_countazimuthal_gapevent_idhorizontal_uncertainty...standard_errorused_phase_countstation_countvertical_uncertaintyupdatedauthoragency_idcreation_timeversionpath
02019-06-21 03:46:25.054642-38.823746175.8646095000.0000002.178845Taupo46.067.671886smi:nz.org.geonet/2019p4625863811.082605...0.27989325.039.00.0000002020-04-29 04:26:43.090682624scevent@kseqp02.geonet.org.nzWEL(GNS_Primary)2019-06-21 03:46:42.845267/2019/06/21/2019-06-21T03-46-25_62586.xml
12019-06-21 03:52:00.142063-38.826271175.8554385000.0000001.230596Taupo21.0163.628212smi:nz.org.geonet/2019p4625966603.341442...0.22919715.015.00.0000002020-04-29 04:26:43.098682624scevent@kseqp02.geonet.org.nzWEL(GNS_Primary)2019-06-21 03:52:21.600176/2019/06/21/2019-06-21T03-52-00_62596.xml
22019-06-21 08:28:16.143864-38.827057175.8660287828.7649151.364495Taupo26.091.194962smi:nz.org.geonet/2019p4631165572.845242...0.25563216.019.08385.6350502020-04-29 04:26:43.106682624scevent@kseqp02.geonet.org.nzWEL(GNS_Primary)2019-06-21 08:28:36.574978/2019/06/21/2019-06-21T08-28-16_63116.xml
32019-06-21 08:29:23.083982-38.827339175.8776705000.0000002.066854Taupo61.040.233490smi:nz.org.geonet/2019p4631183076.595177...0.35076937.053.00.0000002020-04-29 04:26:43.130682368scevent@kseqp02.geonet.org.nzWEL(GNS_Primary)2019-06-21 08:29:41.755653/2019/06/21/2019-06-21T08-29-23_63118.xml
42019-06-21 09:07:54.730731-38.819088175.9041595000.0000001.420447Taupo32.076.976382smi:nz.org.geonet/2019p4631904347.700016...0.20613918.028.00.0000002020-04-29 04:26:43.142682624scevent@kseqp02.geonet.org.nzWEL(GNS_Primary)2019-06-21 09:08:14.577520/2019/06/21/2019-06-21T09-07-54_63190.xml
52019-06-21 15:34:49.878441-38.764454175.615082113987.5336001.530488Taupo21.0265.354118smi:nz.org.geonet/2019p46391611605.852290...0.40686021.014.013899.0741902020-04-29 04:26:43.146682624scevent@kseqp02.geonet.org.nzWEL(GNS_Primary)2019-06-21 15:35:31.012134/2019/06/21/2019-06-21T15-34-49_63916.xml
62019-06-22 09:35:27.682975-38.818531175.86640913595.7679701.880393Taupo39.056.649483smi:nz.org.geonet/2019p4659445168.447127...0.47810226.034.07484.6349622020-04-29 04:26:43.162682368scevent@kseqp02.geonet.org.nzWEL(GNS_Primary)2019-06-22 09:35:48.159662/2019/06/22/2019-06-22T09-35-27_65944.xml
\n", "

7 rows × 28 columns

\n", "
" ], "text/plain": [ " time latitude longitude depth magnitude \\\n", "0 2019-06-21 03:46:25.054642 -38.823746 175.864609 5000.000000 2.178845 \n", "1 2019-06-21 03:52:00.142063 -38.826271 175.855438 5000.000000 1.230596 \n", "2 2019-06-21 08:28:16.143864 -38.827057 175.866028 7828.764915 1.364495 \n", "3 2019-06-21 08:29:23.083982 -38.827339 175.877670 5000.000000 2.066854 \n", "4 2019-06-21 09:07:54.730731 -38.819088 175.904159 5000.000000 1.420447 \n", "5 2019-06-21 15:34:49.878441 -38.764454 175.615082 113987.533600 1.530488 \n", "6 2019-06-22 09:35:27.682975 -38.818531 175.866409 13595.767970 1.880393 \n", "\n", " event_description associated_phase_count azimuthal_gap \\\n", "0 Taupo 46.0 67.671886 \n", "1 Taupo 21.0 163.628212 \n", "2 Taupo 26.0 91.194962 \n", "3 Taupo 61.0 40.233490 \n", "4 Taupo 32.0 76.976382 \n", "5 Taupo 21.0 265.354118 \n", "6 Taupo 39.0 56.649483 \n", "\n", " event_id horizontal_uncertainty ... standard_error \\\n", "0 smi:nz.org.geonet/2019p462586 3811.082605 ... 0.279893 \n", "1 smi:nz.org.geonet/2019p462596 6603.341442 ... 0.229197 \n", "2 smi:nz.org.geonet/2019p463116 5572.845242 ... 0.255632 \n", "3 smi:nz.org.geonet/2019p463118 3076.595177 ... 0.350769 \n", "4 smi:nz.org.geonet/2019p463190 4347.700016 ... 0.206139 \n", "5 smi:nz.org.geonet/2019p463916 11605.852290 ... 0.406860 \n", "6 smi:nz.org.geonet/2019p465944 5168.447127 ... 0.478102 \n", "\n", " used_phase_count station_count vertical_uncertainty \\\n", "0 25.0 39.0 0.000000 \n", "1 15.0 15.0 0.000000 \n", "2 16.0 19.0 8385.635050 \n", "3 37.0 53.0 0.000000 \n", "4 18.0 28.0 0.000000 \n", "5 21.0 14.0 13899.074190 \n", "6 26.0 34.0 7484.634962 \n", "\n", " updated author \\\n", "0 2020-04-29 04:26:43.090682624 scevent@kseqp02.geonet.org.nz \n", "1 2020-04-29 04:26:43.098682624 scevent@kseqp02.geonet.org.nz \n", "2 2020-04-29 04:26:43.106682624 scevent@kseqp02.geonet.org.nz \n", "3 2020-04-29 04:26:43.130682368 scevent@kseqp02.geonet.org.nz \n", "4 2020-04-29 04:26:43.142682624 scevent@kseqp02.geonet.org.nz \n", "5 2020-04-29 04:26:43.146682624 scevent@kseqp02.geonet.org.nz \n", "6 2020-04-29 04:26:43.162682368 scevent@kseqp02.geonet.org.nz \n", "\n", " agency_id creation_time version \\\n", "0 WEL(GNS_Primary) 2019-06-21 03:46:42.845267 \n", "1 WEL(GNS_Primary) 2019-06-21 03:52:21.600176 \n", "2 WEL(GNS_Primary) 2019-06-21 08:28:36.574978 \n", "3 WEL(GNS_Primary) 2019-06-21 08:29:41.755653 \n", "4 WEL(GNS_Primary) 2019-06-21 09:08:14.577520 \n", "5 WEL(GNS_Primary) 2019-06-21 15:35:31.012134 \n", "6 WEL(GNS_Primary) 2019-06-22 09:35:48.159662 \n", "\n", " path \n", "0 /2019/06/21/2019-06-21T03-46-25_62586.xml \n", "1 /2019/06/21/2019-06-21T03-52-00_62596.xml \n", "2 /2019/06/21/2019-06-21T08-28-16_63116.xml \n", "3 /2019/06/21/2019-06-21T08-29-23_63118.xml \n", "4 /2019/06/21/2019-06-21T09-07-54_63190.xml \n", "5 /2019/06/21/2019-06-21T15-34-49_63916.xml \n", "6 /2019/06/22/2019-06-22T09-35-27_65944.xml \n", "\n", "[7 rows x 28 columns]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "bank.put_events(catalog)\n", "bank.read_index()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we want to populate the database with templates: at the moment the database only contains event\n", "files. Note that this might take a while - it will download the required data and process it. \n", "The same arguments used by EQcorrscan's [Tribe.construct](https://eqcorrscan.readthedocs.io/en/latest/submodules/core.match_filter.Tribe.html#eqcorrscan.core.match_filter.Tribe.construct) \n", "are supported by `Template_bank.make_templates`." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "No data downloaded for NZ.NNVZ.10.EHT\n", "No data downloaded for NZ.BKZ.10.HHR\n", "No data downloaded for NZ.BKZ.10.HHT\n", "No data downloaded for NZ.KWHZ.10.EHR\n", "No data downloaded for NZ.MTHZ.10.EHR\n", "No data downloaded for NZ.HIZ.10.HHT\n", "No data downloaded for NZ.HIZ.10.HHT\n", "No data downloaded for NZ.TSZ.10.HHR\n", "No data downloaded for NZ.RITZ.10.EHT\n", "No data downloaded for NZ.WATZ.10.EHT\n", "No data downloaded for NZ.WHTZ.10.EHT\n", "No data downloaded for NZ.KATZ.10.EHT\n", "No data downloaded for NZ.TMVZ.10.HHT\n", "No data downloaded for NZ.ETVZ.10.HHR\n", "No data downloaded for NZ.ETVZ.10.HHR\n", "No data downloaded for NZ.NNVZ.10.EHT\n", "No data downloaded for NZ.OTVZ.10.HHT\n", "No data downloaded for NZ.SNVZ.10.EHT\n", "No data downloaded for NZ.MRHZ.10.EHR\n", "Request would result in too much data. Denied by the datacenter. Split the request in smaller parts\n", "Detailed response of server:\n", "\n", "\n", "/home/chambeca/miniconda3/envs/eqcorrscan/lib/python3.7/site-packages/obspy/io/mseed/core.py:790: UserWarning: The encoding specified in trace.stats.mseed.encoding does not match the dtype of the data.\n", "A suitable encoding will be chosen.\n", " warnings.warn(msg, UserWarning)\n" ] }, { "data": { "text/plain": [ "Tribe of 7 templates" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "bank.make_templates(\n", " catalog=catalog, client=client, lowcut=2., highcut=15.,\n", " samp_rate=50., filt_order=4, prepick=0.5, length=3, swin=\"all\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This both returns the `Tribe` of templates that are created, and writes them to disk for later retrieval.\n", "Note that lots of warnings are raised by this due to missing links between picks. These can be\n", "ignored for this tutorial." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Tribe of 7 templates\n" ] } ], "source": [ "import warnings\n", "\n", "with warnings.catch_warnings():\n", " warnings.simplefilter(\"ignore\")\n", " tribe = bank.get_templates()\n", "print(tribe)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`get_templates` supports filtering using event queries:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Tribe of 3 templates" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "bank.get_templates(starttime=UTCDateTime(2019, 6, 21, 9))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Usage in RT-EQcorrscan\n", "\n", "A `TemplateBank` is required to use the `CatalogListener` and `Reactor` classes. Both these classes will *listen* to a remote client service for new events and add those events to the `TemplateBank`. The `Reactor` class will also react to any events that meet user-defined thresholds, collect the appropriate temlates using the `TemplateBank`, and run the real-time matched-filter process.\n", "\n", "## Usage outside of RT-EQcorrscan\n", "\n", "Because `TemplateBanks` return `Tribe`s, `TemplateBanks` represent an efficient means of storing `Tribes` on disk. `Tribe`s can be used offline for matched-filter detection." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.7" } }, "nbformat": 4, "nbformat_minor": 2 }