seismic.inventory package¶
Subpackages¶
Submodules¶
seismic.inventory.add_time_corrections module¶
Add GPS clock time correction csv_data into inventory file to get a modified station xml file
- CreationDate:
24/02/2020
- Developer:
-
seismic.inventory.add_time_corrections.
add_gpscorrection_into_stationxml
(csv_file, input_xml, out_xml=None)[source]¶ Read in the correction CSV data from a file, get the station metadata node from input_xml file, then add the CSV data into the station xml node to write into out_xml
- Parameters
csv_file – input csv file with correction data
input_xml – input original stationXML file which contains the metadata for the network and station of csv_file
out_xml – Directory of the output xml file
- Returns
full path of the output xml file
-
seismic.inventory.add_time_corrections.
extract_csvdata
(path2xml)[source]¶ Read the station xml file and extract the csv data to be parsed by pandas
- Parameters
path2xml – path_to_stationxml
- Returns
csv_str
-
seismic.inventory.add_time_corrections.
get_csv_correction_data
(path_csvfile)[source]¶ - Read in the csv data from an input file, get the network_code, station_code, csv_data. Format::
$ head 7D.DE43_clock_correction.csv net,sta,date,clock_correction 7D,DE43,2012-11-27,1.0398489013215846 7D,DE43,2012-11-28,0.9408504322549281 7D,DE43,2012-11-29,0.8418519631882714 7D,DE43,2012-11-30,0.7428534941216148 7D,DE43,2012-12-01,0.6438550250549583
- Parameters
path_csvfile – input csv file in /g/data/ha3/Passive/SHARED_DATA/GPS_Clock/corrections/
- Returns
(network_code, station_code, csv_data)
seismic.inventory.engd2stxml module¶
Engdahl and ISC STN data conversion to FDSN station XML¶
Creates database of stations from .STN files, curates the data using heuristic rules, and exports new stations to FDSN station XML format network with non-empty, nominal instrument response data.
Cleanup steps applied: * Removes “blacklisted” networks that add little value and cause problems due to station code conflicts. * Add default station dates where missing. * Make “future” station end dates consistent to max Pandas timestamp. * Remove records with illegal station codes. * Remove duplicate station records. * Merge overlapping channel dates for given NET.STAT.CHAN to a single epoch.
-
seismic.inventory.engd2stxml.
cleanup_database
(df)[source]¶ Main cleanup function encompassing the sequential data cleanup steps.
- Parameters
df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe of station records to clean up
- Returns
Cleaned up dataframe of station records
- Return type
pandas.DataFrame conforming to table_format.TABLE_SCHEMA
-
seismic.inventory.engd2stxml.
compute_neighboring_station_matrix
(df)[source]¶ Compute sparse matrix representing index of neighboring stations.
Ordering of matrix corresponds to ordering of Dataframe df, which is expected to be sequential integer indexed. For a given station index i, then the non-zero off-diagonal entries in row i of the returned matrix indicate the indices of adjacent, nearby stations.
- Parameters
df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe containing station records.
- Returns
Sparse binary matrix having non-zero values at indices of neighboring stations.
- Return type
-
seismic.inventory.engd2stxml.
latlong_to_cosinedistance
(latlong_deg_set1, latlong_deg_set2)[source]¶ Compute the approximate cosine distance between each station of 2 sets.
Each set is specified as a numpy column vector of [latitude, longitude] positions in degrees.
This function performs an outer product and will produce matrix of size N0 x N1, where N0 is the number of rows in latlong_deg_set1 and N1 is the number of rows in latlong_deg_set2.
Returns np.ndarray containing cosines of angles between each pair of stations from the input arguments. If input is 1D, convert to 2D for consistency of matrix orientations.
- Parameters
latlong_deg_set1 (np.ndarray) – First set of numpy column vector of [latitude, longitude] positions in degrees
latlong_deg_set2 (np.ndarray) – Second set of numpy column vector of [latitude, longitude] positions in degrees
- Returns
Array containing cosines of angles between each pair of stations from the input arguments.
- Return type
np.ndarray
-
seismic.inventory.engd2stxml.
main
(iris_xml_file, stations_folder, output_directory, test_mode=False)[source]¶ Main entry point.
- Parameters
iris_xml_file (str or pathlib.Path) – Name of IRIS xml file to load (generated by script update_iris_inventory.py)
stations_folder (str or pathlib.Path) – Path to folder containing STN files to process
-
seismic.inventory.engd2stxml.
merge_overlapping_channel_epochs
(df)[source]¶ Removed overlapping time intervals for a given network.station.channel, as this needs to be unique.
This function expects the input DataFrame to have a sequential integer index.
- Parameters
df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe of station records in which to merge overlapping channel dates
-
seismic.inventory.engd2stxml.
populate_default_station_dates
(df)[source]¶ Replace all missing station start and end dates with default values. Replace all missing channel start and end dates with their corresponding station/end dates.
- Parameters
df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe in which to fill in missing station start and end dates.
-
seismic.inventory.engd2stxml.
read_eng
(fname)[source]¶ Read Engdahl STN file having the following format of fixed width formatted columns:
:: AAI Ambon BMG, Indonesia, IA-Ne -3.6870 128.1945 0.0 2005001 2286324 I :: AAII -3.6871 128.1940 0.0 2005001 2286324 I :: AAK Ala Archa Kyrgyzstan 42.6390 74.4940 0.0 2005001 2286324 I :: ABJI -7.7957 114.2342 0.0 2005001 2286324 I :: APSI -0.9108 121.6487 0.0 2005001 2286324 I :: AS01 Alice Springs Arra -23.6647 133.9508 0.0 2005001 2286324 I :: 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890 :: 10 20 30 40 50 60 70 80
Each Station Code (first column) might NOT be unique, and network codes are missing here, so all records are placed under ‘GE’ network.
- Parameters
fname (str) – STN file name to load
- Returns
Pandas Dataframe containing the loaded data in column order of TABLE_COLUMNS.
- Return type
pandas.DataFrame conforming to table_format.TABLE_SCHEMA
-
seismic.inventory.engd2stxml.
read_isc
(fname, use_pickle=False)[source]¶ Read ISC station inventory having such format and convert to Pandas DataFrame:
:: 109C 32.8892 -117.1100 0.0 2006-06-01 04:11:18 2008-01-04 01:26:30 :: 109C 32.8882 -117.1050 150.0 2008-01-04 01:26:30 :: FDSN 109C TA – BHZ 2004-05-04 23:00:00 2005-03-03 23:59:59 :: FDSN 109C TA – LHZ 2004-05-04 23:00:00 2005-03-03 23:59:59 :: FDSN 109C TA – BHZ 2005-04-11 00:00:00 2006-01-25 22:31:10 :: FDSN 109C TA – LHZ 2005-04-11 00:00:00 2006-01-25 22:31:10 :: 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890 :: 10 20 30 40 50 60 70 80
The lines starting with a station code are HEADER rows, and provide the station coordinates. The idented lines starting with FDSN provide distinct station, network and channel data for the given station location.
- Parameters
fname (str) – STN file name to load
- Returns
Pandas Dataframe containing the loaded data in column order of TABLE_COLUMNS.
- Return type
pandas.DataFrame conforming to table_format.TABLE_SCHEMA
-
seismic.inventory.engd2stxml.
remove_blacklisted
(df)[source]¶ Remove network codes that are explicitly blacklisted due to QA issues or undesirable overlap with trusted FDSN station codes.
- Parameters
df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe of initially loaded data from STN files
-
seismic.inventory.engd2stxml.
remove_duplicate_stations
(df, neighbor_matrix)[source]¶ Remove stations which are identified as duplicates: * Removes duplicated stations in df based on station code and locality of lat/long coordinates. * Removes duplicated stations based on codes and channel data matching, IRRESPECTIVE of locality
- Parameters
df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe containing station records. Is modified during processing.
neighbor_matrix (scipy.sparse.csr_matrix) – Sparse binary matrix having non-zero values at indices of neighboring stations.
- Returns
Dataframe containing station records with identified duplicates removed.
- Return type
pandas.DataFrame conforming to table_format.TABLE_SCHEMA
-
seismic.inventory.engd2stxml.
remove_illegal_stationNames
(df)[source]¶ Remove records for station names that do not conform to expected naming convention. Such names can cause problems in downstream station, in particular names with asterisk.
- Parameters
df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe containing station records from which illegal station codes should be removed (modified in-place)
-
seismic.inventory.engd2stxml.
reportUnpickleFail
(filename)[source]¶ Standard failure report message when trying to unpickle file.
- Parameters
filename (str) – The name of the file that failed to unpickle.
-
seismic.inventory.engd2stxml.
write_portable_inventory
(df, fname)[source]¶ Write the final database to re-usable file formats.
- Parameters
df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe containing complete inventory of station records.
fname (str) – Base filename to export to. Output filename will be appended with file format extensions.
seismic.inventory.fdsnxml_convert module¶
Helper functions to convert FDSN station XML files to Seiscomp3 SC3ML format.
- Can be used as a standalone tool as well:
fdsnxml_convert.py src_path dst_path
-
seismic.inventory.fdsnxml_convert.
sc3_conversion_available
()[source]¶ - Check whether conversion to seiscomp3 format is available os this system.
Only works for Python 3, for Python 2 it always returns True (i.e. give it a try).
- Returns
True if conversion to sc3ml can be performed on this system, False otherwise.
- Return type
-
seismic.inventory.fdsnxml_convert.
toSc3ml
(src_path, dst_path, response_fdsnxml=None)[source]¶ Convert file(s) in src_path from FDSN station XML to SC3ML and emit result(s) to dst_path.
If src_path is a file, dst_path will be treated as a file. If dst_path already exists as a folder, an exception is raised.
If src_path is a folder, dst_path will be treated as a folder. If dst_path already exists as a file, an exception is raised. The src_path directory hierarchy will be walked to find all .xml files, each of which will be converted to a mirrored relative path under dst_path.
- Parameters
src_path (str or pathlib.Path) – The source path from which to input XML file(s).
dst_path (str or pathlib.Path) – The destination path into which converted sc3ml file(s) are output.
response_fdsnxml (str or Path) – Path to existing for containing dummy response to insert into records where instrument response is missing.
- Raises
OSError, FileNotFoundError, RuntimeError, FileExistsError
seismic.inventory.generate_network_plots module¶
Load inventory file from hdf5 format and export station location plots to graphics files.
- Usage::
python generate_network_plots.py inventory_20190206.h5
where inventory_20190206.h5 here is an example file name. The output folder for the graphics files is inferred from the inventory file name, which would be “inventory_20190206” in this example.
seismic.inventory.inventory_merge module¶
Merge a master inventory (e.g. from IRIS) with custom inventories.
-
seismic.inventory.inventory_merge.
inventory_merge
(iris_inv, custom_inv, output_file, test_mode=False)[source]¶ - Merge an IRIS inventory with a custom inventory, filtering out any custom inventory records
that duplicate IRIS records.
- Parameters
-
seismic.inventory.inventory_merge.
mean_lat_long
(network)[source]¶ Compute mean of station lat/long coordinates within a network
- Parameters
network (obspy.core.inventory.network.Network) – Network on which to compute mean lat/long
-
seismic.inventory.inventory_merge.
prune_iris_duplicates
(db_other, db_iris)[source]¶ Prune the records out of db_other which duplicate records that exist in db_iris.
Filtering concepts: Station codes and channel codes are reliable, but network codes and station/channel dates are not. Station lat/lon locations are approximately reliable. Therefore the merge method here matches records with matching station code, channel code and approximate location. For each match, we examine station dates, and discard any records of db_other that overlap with db_iris.
db_iris is not changed by this function.
- Parameters
db_other (pandas.DataFrame) – DataFrame containing metadata records of custom inventory
db_iris (pandas.DataFrame) – DataFrame containing metadata records of IRIS inventory
- Returns
DataFrame containing records from db_other which are unique and do not appear in IRIS.
- Return type
seismic.inventory.inventory_split module¶
Split a station inventory file into a separate file per network.
-
seismic.inventory.inventory_split.
inventory_split
(inv_file, output_folder, sc3ml=False)[source]¶ Split an inventory file (station XML) into separate inventory file per network, stored in folder output_folder.
- Parameters
inv_file (str or path) – Station inventory to be split. This file is not changed by the script.
output_folder (str or path to folder) – Folder where per-network station xml files will be generated.
sc3ml (bool, optional) – If True, try to convert output files to sc3ml format if possible using seiscomp3. Defaults to False.
-
seismic.inventory.inventory_split.
split_inventory_by_network
(obspy_inv, output_folder, validate=False)[source]¶ Export a station XML file per network for each network in given obspy Inventory.
- Parameters
obspy_inv (obspy.core.inventory.inventory.Inventory) – Obspy Inventory containing the networks to export to file.
output_folder (str or Path) – Folder in which to output the per-network XML files. Will be created if doesn’t yet exist.
validate (bool, optional) – Whether to validate the station data on write, defaults to False
seismic.inventory.inventory_util module¶
Utility functions and constants shared amongst inventory management modules.
-
class
seismic.inventory.inventory_util.
Instrument
(sensor, response)¶ Bases:
tuple
-
property
response
¶ Alias for field number 1
-
property
sensor
¶ Alias for field number 0
-
property
-
seismic.inventory.inventory_util.
extract_unique_sensors_responses
(inv, req, show_progress=True, blacklisted_networks=None, test_mode=False)[source]¶ For the channel codes in the given inventory, determine a nominal instrument response suitable for that code. Note that no attempt is made here to determine an ACTUAL correct response for a given network and station. The only requirement here is to populate a plausible, non-empty response for a given channel code, to placate Seiscomp3 which requires that an instrument response always be present.
- Parameters
inv (obspy.Inventory) – Seismic station inventory
req (Object conforming to interface of 'requests' library) – Request object to use for URI query
- Returns
Python dict of (obspy.core.inventory.util.Equipment, obspy.core.inventory.response.Response) indexed by str representing channel code
- Return type
{str: Instrument(obspy.core.inventory.util.Equipment, obspy.core.inventory.response.Response) } where Instrument is a namedtuple(“Instrument”, [‘sensor’, ‘response’])
-
seismic.inventory.inventory_util.
load_station_xml
(inventory_file)[source]¶ Load a stationxml file
- Parameters
inventory_file (str or path) – [description]
-
seismic.inventory.inventory_util.
obtain_nominal_instrument_response
(netcode, statcode, chcode, req)[source]¶ For given network, station and channel code, find suitable response(s) in IRIS database and return as dict of obspy instrument responses.
- Parameters
- Returns
Dictionary of instrument responses from IRIS for given network(s), station(s) and channel(s).
- Return type
dict of {str, Instrument(obspy.core.inventory.util.Equipment, obspy.core.inventory.response.Response)}
seismic.inventory.iris_query module¶
Helper functions for making and managing web queries to IRIS web service.
-
seismic.inventory.iris_query.
form_channel_request_url
(netmask='*', statmask='*', chanmask='*')[source]¶ Form request URL to download station inventory in stationxml format, down to channel level, with the given filters applied to network codes, station codes and channel codes.
- Parameters
netmask – Pattern of network codes to match, comma separated with wildcards, defaults to “*”
netmask – str, optional
statmask – Pattern of station codes to match, comma separated with wildcards, defaults to “*”
statmask – str, optional
chanmask – Pattern of channel codes to match, comma separated with wildcards, defaults to “*”
chanmask – str, optional
- Returns
Fully formed URL to perform IRIS query and get back FDSN station XML result.
- Return type
-
seismic.inventory.iris_query.
form_response_request_url
(netmask, statmask, chanmask)[source]¶ Form request URL to download station inventory in stationxml format, down to response level, for the given network, station and channel codes.
- Parameters
netmask – Pattern of network codes to match, comma separated with wildcards
netmask – str, optional
statmask – Pattern of station codes to match, comma separated with wildcards
statmask – str, optional
chanmask – Pattern of channel codes to match, comma separated with wildcards
chanmask – str, optional
- Returns
Fully formed URL to perform IRIS query and get back FDSN station XML result.
- Return type
seismic.inventory.pdconvert module¶
Helper functions for converting between Pandas dataframe and FDSN Inventory, Network, Station and Channel objects.
-
seismic.inventory.pdconvert.
dataframe_to_fdsn_station_xml
(inventory_df, nominal_instruments, filename, show_progress=True)[source]¶ Export dataframe of station metadata to FDSN station xml file
- Parameters
inventory_df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe containing all the station records to export.
nominal_instruments ({str: Instrument(obspy.core.inventory.util.Equipment, obspy.core.inventory.response.Response) }) – Dictionary mapping from channel code to nominal instrument characterization
filename (str or path) – Output filename
-
seismic.inventory.pdconvert.
dataframe_to_network
(netcode, network_df, instrument_register, progressor=None)[source]¶ Convert Pandas dataframe with unique network code to obspy Network object.
- Parameters
netcode (str) – Network code
network_df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe containing records for a single network code.
instrument_register – Dictionary of nominal instrument responses indexed by channel code, defaults to None
instrument_register – dict of {str, Instrument(obspy.core.inventory.util.Equipment, obspy.core.inventory.response.Response)}, optional
progressor – Progress bar functor to receive progress updates, defaults to None
progressor – Callable object receiving incremental update on progress, optional
- Returns
Network object containing the network information from the dataframe
- Return type
-
seismic.inventory.pdconvert.
inventory_to_dataframe
(inv_object, show_progress=True)[source]¶ Convert a obspy Inventory object to a Pandas Dataframe.
- Parameters
inv_object (obspy.core.inventory.inventory.Inventory) – Obspy inventory object to convert to dataframe
show_progress – Whether to use a progress bar, defaults to True
show_progress – bool, optional
- Returns
Pandas Dataframe with sequential integer index and sorted by [NetworkCode, StationCode]. Only populates entries for non-empty channels.
- Return type
pandas.DataFrame conforming to table_format.TABLE_SCHEMA
seismic.inventory.plotting module¶
Helper functions for plotting FDSN Network objects.
-
seismic.inventory.plotting.
save_network_local_plots
(df, plot_folder, progressor=None, include_stations_list=True)[source]¶ Save visual map plot per network, saved to file netcode.png.
- Parameters
df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe of station records to save.
plot_folder (str) – Name of output folder
progressor – Progress bar functor to receive progress updates, defaults to None
progressor – Callable object receiving incremental update on progress, optional
include_stations_list – If True, also export stationtxt file alongside each png file, defaults to True
include_stations_list – bool, optional
-
seismic.inventory.plotting.
save_station_local_plots
(df, plot_folder, progressor=None, include_stations_list=True)[source]¶ Save visual map plot per station, saved to file netcode.stationcode.png.
- Parameters
df (pandas.DataFrame conforming to table_format.TABLE_SCHEMA) – Dataframe of station records to save.
plot_folder (str) – Name of output folder
progressor – Progress bar functor to receive progress updates, defaults to None
progressor – Callable object receiving incremental update on progress, optional
include_stations_list – If True, also export stationtxt file alongside each png file, defaults to True
include_stations_list – bool, optional
seismic.inventory.response module¶
- Description:
Implements a Class for reading/storing responses from a number of sources. The ResponseFactory class is used for attaching ‘bogus’ responses to station inventories that lack them.
References:
CreationDate: 14/02/19
Developer: rakib.hassan@ga.gov.au
- Revision History:
LastUpdate: 14/02/19 RH LastUpdate: dd/mm/yyyy Who Optional description
-
class
seismic.inventory.response.
ResponseFactory
[source]¶ Bases:
object
The ResponseFactory class encapsulates the generation of a collection of named Instrument Response Objects from a variety of sources. Currently it provides the facility to create Response objects from two sources, namely, Poles and Zeroes supplied by the user and from StationXML files generated from RESP files using the PDCC tool (link below). The conversion of RESP files into a corresponding StationXML file, at this stage, must take place externally, because ObsPy lacks that functionality. The intended usage of this class during the creation of an ASDF dataset is as follows:
1. User creates a number of uniquely named Response objects (see associated tests as well) pertaining to different channels in a given survey. 2. User fetches these Response objects from an instance of ResponseFactory as needed, while creating ObsPy Channel objects, during which Response objects can be passed in as an argument. 3. User builds a hierarchy of channel->station->network inventories, with the appropriate instrument response information embedded 4. The master FDSN StaionXML file output after step 3 can then be converted into an SC3ML file (which can be ingested by SeisComp3) using the fdsnxml2inv tool.
PDCC tool: https://ds.iris.edu/ds/nodes/dmc/software/downloads/pdcc/
-
CreateFromInventory
(name, obspy_inventory)[source]¶ Create response from an Inventory
- Parameters
name (str) – Name of the response for later retrieval
obspy_inventory (obspy.core.inventory.inventory.Inventory) – Inventory from which to extract response
-
CreateFromPAZ
(name, pzTransferFunctionType, normFactor, normFreq, stageGain, stageGainFreq, poles, zeros)[source]¶
-
class
ResponseFromInventory
(source_inventory)[source]¶ Bases:
object
Helper class to get Obspy Response object from an Inventory
- Raises
RuntimeError – Raises error if response not found
-
class
ResponseFromPAZ
(pzTransferFunctionType='LAPLACE (RADIANS/SECOND)', normFactor=80000.0, normFreq=0.01, stageGain=2000.0, stageGainFreq=0.01, poles=[0j], zeros=[0j])[source]¶ Bases:
object
-
class
ResponseFromStationXML
(respFileName)[source]¶ Bases:
seismic.inventory.response.ResponseFactory.ResponseFromInventory
Helper class to get Obspy Response object from a station xml file
-
getResponse
(name)[source]¶ Retrieve response by name
- Parameters
name (str) – Name given to response at creation time
- Raises
RuntimeError – Raises error if name is not recognized
- Returns
The requested response
- Return type
-
seismic.inventory.table_format module¶
Common format for Pandas Dataframe column ordering of Network, Station and Channel metadata.
seismic.inventory.update_iris_inventory module¶
Automatically update IRIS-ALL.xml file from IRIS web portal.
Outout file is saved as FDSN station xml. Script also generates human readable form as IRIS-ALL.txt.
Example usages:¶
python update_iris_inventory.py
python update_iris_inventory.py -o outfile.xml
python update_iris_inventory.py –netmask=U* –statmask=K*
python update_iris_inventory.py –netmask=UW,LO –output outfile.xml
-
seismic.inventory.update_iris_inventory.
cleanup
(tmp_filename)[source]¶ Helper function to clean up temporary file on disk.
- Parameters
tmp_filename (str) – File name to clean up
-
seismic.inventory.update_iris_inventory.
regenerate_human_readable
(iris_data, outfile)[source]¶ Generate human readable, tabular version of the IRIS database.
-
seismic.inventory.update_iris_inventory.
repair_iris_metadata
(iris)[source]¶ Perform text subtitutions to fix known errors in station xml returned from IRIS.
- Parameters
iris (requests.models.Response) – Response to IRIS query request containing response text
- Returns
The text from the response with known faulty data substituted with fixed data.
- Return type
str (Python 3) or unicode (Python 2)
-
seismic.inventory.update_iris_inventory.
update_iris_station_xml
(req, output_file, options=None)[source]¶ Pull the latest IRIS complete station inventory (down to station level, not including instrument responses) from IRIS web service and save to file in FDSN station xml format.
- Parameters
req (Object conforming to interface of 'requests' library) – Request object to use for URI query
output_file (str) – Destination file to generate
options – Filtering options for network, station and channel codes, defaults to None
options – Python dict of key-values pairs matching command line options, optional