Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

There exists the ability to store transfer functions in an MTH5 for a comprehensive representation of data. The transfer functions are stored at the Station level in a group called TransferFunctionsGroup. Within this group station transfer functions are stored as a single group for each transfer function TransferFunctionGroup. Each TransferFunctionGroup has a data set for each statistical estimate provided within the transfer function. Currently supported estimates are

EstimateDescriptionShapeData Type
transfer_functionFull transfer functionn_periods x n_inputs x n_outputscomplex
transfer_function_errorFull transfer function error estimationn_periods x n_inputs x n_outputsreal
impedanceOnly horizontal components, traditional impedance tensorn_periods x 2 x 2complex
impedance_errorOnly horizontal components, traditional impedance tensor errorn_periods x 2 x 2real
tipperhorizontal and vertical magnetic transfer function, Tippern_periods x 1 x 2complex
tipper_errorhorizontal and vertical magnetic transfer function, Tipper errorn_periods x 1 x 2real
inverse_signal_powercovariance of input channels (sources)n_periods x n_inputs x n_inputscomplex
residual_covariancecovariance of output channels (responses)n_periods x n_outputs x n_outputscomplex

Note: There are plans to add phase tensor and resitivity/phase estimations in the future.

This examples demonstrates how transfer functions can be added to an MTH5 file.

TF object

The TF object comes from mt_metadata.transfer_functions.core.TF and is meant to be the common container for transfer functions. It has readers for:

  • EDI

  • EMFT XML

  • Z-files (EMTF output)

  • J-files (BIRRP output)

  • AVG files (Zonge output)

The TF object has two important metadata objects survey_metadata and station_metadata. Metadata from the other files are translated into these containers and translated back when writing a file.

The statistical estimates are stored as xarray.Datasets that have coordinates of period, input_channels, output_channels. This way the transfer function can be generalized. impedance and tipper are stored in transfer_function and TF provides convenience functions to access impedance and tipper and associated errors. Variances are stored as covariances for input channels (inverse_signal_power) and output channels (residual_covariance) when possible, and the transfer_function_error is stored as well.

Note: There are future plans to include phase tensor and resistivity/phase representation as well.

from mth5.mth5 import MTH5

from mt_metadata import TF_XML, TF_EDI_SPECTRA, TF_ZMM, TF_EDI_CGG
from mt_metadata.transfer_functions.core import TF
2023-04-21 10:31:36,442 [line 141] mth5.setup_logger - INFO: Logging file can be found C:\Users\jpeacock\OneDrive - DOI\Documents\GitHub\mth5\logs\mth5_debug.log

m = MTH5(file_version="0.2.0")
m.open_mth5(r"transfer_function_example.h5", "w")

2023-04-21 10:31:42,346 [line 672] mth5.mth5.MTH5._initialize_file - INFO: Initialized MTH5 0.2.0 file transfer_function_example.h5 in mode w

Read in Transfer functions

  • TF_XML: An example of EMTF XML format, the preferred format for archiving

  • TF_EDI_SPECTRA: An example of an EDI file stored as spectra

  • TF_EDI_CGG: An example of an output file from a contractor

  • TF_ZMM: An example of an output file from EMFT

tf1 = TF(TF_XML)
tf1.read_tf_file()

tf2 = TF(TF_EDI_SPECTRA)
tf2.read_tf_file()

tf3 = TF(TF_EDI_CGG)
tf3.read_tf_file()

tf4 = TF(TF_ZMM)
tf4.read_tf_file()

Add TF_XML to the MTH5

When we add a transfer function to an MTH5, it looks for the survey.id and station.id, if it doesn’t find any then they are created. If information is provided on which runs were processed and channels used those are filled in as well.

Note: If you have multiple transfer functions for a given station be sure to rename the file, for an EDI this is in HEADER under the attribute DATAID. Name it something like station_sample_rate or station_runs

tf_group_01 = m.add_transfer_function(tf1)
tf_group_01
2023-04-21 10:31:49,566 [line 1054] mth5.mth5.MTH5.get_survey - WARNING: /Experiment/Surveys/CONUS_South does not exist, check survey_list for existing names.
/Experiment/Surveys/CONUS_South/Stations/NMX20/Transfer_Functions/NMX20: ==================== --> Dataset: inverse_signal_power ................................... --> Dataset: period ..................... --> Dataset: residual_covariance .................................. --> Dataset: transfer_function ................................ --> Dataset: transfer_function_error ......................................
est = tf_group_01.get_estimate("transfer_function")
tf_group_01.has_estimate("covariance")
True

Have a look at what was added to the MTH5

Note that an EMTF XML has comprehensive metadata which can be used to populate the MTH5 as necessary, including Runs and Channels

Add an example EDI

Here the survey is not specified therefore we need to fill that information in ourselves otherwise an error is raised, see below.

#tf_group_02 = m.add_transfer_function(tf2)

Here we give the survey the id unknown_survey. Also note that because the data are stored as spectra in the EDI we can calculate the inverse_signal_power and residual_covariance.

tf2.survey_metadata.id = "unknown_survey"
tf_group_02 = m.add_transfer_function(tf2)
print(tf_group_02.has_estimate("covariance"))
2023-04-21 10:32:07,195 [line 1054] mth5.mth5.MTH5.get_survey - WARNING: /Experiment/Surveys/unknown_survey does not exist, check survey_list for existing names.

True

Add typical EDI file

This file only has impedance and tipper and minimal metadata, which are converted into a full transfer function for storage.

tf3.survey_metadata.id = "unknown_survey"
tf_group_03 = m.add_transfer_function(tf3)
tf_group_03.has_estimate("covariance")
2023-04-21 10:32:11,616 [line 222] mth5.groups.base.TransferFunction.get_estimate - ERROR: residual_covariance does not exist, check groups_list for existing names
False

Add an output from EMTF

A ZMM file contains the full covariance and transfer functions but has minimal metadata

print(tf4)
Station: 300
--------------------------------------------------
	Survey:            None
	Project:           None
	Acquired by:       None
	Acquired date:     1980-01-01
	Latitude:          34.727
	Longitude:         -115.735
	Elevation:         0.000
	Declination:   
		Value:     13.1
		Model:     WMM
	Coordinate System: geographic
	Impedance:         True
	Tipper:            True
	N Periods:     38
	Period Range:
		Min:   1.16364E+00 s
		Max:   1.09227E+04 s
	Frequency Range:
		Min:   9.15527E-05 Hz
		Max:   8.59372E-01 Hz

tf4.survey_metadata.id = "unknown_survey"
tf_group_04 = m.add_transfer_function(tf4)
tf_group_04.has_estimate("impedance")
True

Have a look at the MTH5 file

Everything has been filled in now in the MTH5 including metadata about runs and channels

m
/: ==================== |- Group: Experiment -------------------- |- Group: Reports ----------------- |- Group: Standards ------------------- --> Dataset: summary ...................... |- Group: Surveys ----------------- |- Group: CONUS_South --------------------- |- Group: Filters ----------------- |- Group: coefficient --------------------- |- Group: fap ------------- |- Group: fir ------------- |- Group: time_delay -------------------- |- Group: zpk ------------- |- Group: Reports ----------------- |- Group: Standards ------------------- --> Dataset: summary ...................... |- Group: Stations ------------------ |- Group: NMX20 --------------- |- Group: NMX20a ---------------- --> Dataset: ex ................. --> Dataset: ey ................. --> Dataset: hx ................. --> Dataset: hy ................. --> Dataset: hz ................. |- Group: NMX20b ---------------- --> Dataset: ex ................. --> Dataset: ey ................. --> Dataset: hx ................. --> Dataset: hy ................. --> Dataset: hz ................. |- Group: Transfer_Functions ---------------------------- |- Group: NMX20 --------------- --> Dataset: inverse_signal_power ................................... --> Dataset: period ..................... --> Dataset: residual_covariance .................................. --> Dataset: transfer_function ................................ --> Dataset: transfer_function_error ...................................... |- Group: unknown_survey ------------------------ |- Group: Filters ----------------- |- Group: coefficient --------------------- |- Group: fap ------------- |- Group: fir ------------- |- Group: time_delay -------------------- |- Group: zpk ------------- |- Group: Reports ----------------- |- Group: Standards ------------------- --> Dataset: summary ...................... |- Group: Stations ------------------ |- Group: 300 ------------- |- Group: 300a -------------- --> Dataset: ex ................. --> Dataset: ey ................. --> Dataset: hx ................. --> Dataset: hy ................. --> Dataset: hz ................. |- Group: Transfer_Functions ---------------------------- |- Group: 300 ------------- --> Dataset: inverse_signal_power ................................... --> Dataset: period ..................... --> Dataset: residual_covariance .................................. --> Dataset: transfer_function ................................ --> Dataset: transfer_function_error ...................................... |- Group: SAGE_2005 ------------------- |- Group: SAGE_2005a -------------------- --> Dataset: ex ................. --> Dataset: ey ................. --> Dataset: hx ................. --> Dataset: hy ................. --> Dataset: hz ................. |- Group: Transfer_Functions ---------------------------- |- Group: SAGE_2005 ------------------- --> Dataset: inverse_signal_power ................................... --> Dataset: period ..................... --> Dataset: residual_covariance .................................. --> Dataset: transfer_function ................................ --> Dataset: transfer_function_error ...................................... |- Group: unknown_survey_001 ---------------------------- |- Group: Filters ----------------- |- Group: coefficient --------------------- |- Group: fap ------------- |- Group: fir ------------- |- Group: time_delay -------------------- |- Group: zpk ------------- |- Group: Reports ----------------- |- Group: Standards ------------------- --> Dataset: summary ...................... |- Group: Stations ------------------ |- Group: TEST01 ---------------- |- Group: TEST01a ----------------- --> Dataset: ex ................. --> Dataset: ey ................. --> Dataset: hx ................. --> Dataset: hy ................. --> Dataset: hz ................. |- Group: Transfer_Functions ---------------------------- |- Group: TEST01 ---------------- --> Dataset: period ..................... --> Dataset: transfer_function ................................ --> Dataset: transfer_function_error ...................................... --> Dataset: channel_summary .............................. --> Dataset: tf_summary .........................

Get a transfer function object from MTH5

To retrieve a transfer function from the MTH5 file a convenience function m.get_transfer_function is supplied. You only need to know the station.id, tf.id, and the survey.id. Here the tf.id is the same as the station.id.

tf1_h5 = m.get_transfer_function(tf1.station_metadata.id, tf1.tf_id, tf1.survey_metadata.id)
print(tf1)
print(tf1_h5)
Station: NMX20
--------------------------------------------------
	Survey:            CONUS South
	Project:           USMTArray
	Acquired by:       National Geoelectromagnetic Facility
	Acquired date:     2020-09-20
	Latitude:          34.471
	Longitude:         -108.712
	Elevation:         1940.050
	Declination:   
		Value:     9.09
		Model:     WMM
	Coordinate System: geographic
	Impedance:         True
	Tipper:            True
	N Periods:     33
	Period Range:
		Min:   4.65455E+00 s
		Max:   2.91271E+04 s
	Frequency Range:
		Min:   3.43323E-05 Hz
		Max:   2.14844E-01 Hz
Station: NMX20
--------------------------------------------------
	Survey:            CONUS South
	Project:           USMTArray
	Acquired by:       National Geoelectromagnetic Facility
	Acquired date:     2020-09-20
	Latitude:          34.471
	Longitude:         -108.712
	Elevation:         1940.050
	Declination:   
		Value:     9.09
		Model:     WMM
	Coordinate System: geographic
	Impedance:         True
	Tipper:            True
	N Periods:     33
	Period Range:
		Min:   4.65455E+00 s
		Max:   2.91271E+04 s
	Frequency Range:
		Min:   3.43323E-05 Hz
		Max:   2.14844E-01 Hz

Summarize what transfer functions are in the file

In the MTH5 file there is a property called tf_summary, this provides an array that can be converted to a pandas.DataFrame of the transfer functions within the file. There is a column called hdf5_reference which can be used to get the transfer function directly. This table is updated when the file is close, so when you open the file next it should be up to date. This table should be read only, if you want to change metadata, you should do it directly in the transfer function object.

m.tf_summary.clear_table()
m.tf_summary.summarize()
tf_df = m.tf_summary.to_dataframe()
tf_df
Loading...

Get TF from reference

tf_object = m.from_reference(tf_df.iloc[0].hdf5_reference)

print(tf_object)
Station: NMX20
--------------------------------------------------
	Survey:            CONUS South
	Project:           USMTArray
	Acquired by:       National Geoelectromagnetic Facility
	Acquired date:     2020-09-20
	Latitude:          34.471
	Longitude:         -108.712
	Elevation:         1940.050
	Declination:   
		Value:     9.09
		Model:     WMM
	Coordinate System: geographic
	Impedance:         True
	Tipper:            True
	N Periods:     33
	Period Range:
		Min:   4.65455E+00 s
		Max:   2.91271E+04 s
	Frequency Range:
		Min:   3.43323E-05 Hz
		Max:   2.14844E-01 Hz

MTpy

To analyze, plot, prepare input files one should look to use MTpy. Note: MTpy version 2.0 will use MTH5 as the storage mechanism and TF to read/write files.

m.close_mth5()
2023-04-21 10:32:58,441 [line 753] mth5.mth5.MTH5.close_mth5 - INFO: Flushing and closing transfer_function_example.h5
Filters
Remove Instrument Response
Reports
Add Report