wiki:import_ida

Version 1 (modified by branden, 9 years ago) (diff)

--

Earthworm Module: import_ida

Contributed by: David Chavez

Function

This program enables an Earthworm to retrieve and process continuous trace data from numerous IDA data servers.

Details

Import_ida acquires data from one or more Project IDA data servers, reformats them into TracePackets and loads them into an Earthworm transport ring. The communication with the data server(s) is done using the idatap-1.2.1 toolkit available from the University of California, San Diego via

ftp://idahub.ucsd.edu:/pub/pickup/idatap-1.2.1.tar.Z

In the import_ida.d parameter file the user specifies a list of servers with which to connect, and the list of stations and channels to acquire from each server. The program forks separate processes to independently manage all communication with the various servers. These processes communicate through a pipe to distinct threads in the parent. As data packets are acquired, each child process passes them back to its parent thread, which loads the packets into a message queue that feeds the transport ring via the main thread. In addition, any child log messages are formatted into special packets that are also passed back to the main thread and logged using the standard Earthworm facilities.

Network timeouts are dealt with transparently to the application by the idatap library, which will retry and reconnect as necessary until successfull or until a more serious error occurs, at which point import_ida assumes the responsibility for reestablishing the connection. The import_ida.d parameter file includes an option for specifying the policy to follow when reconnecting after a loss of communication. One may chose to request the next packet in sequence (to minimize gaps in the acquired data set) or to request the most recently available data (to minimize delays).

Helpful Hints