Changes between Version 3 and Version 4 of wave_serverV


Ignore:
Timestamp:
02/27/12 18:38:22 (9 years ago)
Author:
branden
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • wave_serverV

    v3 v4  
    22 
    33= [wiki:Earthworm Earthworm] Module: wave_serverV = 
    4 '''Contributed by: Will Kohler, Alaska Geophysical Institute; Lynn Dietz; Kent Lindquist; Alex Bittenbinder; Mac McKenzie; Eureka Young; Dave Kragness; Pete Lombard (for more details see "History" below''' 
     4'''Contributed by: Will Kohler, Alaska Geophysical Institute; Lynn Dietz; Kent Lindquist; Alex Bittenbinder; Mac !McKenzie; Eureka Young; Dave Kragness; Pete Lombard (for more details see "History" below)''' 
    55 
    66== Function == 
     
    3939The server runs through the list of tanks from the structure file, verifying that each SCNL is listed in the config file. The one piece of information that wave_serverV takes from the config file in this stage is the index file size. Wave_serverV tries to open each the tank file and its index. If index files are missing or out of date, they are recreated by reading through the tank file. Depending on the amount of reconstruction needed for an index, this process may take several minutes for each tank. Note that there is no provision for checking the insertion point, read from the structure file, against the tank file. 
    4040 
    41 If the tank and index files are read successfully, then that tank is marked as OK. If there are errors opening these files for a tank, then one of two things may happen. If !ReCreateBadTanks is set, then new (empty) tank and index files are created using the information from the tank structure file. If ReCreateBadTanks is not set, then that tank is marked as BAD for later disposition. Once this loop has been completed for all the tanks listed in the structure file, stage one is complete. 
     41If the tank and index files are read successfully, then that tank is marked as OK. If there are errors opening these files for a tank, then one of two things may happen. If !ReCreateBadTanks is set, then new (empty) tank and index files are created using the information from the tank structure file. If !ReCreateBadTanks is not set, then that tank is marked as BAD for later disposition. Once this loop has been completed for all the tanks listed in the structure file, stage one is complete. 
    4242 
    4343For stage two of the startup sequence, wave_serverV scans the list of tanks in the config file. Any tanks that were not already found in the structure file will be created using the parameters listed in the config file. Any SCNLs added to the config file since wave_serverV was last run will be created now. If any errors are encountered creating new files here, that tank will be marked as BAD. Since the config file does not list the insertion point, any data that is in existing tank files for this SCNL will be effectively erased. 
     
    5959=== How to Make Configuration Changes === 
    6060 
    61 After you have been running wave_serverV for a while, you will eventually find that you need to make come configuration changes. While you can always shut down the server, delete all the tank, index, and structure files, and start with a new configuration, this is usually not necessary or desirable. Depending on what changes you need to make, existing tank files can often be preserved. Below is a description of how to change each of the tank, index and structure file parameters. '''These procedures depend on having !PleaseContinue set to one, and !ReCreateBadTanks not set.''' If your configuration file does not currently have these values, change the file now to include these values. When you restart wave_serverV as one of the steps below the new values for PleaseContinue and ReCreateBadTanks will take affect immediately. 
     61After you have been running wave_serverV for a while, you will eventually find that you need to make come configuration changes. While you can always shut down the server, delete all the tank, index, and structure files, and start with a new configuration, this is usually not necessary or desirable. Depending on what changes you need to make, existing tank files can often be preserved. Below is a description of how to change each of the tank, index and structure file parameters. '''These procedures depend on having !PleaseContinue set to one, and !ReCreateBadTanks not set.''' If your configuration file does not currently have these values, change the file now to include these values. When you restart wave_serverV as one of the steps below the new values for !PleaseContinue and !ReCreateBadTanks will take affect immediately. 
    6262 
    6363Be sure you understand how to '''restart''' wave_serverV. If you inadvertently shut down wave_serverV without letting it go through its normal shutdown sequence, you risk doing damage to the tank, index and structure files. In the following discussion, restart means a quick but graceful shutdown and startup of wave_serverV, using the method appropriate for your platform and wave_serverV version. When some action must be taken between wave_serverV shutdown and startup, that will be spelled out. 
     
    6666 
    6767 * '''Add an SCNL (tank)?''' Just add a new Tank to the config file, restart wave_serverV, and the tank for the new SCNL will be created. 
     68 
    6869 * '''Delete an SCNL?''' Delete the Tank line for that SCNL from the config file and shut down the wave_server. Delete (or move) the tank file and its associated index file(s). Then start the wave server. 
    69  * '''Change the Station/Component/Network of a tank?''' Since wave_serverV uses the Station, Component, Network, and Location (SCNL) names to uniquely identify trace data, a change to any one of these must be done by deleting the old SCNL and adding the new one. If there is useful data in the tank file for the old SCNL, you probably would want to keep the old SCNL in wave_serverV for a few days after you add the new one. Unless you have near the maximum allowed number of tank files or are running out of disk space, a tank file can be kept in wave_serverV indefinitely without having new data added to it. 
     70 
     71 * '''Change the Station/ Component/ Network of a tank?''' Since wave_serverV uses the Station, Component, Network, and Location (SCNL) names to uniquely identify trace data, a change to any one of these must be done by deleting the old SCNL and adding the new one. If there is useful data in the tank file for the old SCNL, you probably would want to keep the old SCNL in wave_serverV for a few days after you add the new one. Unless you have near the maximum allowed number of tank files or are running out of disk space, a tank file can be kept in wave_serverV indefinitely without having new data added to it. 
     72 
    7073 * '''Change the pin number for a tank?''' Although pin numbers are (sometimes) recorded by wave_serverV and reported by getmenu, these numbers are not currently used by wave_serverV. So no special action is needed for wave_serverV when you change the pin number assigned to an SCNL. 
     74 
    7175 * '''Change the sample rate for an SCNL?''' The sample rate is a critically important value that wave_serverV reads from the trace_buf messages it records. It uses this number for its conversions between time and sample number when wave_serverV responds to requests for trace data in ASCII format. So it is important not to mix trace data of different sample rates within one tank file. Usually, a change of sample rate will require a change in the component designation (such as is specified by Appendix A of the SEED manual); but this may not always be true. 
     76 
    7277 * '''Change recsize (record size) of a tank?''' Wave_serverV considers a tank file to be made up of a sequence of records, all of them `recsize' bytes long. One of these records holds one trace_buf message. If you need to change the records size, you must get rid of the old tank file and the information for that tank in the structure file(s). You must shut down wave_serverV, remove the tank and index file(s) for that SCNL, and then start wave_serverV. That will remove the old information from the structure file(s). To get the new tank going with the new information, you change the recsize value in the config file, and then restart wave_serverV. 
     78 
    7379 * '''Change tank file size?''' The tank file has a moving `starting point': as new data is added within the tank, old data is overwritten. So there is no way to extend the size of an existing tank. If you need to change the tank size, you must get rid of the old tank file and the information for that tank in the structure file(s). You must shut down wave_serverV, remove the tank and index file(s) for that SCNL, and then start wave_serverV. That will remove the old information from the structure file(s). To get the new tank going with the new information, you change the recsize value in the config file, and then restart wave_serverV. 
     80 
    7481 * '''Change installation ID or module ID for a tank?''' The installation ID and module ID in the Tank command are used as filters to select which source module should be used for trace data. This information is stored in the tank structure file(s) for wave_serverV. There is currently no way to change this information without effectively destroying the information in the tank file. You must shut down wave_serverV, remove the tank and index file(s) for that SCNL, and then start wave_serverV. That will remove the old information from the structure file(s). To get the new tank going with the new information, you change the installation or module ID values in the config file, and then restart wave_serverV. 
     82 
    7583 * '''Change index file size?''' The index file is always written in order, newest entry to oldest, so changing the file size is quite easy. Make the desired change in the config file and restart wave_serverV. If for some reason you are reducing the index file size, the oldest entries may get deleted. That will make any trace data to which those index entries refer inaccessible to wave_serverV. But it will not otherwise affect wave_serverV operation. 
     84 
    7685 * '''Change tank file name or location?''' There is no way to change the name or location of an existing tank file. You must shut down wave_serverV, remove the tank and index file(s) for that SCNL, and then start wave_serverV. That will remove the old information from the structure file(s). To get the new tank going with the new information, you change the tank file name in the config file, and then restart wave_serverV. 
     86 
    7787 * '''Change structure file name or location?''' The pathname of the structure file is recorded only in the configuration file, so changing it is possible. Make the change in the config file, shut down wave_serverV, move or rename the structure file to the desired location, and start wave_serverV. Make sure when you rename the structure file that it exactly matches the name that you give in the config file. Also note there is one command to name each of the structure files (if you are using two of them.) 
     88 
    7889 * Change the number of index files? Changing the number of index files is simple: make the change to the config file and restart wave_serverV. If you have turned on redundant index files, new ones will be created for each tank. If you have turned off redundant index files, the second file for each tank will be ignored. But why would you want to do that? 
     90 
    7991 * '''Change the number of structure files?''' There is no provision in wave_serverV to generate a structure file from existing tank tiles. If you need to add a redundant structure file, make the changes to the config file. This requires both setting !RedundantTankStructFiles to 1, and specifying the name with !TankStructFile2. Then shut down wave_serverV, copy the existing tank structure file to the new name, and start wave_serverV. 
     92 
    8093 * '''Change How Asynchronous Data is Handled?''' Wave_serverV can now be configured to deal with asynchronous data. Data packets that overlap in the data stream can be buffered to a database file. As data is supplied to client applications this data can be resynchronized into the stream. There are a few configuration commands that can be used to control this behavior. To turn on this processing set !UsePacketSyncDb to 1. The database name defaults to TB2PACKETS.SL3DB but path and filename may be specified with the !PacketSyncDbFile setting. The packet database is purged occasionally during runtime, deleting any packets older than the oldest packet in the tanks. The !PurgePacketSyncDb may be used to tell wave_serverV to purge the database on startup. These settings may be modified in the configuration file followed by restarting wave_serverV. 
     94 
    8195 * '''Change any other wave_serverV parameter?''' All other wave_serverV configuration parameters not listed above may be changed using this procedure: make the change to the config file and restart wave_serverV. 
    8296 
     
    8599Three simple utilities are available to assist with wave_serverV problems. They basicly read the tank, index, and structure files and write out their contents in human-readable form. 
    86100 
    87 * Inspect_tank will read a tank file, write out a summary of the contents similar to read_index, and optionally write a new index file. This can be used to generate a replacement index file off-line from wave_serverV. Usage is: 
     101 * Inspect_tank will read a tank file, write out a summary of the contents similar to read_index, and optionally write a new index file. This can be used to generate a replacement index file off-line from wave_serverV. Usage is: 
    88102{{{ 
    89103inspect_tank [-g gap-size] rec-size tanksize tankfile-name 
     
    182196 
    183197The original Wave_server module was written by Will Kohler in a rather spectacularly short time: We came in on a Monday to find Will comatose, all waste cans full of espresso cups, and a working wave server. The motivation was to support the effort with the Alaska Geophysical Institute to integrate Earthworm with DataScope and to provide a playback facility for testing real-time algorithms. Lynn Dietz then proceeded to add numerous enhancements. It stores all trace data messages received, and servers all trace data received during a specified time interval. It can thus be used to recreate the the pattern of trace data messages inside an Earthworm system during a specified period of time. This has proven to be valuable for testing and debugging, and this module still exists as "wave_server". 
    184 Kent Lindquist then produced an enhanced version, including the idea of segmenting the tank into one partition for each trace. Since then, several authors were involved in wave_server development: Alex Bittenbinder wrote the main thread; Mac McKenzie wrote the parser of the client thread (server_thread.c); Eureka Young wrote the server routines; Dave Kragness pretty much re-wrote the main thread while implementing crash-recovery, and Pete Lombard produced the suite of associated client routines. 
     198Kent Lindquist then produced an enhanced version, including the idea of segmenting the tank into one partition for each trace. Since then, several authors were involved in wave_server development: Alex Bittenbinder wrote the main thread; Mac !McKenzie wrote the parser of the client thread (server_thread.c); Eureka Young wrote the server routines; Dave Kragness pretty much re-wrote the main thread while implementing crash-recovery, and Pete Lombard produced the suite of associated client routines. 
    185199 
    186200=== TROUBLESHOOTING === 
    187201 
    188 From Lynn Dietz 8/07: The waveserver messages "Circular queue lapped : 3456 messages lost." mean that waveserver is not writing data to its tanks fast enough. This is an obvious source of gaps in your waveserver tanks. [To fix this you] can try increasing "InputQueueLen". Also, 270 channels is a lot for one waveserver process to handle. We usually try to limit each waveserver process to ~100 channels on one disk dedicated to that process. 
     202From Lynn Dietz 8/07: The waveserver messages "Circular queue lapped : 3456 messages lost." mean that waveserver is not writing data to its tanks fast enough. This is an obvious source of gaps in your waveserver tanks. [To fix this you] can try increasing "!InputQueueLen". Also, 270 channels is a lot for one waveserver process to handle. We usually try to limit each waveserver process to ~100 channels on one disk dedicated to that process. 
    189203 
    190204== More Details ==