Changes between Initial Version and Version 1 of v6.3

01/17/12 12:56:45 (10 years ago)



  • v6.3

    v1 v1  
     3Earthworm Release Notes V6.3 
     5(Sept 26, 2005) 
     8Release Notes:  Earthworm Version "working" 
     9Cleaned out after release of v6.2 on 4/15/03 
     11Modified Exlusively by Paul Friberg  (PAF) after 4/15/2004 
     12to build interim version 6.3 release. Between 4/15/2003 
     13and 4/15/2004 the notes were updated as part of regular 
     14bug fixes and mods so any notes during that time are 
     15attributable to the original author. After 4/15/2004 
     16any notes are from PAF with respect to incorporating 
     17new code and bug fixes between v6.2 and v6.3. 
     21This will be the LAST SCN version of earthworm and 
     22the v7.0 will contain Location Codes for all programs. 
     24NEW MODULES: 
     28(added to CVS by Paul Friberg 6/29/2005, authored by Mark Morrison) 
     29From the README.txt found in the src/system_control/startstop_sevice: 
     30startstop_service is identical to the old startstop_nt, except that 
     31it runs as a Windows service.  This means that the parameters are all the 
     32same - read from startstop_nt.d - but that startstop isn't just executed 
     33from a command window or via the scheduler.  Note that this version is 
     34taken from startstop_nt, so I haven't compiled or tested any of this under 
     35Solaris or other systems.  I believe it's generally felt, however, 
     36that these modifications don't apply to Solaris. 
     39(added to CVS by Paul Friberg, authored by Hal Schehner of 
     40This is a EW module which pulls data from a Quanterra 330 
     41digitizer and outputs trace_buf packets onto a ring, much like  q2ew 
     42does for older quanterras. 
     44srparxchewsend - Symmetric Research Digitizer module in 
     45data_sources for use with their PC-based A/D boards. For more info 
     46see them at This compiles under Linux and Windows, 
     47but not yet under Solaris. 
     49slink2ew - Chad Trabant contributed this application to pull data from 
     50seedlink into Earthworm. The version provided with EW v6.3 is  
     51version 1.2.2 last modified by Chad on 2005.07.14 
     52Here's the slink2ew note from the v7.0 release of EW for future reference: 
     53        A seedlink client for acquiring data from seedlink servers (e.g. IRIS) 
     54        and converting to tracebuf2 (or optionally tracebuf). Much more robust 
     55        than liss2ew. Written by Chad Trabant of IRIS; modified for tracebuf2 
     56        and debugged by Chad Trabant and Jim Luetgert. 
     57        JHL 4/19/05 
     60(created by Alexandre Nercessian), imported into Earthworm by PAF. 
     61This version compiles and runs under Linux. It does not have correct 
     62reporting of run time for a module, but otherwise works. It uses and 
     63looks for a startstop_unix.d file (not a startstop_sol.d file as in the 
     64previous version with ew6.2-lnx). PAF 7/21/2005 
     66grf2ew - Generic Recording Format to earthworm. This is for DAQ systems 
     67digitizers. This compiles under Linux and should under Windows too. I 
     68only installed this in the release and did not test it. PAF 9/10/2005 
     73Had a restart option added in by Murray McGowan of GSL (9/6/2005). This allows 
     74a user to restart any module from startstop by typing "restart nnn" followed 
     75by the module name or the process id. 
     78Added in the same restart command as above. It's a nice feature to have. (9/7/2005) 
     81Added: #include  
     82Function tport_bufthr() now returns NULL.  WMK 3/22/04 
     85Added: #include  
     88Changed: "static struct k_buf" to "struct k_buf"   WMK 3/22/04 
     91Added: #include    WMK 3/22/04 
     94Added a "Total Gap" column to the output tables.  "Total Gap" is the 
     95total, in seconds, of the length of all gaps for a particular channel. 
     96"Total Gap" gets reset to 0 when the day rolls over.  Also, added a 
     97column named "Dead Time", which is the elapsed time since a channel 
     98died.  If "Dead Time" is blank, the channel is currently alive. 
     99"Dead Time" does NOT reset to 0 when the day rolls over. WMK 1/8/04 
     101gaplist: Slight table reformat.  WMK 1/20/04 
     105Increased BUFFSIZE from 200 to 250, to accomodate more email recipients. 
     106This is a kludgy fix.  It might better to allocate the buffer on the fly, 
     107to accommodate any number of email messages.  Also, note that I did not 
     108increase BUFFSIZE in libsrc/solaris/sendmail.c  WMK 6/11/03 
     111Fixed endTime bug in makeEwSnippet(). Thanks Jim L. 
     112Alex 5/9/03 
     115The last two arguments to ewdb_api_GetSMDataWithChannelInfo were 
     116reversed. That meant that the number of returned SM messages would be 
     117incorrect IF that number exceeded MaxDataPerEvent. The result would be an 
     118out-of-bounds memory access and corrupted SM data. PNL 6/9/03 
     121Changed the sorting used in Filter_SM_data. Now SM messages are sorted by SCNL 
     122instead of by idchan. That means that the output XML will have components of 
     123one station listed consecutively, a big help for shakemap. PNL 10/30/03 
     126Changed the length of the long station name in the mapping file from  
     12720 to 50 characters.  LDD 2/5/04 
     129skakemapfeed:  (EWDB API - Strong Motion) ewdb_internal_GetAllSMMessages() 
     130(Note: This function is called by ewdb_api_GetSMDataWithChannelInfo()) 
     131Fixed a bug in PostGetAllSMMessages().  The idEvent field may be null  
     132for some records, and Oracle  does not appear to be writing a 0 value to  
     133the memory area for the null field, so that record assumes the  
     134idEvent of the previous record located at that point 
     135in the internal buffer.  The result is data corruption, such that you get 
     136messages that are unassociated with an Event, but they appear to be associated 
     137with an event. 
     138DK 02/12/04 
     140shakemapfeed: Added new optional command, SMQueryMethod, to control how SM data 
     141is requested from the DBMS. Default behavior is original 2-stage query (get 
     142all SM data associated with this eventid, then get all UNassociated SM data 
     143in a space/time box). If SMQueryMethod is non-zero, shakemapfeed will request 
     144all SM data in a space/time box. This will return SM data associated with this 
     145event, SM data associated with other events, and UNassociated SM data.  
     146Regardless of query method, post-query filtering will be performed on eventid, 
     147external eventid, and external author.  SM data associated with a different 
     148eventid but with the same author will be rejected.  Data with a different 
     149author will be allowed - this is most likely data imported from another 
     150network and its eventids are meaningless in our DBMS.  LDD 02/13/2004 
     152shakemapfeed: Changed the auto-feed scheduling logic to allow for variable 
     153time intervals between feeds to shakemap.  Feeds are scheduled at 
     154fixed times after the arrival of the archive message by using the new 
     155command, ScheduleFeed (one time per ScheduleFeed command, up to 20 allowed).  
     156Auto-feeds will continue at the last ScheduleFeed interval until UpdateDuration  
     157minutes after the event origin time.  ScheduleFeed replaces the original  
     158DelayFirstFeed and UpdateInterval commands. These original commands are still  
     159accepted and will set up the auto-feed schedule in the original manner in the  
     160absence of ScheduleFeed commands.  LDD 3/4/2004 
     163The channel bitmap bug reported below has been fixed. K2ew now reads the 
     164correct bitmap to determine which channels are streaming. PNL 6/9/03 
     166liss2ew, ew2liss, dumpseed: 
     167There was a bug on Microsoft based systems that caused the user defined 
     168types BYTE, WORD, and LONG to be ignored.  This was a problem specifically 
     169with the type WORD, which was changed from a signed 16-bit value to an 
     170unsigned 16-bit value, causing many problems, most noteably with the  
     171signed seed varibles for sample rate and sample rate multiplier.  All  
     172liss based modules have had this problem corrected. JMP 06/17/2003 
     175There was a feature in getfile_ew that could cause more cpu to be 
     176used than necessary by checking time since last heartbeat more 
     177often than necessary.  Added sleep_ew(1000) within heartbeat 
     178loop. MMW 06/29/2003 
     181Ported getfileII to FreeBSD.  Two new files are in the source directory: 
     182makefile.bsd and socket_bsd.c.  No other changes to files. WMK 11/25/03 
     185Added functions to properly initialize SUDS structures to  
     186Banfill's default values. CJB 10/3/2003 
     189Updated to make more compatible with suds.h in current 
     190release of PCSUDS. CJB 10/3/2003 
     193Fixed bug in raw2ew.c which caused file2ew to exit when the  
     194SuffixType command contained only 2 arguments.  The third  
     195argument (installation ID) was supposed to be optional, and  
     196now is!  LDD 10/3/2003 
     199Fixed bug in timing of heartbeat file generation. 
     200Had been using HeartBeatInt instead of HeartFileInt to control 
     201the production loop.  Caused continuous heartbeat file generation 
     202when HeartFileInt==0.  LDD 11/24/2003 
     205Changed to sniff headers of both TYPE_TRACEBUF and TYPE_TRACE_COMP_UA msgs. 
     206Will only print data values for TYPE_TRACEBUF messages.  Changed to accept 
     207the any of these strings as wildcards in SCN: "wild", "WILD", "*". 
     208LDD 10/31/2003 
     211Added directives for STANDALONE_MODULES (right now getfileII and sendfileII) 
     212so that their executables, which are created in their source directories,  
     213are also copied to the vX.X/bin directory.  LDD 11/27/2003 
     216Changed rd_strongmotionII() to interpret both "-" and "?" as NULL strings 
     217when reading the QID: line qid and qauthor fields.  Previously only 
     218accepted "?" as a NULL string.  Change was required by CISN data which  
     219uses "-" as a NULL string in those fields.  LDD 2/13/2004 
     222Ported to BSD Unix.  Added a new makefile, named makefile.bsd. 
     223To compile, type "make -f makefile.bsd".  Otherwise, everything is the 
     224same as on Solaris.  WMK 2/17/04. 
     226EWDB - Station Information: 
     227Modified the two stored procedures that handle the processing of SCNL 
     228from station list tools (stalist_XXX2ora) and automatic data (trace feed, 
     229automatic pick feed, strong motion, etc.). 
     230Modified the procedures to properly process the Location Code of the 
     231SCNL.  Prior to this, Location codes weren't properly being handled. 
     232Because this change is to SQL procedures and not C code, you must  
     233refresh the DB stored procedures by updating with the SQL scripts from 
     234"working".  (run ewdb_load_external_utils.sql from schema/sql_scripts). 
     235DK 02/17/04 
     241shakemapfeed.d: Added new optional command, SMQueryMethod, to control how SM 
     242data is requested from the DBMS. Default behavior (SMQueryMethod 0) is original  
     2432-stage query (get all SM data associated with this eventid, then get all  
     244UNassociated SM data in a space/time box). If SMQueryMethod is non-zero,  
     245shakemapfeed will request all SM data in a space/time box. This will return  
     246SM data associated with this event, SM data associated with other events,  
     247and UNassociated SM data. 
     248  Regardless of query method, post-query filtering will be performed on eventid, 
     249external eventid, and external author.  SM data associated with a different 
     250eventid but with the same author will be rejected.  Data with a different 
     251author will be allowed - this is most likely data imported from another 
     252network and its eventids are meaningless in our DBMS.  LDD 02/13/2004 
     254shakemapfeed.d: New command "ScheduleFeed " 
     255sets the shakemap feeding schedule. Up to 20 ScheduleFeed commands are 
     256allowed, setting one time per command.  Auto-feeds will continue at the 
     257last ScheduleFeed interval until UpdateDuration minutes after the event 
     258origin time. ScheduleFeed replaces the original DelayFirstFeed and UpdateInterval  
     259commands. These original commands are still accepted and will set up  
     260the auto-feed schedule in the original manner in the absence of ScheduleFeed  
     261commands.  LDD 3/4/2004 
     265KNOWN BUGS: 
     269liss2ew has been observed intermittantly producing malformed     
     270TRACE_BUF messages.  Currently the conditions for causing this problem    
     271are unknown.  Due to this, liss2ew should be treated as suspect.  Use at  
     272your own risk.  JMP 6-18-2003                                             
     275k2ew uses the k2hdr.rwParms.misc.channel_bitmap parameter to decide 
     276which channels it will see as streams from the K2.  This is actually the 
     277parameter which the K2 uses to decide which channels to record in an event 
     278file (see K2's  command).  The K2's  is the parameter 
     279that shows which channels the K2 is streaming out.  This is the parameter 
     280that k2ew should be using to decide what data it will see.  The big problem 
     281is that none of the header files from Kinemetrics (nkwhdrs.h) seem to  
     282show this parameter anywhere. This issue is only a problem for those folks 
     283who want to record more channels on the K2 than they want to stream continuously. 
     284Terry Dye (Univ of Utah) discovered this problem when trying to stream only 
     2853 channels of a 12 channel K2.  LDD 4/11/2002    FIXED PNL, 6/9/03 
     289it occasionally sends the following error to statmgr: 
     290   UTC_Thu Sep 06 03:30:14 2001  WSV1/wave_serverV_nano ReadBlockData 
     291   failed for tank [z:\nano57.tnk], with offset[999908] and record 
     292   size[64]! errno[0] Mail sent. 
     293The nominal tank size is 1 megabyte, and the actual tank size is 999908. 
     294It looks like waveserver is trying to read off the end of the tank. 
     295WMK 9/6/01 
     298appears not to reply to requests for a single sample of data. I noticed  
     299when testing wave_viewer, that if the start time and end time of a request were  
     300equal (in which case there should be one sample of data) then wave_serverV did  
     301not reply to the request (ASCII request) at all. No Data, No Flags, No Reply,  
     302No Nothing.  It needs to issue a reply to every request. 
     303DavidK 09/25/01 
     305Automatic restarts of adsend (using the "restartMe" line in the descriptor 
     306file) can cause an NT system to hang. Therefore, you should never 
     307use the autorestart feature with adsend, but you should bring down 
     308the entire Earthworm system if adsend needs to be restarted. 
     309LDD 5/31/2000 Comments added to adsend.desc, but leave this warning here! 
     311libsrc/utils/site.c: The strings used for station, channel and network are 
     312required to be fixed length with trailing spaces added to short names. If 
     313the strings given to site_index do not have these trailing blanks, SCN names 
     314will not match. This is not documented anywhere.  PNL 10/15/00 
     316socket_ew: (libsrc/util/socket_ew_common.c libsrc/solaris/socket_ew.c 
     317            libsrc/winnt/socket_ew.c include/socket_ew.h) 
     318Fixed a bug in the connect_ew function(). 
     319When run in non-blocking mode (clients connecting to 
     320a server - using a timeout value), there was a bug in 
     321the connection code(discovered on Solaris) that caused 
     322the function to return a timeout-error when there was 
     323any kind of error during connection. 
     324The bug was discovered when connecting to a non-existent socket. 
     325When trying to connect to a non-existent wave_server on a machine, 
     326the underlying socket library was returning a Connection-Refused error, 
     327but the socket_ew library was passing back a timeout error. 
     329The change only applies when a socket-error occurs while the function is 
     330waiting for the connect to happen.  The instance where you will most-likely 
     331see a difference, is when you try connecting to a non-existent socket. 
     332Previously the function would return TIMEOUT, now it will return 
     333connection REFUSED. 
     335Added a new function: socketSetError_ew(), 
     336and defined a new socket_ew return code: CONNREFUSED_EW. 
     337DK 2003/02/04 
     340There is a bug in sm_ew2ora that involves having multipe time intervals for 
     341components and channels.  If a strong motion message containing information 
     342for a channel that the DB has never seen before is loaded into the DB, and 
     343then later another message for the same channel with an earlier timestamp 
     344is loaded, the load of the second message will fail, due to problems with 
     345overlapping time intervals, and the call that sm_ew2ora uses to create 
     346those time intervals.  This problem only affects stations that were 
     347not previously loaded via one of the station loader programs stalist*2ora, 
     348and only when receiving SM data that is timestamped with a time that is 
     349prior to the original time for that channel.  The bug lies in the logic 
     350of ewdb_api_PutSMMessage(), and not in the underlying code. 
     351Davidk 05/25/01 
     353A change was made to ewdb_api_PutSMMessage() that dramatically affects 
     354sm_ew2ora.  Please see the note about that function.  Davidk 2001/07/26  
     356ewdb_api_CreateWaveform()  (ewdb_internal_CreateSnippet.c  
     357                            ewdb_internal_CreateWaveformDesc.c) 
     358Added a call to release the cursors used by the internal functions, when they 
     359fail.  Fixes a bug which resulted in a DB cursor leak when a call to stuff 
     360a snippet into the DB failed. 
     361DK 2003/02/04 
     3641) Added a new function logit_core() 
     365int logit_core( char *flag, char *format, va_list ap); 
     366This function is the same as logit(), except that it takes a va_list argument 
     367for the variable length parameter instead of '...', and it has an int  
     368return value. 
     369This function can be called by other functions that receive a '...' variable  
     370argument list, where as logit() cannot.  logit_core() is to logit()  
     371what vsprintf() is to sprintf(). 
     3732) Moved all of the functionality in logit() to logit_core(), and modify  
     374logit to call logit_core(). 
     3763)  Added a new function get_prog_name2(), as an intended replacement  
     377of get_prog_name().  get_prog_name2() includes an additional 
     378parameter (the buffer length of the output buffer). 
     3804)  Modified logit_init() to use get_prog_name2() instead of get_prog_name() 
     382All four(4) of these changes should be backwards compatible with all existing 
     383earthworm code, and the modified logit.c has already been tested with several 
     384earthworm modules. 
     389Added new function ewdb_api_GetEventSummaryInfo() to retrieve summary 
     390information for an event(including the preferred szSource and szSourceEventID 
     391of the event).   011904 DK 
     398In Windows NT, the time resolution of sleep_ew() is about 16 msec (one clock 
     399tick).  On Solaris, the resolution is about 10 msec.  This is a problem for  
     400ringtocoax, since packet delays need to be set to a few milliseconds. 
     402statmgr: A space is needed between "tsec:" and the value.  
     403If it isn't there, things fail. Artifcat of the kom routines. (Alex) 
     405threads functions: The KillThread function on WindowsNT and Solaris 
     406terminate the thread without ensuring that no mutexes are held. If a thread 
     407holds a mutex when it dies, no other thread can get that mutex. PNL 1/12/2000 
     410The system time must be set to GMT and ew_nt.cmd must have  
     411TZ=GMT for carlsubtrig to work.  Comments in ew_nt.cmd done 5/25/00. Barbara 
     414needs GMT set on the system 
     417on NT, exits with horrible crash when system is stopped. 
     420Under Solaris 2.6 (and probably other versions as well), the maximum number  
     421of shared memory segments is six. This means that on an out-of-the-box machine 
     422you can only configure six rings. If you try to configure more than that, you 
     423will see a cryptic message from tport_create about too many open files.  The 
     424fix to this problem is to add the following lines to the /etc/system 
     425file, and then reboot the system. 
     427 set shmsys:shminfo_shmmax = 4294967295 
     428 set shmsys:shminfo_shmmin = 1 
     429 set shmsys:shminfo_shmmni = 100 
     430 set shmsys:shminfo_shmseg = 20 
     431 set semsys:seminfo_semmns = 200 
     432 set semsys:seminfo_semmni = 70 
     434This allows for 20 rings. 
     436     Lucky Vidmar (7/6/2000) 
     439Fixed bug in call to logit_init(). 
     440The program was printing error messages: 
     441  Invalid arguments passed in. 
     442  Call to get_prog_name failed. 
     443  WARNING: Call logit_init before logit. 
     446There MAY be a problem with the signal that  
     447startstop sends to modules during the shutdown sequence. The shutdown  
     448sequence is started (after typing "quit" to startstop or running "pau") 
     449by startstop placing a terminate message on all transport rings. Modules 
     450should see this message and start their own shutdown. After a configurable 
     451delay, startstop checks to see that all modules have exitted. Any that are  
     452still running are sent a signal to terminate them. Currently that signal 
     453is SIG_TERM. But since wave_serverV has a handler for SIG_TERM, wave_serverV 
     454sees that as essentially the same as a terminate message. So if wave_serverV 
     455is having problems completing its shutdown, SIG_TERM won't do anything. The 
     456result is that startstop may give up and exit, leaving wave_serverV running. 
     457If that happens, the operator will have to terminate wave_serverV by doing 
     458"kill -9 ". That may leave shared memory and semaphores 
     459stranded in the kernel: run the command "ipcs -a" to see. If necessary, 
     460the stranded shared memory and semaphores may be cleaned up with the 
     461ipcrc command; must be run as root; see the man page. 
     462This problem only exists on Solaris/Unix, not on WindowsNT. 
     463PNL, 10/4/2000 
     465libsrc/utils/kom.c:  fix comment in k_open() 
     467The comment above k_open() says that only one file can 
     468be open at a time. Yet the Kbuf array has slots for MAXBUF (currently 4) open 
     469files. Does this work, or is the comment to be taken at it's word? 
     470PNL 10/15/00 
     472libsrc/utils/logit.c: logit_init() requires a module_id number, which it uses 
     473to construct the log file name. This is not helpful, since the module_id 
     474number is not meaningful to people. Worse, it requires that the config file be 
     475read and earthworm.d lookups be completed before logit calls can be made. Thus 
     476errors in the config file can only be reported to stderr or stdout instead of 
     477being saved in a file.  PNL 11/29/00 
     479libsrc/util/k2evt2ew.c: This library supports a maximum EVT data size 
     480of 800000 samples per channel per EVT file.  This value is hardcoded 
     481as MAXTRACELTH in include/k2evt2ew.h.  k2evt2ew() will return a warning 
     482if the EVT file exceeds this size, and process as much as the EVT file 
     483as the hardcoded limit allows. 
     484DK 2003/01/18 
     487TRACEBUF messages. 
     488The definition of `endtime' of the TRACEBUF message is not documented. 
     489Some programmers are taking it as the "expected start time of the next 
     490TRACEBUF packet (if the sample interval is uniform.)" The more accepted 
     491practice is that `endtime' is the time of the last sample of the current 
     492TRACEBUF packet; that is, one sample interval less than the expected 
     493start time of the next TRACEBUF messsage. Using this last definition, if a 
     494TRACEBUF packet has exactly one sample, then its starttime and endtime are 
     495the identical. Clearly this distinction needs to be documented. The file 
     496waveform_format (in the /home/earthworm/DOC directory) gives no specifics  
     497about start or end times.  PNL 1/24/01