Changes between Version 1 and Version 2 of sgram


Ignore:
Timestamp:
02/26/12 13:12:35 (10 years ago)
Author:
branden
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • sgram

    v1 v2  
    44'''Contributed by: Menlo Park''' 
    55== Function == 
    6 Spectrograms. Official version is in .../src/display/ 
     6Creates spectrogram displays for the WWW. Official version is in .../src/display/ 
    77== Details == 
    88 
     9A few initial caveats: The present version is Solaris only. An NT version will be produced. 
     10 
     11Sgram is designed to rapidly provide GIF images of spectrograms for display on the WWW. Sgram periodically retrieves current data for a selected list of stations and updates GIF images which are transferred to webserver(s). The required waveforms are obtained via periodic requests to Earthworm waveserver(s). Spectra are computed, converted to a color scale and plotted directly to GIF format image files. As the required GIF files are constructed, they are copied via rcp to webserver(s) for viewing. Perl Scripts are used on the webserver(s) to dress up the data displays. Optionally, .html wrappers may be built and copied via rcp to the webserver(s) for viewing. 
     12 
     13The optional structure for the webserver consists of an index.html page listing the stations and dates available for viewing. The dates are each a hotlink to a station/date spectrogram GIF. 
     14 
     15=== How To Set It Up === 
     16 
     17The first task is to set up a directory as a holding tank for the GIF image and .html files to reside while they are waiting to be dispatched to the webserver(s). I have found it useful to set up a directory and sub-directory of form .../gifs/sgram/. This provides the opportunity for other subdirectories of /gifs/ for other image creating modules. If you are going to display a logo on your helicorder gifs, put a copy of the logo.gif in this directory. If you are using a Perl Script on the webserver(s) to generate your html display, you need do nothing further. If you are relying on sgram to create and ship your .html wrapper and you want to customize it for your site, you may need to add files indexa.html and indexb.html as custom prolog and epilog for the index.html which is shipped. 
     18 
     19Examples of a Perl Script, indexa.html and indexb.html are included. 
     20 
     21=== Setting Up Earthworm to Run Sgram. === 
     22 
     231. Make sure that your earthworm.d file contains a module id for sgram. It should be MOD_SGRAM. 
     24 
     252. Insert an entry for sgram in your startstop_sol.d file. 
     26 
     273. Now comes the fun part; setting up the configuration files. 
     28{{{ 
     29************************************************************************ 
     30# 
     31# This is the spectrogram parameter file. This module gets data gulps 
     32# from the waveserver(s), and computes spectrograms 
     33 
     34 
     35#  Basic Earthworm setup: 
     36# 
     37LogSwitch    1              # 0 to completely turn off disk log file 
     38MyModuleId   MOD_SGRAM      # module id for this instance of report 
     39RingName     HYPO_RING      # ring to get input from 
     40HeartBeatInt 15             # seconds between heartbeats 
     41 
     42>>>> 
     43>>>> The first four config variables are standard earthworm. 
     44>>>> 
     45>>>> We need to talk to the transport ring so that StartStop 
     46>>>> can terminate this process. 
     47>>>> 
     48 
     49wsTimeout 12 # time limit (secs) for any one interaction with a wave server. 
     50 
     51>>>> 
     52>>>> We need to know a reasonable time limit for getting 
     53>>>> a response from any wave server. 
     54>>>> 
     55 
     56# List of wave servers (ip port comment) to contact to retrieve trace data. 
     57 WaveServer aaa.bbb.ccc.ddd  17121      "wsv2 - ad1" 
     58 WaveServer eee.fff.ggg.hhh  18004      "wsv2 - ad2" 
     59 
     60>>>> 
     61>>>> This is the list of waveservers to try.  If the requested 
     62>>>> data for a given SCN is on more than one waveserver, the 
     63>>>> first listed will be preferentially used.  If a request 
     64>>>> fails (gap in the data, broken connection) the alternate 
     65>>>> server(s) will be tried.  The comment field is optional; 
     66>>>> it makes error messages a bit more helpful.  The maximum 
     67>>>> number of waveservers is specified by MAX_WAVESERVERS in 
     68>>>> sgram.h. 
     69>>>> 
     70 
     71# List of target computers/directories to place output. 
     72#       UserName  IP Address            Directory 
     73Target myname@webservera:/home/www/waveforms/sgram/ 
     74Target myname@webserverb:/home/www/waveforms/sgram/ 
     75 
     76>>>> 
     77>>>> Here we specify the complete location of our web pages. 
     78>>>> When setting up a directory structure on your webserver, 
     79>>>> I would recommend a directory dedicated to GIF displays. 
     80>>>> with a sub-directory dedicated to spectrogram displays. 
     81>>>> The usual stuff about making sure that all your .hosts 
     82>>>> and .rhosts tables are in order applies.  The maximum 
     83>>>> number of targets is specified by MAX_TARGETS in sgram.h. 
     84>>>> 
     85 
     86# Filename prefix on target computer.  This is useful for identifying 
     87# files for automated deletion via crontab. 
     88Prefix nc 
     89 
     90>>>> 
     91>>>> For each SCN/date, a (sometimes large) GIF file and an (optional) .html 
     92>>>> wrapper are created.  Unless there is some mechanism in place 
     93>>>> to blow away out-of-date files, the webserver will eventually 
     94>>>> drown in old files.  To fix this problem, these two types of 
     95>>>> file are assigned a prefix to readily identify them as volatile 
     96>>>> and a crontab process is used to blow them away. e.g. 
     97>>>> 15 1 * * 0-6 find /home/ehzweb/waveforms/sgram -name nc\* -mtime +7 -exec rm {} \; 
     98>>>> will remove files starting with nc which are older than 7 days. 
     99>>>> 
     100 
     101# Directory in which to store the temporary .gif, .link and .html files. 
     102GifDir   /home/www/gifs/sgram/ 
     103 
     104>>>> 
     105>>>> This is the directory on your local machine which is used to 
     106>>>> store the working versions of the GIF and .html files. 
     107>>>> For each SCN, there may also be a SCN.hist file which keeps 
     108>>>> track of which dates are available for display. 
     109>>>> This is used to construct the optional index.html file. 
     110>>>> If SaveDrifts (see below under optional commands) is set, 
     111>>>> a file SCN.drft is created for each SCN.  Each time a new 
     112>>>> spectrogram hour is started, the min and max spectral 
     113>>>> values for the previous hour of data are recorded. 
     114>>>> 
     115 
     116# Plot Parameters - sorry it's so complex, but that's the price of versatility 
     117        # The following is designed such that each SCN creates it's own 
     118        # spectrogram display; one per HoursPerPlot hours of data. 
     119# S                  Site 
     120# C                  Component 
     121# N                  Network 
     122 
     123>>>> 
     124>>>> These are the SCN identifiers as they appear in the 
     125>>>> waveserver menu. 
     126>>>> 
     127 
     128# 04 HoursPerPlot    Total number of hours per gif image 
     129 
     130>>>> 
     131>>>> I have been using 24 hours per image for overviews and 
     132>>>> 2 hours per image for detailed spectrograms. 
     133>>>> 
     134 
     135# 05 Plot Previous   On startup, retrieve and plot n previous hours from tank. 
     136 
     137>>>> 
     138>>>> This lets you fill in those little gaps left by system crashes, 
     139>>>> power outages, silly mistakes, etc.  I usually leave this 
     140>>>> set to 1 so that gaps get filled automatically if something 
     141>>>> bad happens and statmgr has to restart sgram.  This will only 
     142>>>> let you fill in past data from the current plot (i.e. <=HoursPerPlot). 
     143>>>> If you need past plots use sgram2, the offline 
     144>>>> spectrogram gif generator.  [when&if I finish it] 
     145>>>> 
     146 
     147# 06 Local Time Diff UTC - Local.  e.g. -7 -> PDT; -8 -> PST 
     148# 07 Local Time Zone Three character time zone name. 
     149# 08 Show UTC        UTC will be shown on one of the time axes. 
     150# 09 Use Local       The pages will start relative to local time rather than UTC time. 
     151 
     152>>>> 
     153>>>> These variables govern how the time axes are displayed on 
     154>>>> your plot and whether the plot starts at 00:00 UTC or local. 
     155>>>> 
     156 
     157# 10 XSize           Overall x-size of plot in inches 
     158 
     159>>>> 
     160>>>> This lets you tailor the width of your display. 
     161>>>> Setting this > 100 will imply pixels 
     162>>>> 
     163 
     164# 11 Pixels/Line     Vertical pixels per line of trace displayed 
     165 
     166>>>> 
     167>>>> I use 1 for the 24 hour displays and 4 or 5 for the 2 hour displays 
     168>>>> 
     169 
     170# 12 Minutes/Line    Number of minutes per line of trace displayed 
     171 
     172>>>> 
     173>>>> I use 1 minute/line 
     174>>>> 
     175>>>> The product of Minute/Line, Pixels/Line, and HoursPerPlot determines 
     176>>>> the overall y-size of the plot in pixels. 
     177>>>> 
     178 
     179# 13 Seconds/Gulp    Number of seconds per fft 
     180 
     181>>>> 
     182>>>> I use 60 second ffts 
     183>>>> 
     184 
     185# 14 Freq Max        Maximum frequency 
     186 
     187# 15 Freq Mute       Hz to mute at low end 
     188 
     189>>>> 
     190>>>> Some stations produce a lot of very low frequency energy which 
     191>>>> is not of interest.  This variable applies a mute at the low end. 
     192>>>> 
     193 
     194# 16 nbw             Display scaling. 1:linear, 2:log [neg for greyscale] 
     195 
     196 
     197# 17 amax            Clipping amplitude applied to spectrum [0 for none] 
     198 
     199>>>> 
     200>>>> The spectral amplitude is expressed as spectral colors ranging from 
     201>>>> blue (low values) to red (high values).  If amax is 0, each time a 
     202>>>> spectrum is calculated (a single horizontal line on the plot), its 
     203>>>> maximum value is determined and assigned to the highest (red) color. 
     204>>>> Each line will have a different relationship between spectral amplitude 
     205>>>> and color, although the maximum will always be red and the minimum 
     206>>>> will always be blue.  To make a spectrogram for which the colors 
     207>>>> are meaningful and consistent we need to establish a constant relationship 
     208>>>> between spectral amplitude and the color spectrum.  To do this, we assign 
     209>>>> a spectral amplitude of 0 to blue and assert a spectral amplitude value 
     210>>>> (amax) to be assigned to red.  Any higher values will be plotted as red. 
     211>>>> The assignment of amax is largely a matter of taste, but you can get 
     212>>>> a start by setting the option SaveDrifts to log the values being calculated. 
     213>>>> As a rule of thumb, low values of amax will make spectrograms sensitive 
     214>>>> to the background noise levels; high values will be senstive to larger events. 
     215>>>> 
     216 
     217# 18 scale           A scaling factor for the wiggle-line trace. 
     218 
     219>>>> 
     220>>>> A data trace is plotted along the right-hand time axis for comparison 
     221>>>> with the spectrogram.  The scaling factor for the amplitude of this 
     222>>>> trace is arbitrarily assigned.  Make a plot and use the above scaling factor 
     223>>>> to adjust the amplitude to taste. 
     224>>>> 
     225 
     226# Comment            A comment for the top of the display. 
     227# 
     228#     S    C   N  04 05 06  07  08 09 10   11  12 13 14 15    16  17       18   Comment 
     229# 
     230 
     231 Plot MCM  VHZ NC 24 2  -7  PDT 1  0  600  1   1  60 10 0.2   2   40000    1.0 "Convict Moraine" 
     232 Plot MCS  VHZ NC 24 2  -7  PDT 1  0  350  1   1  60 10 0.0   1   30000    1.0 "Casa Diablo Hot Springs" 
     233 Plot MDP  VHZ NC 24 2  -7  PDT 1  0  350  1   1  60 10 0.0   2   30000    1.0 "Devil's Postpile" 
     234 Plot MDP  DLI NC 24 2  -7  PDT 1  0  350  1   1  60 10 0.0   2   30000    1.0 "Devil's Postpile" 
     235 
     236    # *** Optional Commands *** 
     237 
     238 Days2Save     7    # Number of days to display on web page; default=7 
     239 
     240>>>> 
     241>>>> This variable governs how many dates are retained in the SCN.hist 
     242>>>> file in your local gifs directory. 
     243>>>> 
     244 
     245 UpdateInt    10    # Number of minutes between updates; default=2 
     246 
     247>>>> 
     248>>>> Each UpdateInt minutes, sgram wakes up and updates all its files. 
     249>>>> 
     250 
     251 RetryCount    2    # Number of attempts to get a trace from server; default=2 
     252 
     253>>>> 
     254>>>> If you can't get it in two trys, there is probably something 
     255>>>> seriously wrong.  Nonetheless, there may be reasons to try some more. 
     256>>>> 
     257 
     258 Logo    smusgs.gif # Name of logo in GifDir to be plotted on each image 
     259 
     260>>>> 
     261>>>> If a Logo is specified, the appropriate gif file should be 
     262>>>> placed in the /GIFs/sgram/ directory so the program can 
     263>>>> find it.  If specified the Logo will be appear in the upper 
     264>>>> left corner of each SCN/date spectrogram plot. 
     265>>>> 
     266 
     267 
     268# We accept a command "SaveDrifts" which logs drifts to GIF directory. 
     269  SaveDrifts 
     270 
     271>>>> 
     272>>>> If SaveDrifts is set, a file SCN.drft is created for each SCN. 
     273>>>> Each time a new spectrogram line is started, the mean, rms, 
     274>>>> min and max values for the first minute of data are recorded. 
     275>>>> 
     276 
     277# We accept a command "Make_HTML" to construct and ship index.html file. 
     278  Make_HTML 
     279 
     280>>>> 
     281>>>> If Make_HTML is set, an index.html file is created.  This file is 
     282>>>> shipped to the target webserver(s) via rcp. 
     283>>>> 
     284 
     285# We accept a command "BuildOnRestart" to totally rebuild images on restart. 
     286  BuildOnRestart 
     287 
     288>>>> 
     289>>>> During the process of tuning the displays this option is useful. 
     290>>>> For production, this should be disabled. 
     291>>>> 
     292 
     293 
     294# We accept a command "Debug" which turns on a bunch of log messages 
     295# Debug 
     296# WSDebug 
     297**************************************************************************************** 
     298}}} 
     299 
     300=== Customizing the .html wrapper. === 
     301 
     302The .html wrapper for the primary web page (index.html) has provision for customization to reflect the identity and flavor of an individual earthworm site. If you do nothing, a generic wrapper is constructed. You may, however, control the top part of the index page and the bottom of the index page by placing the .html code you want in files named indexa.html and indexb.html respectively in the GIFs/sgram/ directory. The module will attempt to open and read indexa.html for the top of the page; if it can't find the file, it generates a generic beginning for the index.html file. It then pastes in links to the spectrogram GIF images. Finally, it looks for indexb.html to finish off the page.  
     303If there is no indexb.html, it provides a generic ending. 
     304 
     305=== Known Bug! === 
     306 
     307The library functions which handle the process of reading an existing .gif file, modifying it, and writing it back out are not good at detecting damaged files. Sometimes, if sgram stops or is halted in mid-stream, we are left with a damaged .gif file. Upon restart, the process will crash and burn without error messages when an attempt to read this file is made. There are two solutions; 1) use the BuildOnRestart option above to force overwriting of the file or 2) go to the gif directory and delete the offending spectrogram .gif file. 
    9308== Helpful Hints ==