Changes between Initial Version and Version 1 of Transport_Layer


Ignore:
Timestamp:
01/19/12 16:15:53 (10 years ago)
Author:
branden
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Transport_Layer

    v1 v1  
     1{{{ 
     2                             TRANSPORT.C                4/24/98  
     3 
     4Table of contents: 
     5  I.  Introduction 
     6  1.  Overview of transport routines 
     7      1.1   Transport.h structures used by the calling program. 
     8      1.2   Initializing/terminating access to shared memory. 
     9      1.3   Writing messages to shared memory. 
     10      1.4   Retrieving messages from shared memory. 
     11      1.5   Buffering messages in a private memory region.       
     12      1.6   Communicating with the shared memory header flag. 
     13      1.7   Error reporting by transport functions. 
     14  2.  Function calls 
     15      2.1   tport_create 
     16      2.2   tport_destroy 
     17      2.3   tport_attach 
     18      2.4   tport_detach 
     19      2.5   tport_putmsg 
     20      2.6   tport_getmsg 
     21      2.7   tport_copyto  
     22      2.8   tport_copyfrom 
     23      2.9   tport_buffer 
     24      2.10  tport_bufthr 
     25      2.11  tport_putflag 
     26      2.12  tport_getflag 
     27      2.13  tport_syserr 
     28      2.14  tport_buferror  
     29  3.  Programming tips 
     30  4.  Bug fixes and program modifications 
     31      4.1   Mishandled shared memory pointer wraps in tport_putmsg. 
     32      4.2   Missing argument to shmctl. 
     33      4.3   Speed enhancement using memcpy. 
     34      4.4   Making tport_putmsg multi-thread safe. 
     35      4.5   Mishandled shared memory pointer resets in tport_getmsg. 
     36      4.6   Minor crack in tport_getmsg and tport_copyfrom. 
     37      4.7   Logo-tracking problem with GET_TOOBIG messages,  
     38            tport_getmsg and tport_copyfrom. 
     39      4.8   Tracking problem when no messages of requested logo 
     40            are ever returned, tport_getmsg and tport_copyfrom. 
     41      4.9   Variable name changed to allow use of C++ compilers. 
     42      4.10  Semaphore operations problem in tport_putmsg and tport_copyto  
     43            (Solaris version). 
     44 
     45 
     46I.  INTRODUCTION 
     47 
     48Transport.c contains a set of functions for accessing System V IPC shared  
     49memory regions under SunOS 4.1.1 and Solaris 2.4.  These routines, with  
     50exactly the same function calls, have also been ported to OS/2 and Windows NT.  
     51  void  tport_create(); 
     52  void  tport_destroy(); 
     53  void  tport_attach(); 
     54  void  tport_detach(); 
     55  int   tport_putmsg(); 
     56  int   tport_getmsg(); 
     57  int   tport_copyto();  
     58  int   tport_copyfrom(); 
     59  void  tport_putflag(); 
     60  int   tport_getflag(); 
     61  void  tport_syserr(); 
     62 
     63In June 1995, a set functions were added to transport.c to create multi- 
     64threaded, message-buffering applications under Solaris 2.4, OS/2, and NT. 
     65(SunOS does not support multi-threaded applications): 
     66  int   tport_buffer(); 
     67  void *tport_bufthr(); 
     68  void  tport_buferror();  
     69 
     70On Solaris, source files using transport functions should include these lines: 
     71    #include <earthworm.h>   /* required by multi-thread transport functions */ 
     72    #include <transport.h> 
     73 
     74On OS/2, source files using transport functions should include these lines 
     75(the first 3 lines must be before the transport.h include line): 
     76    #define INCL_DOSMEMMGR                        
     77    #define INCL_DOSSEMAPHORES               
     78    #include <os2.h>              
     79    #include <earthworm.h>   /* required by multi-thread transport functions */ 
     80    #include <transport.h> 
     81 
     82 
     831.  OVERVIEW OF TRANSPORT ROUTINES 
     84 
     85In the following paragraphs, anything written in all capital letters is 
     86defined in transport.h.   The following topics are explained in more detail 
     87below: 
     88  1.1   Transport.h structures used by the calling program. 
     89  1.2   Initializing/terminating access to shared memory. 
     90  1.3   Writing messages to shared memory. 
     91  1.4   Retrieving messages from shared memory. 
     92  1.5   Buffering messages in a private memory region.   
     93  1.6   Communicating with the shared memory header flag. 
     94  1.7   Error reporting by transport functions. 
     95 
     96 
     971.1   Transport.h structures used by the calling program. 
     98 
     99        Many constants and five structure types and are defined in transport.h.  
     100Two of the structure types are used as arguments to transport functions.  The  
     101other defined structure types are used internally by the transport functions; 
     102for more information on those, please read the comments in transport.h.  The 
     103first structure type used as an argument in transport calls is a shared memory 
     104information structure: 
     105 
     106   Solaris version: 
     107       typedef struct {                     
     108             SHM_HEAD  *addr;     /* pointer to beginning of memory region */ 
     109             long       key;      /* key to shared memory region           */ 
     110             long       mid;      /* shared memory region identifier       */ 
     111             long       sid;      /* associated semaphore identifier       */ 
     112       } SHM_INFO;       
     113 
     114   OS/2 version: 
     115       typedef struct {                     
     116             SHM_HEAD  *addr;     /* pointer to beginning of memory region */ 
     117             long       key;      /* key to shared memory region           */ 
     118             PVOID      objAlloc; /* pointer to memory object              */ 
     119             HMTX       hmtx;     /* mutex semaphore handle                */ 
     120       } SHM_INFO;                        
     121 
     122All the values in this structure are set within function tport_create or 
     123tport_attach.  It contains all the information needed to identify and use the 
     124memory region in all other transport function calls.                   
     125 
     126The second structure type used as an argument is the message logo structure: 
     127       typedef struct {                      
     128             unsigned char  type;    /* message is of this type       */ 
     129             unsigned char  mod;     /* was created by this module id */ 
     130             unsigned char  instid;   /* at this installation          */ 
     131       } MSG_LOGO;            
     132This structure describes the message it is associated with.  A single  
     133MSG_LOGO structure is passed an argument to tport_putmsg.  tport_getmsg  
     134takes an array of MSG_LOGO structures as a list of requested logos and it  
     135sets values in an individual MSG_LOGO structure to identify the retrieved 
     136message. 
     137 
     138 
     1391.2   Initializing/terminating access to shared memory. 
     140 
     141        Four of the transport functions deal with getting a program ready to  
     142use or to finish with a shared memory region.  tport_create() creates the  
     143memory region given a unique "key" to identify the region and the size (in  
     144bytes) of the region. The created memory region consists of 2 parts: a header  
     145section (SHM_HEAD) for keeping track of pointers, etc., and a circular buffer  
     146area for storing variable-length messages.  The region should be made large  
     147enough compared to the size of the messages it holds to give each message a  
     148reasonable residence time in the memory before it is overwritten.  All  
     149information needed to identify and use the memory region in other transport  
     150function calls is stored in a shared memory information structure (SHM_INFO).  
     151To access an existing shared memory region, a program must first attach to it  
     152by passing tport_attach() the region's unique key.  tport_attach then sets up  
     153the shared memory information structure. Note: A program should call EITHER tport_create() to create and attach to a memory region OR tport_attach() to attach to an existing region. It should never call both. 
     154        Just before exitting, a program that had attached to a memory region  
     155should detach from it using tport_detach() and one that had created it should  
     156destroy it using tport_destroy().   
     157        None of these four functions has a return value; if a system error  
     158occurs, they will write a message to stdout and exit. 
     159 
     160 
     1611.3   Writing messages to shared memory. 
     162 
     163        Messages are written to a shared memory region using tport_putmsg() or  
     164tport_copyto(), given the region's shared memory information structure.  When  
     165one tport_putmsg or tport_copyto is writing to memory, no other tport_putmsg or  
     166tport_copyto can access the same region.  Both functions write a transport  
     167layer header (TPORT_HEAD) in front of each message in shared memory. The first 
     168byte of this header is always set to FIRST_BYTE to signal the beginning of a  
     169new message.  The header also includes the length of the following message,  
     170its "message logo" (MSG_LOGO; its message type, module id and installation id),  
     171and a sequence number.  If tport_copyto is used, the sequence number is passed  
     172as an argument to the function, and sequencing from another source can be  
     173preserved.  If tport_putmsg is used, the sequence number is assigned and  
     174tracked by tport_putmsg; any previous sequencing of messages will be lost.   
     175tport_putmsg has a limit to the number of different logos for which it can  
     176keep track of sequences numbers (NTRACK_PUT).  If this limit is exceeded,  
     177tport_putmsg will not write messages with new logos to memory; it will return  
     178PUT_NOTRACK, write a warning to stdout and continue.  tport_copyto has no  
     179tracking limits.  tport_putmsg and tport_copyto are multi-thread safe (they  
     180can be used by multiple threads of the same process without problems). 
     181 
     182 
     1831.4   Retrieving messages from shared memory. 
     184 
     185        Messages of a given logo are retrieved from a shared memory region  
     186using tport_getmsg() or tport_copyfrom(). A single logo can be requested or  
     187an array of logos can be requested. Additionally, any or all components (type, 
     188module, instid) of the requested message logo(s) can be wildcarded (set to  
     189WILD).  tport_getmsg or tport_copyfrom will return when it has found the first 
     190message which matches any of the requested logos.  Both functions also keep 
     191track of the sequence number they expect to see for the next message of each 
     192logo; therefore, tport_getmsg or tport_copyfrom can tell if they have missed  
     193any messages.  If tport_getmsg misses messages, it returns GET_MISS; if 
     194tport_copyfrom misses messages, it returns either GET_MISS_LAPPED (if memory  
     195was over-written by tport_putmsg or tport_copyto) or GET_MISS_SEQGAP (if a  
     196gap in sequence numbers was passed along by tport_copyto).  There is a limit 
     197(NTRACK_GET) to the number of logos for which tport_getmsg or tport_copyfrom  
     198can track sequence numbers.  If this limit is exceeded, both functions will  
     199still return a message matching any requested logo, but they won't know if  
     200they have missed any; they will return GET_NOTRACK, write a warning to stdout  
     201and continue.  Both functions write the message logo, length (bytes), and  
     202message to addresses in their argument lists.  tport_copyfrom has one  
     203additional address argument to which it writes the TPORT_HEAD sequence number  
     204of the returned message.  Since both functions have their own private tracking  
     205variables, it is very important that each module use only one of these  
     206functions to grab messages for a given region-logo combination.  Otherwise,  
     207the module may see the same message twice!  tport_getmsg and tport_copyfrom  
     208are not multi-thread safe; they cannot be used safely by two threads of the  
     209same process. 
     210 
     211 
     2121.5   Buffering messages in a private memory region. 
     213         
     214        Several functions have been added to transport.c to give modules a 
     215multi-threaded message-buffering capability.  After attaching to or creating  
     216a public shared memory region and creating (tport_create) a private shared  
     217memory region, a module can call tport_buffer() to start the buffering thread, 
     218passing it 2 shared memory information structures (public and private), an  
     219array of logos, and the module id and installation id of the calling module.   
     220tport_buffer creates a thread, tport_bufthr(), which uses tport_copyfrom and  
     221tport_copyto to transfer all messages of the given logo(s) from the public  
     222region to the private region.  All sequence numbers from the public region  
     223are preserved in the private region.  The buffering-thread reports errors by  
     224calling tport_buferror(), which writes error messages, labeled with the main  
     225thread's module id and installation, to the public region using tport_putmsg.   
     226The main thread must retrieve all of its buffered messages from the private  
     227region using tport_getmsg. [tport_copyfrom and tport_getmsg are not multi- 
     228thread safe, and since the buffering-thread is hard-wired to call  
     229tport_copyfrom, the main thread must use tport_getmsg.]  The buffering-thread 
     230will exit when the shared memory header flag in the public region is set to 
     231TERMINATE.  The main thread must destroy its private buffering region  
     232(tport_destroy) before it exits. 
     233 
     234 
     2351.6   Communicating with the shared memory header flag. 
     236 
     237        Two transport functions deal only with the flag in the shared memory  
     238header structure.  This flag is included as a means of communication between  
     239different programs accessing the same region.  For instance, if the flag is  
     240set to a certain value, all attached programs should detach and terminate. To  
     241change the value of the flag in a given region, use tport_putflag().  To find  
     242out the current value of the flag, use tport_getflag(). 
     243 
     244 
     2451.7   Error reporting by transport functions. 
     246 
     247        Transport routines report errors by use of one of 2 functions,  
     248tport_syserr() or tport_buferror().  Both are meant for internal use only by  
     249the other transport functions.  tport_syserr is called when a system error has 
     250occurred; it writes a message to stdout and exits.  tport_buferror is called by  
     251tport_bufthr (the buffering-thread) when return values from other transport  
     252routines indicate a problem.  tport_buferror writes an error message, tagged  
     253with the main thread's module id and installation id, to the public shared  
     254memory region using tport_putmsg and then it returns. 
     255 
     256 
     257 
     258 
     2592.  FUNCTION CALLS 
     260           
     261Below are the function calls, return values and comment lines from the  
     262transport.c source code. They provide a general description of each  
     263function's purpose and its program flow.  
     264 
     265  2.1   tport_create 
     266  2.2   tport_destroy 
     267  2.3   tport_attach 
     268  2.4   tport_detach 
     269  2.5   tport_putmsg 
     270  2.6   tport_getmsg 
     271  2.7   tport_copyto  
     272  2.8   tport_copyfrom 
     273  2.9   tport_buffer 
     274  2.10  tport_bufthr 
     275  2.11  tport_putflag 
     276  2.12  tport_getflag 
     277  2.13  tport_syserr 
     278  2.14  tport_buferror  
     279 
     280 
     2812.1   tport_create:  create a shared memory region & its semaphore, attach 
     282                     to it and initialize shared memory header values. 
     283 
     284void tport_create( SHM_INFO *region,   /* info structure for memory region  */ 
     285                   long      nbytes,   /* size of shared memory region      */ 
     286                   long      memkey )  /* key to shared memory region       */  
     287 
     288Arguments used as passed:  nbytes, memkey 
     289 
     290Arguments reset by function:  *region  
     291 
     292Return Value: None. If any system error occurs during its execution,  
     293              tport_create writes a message to stdout and exits. 
     294 
     295Program flow: 
     296/* Destroy memory region if it already exists */ 
     297/* Create shared memory region */ 
     298/* Attach to shared memory region */ 
     299/* Initialize shared memory region header */ 
     300/* Make semaphore for this shared memory region & set semval = SHM_FREE */ 
     301/* set values in the shared memory information structure */ 
     302 
     303 
     304 
     3052.2   tport_destroy:  destroy a shared memory region. 
     306 
     307void tport_destroy( SHM_INFO *region )  /* info structure for memory region */ 
     308 
     309Arguments used as passed:  region 
     310 
     311Arguments reset by function:  none  
     312 
     313Return Value: None. If any system error occurs during its execution,  
     314              tport_destroy writes a message to stdout and exits. 
     315 
     316Program flow: 
     317/* Set kill flag, give other attached programs time to terminate */ 
     318/* Detach from shared memory region */ 
     319/* Destroy semaphore set for shared memory region */ 
     320/* Destroy shared memory region */ 
     321 
     322 
     323 
     3242.3   tport_attach:  map to an existing shared memory region. 
     325 
     326void tport_attach( SHM_INFO *region,   /* info structure for memory region  */ 
     327                   long      memkey )  /* key to shared memory region       */ 
     328 
     329Arguments used as passed:  memkey 
     330 
     331Arguments reset by function:  *region  
     332 
     333Return Value: None. If any system error occurs during its execution,  
     334              tport_attach writes a message to stdout and exits. 
     335 
     336Program flow: 
     337/* attach to header; find out size memory region; detach */ 
     338/* reattach to the entire memory region; get semaphore */ 
     339/* set values in the shared memory information structure */ 
     340 
     341 
     342 
     3432.4   tport_detach:  detach from a shared memory region.  
     344 
     345void tport_detach( SHM_INFO *region )   /* info structure for memory region  */ 
     346 
     347Arguments used as passed:  region 
     348 
     349Arguments reset by function:  none  
     350 
     351Return Value: None. If any system error occurs during its execution,  
     352              tport_detach writes a message to stdout and exits. 
     353 
     354 
     355 
     3562.5   tport_putmsg:  write a message into a shared memory region. 
     357 
     358int tport_putmsg( SHM_INFO *region,   /* info structure for memory region   */ 
     359                  MSG_LOGO *putlogo,  /* type,module,instid of incoming msg */ 
     360                  long      length,   /* size of incoming message           */ 
     361                  char     *msg )     /* pointer to incoming message        */ 
     362 
     363Arguments used as passed:  region, putlogo, length, msg 
     364 
     365Arguments reset by function:  none  
     366 
     367Return values: PUT_OK if it put the message in memory with no problems. 
     368               PUT_NOTRACK if it did not put the message in memory because 
     369                   its sequence number tracking limit (NTRACK_PUT) was  
     370                   exceeded. 
     371               PUT_TOOBIG if it did not put the message in memory because 
     372                   it was too long to fit in the region. 
     373 
     374If a system error occurs while tport_putmsg is executing or if a 
     375pointer into the memory region gets lost (doesn't point to a FIRST_BYTE), 
     376tport_putmsg writes a message to stdout and exits. 
     377 
     378Program flow: 
     379/* First time around, init the sequence counters, semaphore controls */ 
     380/* Set up pointers for shared memory, etc. */ 
     381/* First, see if the incoming message will fit in the memory region */ 
     382/* Change semaphore; let others know you're using tracking structure & memory */ 
     383/* Next, find incoming logo in list of combinations already seen */ 
     384/* Incoming logo is a new combination; store it, if there's room */ 
     385/* Store everything you need in the transport header */ 
     386/* First see if keyin will wrap; if so, reset both keyin and keyold */ 
     387/* Then see if there's enough room for new message in shared memory */ 
     388/*      If not, "delete" oldest messages until there's room         */ 
     389/* Now copy transport header into shared memory by chunks... */ 
     390/* ...and copy message into shared memory by chunks */ 
     391/* Finished with shared memory, let others know via semaphore */ 
     392 
     393 
     394 
     3952.6   tport_getmsg:  read a message out of shared memory. 
     396 
     397int tport_getmsg( SHM_INFO  *region,   /* info structure for memory region  */ 
     398                  MSG_LOGO  *getlogo,  /* requested logo(s)                 */ 
     399                  short      nget,     /* number of logos in getlogo        */ 
     400                  MSG_LOGO  *logo,     /* logo of retrieved msg             */ 
     401                  long      *length,   /* size of retrieved message         */ 
     402                  char      *msg,      /* retrieved message                 */ 
     403                  long       maxsize ) /* max length for retrieved message  */ 
     404 
     405Arguments used as passed:  region, getlogo, nget, maxsize 
     406 
     407Arguments reset by function:  *logo, *length, *msg  
     408 
     409Return values: GET_OK   if it got a message of requested logo(s). 
     410               GET_NONE if there were no new messages of requested logo(s). 
     411               GET_MISS if it got a message, but missed some.  Messages could 
     412                   be missed for one of 3 reasons: 
     413                   1) memory was overwritten before the message was retrieved. 
     414                   2) message was lost before being written to memory and a   
     415                      sequence # gap was passed to memory by tport_copyto. 
     416                   3) previous message of returned logo was skipped because 
     417                      it was longer than maxsize. 
     418               GET_NOTRACK if it got a message, but couldn't tell if it 
     419                   had missed any because its sequence # tracking limit 
     420                   (NTRACK_GET) was exceeded. 
     421               GET_TOOBIG if it found a message of requested logo(s) but 
     422                   it was too long to fit in caller's buffer. No message 
     423                   returned, but length and logo of the "toobig" message 
     424                   are returned. 
     425 
     426If a pointer into the memory region gets lost (doesn't point to a FIRST_BYTE), 
     427tport_getmsg writes a message to stdout and exits. 
     428 
     429Program flow: 
     430/* Get the pointers set up */ 
     431/* First time around, initialize sequence counters, outpointers */ 
     432/* find latest starting index to look for any of the requested logos */ 
     433/* See if keyin and keyold were wrapped and reset by tport_putmsg; */ 
     434/*       If so, reset trak[xx].keyout and go back to findkey       */ 
     435/* Find next message from requested type, module, instid */ 
     436   /* make sure you haven't been lapped by tport_putmsg */ 
     437   /* load next header; make sure you weren't lapped */ 
     438   /* make sure it starts at beginning of a header */ 
     439   /* see if this msg matches any requested type */ 
     440/* Found a message of requested logo; retrieve it! */  
     441        /* complain if retrieved msg is too big */       
     442        /* copy message by chunks to caller's address */ 
     443        /* see if we got run over by tport_putmsg while copying msg */ 
     444        /* if we did, go back and try to get a msg cleanly          */ 
     445        /* set other returned variables */ 
     446        /* find logo in tracked list */ 
     447        /* new logo, track it if there's room */    
     448        /* check if sequence #'s match; update sequence # */ 
     449        /* Ok, we're finished grabbing this one */ 
     450/* If you got here, there were no messages of requested logo(s) */ 
     451/* update outpointer ->msg after retrieved one for all requested logos */ 
     452 
     453 
     454 
     4552.7   tport_copyto:  put a message into a shared memory region; preserve the  
     456                     sequence number (passed as an argument) as the transport   
     457                     layer sequence number. 
     458 
     459int tport_copyto( SHM_INFO     *region,  /*info structure for memory region   */ 
     460                  MSG_LOGO     *putlogo, /*type,module,instid of incoming msg */ 
     461                  long          length,  /*size of incoming message           */ 
     462                  char         *msg,     /*pointer to incoming message        */ 
     463                  unsigned char seq )    /*preserve as sequence# in TPORT_HEAD*/ 
     464 
     465Arguments used as passed:  region, putlogo, length, msg, seq 
     466 
     467Arguments reset by function:  none  
     468 
     469Return values: PUT_OK if it put the message in memory with no problems. 
     470               PUT_TOOBIG if it did not put the message in memory because 
     471                   it was too long to fit in the region. 
     472 
     473If a system error occurs while tport_copyto is executing or if a 
     474pointer into the memory region gets lost (doesn't point to a FIRST_BYTE), 
     475tport_copyto writes a message to stdout and exits. 
     476 
     477Program flow: 
     478/* First time around, initialize semaphore controls */ 
     479/* Set up pointers for shared memory, etc. */ 
     480/* First, see if the incoming message will fit in the memory region */ 
     481/* Store everything you need in the transport header */ 
     482/* Change semaphore to let others know you're using memory */ 
     483/* First see if keyin will wrap; if so, reset both keyin and keyold */ 
     484/* Then see if there's enough room for new message in shared memory */ 
     485/*      If not, "delete" oldest messages until there's room         */ 
     486/* Now copy transport header into shared memory by chunks... */ 
     487/* ...and copy message into shared memory by chunks */ 
     488/* Finished with shared memory, let others know via semaphore */ 
     489 
     490 
     491 
     4922.8   tport_copyfrom:  get a message out of public shared memory; save the 
     493                       sequence number from the transport layer. 
     494 
     495int tport_copyfrom( SHM_INFO  *region,   /* info structure for memory region */ 
     496                    MSG_LOGO  *getlogo,  /* requested logo(s)                */ 
     497                    short      nget,     /* number of logos in getlogo       */ 
     498                    MSG_LOGO  *logo,     /* logo of retrieved message        */ 
     499                    long      *length,   /* size of retrieved message        */ 
     500                    char      *msg,      /* retrieved message                */ 
     501                    long       maxsize,  /* max length for retrieved message */ 
     502                    unsigned char *seq ) /* TPORT_HEAD seq# of retrieved msg */ 
     503 
     504Arguments used as passed:  region, getlogo, nget, maxsize 
     505 
     506Arguments reset by function:  *logo, *length, *msg, *seq 
     507 
     508Return values: GET_OK   if it got a message of requested logo(s). 
     509               GET_NONE if there were no new messages of requested logo(s). 
     510               GET_MISS_LAPPED if it got a message, but missed some due to  
     511                   msgs being overwritten (by tport_putmsg or tport_copyto)   
     512                   before it got to them. 
     513               GET_MISS_SEQGAP if it got a message, but noticed a gap in the 
     514                   sequence numbers in the ring.  This means one of 2 things: 
     515                   1) a msg was lost before being placed in shared memory and  
     516                   the sequence gap was transferred into shared memory by  
     517                   tport_copyto. 
     518                   2) the previous message of the returned logo was skipped  
     519                   because it was longer than maxsize. 
     520               GET_NOTRACK if it got a message, but couldn't tell if it 
     521                   had missed any because its sequence # tracking limit 
     522                   (NTRACK_GET) was exceeded. 
     523               GET_TOOBIG if it found a message of requested logo(s) but 
     524                   it was too long to fit in caller's buffer. No message 
     525                   returned, but length and logo of the "toobig" message 
     526                   are returned. 
     527 
     528If a pointer into the memory region gets lost (doesn't point to a FIRST_BYTE), 
     529tport_getmsg writes a message to stdout and exits. 
     530 
     531Program flow: 
     532Same as tport_getmsg program flow (see section 2.6). 
     533 
     534 
     535 
     5362.9   tport_buffer:  initialize the input buffering thread. 
     537 
     538int tport_buffer( SHM_INFO  *region1,      /* transport ring             */ 
     539                  SHM_INFO  *region2,      /* private ring               */ 
     540                  MSG_LOGO  *getlogo,      /* array of logos to copy     */ 
     541                  short      nget,         /* number of logos in getlogo */ 
     542                  unsigned   maxMsgSize,   /* size of message buffer     */ 
     543                  unsigned char module,    /* module id of main thread   */ 
     544                  unsigned char instid )   /* inst id of main thread     */ 
     545 
     546Arguments used as passed:  region1, region2, getlogo, nget, maxMsgSize,  
     547                           module, instid 
     548 
     549Arguments reset by function:  none 
     550 
     551Return values:   0 if there were no errors. 
     552                -1 if there was an error allocating the internal message buffer, 
     553                   or if there was an error creating the thread. 
     554                 
     555Program flow: 
     556/* Allocate internal message buffer */ 
     557/* Copy function arguments to global variables */ 
     558/* Start the input buffer thread, tport_bufthr */ 
     559/* Yield to the buffer thread */ 
     560 
     561 
     562 
     5632.10  tport_bufthr:  thread to buffer input from one transport region to another. 
     564 
     565void *tport_bufthr( void *dummy ) 
     566 
     567Arguments:  none 
     568 
     569Return values:  none 
     570 
     571Program flow: 
     572This function is an infinite loop which will exit only when the termination 
     573flag is set in the public shared memory region's header: 
     574/* Check the flag in the public region; exit if it's set to TERMINATE */ 
     575/* Try to copy a message from the public memory region with tport_copyfrom */ 
     576/* Handle return values from tport_copyfrom */ 
     577/* If you did get a message, copy it to private ring with tport_copyto */ 
     578 
     579 
     580 
     5812.11  tport_putflag:  set the flag in a shared memory header.        
     582 
     583void tport_putflag( SHM_INFO *region,  /* shared memory info structure  */ 
     584                    short     flag )   /* value to set header flag to   */ 
     585 
     586Arguments used as passed:  region, flag 
     587 
     588Arguments reset by function:  none  
     589 
     590Return Value: none  
     591 
     592 
     593 
     5942.12  tport_getflag:  return the value of the flag from a shared memory header. 
     595 
     596int tport_getflag( SHM_INFO *region )   /* shared memory info structure */ 
     597   
     598Arguments used as passed:  region 
     599 
     600Arguments reset by function:  none  
     601 
     602Return value: The value of the shared memory header flag. 
     603 
     604 
     605 
     6062.13  tport_syserr:  print a system error and exit. 
     607 
     608void tport_syserr( char *msg,   /* message to print */ 
     609                   long  key )  /* key of memory region that had an error  */ 
     610 
     611Arguments used as passed:  msg, key 
     612 
     613Arguments reset by function:  none  
     614 
     615Return Value: None. In fact it never returns, but always exits after 
     616              writing the error message to stdout. 
     617 
     618 
     619 
     6202.14  tport_buferror:  build an ascii earthworm error message and put it  
     621                       in the public memory region using tport_putmsg. 
     622 
     623void tport_buferror( short  ierr,       /* 2-byte error word       */ 
     624                     char  *note  )     /* string describing error */ 
     625 
     626Arguments used as passed:  ierr, note 
     627 
     628Arguments reset by function:  none  
     629 
     630Return Value:  none 
     631 
     632 
     633 
     6343.  PROGRAMMING TIPS 
     635      
     636Here are some tips for writing and running programs using transport.c: 
     637 
     638Region key(s) should be defined in a .h file which is included by all  
     639programs that will access the region(s). One program should create the  
     640memory region(s) (tport_create); other programs accessing those regions  
     641will attach to them (tport_attach). The "creator" can also be a "putter"  
     642or "getter" or it can be a program with no purpose other than 
     643creating/destroying memory regions. 
     644 
     645When deciding how large to make a memory region (tport_create), remember 
     646that the transport layer uses a portion of the memory region for its own 
     647bookkeeping.  The region size is NOT required to be an even multiple of the 
     648size of the messages it will contain.  However, suppose a user wants the 
     649region to be exactly large enough to store NUMRING messages of size MSGSIZE. 
     650To include space for transport bookkeeping too, the region size should be: 
     651   sizeof(SHM_HEAD) + NUMRING * ( sizeof(TPORT_HEAD) + MSGSIZE )  
     652 
     653At run time, the "creator" must be started first.  A few seconds should  
     654be allowed for the regions to be set up before starting "attachers".  
     655Otherwise the "attachers" will exit immediately because they can't find  
     656the memory regions. 
     657 
     658Any program accessing shared memory should periodically look at the flag  
     659in the memory's header structure (tport_getflag).  If the flag is set to 
     660TERMINATE, any "attacher" should detach from memory (tport_detach) and  
     661exit, and the "creator" should destroy the memory region(s) (tport_destroy)  
     662and exit.  
     663  
     664To initiate such a polite termination of all programs, one program 
     665must set that termination flag (tport_putflag).  A "killer" program, 
     666whose only purpose is to attach to a region and set the flag, is a useful 
     667tool for keyboard-initiated exits. 
     668 
     669Simple examples of these types of programs reside in the same directory 
     670as transport.c.  They are: 
     671  putter1.c   creates regions and writes messages as module 1. 
     672  putter2.c   attaches to regions and writes messages as module 2. 
     673  getter.c    attactes to regions and retrieves messages, printing them. 
     674  killer.c    sets terminate flag to stop all programs. 
     675  keys.h      include file defining shared memory region keys. 
     676  go          simple script to start the programs. 
     677  Makefile 
     678 
     679Note: Transport.c was designed to work in programs which run continuously. 
     680If, however, a putter or getter is a transient beast that is run only 
     681intermittently, the getter may return the "GET_MISS" status without actually 
     682missing any messages.  This is due to the fact that every time a putter or  
     683starts up, its sequence # trackers are set to 0. 
     684 
     685 
     686 
     6874.  BUG FIXES AND PROGRAM MODIFICATIONS 
     688    
     689   4.1   Mishandled shared memory pointer wraps in tport_putmsg. 
     690   4.2   Missing argument to shmctl. 
     691   4.3   Speed enhancement using memcpy. 
     692   4.4   Making tport_putmsg multi-thread safe. 
     693   4.5   Mishandled shared memory pointer resets in tport_getmsg. 
     694   4.6   Minor crack in tport_getmsg and tport_copyfrom. 
     695   4.7   Logo-tracking problem with GET_TOOBIG messages,  
     696         tport_getmsg and tport_copyfrom. 
     697   4.8   Tracking problem when no messages of requested logo 
     698         are ever returned, tport_getmsg and tport_copyfrom. 
     699   4.9   Variable name changed to allow use of C++ compilers. 
     700   4.10  Semaphore operations problem in tport_putmsg and tport_copyto  
     701         (Solaris version). 
     702 
     703 
     7044.1   Mishandled shared memory pointer wraps.   
     705Problem: tport_putmsg mishandled wraps in the shared memory header's 
     706         unsigned long keyin and keyold.  The caused the transport layer to 
     707         lose its place in the memory ring and die. 
     708 
     709The Fix: After resetting keyin and keyold, check to make sure that keyin is 
     710         larger than keyold.  If not make keyin = keyin + keymax. 
     711         Change made in tport_putmsg on 10/24/94 by Lynn Dietz. 
     712 
     713         I also changed transport.c so that it writes warning and error 
     714         messages to stdout (instead of stderr as it was doing) so that the 
     715         messages can easily be redirected to a log file. 
     716         Change made in transport.c on 10/24/94 by Lynn Dietz. 
     717 
     718 
     7194.2   Missing argument to shmctl. 
     720Problem: tport_create and tport_destroy each have a call to shmctl().  
     721         Shmctl() takes 3 arguments, but I only had the first two passed.   
     722         The compiler under SunOS never complained about it, but the 
     723         Solaris compiler 3.0.1 did. 
     724 
     725The Fix: I added the 3rd argument (struct shmid_ds shmbuf) to both of 
     726         the shmctl() calls. 
     727         Change made in transport.c on 3/28/95 by Lynn Dietz. 
     728 
     729 
     7304.3   Speed enhancement using memcpy. 
     731Problem: I noticed that coaxtoring, a program that just reads messages  
     732         from ethernet and puts them into shared memory using tport_putmsg,  
     733         took a big chunk of the cpu on a Sparc2 when handling large  
     734         messages (>50,000 bytes).  Suspect that something isn't optimized.  
     735 
     736The Fix: I changed how tport_putmsg and tport_getmsg copy messages from 
     737         one address to another. A byte-by-byte for loop was replaced with 
     738         one or two (if the message was wrapped around the end of the ring) 
     739         calls to memcpy().  This sped up the coaxtoring program by 20-30%. 
     740         Change made in transport.c on 6/20/95 by Lynn Dietz. 
     741 
     742 
     7434.4   Making tport_putmsg multi-thread safe. 
     744Problem: Previously, the semaphore was set in tport_putmsg after the incoming 
     745         logo was found in the tracking list.  If more than one thread of the 
     746         same process was using tport_putmsg, they could have competed for 
     747         access to the tracking structure, potentially causing duplicated 
     748         sequence numbers or other errors. 
     749 
     750The Fix: tport_putmsg now sets the semaphore before it looks for the logo in 
     751         the tracking structure.  Since only one tport_putmsg can access the  
     752         tracking structure at a time, multiple threads of one process can  
     753         safely use the same routine. 
     754         Change made in transport.c on 6/27/95 by Lynn Dietz. 
     755 
     756 
     7574.5   Mishandled shared memory pointer resets in tport_getmsg. 
     758Problem: Each tport_getmsg() and tport_copyfrom() must reset its tracking  
     759         pointers (trak[xx].keyout) after shared memory header keyin & keyold  
     760         are wrapped and reset (by tport_putmsg or tport_copyto).  Sometimes, 
     761         keyout was mistakenly reset to a number less than keyold, causing  
     762         the getter to grab messages from the ring starting with the oldest  
     763         complete message in the ring.  This results in a "missed message"  
     764         error, because of a gap in transport sequence numbers.  It also 
     765         causes some messages to be processed twice. 
     766 
     767The Fix: After resetting a keyout value in tport_getmsg() and tport_copyfrom(), 
     768         first see if it still points to the FIRST_BYTE of a message.   
     769         If it does, make sure the value of keyout lies between keyold and  
     770         keyin.  If it doesn't point to a FIRST_BYTE, the getter was lapped  
     771         by a putter; reset keyout to keyold. 
     772         Change made in transport.c on 1/17/96 by Lynn Dietz 
     773 
     774 
     7754.6   Minor crack in tport_getmsg and tport_copyfrom. 
     776Problem: When reading shared memory, both tport_getmsg and tport_copyfrom use 
     777         this logic: make sure I haven't been lapped by a putter, grab a  
     778         TPORT_HEAD from the ring, make sure that the TPORT_HEAD starts with  
     779         a FIRST_BYTE.  On very rare occassions, a putter will overwrite the 
     780         first byte (or the TPORT_HEAD) between the getter's lap-check an its  
     781         grabbing the header from the ring.  In this case, the getter will 
     782         complain that the header doesn't begin with a FIRST_BYTE and it will 
     783         exit. 
     784 
     785The Fix: Add another lap-check just after tport_getmsg and tport_copyfrom 
     786         grab a TPORT_HEAD from the ring.  Their logic now looks like this: 
     787         make sure I haven't been lapped by a putter, grab a TPORT_HEAD from  
     788         the ring, make sure I haven't been lapped by a putter, make sure  
     789         that the TPORT_HEAD starts with a FIRST_BYTE.  Note: another lap- 
     790         check is done after each message is grabbed from the ring. 
     791 
     792 Change: In a move totally unrelated to the above problem, I changed  
     793         the word "WARNING" to "NOTICE" in all references to wraps of 
     794         keyin/keyout/keyget to reflect the fact that this is really a  
     795         normal, albeit rare, occurrance.  
     796         Changes made in transport.c on 6/12/96 by Lynn Dietz 
     797 
     798 
     7994.7   Logo-tracking problem with GET_TOOBIG messages,  
     800      tport_getmsg and tport_copyfrom. 
     801Problem: Whenever tport_getmsg or tport_copyfrom find a message that matches 
     802         the requested logo(s) but is too long for the target address, they  
     803         return the logo and length of the message, but they never enter the 
     804         logo-tracking part of the routine.  This causes a problem only if the 
     805         very first message is GET_TOOBIG; since no logos are being tracked,  
     806         these functions don't record the fact that they've looked at this  
     807         GET_TOOBIG message already.  On the next call, they look at the same  
     808         GET_TOOBIG message, and thus get stuck looking at this same message  
     809         forever... (which may put you into an infinite loop depending on 
     810         how you handled the return codes). 
     811 
     812The Fix: Modify the program flow of tport_getmsg and tport_copyfrom such that 
     813         after a TOOBIG message is found, they enter the logo-tracking part 
     814         of the routine.  Also make sure that the return code does NOT get 
     815         changed from GET_TOOBIG! 
     816         Changes made in transport.c on 6/12/96 by Lynn Dietz 
     817 
     818 
     8194.8   Tracking problem when no messages of requested logo are ever returned,  
     820      tport_getmsg and tport_copyfrom. 
     821Symptom: If a module never finds a message of any requested logo in a given 
     822         memory region, that module eventually becomes a CPU hog.  We know  
     823         something is wrong because the module has nothing to process; it  
     824         should be doing a loop something like: call tport_getmsg, get a  
     825         return code of GET_NONE, sleep a little bit, try again.  Where is  
     826         the CPU going? 
     827 
     828Problem: The problem is essentially the same as that described in section 4.7. 
     829         No entries exist in the logo-tracking list until a message of a 
     830         requested logo is actually found in shared memory.  If no such  
     831         message has been found, tport_getmsg and tport_copyfrom have no way 
     832         to record the position in shared memory of the last message that 
     833         they considered (and rejected).  So on every single call, tport_getmsg  
     834         or tport_copyfrom start at the oldest complete message in memory and  
     835         look at every single one (even though they've probably seen most of  
     836         them already...) before concluding that none of them match their  
     837         request.  If the memory region is large and there are a lot of  
     838         little messages in it, this can take a lot of CPU! 
     839 
     840The Fix: Modify tport_getmsg and tport_copyfrom so that the first thing they 
     841         do is verify that each of the requested logos is entered in the         
     842         logo-tracking list.  This way, even if none of the requested logos  
     843         is found, there is place to record the position of the last message  
     844         that was considered for each requested logo.  (The sequence number 
     845         tracking for each logo remains "inactive" until the first message 
     846         with that logo is found).  On subsequent calls, tport_getmsg and 
     847         tport_copyto will only look at messages they haven't seen before. 
     848         Changes made in transport.c on 6/18/96 by Lynn Dietz 
     849 
     850 
     8514.9   Variable name changed to allow use of C++ compilers. 
     852Problem: We had used "class" as the variable name for the installation in  
     853         the MSG_LOGO structure.  However, "class" is also a keyword in C++, 
     854         so if you want to use a C++ compiler, you cannot use "class" as 
     855         a variable name.  
     856 
     857The Fix: Change all references to "class" to "instid" to allow this software 
     858         to be compiled with a C++ compiler. 
     859         Changes made in transport.c and transport.h on 3/13/97 by Lynn Dietz 
     860 
     861 
     8624.10  Semaphore operations problem in tport_putmsg and tport_copyto (Solaris). 
     863Symptom: All modules attached to a given transport ring (running on a Solaris  
     864         system) suddenly die with a message like:  
     865           "ERROR: tport_getmsg; keyget not at FIRST_BYTE, Region xxxx" 
     866         This message implies that the transport ring is corrupted. The symptom  
     867         was first noticed when Doug Neuhauser ran his transport-based UCB code  
     868         on a dual-processor Ultra.  A dual-processor X86 Solaris machine has  
     869         also exhibited this symptom while running Earthworm v3.1 code.   
     870 
     871Problem: Many thanks go to Doug Neuhauser for tracking down the bug!  In both  
     872         tport_putmsg and tport_copyto, the structure sembuf sops, used as 
     873         an argument to the semaphore operation function semop(), had been  
     874         declared as a static struct. In multi-threaded code, you can have two  
     875         simultaneous invocations of tport_putmsg(), eg one for a heartbeat  
     876         and one for data.  Each one will overwrite the values of sops for the 
     877         other thread. This bug shows up readily on a multi-processor machine 
     878         on which two threads can really run simultaneously. It could also  
     879         presumably occur on a single-processor machine, but we've never 
     880         experienced it yet. This bug can manifest itself with these symptoms: 
     881         1) a corrupted transport ring, from both threads writing to the  
     882            ring at the same time, and  
     883         2) deadlock, where both threads are waiting for the semaphore. 
     884 
     885The Fix: Remove "static" from the declaration of "struct sembuf sops;" in  
     886         tport_putmsg and tport_copyto.  Also, pull the initialization of  
     887         sops structure members out of one-time-only initialization loops. 
     888         Changes made in solaris/transport.c on 4/24/98 by Lynn Dietz 
     889 
     890      
     891For more information contact:  Lynn Dietz 
     892                               dietz@andreas.wr.usgs.gov 
     893                               415-329-5520 
     894}}}