wiki:ringtocoaxII

Version 2 (modified by branden, 9 years ago) (diff)

--

Earthworm Module: ringtocoaxII

Contributed by: John Patton

Function

Same as ringtocoax but with an input queue (new in EW v7.0)

Details

John Patton rewrote ringtocoax as ringtocoaxII for HYDRA EW and it was contributed back for the EW v7.0 release. This has only been run and tested under Windows. Here are some notes from John on the improvements:

        There are basically 3 differences between ring2coax and ring2coaxII:
       
                1. ring2coaxII is multithreaded.
                2. ring2coaxII has an input queue
                3. ring2coaxII has my attempt at an adaptive algorithm that increases 
                the outgoing udp data rate as the queue fills up (within configurable 
                limits).  It also tries to "smooth" the udp message load so that we're 
                at a constant level rather then spikes and troughs (not very good at 
                this, but better than nothing).

        The design idea behind it was that we had problems with ring2coaxII getting cpu 
        starved for a few seconds and loosing data, thus the input queue.  The other half 
        was how our udp load on our network would change drastically from second to second 
        from very high to very low, we wanted some sort of way to "smooth" it out.  I've 
        had limited success on that one.

The old description:

Program to broadcast Earthworm messages from a shared memory transport ring onto Ethernet. Messages may be of any installation id, message type, and module id. To prevent bursty data on the Ethernet, the program sleeps between outgoing packets. A parameter in the configuration file controls whether heartbeat and error messages are passed or blocked. If heartbeats and error messages are not sent, they should be handled by a local copy of the statmgr program.

This module does connection-less broadcasts. In normal usage, it broadcasts onto an isolated, dedicated class b subnet. There must be no routing to or from that subnet. The broadcast is restricted to the subnet specified in the configuration file. Note that the companion module, coaxtoring, has strange system-specific addressing requirements.

NB: This was updated with the ringtocoaxII.d file, but the commands were not updated. Note the new INPUT QUEUE commands and the Min/MaxPacketsPerBurst? commands in red font below in the .d file example.

On startup, ringtocoaxII reads the configuration file named on the command line. Commands in this file set up all parameters used in reading Earthworm messages from shared memory and broadcasting them onto ethernet as UDP packets. In the control file, lines may begin with a valid ringtocoaxII command (listed below) or with one of 2 special characters:

#  marks the line as a comment (example: # This is a comment).
   
@  allows control files to be nested; one control file can be 
   accessed from another with the command "@" followed by 
   a string representing the path name of the next control file 
   (example: @model.d).

Command names must be typed in the control file exactly as shown in this document (upper/lower case matters!).

EXAMPLE CONFIGURATION FILE

#
#This is RingtocoaxII's parameter file
#
# This program gets earthworm messages from a transport ring and broadcasts
# them to ethernet.
#
#       *** WARNING ***
#       This module does connection-less broadcasts.
#       In normal usage, it broadcasts onto an isolated, dedicated class b
#       subnet. There must be no routing to or from that subnet.
#       The broadcast is restricted to that subnet as stated below.
#       Note that the companion module, coaxtoring, has strange
#       system-specific addressing requirements

# General Info
MyModuleId        MOD_RINGTOCOAX   # Module id for this instance of ringtocoax
RingName          WAVE_RING        # Shared memory ring for input/output
HeartBeatInterval 30               # Seconds between heartbeats
LogSwitch         1                # If 0, turn off disk log file
                                   # If 1, log to disk file and stderr/stdout
                                   # If 2, log only disk file
Debug             1                # If 1, print Adaptive PacketsPerBurst info
                                   # If 2, also print message logos on screen
                                   # If 0, don't print debug info.
                                   # NOTE: Logging logos / Adaptive info to disk can 
                                   # slow down RingtocoaxII's performance.  Debug
                                   # should only be used for troubleshooting.
                                   # OPTIONAL, defaults to 0.
# Message Settings
MsgMaxBytes       4096             # Size of biggest message to be sent
CopyStatus        0                # If 0, don't pass heartbeats, errors, or restarts.  
                                   # If 1, pass them.
MaxLogo           256              # Maximum number of logos we can handle
                                   # Set to a number >= 0
# List each logo that you want to broadcast in a BroadcastLogo command.
# If the BroadcastLogo is omitted, all messages in RingName will be shipped.
#               installation   module_id     message_type 
#               ------------   ---------     ------------
BroadcastLogo   INST_WILDCARD  MOD_WILDCARD  TYPE_TRACEBUF

# Network Settings
OutAddress        192.168.14.255   # IP address of net to broadcast to.
                                   # Set the node number to 255
                                   # eg. OutAddress    192.168.4.255
OutPortNumber     10001            # The well-known port number to broadcast on.
                                   # Note: Under Solaris, if the port number is at or below 
                                   #       1024, ringtocoax must run as root.
# Queue Settings
InputQueueLen       1000           # The Desired Size of the Input Queue in Messages,
                                   # Note: Actual size is InputQueueLen-1.
QueueOptimumPercent 10             # Optimum queue level in percent.  Used to determine 
                                   # percent excedence.  OPTIONAL, defaults to 10
ExcedenceConstant   1              # Constant to multiply the percent excedence by to control
                                   # how much the desired excedence rate can change.
                                   # OPTIONAL, defaults to 1.
STAInterval         1.5            # Interval in seconds over which the short term average
                                   # of the queue height is computed. 
                                   # OPTIONAL, defaults to 1.5.
QueueReportInt    20             # Seconds between queue size reports
                                   # OPTIONAL, defaults to 300 seconds.
# Data Export Rate Settings
MaxPacketsPerBurst  200            # Maximum allowed Number of packets per burst the reciver 
                                             # can send
MinPacketsPerBurst  10             # Minmum allowed Number of packets per burst the reciver
                                             # can send. OPTIONAL, defaults to 1.
SqrtCount           2000           # Number of square roots between packets within a burst.
BurstInterval       10             # Sleep this many msec between bursts
ThroughputReportInt 20             # Seconds between throughput reports
                                   # OPTIONAL, defaults to 300 seconds.

FUNCTIONAL COMMAND LISTING

Below are the commands recognized by ringtocoax, grouped by the function they influence. All of the commands are required; they must be specified in the control file in order for ringtocoax to operate.

         Earthworm system setup:
		HeartBeatInterval 	required
 		MyModName	   	required
		RingName	   	required

	Network Information:
		OutAddress	   	required
		OutPortNumber	   	required

	Message Handling:
		CopyStatus		required
		MaxLogo			required
		MsgMaxBytes	   	required
		BurstInterval		required
		BurstCount		required
		SqrtCount		required
		BroadcastLogo

	Output Control:
		ScrnMsg	   		required
		LogSwitch		required

ALPHABETIC COMMAND LISTING & DESCRIPTION

In the following section, all configuration file commands are listed in alphabetical order. Listed along with the command (bold-type) are its arguments (in red), the name of the subroutine that processes the command, and the function within the module that the command influences. A detailed description of the command and is also given. Default values and example commands are listed after each command description.

The following list is organized by:

command [argument here]

BroadcastLogo [instid modid msgtype]
Processed by: ringtocoax_config
Function: msg handling

Specifies one message logo (instid, modid, and msgtype) to read from the shared memory region (named in the "RingName" command) and to broadcast onto ethernet. The instid must be a valid string in earthworm.h; modid and msgtype must be valid strings in earthworm.d. List one logo per "BroadcastLogo" command, and use as many commands as needed. If msgtype is TYPE_HEARTBEAT or TYPE_ERROR, be sure that the CopyStatus? value is non-zero. If no "BroadcastLogo" command is given, ringtocoax defaults to broadcasting all messages in the designated shared memory region.

Example:  BroadcastLogo INST_MENLO MOD_WILDCARD TYPE_TRACEBUF

BurstCount [BurstCount]
Processed by: ringtocoax_config
Function: msg handling Defines the number of UDP packets, BurstCount, in a broadcast "burst". Three commands, "BurstCount", "SqrtCount", and "BurstInterval", control the temporal spacing of UDP packets broadcast by ringtocoax. Ringtocoax broadcasts messages in groups of BurstCount? packets; between the packets within a "burst", ringtocoax loops thru SqrtCount square root calculations to waste a little time. After broadcasting a "burst" of packets, ringtocoax sleeps for BurstInterval? milliseconds, then it starts sending the next burst. The goal of spacing packets a tiny bit in time is to give the receiving program a better chance of catching every packet.

UDP packets are variable in length up to a maximum of 1472 bytes (6 bytes of header, 1466 bytes of data). If an Earthworm message has a length less than or equal to 1466 bytes, it will be broadcast in a single UDP packet. If an Earthworm message is longer than 1466 bytes, it will be fragmented by ringtocoax, broadcast in multiple UDP packets, and reassembled by the receiving program (coaxtoring). Ringtocoax will never pack more than one Earthworm message in a single UDP packet.

It is very important that the overall rate of broadcast is as at least as fast as the rate of local information production. For example, if your system produces 256 TYPE_TRACEBUF messages per second (464 bytes each = 1 UDP packet each), then the BurstCount, SqrtCount, and BurstInterval commands should be configured such that ringtocoax will broadcast at least 256 UDP packets per second. System clock resolution must be considered when configuring the three commands, especially BurstInterval. As of April 1999, the time resolution of each sleep on Windows NT is about 16 msec (one clock tick). On Solaris, the resolution is about 10 msec.

Default:  none
Example:  BurstCount 3

BurstInterval [BurstInterval]
Processed by: ringtocoax_config
Function: msg handling

Defines the minimum interval, in milliseconds, between "bursts" of UDP broadcasts. See the BurstCount command for a detailed description of ringtocoax's broadcasting scheme. BurstInterval must be a number greater than or equal to zero. Remember to consider the system clock resolution when configuring BurstInterval?. As of April 1999, the clock-tick resolution is about 16 msec on Windows NT, and about 10 msec on Solaris.

Default:  none
Example:  BurstInterval 10

CopyStatus [CopyStatus]
Processed by: ringtocoax_config
Function: msg handling

Sets the CopyStatus switch to control whether ringtocoax broadcasts TYPE_ERROR and TYPE_HEARTBEAT messages. If no "BroadcastLogo" command is present, then ringtocoax will default to reading all messages in the shared memory region specified by the "RingName" command. If CopyStatus is non-zero, all of these messages will be broadcast onto ethernet. If CopyStatus? is 0, TYPE_HEARTBEAT and TYPE_ERROR will not be broadcast, but all other messages will.

Default:  none
Example:  CopyStatus 0

HeartBeatInterval [nsec]
Processed by: ringtocoax_config
Function: Earthworm setup Defines the number of seconds nsec between TYPE_HEARTBEAT messages issued by ringtocoax.

Default:  none				
Example:  HeartBeatInterval 15

LogSwitch [switch]
Processed by: ringtocoax_config
Function: output Sets the on-off switch for writing a log file to disk. If switch is 0, no log file will be written. If switch is 1, ringtocoax will write a daily log file(s) called ringtocoaxxx.log_ccyymmdd where xx is ringtocoax's module id (set with "MyModName" command) and ccyymmdd is the current UTC date (ex: 19990123) on the system clock. The file(s) will be written in the EW_LOG directory (environment variable).

Default:  none

MaxLogo [nlogo]
Processed by: ringtocoax_config
Function: msg handling

Sets the maximum number, nlogo, of different message logos that ringtocoax can process. Ringtocoax assigns a sequence number (unsigned char) to each message based on its logo before it broadcasts the message. Space is allocated to store the last sequence number used for up to nlogo different logos. If the tracking arrays are full and ringtocoax gets a never-seen-before message logo from shared memory, ringtocoax will not broadcast that message, but it will issue a TYPE_ERROR message explaining the problem.

Default:  none
Example:  MaxLogo 256

MsgMaxBytes [nbytes]
Processed by: ringtocoax_config
Functio: msg handling

Sets the size, in bytes, of the buffer that ringtocoax uses for storing Earthworm messages read from shared memory. nbytes should be the size of the largest message you expect ringtocoax to receive. If ringtocoax receives a message larger than nbytes, it will drop the entire message and issue a TYPE_ERROR message containing the string "too big for Buffer" and the logo of the dropped message.

Default:  none
Example:  MsgMaxBytes 51740

MyModName [mod_id]
Processed by: ringtocoax_config
Function: Earthworm setup Sets the module id for labeling all outgoing heartbeat and error messages. mod_id is a character string (valid strings are listed in earthworm.d) that relates (in earthworm.d) to a unique single-byte number. All messages that ringtocoax receives from shared memory are tagged with their original logos when they are broadcast onto ethernet.

Default:  none				
Example:  MyModName MOD_RINGTOCOAX

OutAddress [a.b.c.d]
Processed by: ringtocoax_config
Function: network info

Tells ringtocoax the TCP/IP address of the ethernet line onto which it should broadcast Earthworm messages. a.b.c.d is a character string in tcp/ip's standard dotted-decimal notation that ringtocoax uses along with the port number from the "OutPortNumber" command to set up a socket to which it will broadcast UDP Datagram packets. By convention, all Earthworm broadcasts have the node ("d" part of address) set to 255 (for example a.b.c.255).

Default:  none			
Example:  OutAddress 192.168.4.255

OutPortNumber [port]
Processed by: ringtocoax_config
Function: network info

Defines the well-known port number port to be used by ringtocoax for broadcasting. port is an integer that ringtocoax uses along with the address from the "OutAddress" command to set up a socket to which it will broadcast UDP Datagram packets.

NOTE: The port number must be greater than 1024. Smaller port numbers require root permission.

Default:  none
Example:  OutPortNumber 2025

RingName [ring]
Processed by: ringtocoax_config
Function: Earthworm setup Tells ringtocoax which shared memory region to use for input/output. ring is a character string (valid strings are listed in earthworm.d) that relates (in earthworm.d) to a unique number for the key to the shared memory region. All messages that ringtocoax reads from shared memory are tagged with their original logos when they are broadcast onto ethernet.

Default:  none				
Example:  RingName WAVE_RING

ScrnMsg [switch]
Processed by: ringtocoax_config
Function: output

Sets a switch for writing messages to the screen. If switch is 1, a line is printed to the screen detailing the logo and the sequence number of each Earthworm message broadcast to ethernet. If switch is 0, no messages are written to the screen.

Default:  none
Example:  ScrnMsg 0

SqrtCount [SqrtCount]
Processed by: ringtocoax_config
Function: msg handling Defines the number of square root calculations ringtocoax performs between broadcasting each UDP packet of a "burst." See the BurstCount command for a detailed description of ringtocoax's broadcasting scheme. SqrtCount must be an integer greater than or equal to zero.

Default:  none
Example:  SqrtCount 2000

Helpful Hints