| Version 4 (modified by branden, 9 years ago) (diff) |
|---|
Earthworm Module: ringtocoaxII
Contributed by: John Patton
Function
Same as ringtocoax but with an input queue (new in EW v7.0)
Details
John Patton rewrote ringtocoax as ringtocoaxII for HYDRA EW and it was contributed back for the EW v7.0 release. This has only been run and tested under Windows. Here are some notes from John on the improvements:
There are basically 3 differences between ring2coax and ring2coaxII:
1. ring2coaxII is multithreaded.
2. ring2coaxII has an input queue
3. ring2coaxII has my attempt at an adaptive algorithm that increases
the outgoing udp data rate as the queue fills up (within configurable
limits). It also tries to "smooth" the udp message load so that we're
at a constant level rather then spikes and troughs (not very good at
this, but better than nothing).
The design idea behind it was that we had problems with ring2coaxII getting cpu
starved for a few seconds and loosing data, thus the input queue. The other half
was how our udp load on our network would change drastically from second to second
from very high to very low, we wanted some sort of way to "smooth" it out. I've
had limited success on that one.
The old description:
Program to broadcast Earthworm messages from a shared memory transport ring onto Ethernet. Messages may be of any installation id, message type, and module id. To prevent bursty data on the Ethernet, the program sleeps between outgoing packets. A parameter in the configuration file controls whether heartbeat and error messages are passed or blocked. If heartbeats and error messages are not sent, they should be handled by a local copy of the statmgr program.
This module does connection-less broadcasts. In normal usage, it broadcasts onto an isolated, dedicated class b subnet. There must be no routing to or from that subnet. The broadcast is restricted to the subnet specified in the configuration file. Note that the companion module, coaxtoring, has strange system-specific addressing requirements.
NB: This was updated with the ringtocoaxII.d file, but the commands were not updated. Note the new INPUT QUEUE commands and the Min/MaxPacketsPerBurst commands in red font below in the .d file example.
On startup, ringtocoaxII reads the configuration file named on the command line. Commands in this file set up all parameters used in reading Earthworm messages from shared memory and broadcasting them onto ethernet as UDP packets. In the control file, lines may begin with a valid ringtocoaxII command (listed below) or with one of 2 special characters:
# marks the line as a comment (example: # This is a comment). @ allows control files to be nested; one control file can be accessed from another with the command "@" followed by a string representing the path name of the next control file (example: @model.d).
Command names must be typed in the control file exactly as shown in this document (upper/lower case matters!).
EXAMPLE CONFIGURATION FILE
#
#This is RingtocoaxII's parameter file
#
# This program gets earthworm messages from a transport ring and broadcasts
# them to ethernet.
#
# *** WARNING ***
# This module does connection-less broadcasts.
# In normal usage, it broadcasts onto an isolated, dedicated class b
# subnet. There must be no routing to or from that subnet.
# The broadcast is restricted to that subnet as stated below.
# Note that the companion module, coaxtoring, has strange
# system-specific addressing requirements
# General Info
MyModuleId MOD_RINGTOCOAX # Module id for this instance of ringtocoax
RingName WAVE_RING # Shared memory ring for input/output
HeartBeatInterval 30 # Seconds between heartbeats
LogSwitch 1 # If 0, turn off disk log file
# If 1, log to disk file and stderr/stdout
# If 2, log only disk file
Debug 1 # If 1, print Adaptive PacketsPerBurst info
# If 2, also print message logos on screen
# If 0, don't print debug info.
# NOTE: Logging logos / Adaptive info to disk can
# slow down RingtocoaxII's performance. Debug
# should only be used for troubleshooting.
# OPTIONAL, defaults to 0.
# Message Settings
MsgMaxBytes 4096 # Size of biggest message to be sent
CopyStatus 0 # If 0, don't pass heartbeats, errors, or restarts.
# If 1, pass them.
MaxLogo 256 # Maximum number of logos we can handle
# Set to a number >= 0
# List each logo that you want to broadcast in a BroadcastLogo command.
# If the BroadcastLogo is omitted, all messages in RingName will be shipped.
# installation module_id message_type
# ------------ --------- ------------
BroadcastLogo INST_WILDCARD MOD_WILDCARD TYPE_TRACEBUF
# Network Settings
OutAddress 192.168.14.255 # IP address of net to broadcast to.
# Set the node number to 255
# eg. OutAddress 192.168.4.255
OutPortNumber 10001 # The well-known port number to broadcast on.
# Note: Under Solaris, if the port number is at or below
# 1024, ringtocoax must run as root.
# Queue Settings
InputQueueLen 1000 # The Desired Size of the Input Queue in Messages,
# Note: Actual size is InputQueueLen-1.
QueueOptimumPercent 10 # Optimum queue level in percent. Used to determine
# percent excedence. OPTIONAL, defaults to 10
ExcedenceConstant 1 # Constant to multiply the percent excedence by to control
# how much the desired excedence rate can change.
# OPTIONAL, defaults to 1.
STAInterval 1.5 # Interval in seconds over which the short term average
# of the queue height is computed.
# OPTIONAL, defaults to 1.5.
QueueReportInt 20 # Seconds between queue size reports
# OPTIONAL, defaults to 300 seconds.
# Data Export Rate Settings
MaxPacketsPerBurst 200 # Maximum allowed Number of packets per burst the reciver
# can send
MinPacketsPerBurst 10 # Minmum allowed Number of packets per burst the reciver
# can send. OPTIONAL, defaults to 1.
SqrtCount 2000 # Number of square roots between packets within a burst.
BurstInterval 10 # Sleep this many msec between bursts
ThroughputReportInt 20 # Seconds between throughput reports
# OPTIONAL, defaults to 300 seconds.
FUNCTIONAL COMMAND LISTING
Below are the commands recognized by ringtocoax, grouped by the function they influence. All of the commands are required; they must be specified in the control file in order for ringtocoax to operate.
Earthworm system setup: HeartBeatInterval required MyModName required RingName required Network Information: OutAddress required OutPortNumber required Message Handling: CopyStatus required MaxLogo required MsgMaxBytes required BurstInterval required BurstCount required SqrtCount required BroadcastLogo Output Control: ScrnMsg required LogSwitch required
ALPHABETIC COMMAND LISTING & DESCRIPTION
In the following section, all configuration file commands are listed in alphabetical order. Listed along with the command (bold-type) are its arguments (in red), the name of the subroutine that processes the command, and the function within the module that the command influences. A detailed description of the command and is also given. Default values and example commands are listed after each command description.
The following list is organized by:
command [argument here]
BroadcastLogo [instid modid msgtype]
Processed by: ringtocoax_config
Function: msg handling
Specifies one message logo (instid, modid, and msgtype) to read from the shared memory region (named in the "RingName" command) and to broadcast onto ethernet. The instid must be a valid string in earthworm.h; modid and msgtype must be valid strings in earthworm.d. List one logo per "BroadcastLogo" command, and use as many commands as needed. If msgtype is TYPE_HEARTBEAT or TYPE_ERROR, be sure that the CopyStatus? value is non-zero. If no "BroadcastLogo" command is given, ringtocoax defaults to broadcasting all messages in the designated shared memory region.
Example: BroadcastLogo INST_MENLO MOD_WILDCARD TYPE_TRACEBUF
BurstCount [BurstCount]
Processed by: ringtocoax_config
Function: msg handling
Defines the number of UDP packets, BurstCount, in a broadcast "burst". Three commands, "BurstCount", "SqrtCount", and "BurstInterval", control the temporal spacing of UDP packets broadcast by ringtocoax. Ringtocoax broadcasts messages in groups of BurstCount? packets; between the packets within a "burst", ringtocoax loops thru SqrtCount square root calculations to waste a little time. After broadcasting a "burst" of packets, ringtocoax sleeps for BurstInterval? milliseconds, then it starts sending the next burst. The goal of spacing packets a tiny bit in time is to give the receiving program a better chance of catching every packet.
UDP packets are variable in length up to a maximum of 1472 bytes (6 bytes of header, 1466 bytes of data). If an Earthworm message has a length less than or equal to 1466 bytes, it will be broadcast in a single UDP packet. If an Earthworm message is longer than 1466 bytes, it will be fragmented by ringtocoax, broadcast in multiple UDP packets, and reassembled by the receiving program (coaxtoring). Ringtocoax will never pack more than one Earthworm message in a single UDP packet.
It is very important that the overall rate of broadcast is as at least as fast as the rate of local information production. For example, if your system produces 256 TYPE_TRACEBUF messages per second (464 bytes each = 1 UDP packet each), then the BurstCount, SqrtCount, and BurstInterval commands should be configured such that ringtocoax will broadcast at least 256 UDP packets per second. System clock resolution must be considered when configuring the three commands, especially BurstInterval. As of April 1999, the time resolution of each sleep on Windows NT is about 16 msec (one clock tick). On Solaris, the resolution is about 10 msec.
Default: none Example: BurstCount 3
BurstInterval [BurstInterval]
Processed by: ringtocoax_config
Function: msg handling
Defines the minimum interval, in milliseconds, between "bursts" of UDP broadcasts. See the BurstCount command for a detailed description of ringtocoax's broadcasting scheme. BurstInterval must be a number greater than or equal to zero. Remember to consider the system clock resolution when configuring BurstInterval?. As of April 1999, the clock-tick resolution is about 16 msec on Windows NT, and about 10 msec on Solaris.
Default: none Example: BurstInterval 10
CopyStatus [CopyStatus]
Processed by: ringtocoax_config
Function: msg handling
Sets the CopyStatus switch to control whether ringtocoax broadcasts TYPE_ERROR and TYPE_HEARTBEAT messages. If no "BroadcastLogo" command is present, then ringtocoax will default to reading all messages in the shared memory region specified by the "RingName" command. If CopyStatus is non-zero, all of these messages will be broadcast onto ethernet. If CopyStatus? is 0, TYPE_HEARTBEAT and TYPE_ERROR will not be broadcast, but all other messages will.
Default: none Example: CopyStatus 0
HeartBeatInterval [nsec]
Processed by: ringtocoax_config
Function: Earthworm setup
Defines the number of seconds nsec between TYPE_HEARTBEAT messages issued by ringtocoax.
Default: none Example: HeartBeatInterval 15
LogSwitch [switch]
Processed by: ringtocoax_config
Function: output
Sets the on-off switch for writing a log file to disk. If switch is 0, no log file will be written. If switch is 1, ringtocoax will write a daily log file(s) called ringtocoaxxx.log_ccyymmdd where xx is ringtocoax's module id (set with "MyModName" command) and ccyymmdd is the current UTC date (ex: 19990123) on the system clock. The file(s) will be written in the EW_LOG directory (environment variable).
Default: none
MaxLogo [nlogo]
Processed by: ringtocoax_config
Function: msg handling
Sets the maximum number, nlogo, of different message logos that ringtocoax can process. Ringtocoax assigns a sequence number (unsigned char) to each message based on its logo before it broadcasts the message. Space is allocated to store the last sequence number used for up to nlogo different logos. If the tracking arrays are full and ringtocoax gets a never-seen-before message logo from shared memory, ringtocoax will not broadcast that message, but it will issue a TYPE_ERROR message explaining the problem.
Default: none Example: MaxLogo 256
MsgMaxBytes [nbytes]
Processed by: ringtocoax_config
Functio: msg handling
Sets the size, in bytes, of the buffer that ringtocoax uses for storing Earthworm messages read from shared memory. nbytes should be the size of the largest message you expect ringtocoax to receive. If ringtocoax receives a message larger than nbytes, it will drop the entire message and issue a TYPE_ERROR message containing the string "too big for Buffer" and the logo of the dropped message.
Default: none Example: MsgMaxBytes 51740
MyModName [mod_id]
Processed by: ringtocoax_config
Function: Earthworm setup
Sets the module id for labeling all outgoing heartbeat and error messages. mod_id is a character string (valid strings are listed in earthworm.d) that relates (in earthworm.d) to a unique single-byte number. All messages that ringtocoax receives from shared memory are tagged with their original logos when they are broadcast onto ethernet.
Default: none Example: MyModName MOD_RINGTOCOAX
OutAddress [a.b.c.d]
Processed by: ringtocoax_config
Function: network info
Tells ringtocoax the TCP/IP address of the ethernet line onto which it should broadcast Earthworm messages. a.b.c.d is a character string in tcp/ip's standard dotted-decimal notation that ringtocoax uses along with the port number from the "OutPortNumber" command to set up a socket to which it will broadcast UDP Datagram packets. By convention, all Earthworm broadcasts have the node ("d" part of address) set to 255 (for example a.b.c.255).
Default: none Example: OutAddress 192.168.4.255
OutPortNumber [port]
Processed by: ringtocoax_config
Function: network info
Defines the well-known port number port to be used by ringtocoax for broadcasting. port is an integer that ringtocoax uses along with the address from the "OutAddress" command to set up a socket to which it will broadcast UDP Datagram packets.
NOTE: The port number must be greater than 1024. Smaller port numbers require root permission.
Default: none Example: OutPortNumber 2025
RingName [ring]
Processed by: ringtocoax_config
Function: Earthworm setup
Tells ringtocoax which shared memory region to use for input/output. ring is a character string (valid strings are listed in earthworm.d) that relates (in earthworm.d) to a unique number for the key to the shared memory region. All messages that ringtocoax reads from shared memory are tagged with their original logos when they are broadcast onto ethernet.
Default: none Example: RingName WAVE_RING
ScrnMsg [switch]
Processed by: ringtocoax_config
Function: output
Sets a switch for writing messages to the screen. If switch is 1, a line is printed to the screen detailing the logo and the sequence number of each Earthworm message broadcast to ethernet. If switch is 0, no messages are written to the screen.
Default: none Example: ScrnMsg 0
SqrtCount [SqrtCount]
Processed by: ringtocoax_config
Function: msg handling
Defines the number of square root calculations ringtocoax performs between broadcasting each UDP packet of a "burst." See the BurstCount command for a detailed description of ringtocoax's broadcasting scheme. SqrtCount must be an integer greater than or equal to zero.
Default: none Example: SqrtCount 2000
