Changes between Version 1 and Version 2 of Import-Export

Timestamp:

02/29/12 15:55:10 (8 years ago)

Author:

branden

Comment:

--

Import-Export

@@ +1 @@
+[[PageOutline]]
+
 = [wiki:Earthworm Earthworm]: Import-Export Overview =
 
@@ -32 +34 @@
   This new version of export expects to receive an acknowledgment packet  for every packet it writes to the socket. Creation of this new version  was desired because we observed during the export of continuous trace data that during a communication failure, export was able to successfully write several messages to the (broken?) socket, but import never received them. When the socket was re-established, those messages were lost forever. 
 
-  Acknowledgments are received asynchronously by the socket-reading thread.  A circular buffer, SendQueue, tracks the status of msgs after they been  pulled from the MessageStacker queue. If the socket is broken/reconnected,  export*ack re-sends any msgs in the SendQueue which haven't been  acknowledged. In normal operation, if export*ack laps the SendQueue and  finds a message that has been sent but not ACKed, it will sleep 10ms and  then check status again, looping until an ACK is received before preparing  to send the next message. 
+  Acknowledgments are received asynchronously by the socket-reading thread.  A circular buffer, !SendQueue, tracks the status of msgs after they been  pulled from the !MessageStacker queue. If the socket is broken/reconnected,  export*ack re-sends any msgs in the !SendQueue which haven't been  acknowledged. In normal operation, if export*ack laps the !SendQueue and  finds a message that has been sent but not ACKed, it will sleep 10ms and  then check status again, looping until an ACK is received before preparing  to send the next message. 
 
-  The SendQueue length is configurable, from 1 to 254 messages (default=100).   Using a shorter SendQueue can cause a decrease in the rate at which export will send messages. A slower rate might be desireable so  that an import system is not flooded by data after a prolonged  communication break. In testing with export and import on the same  machine, these are export's approximate maximum data rates:
+  The !SendQueue length is configurable, from 1 to 254 messages (default=100).   Using a shorter !SendQueue can cause a decrease in the rate at which export will send messages. A slower rate might be desireable so  that an import system is not flooded by data after a prolonged  communication break. In testing with export and import on the same  machine, these are export's approximate maximum data rates:
 
 {{{
@@ -42 +44 @@
 }}}
 
-Using the default SendQueueLength, export*ack's data rate was not observably different than export_generic's data rate.  
+Using the default !SendQueueLength, export*ack's data rate was not observably different than export_generic's data rate.  
 
-  export*ack requires using the new import_ack to communicate properly. Since the receipt of ACKs is asynchronous, we may now have the opposite  problem to that we had before (duplicate data instead of lost data). It  is now possible that import_ack actually received a packet, but export_ack didn't get the ACK before the socket broke. In this is case, export_ack 
-will re-send the packet and import_ack will see the same packet more than  once. The system running import_ack must be prepared for duplicate data.
+export*ack requires using the new import_ack to communicate properly. Since the receipt of ACKs is asynchronous, we may now have the opposite  problem to that we had before (duplicate data instead of lost data). It  is now possible that import_ack actually received a packet, but export_ack didn't get the ACK before the socket broke. In this is case, export_ack will re-send the packet and import_ack will see the same packet more than  once. The system running import_ack must be prepared for duplicate data.
 
 == import_ack ==