Discussion:
[omniORB] omniORB timeout problems on systems with two network interface cards
Nikola Radovanovic
2009-03-03 19:06:44 UTC
Permalink
Hi,
we have omniORB timeout problems on systems with two network interface
cards (NIC)

We have system consisting out of 2 PCs running windows XP Pro 32 bit,
each with two NIC's.
Our software uses omniORB-4.1.1 with following params:
resetTimeOutOnRetries 1
verifyObjectExistsAndType 0
traceLevel 25

also, DisableDHCPMediaSense is set to 1 in
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

We wanted to see how omniORB behaves in cases when we plug out one
(primary) network cable.

Main test scenario:
On one PC runs CORBA server, binded to both NICs via ORBendpoints passed
through command line.
We than pull out primary network cable on server PC and try to run
client on second PC. Client will try to obtain server object reference
from omniNames and call some (simple) method on it.
We have tried different timeout configurations and get the following
results:

1. all timeouts are disabled (this includes connection and both
(global and per reference) call timeouts, per thread call timeout is
disabled in omniORB configuration)
When opening connection to omniNames it takes (roughly) 5 secs
(probably due to scanGranularity which is 5 secs by default) for omniORB
on client side to figure out that connection to omniNames via primary
connection can not be established. Switch rope occurs and connection
will be established via secondary network. Reference is obtained via
that connection. When method is called on that server reference object,
it takes roughly 5 sesc for omniORB on client side to figure out that
connection to server via primary connection can not be established.
Switch rope will occur and connection is established via secondary
network. After that, method call on server reference will be executed
successfully.

2. connection timeout is set to value less than 5 secs (e.g. 2 or 3
secs); global and per reference call timeouts are disabled
Behavior is as expected: it takes specified amount of time for
client to figure out that connection to server via primary connection
could not be established.
Beside that, behavior is like in previous scenario.

3. global call timeout is set to 10 secs, connection timeout is 3
secs and per reference call timeout is not specified
Connection timeout would not be detected (3 secs as expected) but
global call timeout (10 secs) will expire.
This is the pattern when global call timeout is set to less than 20
secs. For global call timeout greater than 20 secs, time to detect
failure while opening connection via primary LAN is roughly 20 secs.
In short, in this scenario connection timeout is totally ignored and
it takes roughly 20 secs to detect failure to open connection (4x more
than in scenario 1)

4. per reference call timeout is set to 10 secs, connection timeout
is 3 secs and global call timeout is not specified
It takes roughly 3 secs (connection timeout occurs) to connect to
omniNames and acquire server object reference. Per reference call
timeout on that reference is set to 10 secs. When method is called on
that reference connection failure will not be detected, instead call
timeout will expire. If per reference call timeout is set to value
greater than 20 secs, time to detect failure while opening connection
via primary LAN is roughly 20 secs.
In short, in this scenario connection timeout is totally ignored on
reference with per reference call timeout set.

thanks for the replies
Telvent DMS
Nikola Radovanovic
2009-03-03 19:08:55 UTC
Permalink
omniORB timeout problems on systems with two network interface cards (NIC)

We have system consisting out of 2 PCs running windows XP Pro 32 bit,
each with two NIC's.
Our software uses omniORB-4.1.1 with following params:
resetTimeOutOnRetries 1
verifyObjectExistsAndType 0
traceLevel 25

also, DisableDHCPMediaSense is set to 1 in
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

We wanted to see how omniORB behaves in cases when we plug out one
(primary) network cable.

Main test scenario:
On one PC runs CORBA server, binded to both NICs via ORBendpoints passed
through command line.
We than pull out primary network cable on server PC and try to run
client on second PC. Client will try to obtain server object reference
from omniNames and call some (simple) method on it.
We have tried different timeout configurations and get the following
results:

1. all timeouts are disabled (this includes connection and both
(global and per reference) call timeouts, per thread call timeout is
disabled in omniORB configuration)
When opening connection to omniNames it takes (roughly) 5 secs
(probably due to scanGranularity which is 5 secs by default) for omniORB
on client side to figure out that connection to omniNames via primary
connection can not be established. Switch rope occurs and connection
will be established via secondary network. Reference is obtained via
that connection. When method is called on that server reference object,
it takes roughly 5 sesc for omniORB on client side to figure out that
connection to server via primary connection can not be established.
Switch rope will occur and connection is established via secondary
network. After that, method call on server reference will be executed
successfully.

2. connection timeout is set to value less than 5 secs (e.g. 2 or 3
secs); global and per reference call timeouts are disabled
Behavior is as expected: it takes specified amount of time for
client to figure out that connection to server via primary connection
could not be established.
Beside that, behavior is like in previous scenario.

3. global call timeout is set to 10 secs, connection timeout is 3
secs and per reference call timeout is not specified
Connection timeout would not be detected (3 secs as expected) but
global call timeout (10 secs) will expire.
This is the pattern when global call timeout is set to less than 20
secs. For global call timeout greater than 20 secs, time to detect
failure while opening connection via primary LAN is roughly 20 secs.
In short, in this scenario connection timeout is totally ignored and
it takes roughly 20 secs to detect failure to open connection (4x more
than in scenario 1)

4. per reference call timeout is set to 10 secs, connection timeout
is 3 secs and global call timeout is not specified
It takes roughly 3 secs (connection timeout occurs) to connect to
omniNames and acquire server object reference. Per reference call
timeout on that reference is set to 10 secs. When method is called on
that reference connection failure will not be detected, instead call
timeout will expire. If per reference call timeout is set to value
greater than 20 secs, time to detect failure while opening connection
via primary LAN is roughly 20 secs.
In short, in this scenario connection timeout is totally ignored on
reference with per reference call timeout set.

thanks for the replies
Telvent DMS
Duncan Grisby
2009-03-13 19:04:15 UTC
Permalink
Post by Nikola Radovanovic
omniORB timeout problems on systems with two network interface cards (NIC)
[...]

Please get traces from the situations you think are problematic, running
the client with -ORBtraceLevel 25 -ORBtraceInvocations 1
-ORBtraceThreadId 1 -ORBtraceTime 1

Cheers,

Duncan.
--
-- Duncan Grisby --
-- ***@grisby.org --
-- http://www.grisby.org --
Loading...