Discussion:
[omniORB] giopServer pd_state ZOMBIE for RootPOA (when giopServer TIMEDOUT before)
Martin Trappel
2008-08-05 22:28:23 UTC
Permalink
[omniORB 4.1.2, Windows XP, VC8 prebuilt DLLs]

I have the situation where I repeatedly initialize and destroy the ORB
in a client application, in between creating one new POA from the
RootPOA and doing a few server calls on a server resolved via
NameService. (The reason for this is a multi threaded environment where,
normally, the init+shutdown would occur once - but
can occur an arbitrary number of times for certain usecases.)

I have successfully synchronized ORB_init and the complete shutdown of
the ORB as well as all POAs via a singleton class. (I'm pretty sure
this synchronization is working OK, having sufficient logs to verify
it.)
i.e. my ORB_init(...) is completely synchronized against my
orb_var->shutdown(true) + orb_var->destroy()

Now I face the problem that I hit an omniORB::fatalException reliably,
after a few cycles of init+shutdow. The error occurs in
giopserver.cc(284) where apparently the endpoint object giopServer of
the root POA is in a ZOMBIE state.
284: OMNIORB_ASSERT(pd_state != ZOMBIE);

The call stack is as follows:
omniORB412_vc8_rtd.dll!omni::assertFail(const char * file=0x67ca8...
omniORB412_vc8_rtd.dll!omni::giopServer::instantiate(const char *...
omniORB412_vc8_rtd.dll!omni::instantiate_endpoint(const char * ur...
omniORB412_vc8_rtd.dll!omni::omniObjAdapter::initialise() Line 4...
omniORB412_vc8_rtd.dll!initialise_poa() Line 2604 C++
omniORB412_vc8_rtd.dll!omni::omniOrbPOA::rootPOA(int init_if_none...
omniORB412_vc8_rtd.dll!omni::resolveRootPOAFn() Line 4159 + 0xa ...
omniORB412_vc8_rtd.dll!omni::resolvePseudo(const char * id=0x04c7...
omniORB412_vc8_rtd.dll!omni::omniInitialReferences::resolve(const...
omniORB412_vc8_rtd.dll!omniOrbORB::resolve_initial_references(con...
ProcessORB.dll!ProcessORB::CreateNewPOA() Line 92 + 0x28 bytes
CDACIC.dll!CACIClient::InitPOA() Line 365 + 0x19 bytes C++
CDACIC.dll!CACIClient::Init() Line 56 C++
...

Under which circumstances would an giopServer endpoint object be left
in the ZOMBIE state? giopServer::remove() will apparently leave the
object in a ZOMBIE state if pd_state is TIMEDOUT. I was able to verify
that if on shutdown the object does not do delete this; because it was
TIMEDOUT, upon the next initialization and call to
resolve_initial_refereces(RootPOA) I get the ZOMBIE error in the
instatiate() method.

It would be helpful if someone could tell me what that that TIMEDOUT
is signifying there in the context of the RootPOA (and maybe how to
reproduce it in a less complicated context for the RootPOA).

any and all pointers and questions welcome!
cheers,
Martin
Martin Trappel
2008-08-06 19:43:01 UTC
Permalink
Post by Martin Trappel
[omniORB 4.1.2, Windows XP, VC8 prebuilt DLLs]
I have the situation where I repeatedly initialize and destroy the ORB
in a client application, (...)
I have now been able to reproduce the problem in a simpler context.
Consider the following IDL:
interface Dummy {
void echo(in string s);
};
interface DummyServer {
void doEcho(in Dummy d);
};

DummyServer is called by the process running a Dummy instance and the
doEcho method just calls echo on the given interface pointer.

And this is the setup I'm using:
A.) Start a process with an instance of DummyServer (and register the
object with the naming service). The server will just keep running and
do nothing else. (just as with the echo example)

B.) The Client Process now does the following *in a loop*:
1.) ORB_init
2.) start a separate omni_thread and call orb->run() there
3.) create a new poa from the root poa with the same policies as the
root poa
4.) Create a new Dummy impl object
5.) activate the object with the created POA
6.) activate the POA
7.) Resolve the DummyServer Object via NamingService
8.) Call dummy_srv_ref->doEcho(my_dummy->_this())
9.) deactivate POA + destroy POA
10.) shutdown(true) orb (afterwards wait for run thread to terminate)
11.) destroy() orb

With this setup I will reliably hit
Caught omniORB::fatalException:
file: giopServer.cc
line: 284
mesg: pd_state != ZOMBIE
after just a few iterations (sometimes even on the first one).

If the doEcho() method in the server does not callback echo() in the
client everything works fine. (I assume no connections are open with the
Dummy POA so there can be no timeouts there.)

So somehow the active POA leads to a TIMEDOUT condition when shutting
down and upon next ORB_init + resolve of rootpoa something is still left
in a ZOMBIE state ... (although the ORB has apparently been destroyed
cleanly)

The related omniTrace message in the last session appears to be:
<omniORB: Timed out. 1 connection and 1 dedicated worker remaining.>
This message is generated from giopServer.cc:653 and if we get a timeout
there, the next session will hit the ZOMBIE state.

Find the omniORB trace log attached.

The only solution at the moment appears to be to never shutdown the ORB
after it has been initialized once :(

( If someone wants to reproduce this I can clean out my testsources and
also post them. )

br,
Martin




-------------- next part --------------
omniORB: Version: 4.1.2
omniORB: Distribution date: Thu Feb 14 14:19:08 GMT 2008 dgrisby
omniORB: Maximum supported GIOP version is 1.2
omniORB: Native char code sets: UTF-8 ISO-8859-1.
omniORB: Transmission char code sets: UTF-8(1.2) UTF-8(1.1) ISO-8859-1(1.2) ISO-8859-1(1.1) ISO-8859-1(1.0).
omniORB: Native wide char code sets: UTF-16.
omniORB: Transmission wide char code sets: UTF-16(1.2).
omniORB: Initialising omniDynamic library.
omniORB: Current configuration is as follows:
omniORB: DefaultInitRef (file) =
omniORB: DefaultInitRef (args) =
omniORB: InitRef = NameService=corbaname::trappelm.ksengineers.at
omniORB: abortOnInternalError = 0
omniORB: abortOnNativeException = 0
omniORB: acceptBiDirectionalGIOP = 0
omniORB: acceptMisalignedTcIndirections = 0
omniORB: bootstrapAgentHostname =
omniORB: bootstrapAgentPort = 900
omniORB: clientCallTimeOutPeriod = 0
omniORB: clientConnectTimeOutPeriod = 0
omniORB: clientTransportRule = * unix,ssl,tcp
omniORB: configFile = [none]
omniORB: connectionWatchImmediate = 0
omniORB: connectionWatchPeriod = 50000
omniORB: copyValuesInLocalCalls = 1
omniORB: diiThrowsSysExceptions = 0
omniORB: dumpConfiguration = 0
omniORB: endPoint = giop:tcp:trappelm.ksengineers.at:
omniORB: endPointPublish = addr
omniORB: giopMaxMsgSize = 2097152
omniORB: giopTargetAddressMode = KeyAddr
omniORB: id = omniORB4
omniORB: idleThreadTimeout = 10
omniORB: immediateAddressSwitch = 0
omniORB: inConScanPeriod = 180
omniORB: lcdMode = 0
omniORB: maxGIOPConnectionPerServer = 5
omniORB: maxGIOPVersion = 1.2
omniORB: maxInterleavedCallsPerConnection = 5
omniORB: maxServerThreadPerConnection = 100
omniORB: maxServerThreadPoolSize = 100
omniORB: maxSocketRecv = 131072
omniORB: maxSocketSend = 131072
omniORB: nativeCharCodeSet = ISO-8859-1
omniORB: nativeWCharCodeSet = UTF-16
omniORB: objectTableSize = 0
omniORB: offerBiDirectionalGIOP = 0
omniORB: oneCallPerConnection = 1
omniORB: outConScanPeriod = 120
omniORB: poaHoldRequestTimeout = 0
omniORB: poaUniquePersistentSystemIds = 1
omniORB: principal = [Null]
omniORB: resetTimeOutOnRetries = 0
omniORB: scanGranularity = 5
omniORB: serverCallTimeOutPeriod = 0
omniORB: serverTransportRule = * unix,ssl,tcp
omniORB: socketSendBuffer = 16384
omniORB: strictIIOP = 1
omniORB: supportBootstrapAgent = 0
omniORB: supportCurrent = 1
omniORB: supportPerThreadTimeOut = 0
omniORB: tcAliasExpand = 0
omniORB: threadPerConnectionLowerLimit = 9000
omniORB: threadPerConnectionPolicy = 1
omniORB: threadPerConnectionUpperLimit = 10000
omniORB: threadPoolWatchConnection = 1
omniORB: traceExceptions = 1
omniORB: traceFile = omniORB412_log_test_dummy_cli.txt
omniORB: traceInvocationReturns = 0
omniORB: traceInvocations = 0
omniORB: traceLevel = 25
omniORB: traceThreadId = 0
omniORB: traceTime = 0
omniORB: unixTransportDirectory = /tmp/omni-%u
omniORB: unixTransportPermission = 777
omniORB: useTypeCodeIndirections = 1
omniORB: verifyObjectExistsAndType = 1
omniORB: Initialising incoming endpoints.
omniORB: Instantiate endpoint 'giop:tcp:trappelm.ksengineers.at:'
omniORB: Explicit bind to host trappelm.ksengineers.at.
omniORB: Bind to address 172.27.169.2
omniORB: Publish specification: 'addr'
omniORB: Try to publish 'addr' for endpoint giop:tcp:trappelm.ksengineers.at:3207
omniORB: Publish endpoint 'giop:tcp:trappelm.ksengineers.at:3207'
omniORB: Starting serving incoming endpoints.
omniORB: AsyncInvoker: thread id = 38 has started. Total threads = 1
omniORB: giopRendezvouser task execute for giop:tcp:trappelm.ksengineers.at:3207
omniORB: Adding root/Automatically_Named_POA_10<0> (activating) to object table.
omniORB: State root/Automatically_Named_POA_10<0> (activating) -> active
omniORB: Creating ref to remote: key<NameService>
target id : IDL:omg.org/CORBA/Object:1.0
most derived id:
omniORB: Initial reference `NameService' resolved from configuration file.
omniORB: Client attempt to connect to giop:tcp:trappelm.ksengineers.at:2809
omniORB: AsyncInvoker: thread id = 39 has started. Total threads = 2
omniORB: Scavenger task execute.
omniORB: Client opened connection to giop:tcp:172.27.169.2:2809
omniORB: sendChunk: to giop:tcp:172.27.169.2:2809 100 bytes
omniORB: inputMessage: from giop:tcp:172.27.169.2:2809 25 bytes
omniORB: Creating ref to remote: key<NameService>
target id : IDL:omg.org/CosNaming/NamingContext:1.0
most derived id:
omniORB: sendChunk: to giop:tcp:172.27.169.2:2809 111 bytes
omniORB: inputMessage: from giop:tcp:172.27.169.2:2809 172 bytes
omniORB: Creating ref to remote: root<0>
target id : IDL:omg.org/CORBA/Object:1.0
most derived id: IDL:DummyServer:1.0
omniORB: ObjRef() -- deleted.
omniORB: Creating ref to local: root/Automatically_Named_POA_10<0>
target id : IDL:Dummy:1.0
most derived id: IDL:Dummy:1.0
omniORB: LocateRequest to remote: root<0>
omniORB: Client attempt to connect to giop:tcp:trappelm.ksengineers.at:3057
omniORB: Client opened connection to giop:tcp:172.27.169.2:3057
omniORB: sendChunk: to giop:tcp:172.27.169.2:3057 38 bytes
omniORB: inputMessage: from giop:tcp:172.27.169.2:3057 20 bytes
omniORB: Send codeset service context: (ISO-8859-1,UTF-16)
omniORB: sendChunk: to giop:tcp:172.27.169.2:3057 252 bytes
omniORB: Server accepted connection from giop:tcp:172.27.169.2:3210
omniORB: AsyncInvoker: thread id = 40 has started. Total threads = 3
omniORB: giopWorker task execute.
omniORB: Accepted connection from giop:tcp:172.27.169.2:3210 because of this rule: "* unix,ssl,tcp"
omniORB: inputMessage: from giop:tcp:172.27.169.2:3210 65 bytes
omniORB: Handling a GIOP LOCATE_REQUEST.
omniORB: sendChunk: to giop:tcp:172.27.169.2:3210 20 bytes
omniORB: inputMessage: from giop:tcp:172.27.169.2:3210 163 bytes
omniORB: Receive codeset service context and set TCS to (ISO-8859-1,UTF-16)
omniORB: sendChunk: to giop:tcp:172.27.169.2:3210 24 bytes
omniORB: inputMessage: from giop:tcp:172.27.169.2:3057 24 bytes
omniORB: Deactivating POAManager for POA(Automatically_Named_POA_10).
omniORB: State root/Automatically_Named_POA_10<0> (active) -> deactivating (OA destruction)
omniORB: State root/Automatically_Named_POA_10<0> (deactivating OA) -> etherealising
omniORB: Removing root/Automatically_Named_POA_10<0> (etherealising) from object table
omniORB: Met detached object. 0 remaining.
omniORB: Wait for 0 detached objects.
omniORB: Destroying POA(Automatically_Named_POA_10).
omniORB: Deactivating all POA(Automatically_Named_POA_10)'s objects.
omniORB: Waiting for requests to complete on POA(Automatically_Named_POA_10).
omniORB: Requests on POA(Automatically_Named_POA_10) completed.
omniORB: Etherealising POA(Automatically_Named_POA_10)'s objects.
omniORB: Wait for 0 detached objects.
omniORB: Destruction of POA(Automatically_Named_POA_10) complete.
omniORB: Preparing to shutdown ORB.
omniORB: Destroying POA(RootPOA).
omniORB: Deactivating all POA(RootPOA)'s objects.
omniORB: Waiting for requests to complete on POA(RootPOA).
omniORB: Requests on POA(RootPOA) completed.
omniORB: Etherealising POA(RootPOA)'s objects.
omniORB: Wait for 0 detached objects.
omniORB: All object adapters inactive. Stopping serving incoming endpoints.
omniORB: giopServer deactivate...
omniORB: Close connections with threads and monitors...
omniORB: sendCloseConnection: to giop:tcp:172.27.169.2:3210 12 bytes
omniORB: Closed 1 connection out of 1.
omniORB: Wait for 1 dedicated thread to finish...
omniORB: Error in network receive (start of message): giop:tcp:172.27.169.2:3210
omniORB: throw giopStream::CommFailure from giopStream.cc:874(0,NO,COMM_FAILURE_UnMarshalArguments)
omniORB: Server connection refcount = 1
omniORB: removeConnectionAndWorker for dedicated worker. 0 remaining.
omniORB: 0 connections and 0 dedicated workers remaining.
omniORB: Server connection refcount = 0
omniORB: Terminate rendezvousers...
omniORB: Server close connection from giop:tcp:172.27.169.2:3210
omniORB: giopRendezvouser for giop:tcp:trappelm.ksengineers.at:3207 terminate...
omniORB: giopRendezvouser for endpoint giop:tcp:trappelm.ksengineers.at:3207 exit.
omniORB: No remaining rendezvousers.
omniORB: Rendezvousers terminated.
omniORB: giopServer deactivated.
omniORB: Destruction of POA(RootPOA) complete.
omniORB: Shutting-down all incoming endpoints.
omniORB: TCP endpoint shut down.
omniORB: Disable ObjRef(IDL:Dummy:1.0) root/Automatically_Named_POA_10<0>
omniORB: Object table entry root/Automatically_Named_POA_10<0> (dead) deleted.
omniORB: Disable ObjRef(IDL:DummyServer:1.0) root<0>
omniORB: omniRemoteIdentity deleted.
omniORB: Disable ObjRef() key<NameService>
omniORB: omniRemoteIdentity deleted.
omniORB: 3 object references present at ORB shutdown.
omniORB: ORB shutdown is complete.
omniORB: ObjRef() -- deleted.
omniORB: Deinitialising omniDynamic library.
omniORB: Release registered value factories.
omniORB: Terminate strand scavenger.
omniORB: Close remaining strands.
omniORB: Shutdown close connection to giop:tcp:trappelm.ksengineers.at:3057
omniORB: sendCloseConnection: to giop:tcp:172.27.169.2:3057 12 bytes
omniORB: Client connection refcount (forced) = 0
omniORB: Client close connection to giop:tcp:172.27.169.2:3057
omniORB: Shutdown close connection to giop:tcp:trappelm.ksengineers.at:2809
omniORB: Client connection refcount (forced) = 0
omniORB: Client close connection to giop:tcp:172.27.169.2:2809
omniORB: 0 remaining bidir ropes deleted.
omniORB: 2 remaining ropes deleted.
omniORB: Clear endPoint options.
omniORB: Wait for 3 invoker threads to finish.
omniORB: AsyncInvoker: thread id = 40 has exited. Total threads = 3
omniORB: AsyncInvoker: thread id = 38 has exited. Total threads = 3
omniORB: AsyncInvoker: thread id = 39 has exited. Total threads = 1
omniORB: Invoker threads finished.
omniORB: AsyncInvoker: deleted.
omniORB: ObjRef(IDL:Dummy:1.0) -- deleted.
omniORB: ObjRef(IDL:DummyServer:1.0) -- deleted.
omniORB: No more references to the ORB -- deleted.
omniORB: Released 2 stub TypeCodes from 'boxesDynSK.cc'.
omniORB: Unregister value factory for 'IDL:omg.org/CORBA/WStringValue:1.0'.
omniORB: Unregister value factory for 'IDL:omg.org/CORBA/StringValue:1.0'.
omniORB: Released 21 stub TypeCodes from 'NamingDynSK.cc'.
omniORB: Released 75 stub TypeCodes from 'irDynSK.cc'.
omniORB: Released 46 stub TypeCodes from 'corbaidlDynSK.cc'.
omniORB: Released 4 stub TypeCodes from 'bootstrapDynSK.cc'.
omniORB: Released 7 stub TypeCodes from 'poa_enumsDynSK.cc'.
omniORB: Released 1 stub TypeCode from 'unknownUserExn.cc'.
omniORB: Released 2 stub TypeCodes from 'policy.cc'.
omniORB: Released 60 stub TypeCodes from 'dynException.cc'.
omniORB: Final clean-up
omniORB: Release value factory table.
omniORB: Released 21 stub TypeCodes from 'typecode.cc'.
omniORB: Deleted 13 nil object references and 4 other tracked objects.
omniORB: Final clean-up completed.
omniORB: Version: 4.1.2
omniORB: Distribution date: Thu Feb 14 14:19:08 GMT 2008 dgrisby
omniORB: Warning: unable to create an IPv6 socket. Unable to obtain the list of IPv6 interface addresses (10047).
omniORB: My addresses are:
omniORB: 172.27.169.2
omniORB: 127.0.0.1
omniORB: Maximum supported GIOP version is 1.2
omniORB: Native char code sets: UTF-8 ISO-8859-1.
omniORB: Transmission char code sets: UTF-8(1.2) UTF-8(1.1) ISO-8859-1(1.2) ISO-8859-1(1.1) ISO-8859-1(1.0).
omniORB: Native wide char code sets: UTF-16.
omniORB: Transmission wide char code sets: UTF-16(1.2).
omniORB: Initialising omniDynamic library.
omniORB: Current configuration is as follows:
omniORB: DefaultInitRef (file) =
omniORB: DefaultInitRef (args) =
omniORB: InitRef = NameService=corbaname::trappelm.ksengineers.at
omniORB: abortOnInternalError = 0
omniORB: abortOnNativeException = 0
omniORB: acceptBiDirectionalGIOP = 0
omniORB: acceptMisalignedTcIndirections = 0
omniORB: bootstrapAgentHostname =
omniORB: bootstrapAgentPort = 900
omniORB: clientCallTimeOutPeriod = 0
omniORB: clientConnectTimeOutPeriod = 0
omniORB: clientTransportRule = * unix,ssl,tcp
omniORB: configFile = [none]
omniORB: connectionWatchImmediate = 0
omniORB: connectionWatchPeriod = 50000
omniORB: copyValuesInLocalCalls = 1
omniORB: diiThrowsSysExceptions = 0
omniORB: dumpConfiguration = 0
omniORB: endPoint = giop:tcp:trappelm.ksengineers.at:
omniORB: endPointPublish = addr
omniORB: giopMaxMsgSize = 2097152
omniORB: giopTargetAddressMode = KeyAddr
omniORB: id = omniORB4
omniORB: idleThreadTimeout = 10
omniORB: immediateAddressSwitch = 0
omniORB: inConScanPeriod = 180
omniORB: lcdMode = 0
omniORB: maxGIOPConnectionPerServer = 5
omniORB: maxGIOPVersion = 1.2
omniORB: maxInterleavedCallsPerConnection = 5
omniORB: maxServerThreadPerConnection = 100
omniORB: maxServerThreadPoolSize = 100
omniORB: maxSocketRecv = 131072
omniORB: maxSocketSend = 131072
omniORB: nativeCharCodeSet = ISO-8859-1
omniORB: nativeWCharCodeSet = UTF-16
omniORB: objectTableSize = 0
omniORB: offerBiDirectionalGIOP = 0
omniORB: oneCallPerConnection = 1
omniORB: outConScanPeriod = 120
omniORB: poaHoldRequestTimeout = 0
omniORB: poaUniquePersistentSystemIds = 1
omniORB: principal = [Null]
omniORB: resetTimeOutOnRetries = 0
omniORB: scanGranularity = 5
omniORB: serverCallTimeOutPeriod = 0
omniORB: serverTransportRule = * unix,ssl,tcp
omniORB: socketSendBuffer = 16384
omniORB: strictIIOP = 1
omniORB: supportBootstrapAgent = 0
omniORB: supportCurrent = 1
omniORB: supportPerThreadTimeOut = 0
omniORB: tcAliasExpand = 0
omniORB: threadPerConnectionLowerLimit = 9000
omniORB: threadPerConnectionPolicy = 1
omniORB: threadPerConnectionUpperLimit = 10000
omniORB: threadPoolWatchConnection = 1
omniORB: traceExceptions = 1
omniORB: traceFile = omniORB412_log_test_dummy_cli.txt
omniORB: traceInvocationReturns = 0
omniORB: traceInvocations = 0
omniORB: traceLevel = 25
omniORB: traceThreadId = 0
omniORB: traceTime = 0
omniORB: unixTransportDirectory = /tmp/omni-%u
omniORB: unixTransportPermission = 777
omniORB: useTypeCodeIndirections = 1
omniORB: verifyObjectExistsAndType = 1
omniORB: Initialising incoming endpoints.
omniORB: Instantiate endpoint 'giop:tcp:trappelm.ksengineers.at:'
omniORB: Explicit bind to host trappelm.ksengineers.at.
omniORB: Bind to address 172.27.169.2
omniORB: Publish specification: 'addr'
omniORB: Try to publish 'addr' for endpoint giop:tcp:trappelm.ksengineers.at:3212
omniORB: Publish endpoint 'giop:tcp:trappelm.ksengineers.at:3212'
omniORB: Starting serving incoming endpoints.
omniORB: AsyncInvoker: thread id = 2 has started. Total threads = 1
omniORB: giopRendezvouser task execute for giop:tcp:trappelm.ksengineers.at:3212
omniORB: Adding root/Automatically_Named_POA_1<0> (activating) to object table.
omniORB: State root/Automatically_Named_POA_1<0> (activating) -> active
omniORB: Creating ref to remote: key<NameService>
target id : IDL:omg.org/CORBA/Object:1.0
most derived id:
omniORB: Initial reference `NameService' resolved from configuration file.
omniORB: Client attempt to connect to giop:tcp:trappelm.ksengineers.at:2809
omniORB: AsyncInvoker: thread id = 3 has started. Total threads = 2
omniORB: Scavenger task execute.
omniORB: Client opened connection to giop:tcp:172.27.169.2:2809
omniORB: sendChunk: to giop:tcp:172.27.169.2:2809 100 bytes
omniORB: inputMessage: from giop:tcp:172.27.169.2:2809 25 bytes
omniORB: Creating ref to remote: key<NameService>
target id : IDL:omg.org/CosNaming/NamingContext:1.0
most derived id:
omniORB: sendChunk: to giop:tcp:172.27.169.2:2809 111 bytes
omniORB: inputMessage: from giop:tcp:172.27.169.2:2809 172 bytes
omniORB: Creating ref to remote: root<0>
target id : IDL:omg.org/CORBA/Object:1.0
most derived id: IDL:DummyServer:1.0
omniORB: ObjRef() -- deleted.
omniORB: Creating ref to local: root/Automatically_Named_POA_1<0>
target id : IDL:Dummy:1.0
most derived id: IDL:Dummy:1.0
omniORB: LocateRequest to remote: root<0>
omniORB: Client attempt to connect to giop:tcp:trappelm.ksengineers.at:3057
omniORB: Client opened connection to giop:tcp:172.27.169.2:3057
omniORB: sendChunk: to giop:tcp:172.27.169.2:3057 38 bytes
omniORB: inputMessage: from giop:tcp:172.27.169.2:3057 20 bytes
omniORB: Send codeset service context: (ISO-8859-1,UTF-16)
omniORB: sendChunk: to giop:tcp:172.27.169.2:3057 248 bytes
omniORB: Server accepted connection from giop:tcp:172.27.169.2:3215
omniORB: AsyncInvoker: thread id = 4 has started. Total threads = 3
omniORB: giopWorker task execute.
omniORB: Accepted connection from giop:tcp:172.27.169.2:3215 because of this rule: "* unix,ssl,tcp"
omniORB: inputMessage: from giop:tcp:172.27.169.2:3215 64 bytes
omniORB: Handling a GIOP LOCATE_REQUEST.
omniORB: sendChunk: to giop:tcp:172.27.169.2:3215 20 bytes
omniORB: inputMessage: from giop:tcp:172.27.169.2:3215 155 bytes
omniORB: Receive codeset service context and set TCS to (ISO-8859-1,UTF-16)
omniORB: sendChunk: to giop:tcp:172.27.169.2:3215 24 bytes
omniORB: inputMessage: from giop:tcp:172.27.169.2:3057 24 bytes
omniORB: Deactivating POAManager for POA(Automatically_Named_POA_1).
omniORB: State root/Automatically_Named_POA_1<0> (active) -> deactivating (OA destruction)
omniORB: State root/Automatically_Named_POA_1<0> (deactivating OA) -> etherealising
omniORB: Removing root/Automatically_Named_POA_1<0> (etherealising) from object table
omniORB: Met detached object. 0 remaining.
omniORB: Wait for 0 detached objects.
omniORB: Destroying POA(Automatically_Named_POA_1).
omniORB: Deactivating all POA(Automatically_Named_POA_1)'s objects.
omniORB: Waiting for requests to complete on POA(Automatically_Named_POA_1).
omniORB: Requests on POA(Automatically_Named_POA_1) completed.
omniORB: Etherealising POA(Automatically_Named_POA_1)'s objects.
omniORB: Wait for 0 detached objects.
omniORB: Destruction of POA(Automatically_Named_POA_1) complete.
omniORB: Preparing to shutdown ORB.
omniORB: Destroying POA(RootPOA).
omniORB: Deactivating all POA(RootPOA)'s objects.
omniORB: Waiting for requests to complete on POA(RootPOA).
omniORB: Requests on POA(RootPOA) completed.
omniORB: Etherealising POA(RootPOA)'s objects.
omniORB: Wait for 0 detached objects.
omniORB: All object adapters inactive. Stopping serving incoming endpoints.
omniORB: giopServer deactivate...
omniORB: Close connections with threads and monitors...
omniORB: sendCloseConnection: to giop:tcp:172.27.169.2:3215 12 bytes
omniORB: Closed 1 connection out of 1.
omniORB: Wait for 1 dedicated thread to finish...
omniORB: Timed out. 1 connection and 1 dedicated worker remaining.
omniORB: Terminate rendezvousers...
omniORB: giopRendezvouser for giop:tcp:trappelm.ksengineers.at:3212 terminate...
omniORB: Error in network receive (start of message): giop:tcp:172.27.169.2:3215
omniORB: throw giopStream::CommFailure from giopStream.cc:874(0,NO,COMM_FAILURE_UnMarshalArguments)
omniORB: Server connection refcount = 1
omniORB: removeConnectionAndWorker for dedicated worker. 0 remaining.
omniORB: giopServer terminate connection from giop:tcp:172.27.169.2:3216.
omniORB: giopRendezvouser for endpoint giop:tcp:trappelm.ksengineers.at:3212 exit.
omniORB: No remaining rendezvousers.
omniORB: Rendezvousers terminated.
omniORB: giopServer deactivated.
omniORB: Destruction of POA(RootPOA) complete.
omniORB: Shutting-down all incoming endpoints.
omniORB: TCP endpoint shut down.
omniORB: Disable ObjRef(IDL:Dummy:1.0) root/Automatically_Named_POA_1<0>
omniORB: Object table entry root/Automatically_Named_POA_1<0> (dead) deleted.
omniORB: Disable ObjRef(IDL:DummyServer:1.0) root<0>
omniORB: omniRemoteIdentity deleted.
omniORB: Disable ObjRef() key<NameService>
omniORB: omniRemoteIdentity deleted.
omniORB: 3 object references present at ORB shutdown.
omniORB: ORB shutdown is complete.
omniORB: ObjRef() -- deleted.
omniORB: Deinitialising omniDynamic library.
omniORB: Release registered value factories.
omniORB: Terminate strand scavenger.
omniORB: Close remaining strands.
omniORB: Shutdown close connection to giop:tcp:trappelm.ksengineers.at:3057
omniORB: sendCloseConnection: to giop:tcp:172.27.169.2:3057 12 bytes
omniORB: Client connection refcount (forced) = 0
omniORB: Client close connection to giop:tcp:172.27.169.2:3057
omniORB: Shutdown close connection to giop:tcp:trappelm.ksengineers.at:2809
omniORB: Client connection refcount (forced) = 0
omniORB: Client close connection to giop:tcp:172.27.169.2:2809
omniORB: 0 remaining bidir ropes deleted.
omniORB: 2 remaining ropes deleted.
omniORB: Clear endPoint options.
omniORB: Wait for 3 invoker threads to finish.
omniORB: AsyncInvoker: thread id = 2 has exited. Total threads = 3
omniORB: AsyncInvoker: thread id = 3 has exited. Total threads = 2
omniORB: Server connection refcount = 0
omniORB: Server close connection from giop:tcp:172.27.169.2:3215
omniORB: AsyncInvoker: thread id = 4 has exited. Total threads = 1
omniORB: Invoker threads finished.
omniORB: AsyncInvoker: deleted.
omniORB: ObjRef(IDL:Dummy:1.0) -- deleted.
omniORB: ObjRef(IDL:DummyServer:1.0) -- deleted.
omniORB: No more references to the ORB -- deleted.
omniORB: Version: 4.1.2
omniORB: Distribution date: Thu Feb 14 14:19:08 GMT 2008 dgrisby
omniORB: Maximum supported GIOP version is 1.2
omniORB: Native char code sets: UTF-8 ISO-8859-1.
omniORB: Transmission char code sets: UTF-8(1.2) UTF-8(1.1) ISO-8859-1(1.2) ISO-8859-1(1.1) ISO-8859-1(1.0).
omniORB: Native wide char code sets: UTF-16.
omniORB: Transmission wide char code sets: UTF-16(1.2).
omniORB: Initialising omniDynamic library.
omniORB: Current configuration is as follows:
omniORB: DefaultInitRef (file) =
omniORB: DefaultInitRef (args) =
omniORB: InitRef = NameService=corbaname::trappelm.ksengineers.at
omniORB: abortOnInternalError = 0
omniORB: abortOnNativeException = 0
omniORB: acceptBiDirectionalGIOP = 0
omniORB: acceptMisalignedTcIndirections = 0
omniORB: bootstrapAgentHostname =
omniORB: bootstrapAgentPort = 900
omniORB: clientCallTimeOutPeriod = 0
omniORB: clientConnectTimeOutPeriod = 0
omniORB: clientTransportRule = * unix,ssl,tcp
omniORB: configFile = [none]
omniORB: connectionWatchImmediate = 0
omniORB: connectionWatchPeriod = 50000
omniORB: copyValuesInLocalCalls = 1
omniORB: diiThrowsSysExceptions = 0
omniORB: dumpConfiguration = 0
omniORB: endPoint = giop:tcp:trappelm.ksengineers.at:
omniORB: endPointPublish = addr
omniORB: giopMaxMsgSize = 2097152
omniORB: giopTargetAddressMode = KeyAddr
omniORB: id = omniORB4
omniORB: idleThreadTimeout = 10
omniORB: immediateAddressSwitch = 0
omniORB: inConScanPeriod = 180
omniORB: lcdMode = 0
omniORB: maxGIOPConnectionPerServer = 5
omniORB: maxGIOPVersion = 1.2
omniORB: maxInterleavedCallsPerConnection = 5
omniORB: maxServerThreadPerConnection = 100
omniORB: maxServerThreadPoolSize = 100
omniORB: maxSocketRecv = 131072
omniORB: maxSocketSend = 131072
omniORB: nativeCharCodeSet = ISO-8859-1
omniORB: nativeWCharCodeSet = UTF-16
omniORB: objectTableSize = 0
omniORB: offerBiDirectionalGIOP = 0
omniORB: oneCallPerConnection = 1
omniORB: outConScanPeriod = 120
omniORB: poaHoldRequestTimeout = 0
omniORB: poaUniquePersistentSystemIds = 1
omniORB: principal = [Null]
omniORB: resetTimeOutOnRetries = 0
omniORB: scanGranularity = 5
omniORB: serverCallTimeOutPeriod = 0
omniORB: serverTransportRule = * unix,ssl,tcp
omniORB: socketSendBuffer = 16384
omniORB: strictIIOP = 1
omniORB: supportBootstrapAgent = 0
omniORB: supportCurrent = 1
omniORB: supportPerThreadTimeOut = 0
omniORB: tcAliasExpand = 0
omniORB: threadPerConnectionLowerLimit = 9000
omniORB: threadPerConnectionPolicy = 1
omniORB: threadPerConnectionUpperLimit = 10000
omniORB: threadPoolWatchConnection = 1
omniORB: traceExceptions = 1
omniORB: traceFile = omniORB412_log_test_dummy_cli.txt
omniORB: traceInvocationReturns = 0
omniORB: traceInvocations = 0
omniORB: traceLevel = 25
omniORB: traceThreadId = 0
omniORB: traceTime = 0
omniORB: unixTransportDirectory = /tmp/omni-%u
omniORB: unixTransportPermission = 777
omniORB: useTypeCodeIndirections = 1
omniORB: verifyObjectExistsAndType = 1
omniORB: Initialising incoming endpoints.
omniORB: Instantiate endpoint 'giop:tcp:trappelm.ksengineers.at:'
omniORB: Assertion failed. This indicates a bug in the application
using omniORB, or maybe in omniORB itself.
file: giopServer.cc
line: 284
info: pd_state != ZOMBIE
omniORB: Released 2 stub TypeCodes from 'boxesDynSK.cc'.
omniORB: Unregister value factory for 'IDL:omg.org/CORBA/WStringValue:1.0'.
omniORB: Unregister value factory for 'IDL:omg.org/CORBA/StringValue:1.0'.
omniORB: Released 21 stub TypeCodes from 'NamingDynSK.cc'.
omniORB: Released 75 stub TypeCodes from 'irDynSK.cc'.
omniORB: Released 46 stub TypeCodes from 'corbaidlDynSK.cc'.
omniORB: Released 4 stub TypeCodes from 'bootstrapDynSK.cc'.
omniORB: Released 7 stub TypeCodes from 'poa_enumsDynSK.cc'.
omniORB: Released 1 stub TypeCode from 'unknownUserExn.cc'.
omniORB: Released 2 stub TypeCodes from 'policy.cc'.
omniORB: Released 60 stub TypeCodes from 'dynException.cc'.
omniORB: ORB not destroyed; no final clean-up.
Martin Trappel
2008-08-08 15:03:44 UTC
Permalink
Post by Martin Trappel
Post by Martin Trappel
[omniORB 4.1.2, Windows XP, VC8 prebuilt DLLs]
I have the situation where I repeatedly initialize and destroy the ORB
in a client application, (...)
I have now been able to reproduce the problem in a simpler context.
interface Dummy {
void echo(in string s);
};
interface DummyServer {
void doEcho(in Dummy d);
};
DummyServer is called by the process running a Dummy instance and the
doEcho method just calls echo on the given interface pointer.
A.) Start a process with an instance of DummyServer (and register the
object with the naming service). The server will just keep running and
do nothing else. (just as with the echo example)
1.) ORB_init
2.) start a separate omni_thread and call orb->run() there
3.) create a new poa from the root poa with the same policies as the
root poa
4.) Create a new Dummy impl object
5.) activate the object with the created POA
6.) activate the POA
7.) Resolve the DummyServer Object via NamingService
8.) Call dummy_srv_ref->doEcho(my_dummy->_this())
9.) deactivate POA + destroy POA
10.) shutdown(true) orb (afterwards wait for run thread to terminate)
11.) destroy() orb
With this setup I will reliably hit
file: giopServer.cc
line: 284
mesg: pd_state != ZOMBIE
after just a few iterations (sometimes even on the first one).
(...)
As a crosscheck I have tried to orb->run() in the same thread as
ORB_init() was called (and launch the server call in a separate thread
instead) -- error still there.

During this I found that if I introduce a slight delay (Sleep_100ms)
before orb->shutdown(true) (step 10) the error is no longer reproducible.

I'll keep you posted if I find something more ...

br,
Martin
Duncan Grisby
2008-08-08 19:04:07 UTC
Permalink
Post by Martin Trappel
I have now been able to reproduce the problem in a simpler context.
[...]
Post by Martin Trappel
( If someone wants to reproduce this I can clean out my testsources
and also post them. )
Please send me the test. That will make it much easier for me to look
into it.

Duncan.
--
-- Duncan Grisby --
-- ***@grisby.org --
-- http://www.grisby.org --
Martin Trappel
2008-08-11 12:28:53 UTC
Permalink
Post by Duncan Grisby
Post by Martin Trappel
I have now been able to reproduce the problem in a simpler context.
[...]
Post by Martin Trappel
( If someone wants to reproduce this I can clean out my testsources
and also post them. )
Please send me the test. That will make it much easier for me to look
into it.
Duncan.
Since last week I have been fiddling around to make a test-project as
simple as possible.
I tried to reproduce the failure without doing a callback, but instead
just having a fixed client-server relationship. With this I can also
reproduce the behavior, but not as reliable as with the callback. (Or,
maybe, it's to do with something else as I just reproduced it 2 times
with the attached test-project.)

Anyway, find attached:
* Test Setup with one Client and one Server, where the client calls a
Disconnect method in the server upon which the server does a
shutdown(false) and after the ORB has shutdown the orb is immediately
restarted. The client will the do a Disconnect again after 10sec.
* It's as a VisualStudio 2005 Project, but I have not used any windows
specifics, so it should compile easy enough anywhere else. (I have
attached the settings files I use if you want to cross-check.)
* I have also attached a test session I just did this morning, where I
hit the fatalException 2 times with the sourcecode exactly as attached.
* Also, find the current omniORB registry settings attached

As for this test-project, I circumvent the nameserver by using
omniINSPOA and the -ORBendPoint option. Both the server and the client
really do not do alot, just one call from the client and then server
shutdown. (As for the logging - there is the now_str() function that
prints timestamps as the omniORB logging does.)

I hope this is of use to you.

cheers,
Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: omniORB_reinit.zip
Type: application/octet-stream
Size: 13383 bytes
Desc: not available
Url : http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20080811/27434c40/omniORB_reinit-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: vss_settings.zip
Type: application/octet-stream
Size: 1507 bytes
Desc: not available
Url : http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20080811/27434c40/vss_settings-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: omniORB test-session 11-08-2008.zip
Type: application/octet-stream
Size: 8219 bytes
Desc: not available
Url : http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20080811/27434c40/omniORBtest-session11-08-2008-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: omniORB (4.1.2 on 11-08-2008).reg.zip
Type: application/octet-stream
Size: 1059 bytes
Desc: not available
Url : http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20080811/27434c40/omniORB4.1.2on11-08-2008.reg-0001.obj
Loading...