[omniORB] ORB broken, stop listening on endpoint port

[VEYSSIERE Nicolas]

Hello,
here is the init trace log of my software using omniORB (without a
naming service) :

omniORB: (0) 2010-08-31 13:59:38.992559: Initialising incoming
endpoints.
omniORB: (0) 2010-08-31 13:59:38.992598: Instantiate endpoint
'giop:tcp::1234'
omniORB: (0) 2010-08-31 13:59:38.992735: Bind to address :: port 1234.
omniORB: (0) 2010-08-31 13:59:38.992867: Publish specification: 'addr'
omniORB: (0) 2010-08-31 13:59:38.992905: Try to publish 'addr' for
endpoint giop:tcp:192.9.10.213:1234
omniORB: (0) 2010-08-31 13:59:38.992929: Publish endpoint
'giop:tcp:192.9.10.213:1234'
omniORB: (0) 2010-08-31 13:59:38.993058: Starting serving incoming
endpoints.
omniORB: (0) 2010-08-31 13:59:38.994207: Adding key<ciorAdmin>
(activating) to object table.
omniORB: (0) 2010-08-31 13:59:38.994241: State key<ciorAdmin>
(activating) -> active
omniORB: (0) 2010-08-31 13:59:38.994400: Creating ref to local:
key<ciorAdmin>
target id : IDL:CLA/CIORAdmin:1.0
most derived id: IDL:CLA/CIORAdmin:1.0
omniORB: (1) 2010-08-31 13:59:38.996625: AsyncInvoker: thread id =
1 has started. Total threads = 1
omniORB: (1) 2010-08-31 13:59:38.996647: giopRendezvouser task
execute for giop:tcp:192.9.10.213:1234
omniORB: (1) 2010-08-31 13:59:39.430031: SocketCollection idle.
Sleeping.


The ORB is waiting on port '1234', with a client sometimes I have the
following error :


omniORB: (0) 2010-09-02 12:42:31.140468: Invoke
'notificationImagerie' on remote:
key<............................RootPOA..............>
omniORB: (0) 2010-09-02 12:42:31.140509: sendChunk: to
giop:tcp:191.0.248.53:2501 573 bytes
omniORB: (0) 2010-09-02 12:42:31.140528: 128 bytes out of 573
4749 4f50 0102 0000 0000 0231 0000 189e GIOP.......1....
0000 0000 0000 0008 0000 0031 afab cb00 ...........1....
0000 0020 d1d4 a95f 0000 0001 0000 0000 ... ..._........
0000 0001 0000 0008 526f 6f74 504f 4100 ........RootPOA.
0000 0008 0000 0001 0000 0000 1400 0002 ................
0000 0015 6e6f 7469 6669 6361 7469 6f6e ....notification
496d 6167 6572 6965 0000 000a 0000 0000 Imagerie........
0000 0010 0000 0015 0000 0118 0000 0000 ................
omniORB: (0) 2010-09-02 12:42:31.140594: Error in network send:
giop:tcp:191.0.248.53:2501
omniORB: (0) 2010-09-02 12:42:31.140611: throw
giopStream::CommFailure from
giopStream.cc:1186(1,NO,COMM_FAILURE_MarshalArguments)
omniORB: (0) 2010-09-02 12:42:31.140833: Client connection refcount = 0
omniORB: (0) 2010-09-02 12:42:31.140851: Client close connection to
giop:tcp:191.0.248.53:2501
omniORB: (0) 2010-09-02 12:42:31.140925: Invoke
'notificationImagerie' on remote:
key<............................RootPOA..............>
omniORB: (0) 2010-09-02 12:42:31.140960: Send codeset service
context: (ISO-8859-1,UTF-16)
omniORB: (0) 2010-09-02 12:42:31.141000: Client attempt to connect
to giop:tcp:191.0.248.53:2501
omniORB: (0) 2010-09-02 12:42:31.141597: Failed to connect (no peer
name): 191.0.248.53
omniORB: (0) 2010-09-02 12:42:31.141641: Switch rope to use address
giop:tcp:191.0.248.53:2501
omniORB: (0) 2010-09-02 12:42:31.141657: Unable to open new
connection: giop:tcp:191.0.248.53:2501
omniORB: (0) 2010-09-02 12:42:31.141673: throw
giopStream::CommFailure from
giopStream.cc:1152(0,NO,TRANSIENT_ConnectFailed)
omniORB: (0) 2010-09-02 12:42:31.141794: throw TRANSIENT from
omniObjRef.cc:813 (NO,TRANSIENT_ConnectFailed)
omniORB: (0) 2010-09-02 12:42:31.142267: omniRemoteIdentity deleted.
omniORB: (0) 2010-09-02 12:42:31.142289: ObjRef(IDL:CLA/Client:1.0)
-- deleted.


A network error happened between the server and the client (in my point
of view).

But there is a problem : the ORB stop listening on port '1234', I and
don't know why, I can't reconnect to this server.


Can some one helps me to find what happens here ?


Thanks.

Regards,

[Bruce Visscher]

Nicolas,

What version of omniORB and what platform is the server running on?

I have had problems in the past on an omniORB server on the OpenVMS platform
where the server will quit accepting incoming connections. The problem was
in src/lib/omniORB/orbcore/SocketCollection.cc. I found that either poll or
select (you can configure the platform to use either) was indicating an
error (negative value returned) but the errno value was other than EBADF.
In the case of OpenVMS, errno was EVMSERR which indicates a platform
specific error condition that is indicated in vaxc$errno. It turns out this
value was 316 which is "%SYSTEM-F-IVCHAN, invalid I/O channel". I patched
SocketCollection.cc to treat this condition the same as EBADF and after that
I found that the server would recover from this.

I will say that prior to modifying this source file that the server was a
little too quiet about what was going wrong here. I don't remember what
trace level was required to see what was going wrong here but I think it
should display *something* in this case unless traceLevel indicates that
output from the server is strictly forbidden. In that case it might be
better if the server would just crash. A server that isn't listening isn't
good for very much, IMHO.

Bruce

Post by VEYSSIERE Nicolas
Hello,
here is the init trace log of my software using omniORB (without a naming
omniORB: (0) 2010-08-31 13:59:38.992559: Initialising incoming
endpoints.
omniORB: (0) 2010-08-31 13:59:38.992598: Instantiate endpoint
'giop:tcp::1234'
omniORB: (0) 2010-08-31 13:59:38.992735: Bind to address :: port 1234.
omniORB: (0) 2010-08-31 13:59:38.992867: Publish specification: 'addr'
omniORB: (0) 2010-08-31 13:59:38.992905: Try to publish 'addr' for
endpoint giop:tcp:192.9.10.213:1234
192.9.10.213:1234'
omniORB: (0) 2010-08-31 13:59:38.993058: Starting serving incoming
endpoints.
omniORB: (0) 2010-08-31 13:59:38.994207: Adding key<ciorAdmin>
(activating) to object table.
omniORB: (0) 2010-08-31 13:59:38.994241: State key<ciorAdmin>
(activating) -> active
key<ciorAdmin>
target id : IDL:CLA/CIORAdmin:1.0
most derived id: IDL:CLA/CIORAdmin:1.0
omniORB: (1) 2010-08-31 13:59:38.996625: AsyncInvoker: thread id = 1 has
started. Total threads = 1
omniORB: (1) 2010-08-31 13:59:38.996647: giopRendezvouser task execute
for giop:tcp:192.9.10.213:1234
omniORB: (1) 2010-08-31 13:59:39.430031: SocketCollection idle.
Sleeping.
The ORB is waiting on port '1234', with a client sometimes I have the
omniORB: (0) 2010-09-02 12:42:31.140468: Invoke 'notificationImagerie'
on remote: key<............................RootPOA..............>
191.0.248.53:2501 573 bytes
omniORB: (0) 2010-09-02 12:42:31.140528: 128 bytes out of 573
4749 4f50 0102 0000 0000 0231 0000 189e GIOP.......1....
0000 0000 0000 0008 0000 0031 afab cb00 ...........1....
0000 0020 d1d4 a95f 0000 0001 0000 0000 ... ..._........
0000 0001 0000 0008 526f 6f74 504f 4100 ........RootPOA.
0000 0008 0000 0001 0000 0000 1400 0002 ................
0000 0015 6e6f 7469 6669 6361 7469 6f6e ....notification
496d 6167 6572 6965 0000 000a 0000 0000 Imagerie........
0000 0010 0000 0015 0000 0118 0000 0000 ................
giop:tcp:191.0.248.53:2501
omniORB: (0) 2010-09-02 12:42:31.140611: throw giopStream::CommFailure
from giopStream.cc:1186(1,NO,COMM_FAILURE_MarshalArguments)
omniORB: (0) 2010-09-02 12:42:31.140833: Client connection refcount = 0
omniORB: (0) 2010-09-02 12:42:31.140851: Client close connection to
giop:tcp:191.0.248.53:2501
omniORB: (0) 2010-09-02 12:42:31.140925: Invoke 'notificationImagerie'
on remote: key<............................RootPOA..............>
(ISO-8859-1,UTF-16)
omniORB: (0) 2010-09-02 12:42:31.141000: Client attempt to connect to
giop:tcp:191.0.248.53:2501
omniORB: (0) 2010-09-02 12:42:31.141597: Failed to connect (no peer
name): 191.0.248.53
omniORB: (0) 2010-09-02 12:42:31.141641: Switch rope to use address
giop:tcp:191.0.248.53:2501
giop:tcp:191.0.248.53:2501
omniORB: (0) 2010-09-02 12:42:31.141673: throw giopStream::CommFailure
from giopStream.cc:1152(0,NO,TRANSIENT_ConnectFailed)
omniORB: (0) 2010-09-02 12:42:31.141794: throw TRANSIENT from
omniObjRef.cc:813 (NO,TRANSIENT_ConnectFailed)
omniORB: (0) 2010-09-02 12:42:31.142267: omniRemoteIdentity deleted.
omniORB: (0) 2010-09-02 12:42:31.142289: ObjRef(IDL:CLA/Client:1.0) --
deleted.
A network error happened between the server and the client (in my point of
view).
But there is a problem : the ORB stop listening on port '1234', I and don't
know why, I can't reconnect to this server.
Can some one helps me to find what happens here ?
Thanks.
Regards,
--
Nicolas V.
_______________________________________________
omniORB-list mailing list
http://www.omniorb-support.com/mailman/listinfo/omniorb-list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20100902/2cdfa3e4/attachment.htm

[VEYSSIERE Nicolas]

Skipped content of type multipart/alternative-------------- next part --------------
A non-text attachment was scrubbed...
Name: nicolas_veyssiere.vcf
Type: text/x-vcard
Size: 381 bytes
Desc: not available
Url : http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20100902/60a40414/nicolas_veyssiere-0001.vcf

[Duncan Grisby]

Post by VEYSSIERE Nicolas
here is the init trace log of my software using omniORB (without a

[...]

Post by VEYSSIERE Nicolas
omniORB: (0) 2010-08-31 13:59:38.992735: Bind to address :: port 1234.

[...]

Post by VEYSSIERE Nicolas
omniORB: (0) 2010-08-31 13:59:38.992929: Publish endpoint
'giop:tcp:192.9.10.213:1234'

[...]

Post by VEYSSIERE Nicolas
The ORB is waiting on port '1234', with a client sometimes I have the
omniORB: (0) 2010-09-02 12:42:31.140468: Invoke
key<............................RootPOA..............>
omniORB: (0) 2010-09-02 12:42:31.140509: sendChunk: to
giop:tcp:191.0.248.53:2501 573 bytes

[...]

Post by VEYSSIERE Nicolas
giop:tcp:191.0.248.53:2501

That's a completely different address. Whichever server it is that has
the failure, it is not the one the first trace is from.

Do you have a log from the server the client is trying to contact?

Cheers,

Duncan.

[VEYSSIERE Nicolas]

Skipped content of type multipart/alternative-------------- next part --------------
A non-text attachment was scrubbed...
Name: nicolas_veyssiere.vcf
Type: text/x-vcard
Size: 392 bytes
Desc: not available
Url : http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20100906/6ad82df8/nicolas_veyssiere.vcf

[jianping jia]

I also met the similar issue once. For some critical errors, omni org
directly shutdown the listening port.
Best Regards
Jia Jian Ping

Post by VEYSSIERE Nicolas
here is the init trace log of my software using omniORB (without a
[...]
omniORB: (0) 2010-08-31 13:59:38.992735: Bind to address :: port 1234.
[...]
omniORB: (0) 2010-08-31 13:59:38.992929: Publish endpoint
'giop:tcp:192.9.10.213:1234'
[...]
The ORB is waiting on port '1234', with a client sometimes I have the
omniORB: (0) 2010-09-02 12:42:31.140468: Invoke
key<............................RootPOA..............>
omniORB: (0) 2010-09-02 12:42:31.140509: sendChunk: to
giop:tcp:191.0.248.53:2501 573 bytes
[...]
giop:tcp:191.0.248.53:2501
That's a completely different address. Whichever server it is that has
the failure, it is not the one the first trace is from.
Do you have a log from the server the client is trying to contact?
Cheers,
Duncan.
A can asure you that this log is not modified, the first block tels us,
that the server is running on IP = 192.9.10.213
and the second that the client is on IP = 191.0.248.53, there is a router
between the 2 networks.
In fact in my first post I forgot to post a log that is quite important and
is exactly what Bruce tels me earlier, the poll() method
returns with a errno = 11, wich means EAGAIN on HP-UX, here is the man from
EAGAIN : Allocation of internal data structures failed due to a temporary
condition. Calling open()/poll() again might succeed.
--
Nicolas VEYSSIERE
Ing?nieur recherche et d?veloppement logiciel
Thales Security Solutions & Services S.A.S
[phone]
_______________________________________________
omniORB-list mailing list
http://www.omniorb-support.com/mailman/listinfo/omniorb-list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20100906/279f12ad/attachment.htm

[Duncan Grisby]

Post by VEYSSIERE Nicolas
A can asure you that this log is not modified, the first block tels
us, that the server is running on IP = 192.9.10.213
and the second that the client is on IP = 191.0.248.53, there is a
router between the 2 networks.

I think that's a terminology issue. What you are calling the client is
acting in the role of a server (presumably due to callbacks).

Post by VEYSSIERE Nicolas
In fact in my first post I forgot to post a log that is quite
important and is exactly what Bruce tels me earlier, the poll() method
returns with a errno = 11, wich means EAGAIN on HP-UX, here is the man

OK, so that's the real issue. Does the problem go away if you remove the
"return 0" from just after the log message about "Error return from
poll()"?

Cheers,

Duncan.

[VEYSSIERE Nicolas]

Skipped content of type multipart/alternative-------------- next part --------------
A non-text attachment was scrubbed...
Name: nicolas_veyssiere.vcf
Type: text/x-vcard
Size: 392 bytes
Desc: not available
Url : http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20100913/99889ee6/nicolas_veyssiere.vcf