[omniORB] OmniORB 4.1.4 assertion failure

Discussion:

Richard Pitts

2010-03-22 14:22:26 UTC

Hi

I've been having problems upgrading one of our server applications to
OmniORB 4.1.4 from OmniORB 3.0.4.

I have a single threaded client testing application using OmniORB 3.0.4
compiled on VC++ 6 and a multi-threaded server
application using OmniORB 4.1.4 built on Visual Studio 2008. The client
repeatedly calls two methods on the server sending
and receiving an XML document as strings. Communication is perfect until
a seemingly random assertion failure on the
server in OmniORB after processing a random number of requests. The
server then processes one last request before ceasing
communication.

The server is logging this:

[<--- Request n]
[<--- Request n + 1]
omniORB: Assertion failed. This indicates a bug in the application
using omniORB, or maybe in omniORB itself.
file: SocketCollection.cc
line: 955
info: s->pd_fd_index >= 0
omniORB: Unexpected exception caught by giopRendezvouser
omniORB: Unrecoverable error for this endpoint:
giop:tcp:10.26.60.62:2503, it will no longer be serviced.
[<--- Request n + 2]
[<--- No more requests]

Searching around the Internet indicates no one has encountered this
before which suggests the issue is with
one of our applications.

I've tried a few things

- Build server with OmniORB 4.1.1, this exhibits the same issue.
- Build server with OmniORB 3.0.4, fails to compile
- Build client with OmniORB 4.1.1 VC++ 6 build, this fails to compile.
- Modified client and server behavior to send and receive empty strings,
efficiently doing nothing but exchange empty strings, the issue is still
present.
- Run the client application against an old version of the server both
OmniORB 3.0.4, no issues.
- Turned the tracing level up on both server and client, nothing looks
obviously wrong.

My initial guess is incompatible between different versions of OmniORB,
then again I'm unsure because it appears to work fine
until a random event. Ideally I'd like to upgrade the client to OmniORB
4.1.4 using Visual Studio 2008 but I can't at the moment.
If the problem persists then this issue should hopefully forces us to do
so sooner.

This is a bit of a show stopping because its preventing us from testing
the server under any realistic high load situations.

If any more information is needed, logs or ORB arguments, please let me
know.

Thanks

Richard P

</PRE>

********************************************************************** 
Please consider the environment - do you really need to print this email? 
 
This email is intended only for the person(s) named above and may contain private and confidential information. If it has come to you in error, please destroy and permanently delete any copy in your possession and contact us on +44 (0) 161 480 4420. The information in this email is copyright © CDL Group Holdings Limited. We cannot accept any liability for any loss or damage sustained as a result of software viruses. It is your responsibility to carry out such virus checking as is necessary before opening any attachment. 
Cheshire Datasystems Limited uses software which automatically screens incoming emails for inappropriate content and attachments. If the software identifies such content or attachment, the email will be forwarded to our Technology Department for checking. You should be aware that any email which you send to Cheshire Datasystems Limited is subject to this procedure. 
Cheshire Datasystems Limited, Strata House, Kings Reach Road, Stockport SK4 2HD 
Registered in England and Wales with Company Number 3991057 
VAT registration: 727 1188 33
<PRE>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20100322/75e5714f/attachment.htm

Duncan Grisby

2010-03-22 23:38:25 UTC

Permalink

On Mon, 2010-03-22 at 09:21 +0000, Richard Pitts wrote:

[...]

Post by Richard Pitts
[<--- Request n]
[<--- Request n + 1]
omniORB: Assertion failed. This indicates a bug in the application
using omniORB, or maybe in omniORB itself.
file: SocketCollection.cc
line: 955
info: s->pd_fd_index >= 0
omniORB: Unexpected exception caught by giopRendezvouser
giop:tcp:10.26.60.62:2503, it will no longer be serviced.
[<--- Request n + 2]
[<--- No more requests]

[...]

Post by Richard Pitts
My initial guess is incompatible between different versions of
OmniORB, then again I'm unsure because it appears to work fine
until a random event. Ideally I'd like to upgrade the client to
OmniORB 4.1.4 using Visual Studio 2008 but I can't at the moment.
If the problem persists then this issue should hopefully forces us to
do so sooner.

Richard Pitts

2010-03-24 19:44:29 UTC

Permalink

Hi

The attached patch worked fine which is good news. Thanks for the help.

Kind Regards

Richard P

-----Original Message-----
From: Duncan Grisby [mailto:***@grisby.org]
Sent: 22 March 2010 18:38
To: Richard Pitts
Cc: omniorb-***@omniorb-support.com
Subject: Re: [omniORB] OmniORB 4.1.4 assertion failure

On Mon, 2010-03-22 at 09:21 +0000, Richard Pitts wrote:

[...]

[...]

Post by Richard Pitts
My initial guess is incompatible between different versions of
OmniORB, then again I'm unsure because it appears to work fine until a
random event. Ideally I'd like to upgrade the client to OmniORB 4.1.4
using Visual Studio 2008 but I can't at the moment.
If the problem persists then this issue should hopefully forces us to
do so sooner.

It's definitely not an incompatibility between omniORB versions. It
should not be possible for a client to crash a server no matter what.

Are you running in thread per connection mode or thread pool mode?
Having analysed the code, I think it is conceivable that there is a race
condition in thread pool mode if the scheduler behaves in an unlikely
way. If you are running in thread pool mode, can you try thread per
connection mode to see if the problem goes away?

Regardless, I would expect the attached patch to fix it. Can you try
that?

Cheers,

Duncan.
--
-- Duncan Grisby --
-- ***@grisby.org --
-- http://www.grisby.org/ --
**********************************************************************
Please consider the environment - do you really need to print this email?

This email is intended only for the person(s) named above and may contain private and confidential information. If it has come to you in error, please destroy and permanently delete any copy in your possession and contact us on +44 (0) 161 480 4420. The information in this email is copyright ? CDL Group Holdings Limited. We cannot accept any liability for any loss or damage sustained as a result of software viruses. It is your responsibility to carry out such virus checking as is necessary before opening any attachment.

Cheshire Datasystems Limited uses software which automatically screens incoming emails for inappropriate content and attachments. If the software identifies such content or attachment, the email will be forwarded to our Technology Department for checking. You should be aware that any email which you send to Cheshire Datasystems Limited is subject to this procedure.

Cheshire Datasystems Limited, Strata House, Kings Reach Road, Stockport SK4 2HD
Registered in England and Wales with Company Number 3991057
VAT registration: 727 1188 33