[omniORB] in MAN_THREAD_MODEL omniORB releases references from other thread

[Michael Kilburn]

Hi,

I have recently discovered that MAN_THREAD_MODEL does not guarantee that all
calls to servant are made on the same thread. In particular:
- in a typical destroy() method implementation disconnecting servant from
CORBA layer does not cause it to be destroyed immediately (which is ok), but
instead it is delayed until some point in future
- and at that point in future release (which usually the last one, and
triggers "delete this") is called from different thread.

It sound almost like a violation of guarantees this thread model is supposed
to give (it was designed to for old non-threaded code after all). What was
the reason for this behavior and can it be "fixed"? (yes, it caused issues
in our product in conjunction with other non-obvious things in other places
of our code, I've spent a lot of time trying to figure out why on rare
occasions our product started eating tons of memory :-D)

[Michael Kilburn]

Any help? At least advice what to do in such situation...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20090111/b15a8537/attachment.htm

[Duncan Grisby]

Post by Michael Kilburn
I have recently discovered that MAN_THREAD_MODEL does not guarantee that all
- in a typical destroy() method implementation disconnecting servant from
CORBA layer does not cause it to be destroyed immediately (which is ok), but
instead it is delayed until some point in future

I assume you mean calling poa->deactivate_object() ?

Post by Michael Kilburn
- and at that point in future release (which usually the last one, and
triggers "delete this") is called from different thread.

Well, servant->remove_ref() is called from the thread that notices that
the last invocation has completed.

Post by Michael Kilburn
It sound almost like a violation of guarantees this thread model is supposed
to give (it was designed to for old non-threaded code after all). What was the
reason for this behavior and can it be "fixed"? (yes, it caused issues in our
product in conjunction with other non-obvious things in other places of our
code, I've spent a lot of time trying to figure out why on rare occasions our
product started eating tons of memory :-D)

The CORBA spec doesn't say a huge amount about the main thread model.
All it says is that "upcalls" are handled by the "main" thread, but it
doesn't specify what constitutes an upcall. omniORB currently considers
only IDL-defined methods to be upcalls.

Now you mention it, though, it does seem logical that the call to
_remove_ref() is also done in on the main thread. I've checked in a
change to CVS that implements that.

If you can't update to the CVS version, you can force use of the main
thread by registering a ServantActivator that is itself activated in a
POA with the main thread policy. That way, servants will be released
using a call the etherealize() on the ServantActivator, which will
happen on the main thread.

Cheers,

Duncan.

[Michael Kilburn]

Post by Duncan Grisby
I assume you mean calling poa->deactivate_object() ?

Yes.

Post by Duncan Grisby
The CORBA spec doesn't say a huge amount about the main thread model.
All it says is that "upcalls" are handled by the "main" thread, but it
doesn't specify what constitutes an upcall.

I noticed it -- that is why I used "almost violation".

Post by Duncan Grisby
you can force use of the main
thread by registering a ServantActivator that is itself activated in a
POA with the main thread policy. That way, servants will be released
using a call the etherealize() on the ServantActivator, which will
happen on the main thread.

Thanks, Duncan!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20090113/b760d1d7/attachment.htm

[Michael Kilburn]

I am terribly sorry to reanimate this thread...

Post by Duncan Grisby

Post by Michael Kilburn
I have recently discovered that MAN_THREAD_MODEL does not guarantee that

all

Post by Michael Kilburn
- in a typical destroy() method implementation disconnecting servant from
CORBA layer does not cause it to be destroyed immediately (which is ok),

but

Post by Michael Kilburn
instead it is delayed until some point in future

I assume you mean calling poa->deactivate_object() ?

Post by Michael Kilburn
- and at that point in future release (which usually the last one, and
triggers "delete this") is called from different thread.

[skip]

Post by Duncan Grisby
Now you mention it, though, it does seem logical that the call to
_remove_ref() is also done in on the main thread. I've checked in a
change to CVS that implements that.

Do you remember in which version of omniORB this problem was fixed?

Post by Duncan Grisby
If you can't update to the CVS version, you can force use of the main
thread by registering a ServantActivator that is itself activated in a
POA with the main thread policy. That way, servants will be released
using a call the etherealize() on the ServantActivator, which will
happen on the main thread.

We've just discovered a small problem with this approach -- in ORB's
shutdown method we do not get a signle call to ServantActivator's
etherealize() method -- all servants active in that POA leak... In fact
activator itself leaks too. My understanding is that when POA is teared
down, it deactivates everything and subsequent etherealize() calls made from
other thread silently fail.

Can you suggest anything to fix this issue, please? I guess I can create
another main-thread POA (if it is possible at all -- it seems illogical) and
register activator there -- but as I understand there are no guarantees on
the order of POAs destruction...

[Michael Kilburn]

Post by Duncan Grisby
Now you mention it, though, it does seem logical that the call to

Post by Duncan Grisby
_remove_ref() is also done in on the main thread. I've checked in a
change to CVS that implements that.

Do you remember in which version of omniORB this problem was fixed?

Post by Duncan Grisby
If you can't update to the CVS version, you can force use of the main
thread by registering a ServantActivator that is itself activated in a
POA with the main thread policy. That way, servants will be released
using a call the etherealize() on the ServantActivator, which will
happen on the main thread.

We've just discovered a small problem with this approach -- in ORB's
shutdown method we do not get a signle call to ServantActivator's
etherealize() method -- all servants active in that POA leak... In fact
activator itself leaks too. My understanding is that when POA is teared
down, it deactivates everything and subsequent etherealize() calls made from
other thread silently fail.

Some help! Please!

[Duncan Grisby]

Post by Duncan Grisby
Now you mention it, though, it does seem logical that
the call to
_remove_ref() is also done in on the main thread. I've
checked in a
change to CVS that implements that.
Do you remember in which version of omniORB this problem was fixed?

It was fixed in 4.1.4.

Post by Duncan Grisby
We've just discovered a small problem with this approach -- in
ORB's shutdown method we do not get a signle call to
ServantActivator's etherealize() method -- all servants active
in that POA leak... In fact activator itself leaks too. My
understanding is that when POA is teared down, it deactivates
everything and subsequent etherealize() calls made from other
thread silently fail.

Can you come up with a minimal example that shows that problem?
etherealize() certainly should be called during shutdown.

Cheers,

Duncan.

[Michael Kilburn]

Skipped content of type multipart/alternative-------------- next part --------------
A non-text attachment was scrubbed...
Name: sample.cpp
Type: application/octet-stream
Size: 3186 bytes
Desc: not available
Url : http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20100507/3cf64831/sample.obj