[omniORB] Optimizing call signature + performance

[Martin B.]

Hi all!

I could need some advice with performance tuning our CORBA calls with
omniORB - find the "details" below:

-------- Original Message --------
Subject: Optimizing call "layout" + performance
Date: Fri, 03 Dec 2010 12:14:54 +0100
From: Martin B. <***@gmx.at>
Newsgroups: comp.object.corba

Hi,

We are currently transferring measurement samples (list of doubles +
timestamp) via a corba call connection.

The current interface looks like this:
...
typedef ... TimestampStruct;
typedef sequence<double> DblValueSeq;
void transferSample(in TimestampStruct ts, in DblValueSeq values);
...

We would like to make sure that our data transfer is as efficient as
possible (as in: transferring samples with several kHz), and so I'm
trying to get opinions on how we could possibly modify the call
signature (if necessary) or which ORB settings influence data transfer
efficiency.

Notes: Typical sequence sizes range from ~10 ... ~hundreds, so each call
transfers very approximately say 100 bytes - 10 kbytes

One specific question that bothers me is whether it could make sense to
block several samples into one call, that is to change the sig to:
...
struct Sample {
TimestampStruct ts;
DblValueSeq values;
};
typedef sequence<Sample> SampleSeq;
void transferSamples(in SampleSeq samples);
...

Any tips and pointers welcome!

cheers,
Martin

[Thomas Lockhart]

An HTML attachment was scrubbed...
URL: http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20101206/023892cc/attachment.htm

[Martin B.]

Post by Martin B.
Notes: Typical sequence sizes range from ~10 ... ~hundreds, so each
call transfers very approximately say 100 bytes - 10 kbytes
One specific question that bothers me is whether it could make sense
to block several samples into one call, that is to change the sig
to: ...

That is almost certainly what you will have to do. I think you will find
the limiting factor is the number of TCP/IP calls per second, not the
size of the data packet (at least on modern machinery). I haven't look
at this for some time, but I have found in the past that 5-10k call per
second was the best you could do (and I may be fudging and scaling that
up a bit; I'm recalling 1-3k calls per second about 10 years ago).

Thanks!

I would suggest running some TCP/IP packet transfer tests, leaving out
CORBA, so you get a sense for the performance of your system. Then when
things are that slow with omniORB, you won't be blaming CORBA for your
performance limits ;)

How large would I make these packets to transfer? When working on a
TCP/IP socket I'm (logically) just "streaming" data, or don't I? I
vaguely remember there being a maximum message size (maybe the term is a
bit different) for TCP/IP messages on Windows as well as some omniORB
setting relating to a max msg size ... ??

cheers,
Martin

[Thomas Lockhart]

Post by Martin B.
How large would I make these packets to transfer? When working on a
TCP/IP socket I'm (logically) just "streaming" data, or don't I? I
vaguely remember there being a maximum message size (maybe the term is
a bit different) for TCP/IP messages on Windows as well as some
omniORB setting relating to a max msg size ... ??

There are maximum sizes, and I have not ever run into them. My systems
have configurable telemetry streams built on top of CORBA Event
Channels, typically with some header info and sequences of doubles or
images. The images can be 128x128 or 256x256 doubles and can have
several in a transfer packet. Past systems have sent even more in a
packet. My point is that if you design for "reasonable sizes" of
hundreds of kilobytes or less, then you will be unlikely to run into
limitations. If you do run into a limitation then you can configure your
system to work around it. You can search the omniORB and TAO mailing
list archives to see discussions of configuring for large packet sizes.

hth

- Tom

[Duncan Grisby]

Post by Martin B.
How large would I make these packets to transfer? When working on a
TCP/IP socket I'm (logically) just "streaming" data, or don't I? I
vaguely remember there being a maximum message size (maybe the term is a
bit different) for TCP/IP messages on Windows as well as some omniORB
setting relating to a max msg size ... ??

omniORB's default maximum message size is 2MB. You can change it with
the giopMaxMsgSize parameter.

Since you want to stream data, it is possibly best to use oneway
messages to avoid the latency of waiting for replies to come back from
the server. The risk with oneways is that when the first message comes
in, a thread starts processing it; while it's still processing, the next
message can arrive, and omniORB might dispatch another thread to handle
it. There's an extension to omniORB that lets you prevent that, and also
to enable Nagle's algorithm that can automatically batch data at a TCP
level, which may or may not be helpful for you. Look in
src/lib/omniORB/connections/README.txt for some details.

Cheers,

Duncan.

[Martin B.]

Post by Duncan Grisby

Post by Martin B.
How large would I make these packets to transfer? When working on a
TCP/IP socket I'm (logically) just "streaming" data, or don't I? I
vaguely remember there being a maximum message size (maybe the term is a
bit different) for TCP/IP messages on Windows as well as some omniORB
setting relating to a max msg size ... ??

omniORB's default maximum message size is 2MB. You can change it with
the giopMaxMsgSize parameter.

Thanks for pointing this out! I may have been able to find the
parameter, but I'm unsure if I'd contextualized it correctly :-)

Post by Duncan Grisby
Since you want to stream data, it is possibly best to use oneway
messages to avoid the latency of waiting for replies to come back from
the server. ...

I'm unsure if I really want to avoid the latency at all.
If I understand this correctly, with a non-oneway call I can be sure the
other party has finished processing the data I sent it so I can be sure
that it makes sense to send the next (batch of) data.

I guess there's nothing for it but to test this, but if anyone has any
opinion on the matter I would appreciate any further info.

Post by Duncan Grisby
... The risk with oneways is that when the first message comes
in, a thread starts processing it; while it's still processing, the next
message can arrive, and omniORB might dispatch another thread to handle
it. There's an extension to omniORB that lets you prevent that, and also
to enable Nagle's algorithm that can automatically batch data at a TCP
level, which may or may not be helpful for you. Look in
src/lib/omniORB/connections/README.txt for some details.

Awesome! Thanks! Good to know this is around.

cheers,
Martin