[omniORB] [Q]report server architecture with omniOrb

[V S P]

Hi,
I am new to the list.
Currently designing a somewhat simple (but distributed) reporting server
that will be using postegres and files joining them together to create
reports.

The way the system would work is there is a table called request_queue
an entry there would indicate a type of report to run and the status of
the entry (submitted, executed, errored, inprogress).

Reports can take several hours to run, so the 'report servers'
have to be asynchronious.

I was thinking I coud use omniOrb to implement the 'distribution'
underpinning. However I am having difficulties invisioning how it can be
done.

a) assuming that I have some kind of a 'dispatcher' client that reads
the data from the queue table, then it needs to figure out which one of
the report serveres is not busy -- and send the request to that one.

b) the server would take the request, will say 'thank you' and work it
on it for a couple of hours, and then update the status of the queue
entry as 'executed'.

I thought I would need Naming Service and notification service for this,
however, my undestanding is that Notification service is not really
supported and certainly not working in Windows with VC++ 9

Wanted to ask if anybody has recommendation of what OmniOrb features I
need to look for/use for the above (or if there are better alternatives)

thank you in advance,
Vlad

[Martin Trappel]

Post by V S P
Hi,
I am new to the list.
Currently designing a somewhat simple (but distributed) reporting server
that will be using postegres and files joining them together to create
reports.
The way the system would work is there is a table called request_queue
an entry there would indicate a type of report to run and the status of
the entry (submitted, executed, errored, inprogress).
Reports can take several hours to run, so the 'report servers'
have to be asynchronious.
I was thinking I coud use omniOrb to implement the 'distribution'
underpinning. However I am having difficulties invisioning how it can be
done.
a) assuming that I have some kind of a 'dispatcher' client that reads
the data from the queue table, then it needs to figure out which one of
the report serveres is not busy -- and send the request to that one.
b) the server would take the request, will say 'thank you' and work it
on it for a couple of hours, and then update the status of the queue
entry as 'executed'.

Since both the Servers and the Dispatcher-Client need to talk to the
postgre Database, you could just leave CORBA out completely and let them
talk via the database ...

Post by V S P
I thought I would need Naming Service and notification service for this,
however, my undestanding is that Notification service is not really
supported and certainly not working in Windows with VC++ 9
Wanted to ask if anybody has recommendation of what OmniOrb features I
need to look for/use for the above (or if there are better alternatives)

I'm not sure you even need the Naming Service.

// Multiple Instances on multiple machines:
interface ReportGenerator {
// return true on OK, false on currently busy
// ReportDispatcher can call al registered servers until one returns true
bool StartProcessingRequest(in request_id id);
}

// Dispatcher Service, starting the report generation.
// Servers look it up via corbaloc.
interface ReportDispatcher {
void RegisterGenerator(in ReportGenerator r);
void UnregisterGenerator(in ReportGenerator r);

void RequestFinished(in request_id id);
}


br,
Martin

[V S P]

Thank you

If the
ReportGenerator ::StartProcessingRequest

returns back 'true', what mechanism should there be
to allow the same ReportGenerator server instance to beging
the several hours work on the report?

In otherwords, can I somehow reuse pre-spawned threads within
the report generator (after replying 'true') and ask them: now start
processing this request.

For example, in older applications I worked with -- there was
a simple thread-safe queue of tasks for the worker threads.

So what I thought, I need, is a way, may be,
for the ReportGenerator ::StartProcessingRequest to post a task
onto the internal OmniOrb queue (if such a thing exists)
such that one thread unlocks and starts processing the task.

or otherwise, how would I commence the processing such that
the 'response' -- 'yes I will take on the work' -- is decoupled
from the actual work.


thank you again,
Vlad

Post by Martin Trappel

Post by V S P
Hi,
I am new to the list.
Currently designing a somewhat simple (but distributed) reporting server
that will be using postegres and files joining them together to create
reports.
The way the system would work is there is a table called request_queue
an entry there would indicate a type of report to run and the status of
the entry (submitted, executed, errored, inprogress).
Reports can take several hours to run, so the 'report servers'
have to be asynchronious.
I was thinking I coud use omniOrb to implement the 'distribution'
underpinning. However I am having difficulties invisioning how it can be
done.
a) assuming that I have some kind of a 'dispatcher' client that reads
the data from the queue table, then it needs to figure out which one of
the report serveres is not busy -- and send the request to that one.
b) the server would take the request, will say 'thank you' and work it
on it for a couple of hours, and then update the status of the queue
entry as 'executed'.

Since both the Servers and the Dispatcher-Client need to talk to the
postgre Database, you could just leave CORBA out completely and let them
talk via the database ...

Post by V S P
I thought I would need Naming Service and notification service for this,
however, my undestanding is that Notification service is not really
supported and certainly not working in Windows with VC++ 9
Wanted to ask if anybody has recommendation of what OmniOrb features I
need to look for/use for the above (or if there are better alternatives)

I'm not sure you even need the Naming Service.
interface ReportGenerator {
// return true on OK, false on currently busy
// ReportDispatcher can call al registered servers until one returns true
bool StartProcessingRequest(in request_id id);
}
// Dispatcher Service, starting the report generation.
// Servers look it up via corbaloc.
interface ReportDispatcher {
void RegisterGenerator(in ReportGenerator r);
void UnregisterGenerator(in ReportGenerator r);
void RequestFinished(in request_id id);
}
br,
Martin

[V S P]

I think I am finally able to distill my question into something I can
describe one sentense :-)

does omniOrb support non-blocking remote calls (from client-to-servants)
and non-blocking calls to functions of the Servant from within that
servant
(using POA if that's relevant for the question)

if yes, is there example/description I can start looking at?


thank you,
Vlad

Post by V S P
Thank you
If the
ReportGenerator ::StartProcessingRequest
returns back 'true', what mechanism should there be
to allow the same ReportGenerator server instance to beging
the several hours work on the report?
In otherwords, can I somehow reuse pre-spawned threads within
the report generator (after replying 'true') and ask them: now start
processing this request.
For example, in older applications I worked with -- there was
a simple thread-safe queue of tasks for the worker threads.
So what I thought, I need, is a way, may be,
for the ReportGenerator ::StartProcessingRequest to post a task
onto the internal OmniOrb queue (if such a thing exists)
such that one thread unlocks and starts processing the task.
or otherwise, how would I commence the processing such that
the 'response' -- 'yes I will take on the work' -- is decoupled
from the actual work.
thank you again,
Vlad

Post by Martin Trappel

Post by V S P
Hi,
I am new to the list.
Currently designing a somewhat simple (but distributed) reporting server
that will be using postegres and files joining them together to create
reports.
The way the system would work is there is a table called request_queue
an entry there would indicate a type of report to run and the status of
the entry (submitted, executed, errored, inprogress).
Reports can take several hours to run, so the 'report servers'
have to be asynchronious.
I was thinking I coud use omniOrb to implement the 'distribution'
underpinning. However I am having difficulties invisioning how it can be
done.
a) assuming that I have some kind of a 'dispatcher' client that reads
the data from the queue table, then it needs to figure out which one of
the report serveres is not busy -- and send the request to that one.
b) the server would take the request, will say 'thank you' and work it
on it for a couple of hours, and then update the status of the queue
entry as 'executed'.

Since both the Servers and the Dispatcher-Client need to talk to the
postgre Database, you could just leave CORBA out completely and let them
talk via the database ...

Post by V S P
I thought I would need Naming Service and notification service for this,
however, my undestanding is that Notification service is not really
supported and certainly not working in Windows with VC++ 9
Wanted to ask if anybody has recommendation of what OmniOrb features I
need to look for/use for the above (or if there are better alternatives)

I'm not sure you even need the Naming Service.
interface ReportGenerator {
// return true on OK, false on currently busy
// ReportDispatcher can call al registered servers until one returns true
bool StartProcessingRequest(in request_id id);
}
// Dispatcher Service, starting the report generation.
// Servers look it up via corbaloc.
interface ReportDispatcher {
void RegisterGenerator(in ReportGenerator r);
void UnregisterGenerator(in ReportGenerator r);
void RequestFinished(in request_id id);
}
br,
Martin

--
V S P
--
http://www.fastmail.fm - Does exactly what it says on the tin
_______________________________________________
omniORB-list mailing list
http://www.omniorb-support.com/mailman/listinfo/omniorb-list

[Martin Trappel]

Post by V S P

Post by V S P
So what I thought, I need, is a way, may be,
for the ReportGenerator ::StartProcessingRequest to post a task
onto the internal OmniOrb queue (if such a thing exists)
such that one thread unlocks and starts processing the task.

(...)

Post by V S P
I think I am finally able to distill my question into something I can
describe one sentense :-)
does omniOrb support non-blocking remote calls (from client-to-servants)
and non-blocking calls to functions of the Servant from within that
servant
(using POA if that's relevant for the question)
if yes, is there example/description I can start looking at?

Your question has nothing to do with CORBA or omniORB.
Exactly What *your* implementation of the interface does is entirely up
to you.
The non-blocking part of the call will be implemented by you, because
you will have to decide if you return true or false after checking your
internal state. CORBA/omniORB just is responsible for the communication
framework, i.e. that you call to StartProcessingRequest gets to the
right implementation.

So it might look like this:
0) Your method gets called by omniORB due to a remote (or local) invocation
1) You check if you're busy
2) If busy: return false
3) If not busy: You start new thread or process to do work
4) return true

br,
Martin

[Wernke Zur Borg (external)]

Post by V S P
In otherwords, can I somehow reuse pre-spawned threads within
the report generator (after replying 'true') and ask them: now start
processing this request.

I would not try to do that. As Martin wrote, use your own threads to
process the requests. You can of course use the omnithread library to
control your threads, but you should not attempt to use existing omniORB
threads to perform your work.

Just my few cents...

Wernke

[V S P]

Ok,
thank you

I understand.
I, thought for some reason, that the corba-prespawned threads are
the ones that also do the actual work. That was wrong.

thank you again,
Vlad



On Fri, 6 Feb 2009 09:59:03 +0100, "Wernke Zur Borg (external)"

Post by Wernke Zur Borg (external)

Post by V S P
In otherwords, can I somehow reuse pre-spawned threads within
the report generator (after replying 'true') and ask them: now start
processing this request.

I would not try to do that. As Martin wrote, use your own threads to
process the requests. You can of course use the omnithread library to
control your threads, but you should not attempt to use existing omniORB
threads to perform your work.
Just my few cents...
Wernke
_______________________________________________
omniORB-list mailing list
http://www.omniorb-support.com/mailman/listinfo/omniorb-list

[Duncan Grisby]

Post by V S P
I think I am finally able to distill my question into something I can
describe one sentense :-)
does omniOrb support non-blocking remote calls (from client-to-servants)
and non-blocking calls to functions of the Servant from within that
servant
(using POA if that's relevant for the question)
if yes, is there example/description I can start looking at?

The usual way to do what you describe is a simple scheme with a
callback. There's no need to use exotic things like asynchronous calls
and so on.

The client submits the request which triggers an application thread to
do the work, then immediately returns. When the server is finished, it
calls the client back to indicate that it's finished.

There's an example of callbacks in src/examples/call_back.

Cheers,

Duncan.