Re: Lily Bulk Load ...which is better REST API / JAVA API (Linux Specs)

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Lily Bulk Load ...which is better REST API / JAVA API (Linux Specs)

Omar Atia
Dears,

Why don't we have components for Lily in Apache camel , so it will ease
writing Lily Clients .

I believe apache camel has Zookeeper and Hbase components .

it is just suggestion :) .

thanks,
Omar Atia

On Mon, Sep 24, 2012 at 11:23 PM, Omar Atia <[hidden email]> wrote:

> Dears,
>
> I  successfully write a client for data migration but unfortunelty during
> loading on  VMWARE Debian 5 (3 laptops), suddenly disconnect from zookeeper
> , session expired .
>
> I beleive this is issue in virtulization which same what you mentioned in
> this thread .
>
>  i figured out that Java API is much faster than REST API :) .
>
> The question now is how we can do bulk delete ? or should we delete record
> by record ?
>
> is there any integration with JMX to know the number of incoming requests
> , I have checked JMX lily part but maninly i find timings max min etc...
>
> thanks,
> Omar Atia
>
>
>
>
>
>
> On Mon, Sep 24, 2012 at 1:53 PM, Bruno Dumon <[hidden email]> wrote:
>
>>  On Mon, Sep 24, 2012 at 11:12 AM, Omar Atia <[hidden email]>wrote:
>>
>>>  *Dears,*
>>> **
>>> *if i connect to lily with JAVA API using Zookeeper :*
>>>
>>> *LilyClient lilyClient = new LilyClient("localhost:2181", 20000);*
>>> *I have one file of 10000 record CDRs for example .*
>>> **
>>> *should I connect to Zookeeper one connection per record means open and
>>> close connection for each record or open once and load everything.*
>>>
>>
>> No, absolutely not. Just create one LilyClient instance per JVM, so
>> manage it as a singleton. It is thread-safe, thus you can call it from any
>> number of threads without any need for synchronization.
>>
>>
>>>  **
>>> *I have downloaded artifacts using maven and i wanna start building
>>> multithreaded client .*
>>> *also if I have 3 Zookeeper , how could I write *
>>> *LilyClient lilyClient = new LilyClient("localhost:2181", 20000); for
>>> three of them.*
>>>
>>
>>
>> new LilyClient("host1,host2,host3", 20000);
>>
>> You don't need to specify the port number if it is the default 2181.
>>
>> --
>>  Bruno Dumon
>> NGDATA - Making Sense of Data
>> http://www.ngdata.com/
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Lily Bulk Load ...which is better REST API / JAVA API (Linux Specs)

Christian Mueller
Administrator
Because until now nobody asked for this integration...
Could you send me a pointer what Lily is about? Do you consider to
contribute such a component to Apache Camel? We love contributions... ;-)

Best,
Christian

On Tue, Sep 25, 2012 at 9:53 AM, Omar Atia <[hidden email]> wrote:

> Dears,
>
> Why don't we have components for Lily in Apache camel , so it will ease
> writing Lily Clients .
>
> I believe apache camel has Zookeeper and Hbase components .
>
> it is just suggestion :) .
>
> thanks,
> Omar Atia
>
> On Mon, Sep 24, 2012 at 11:23 PM, Omar Atia <[hidden email]>
> wrote:
>
> > Dears,
> >
> > I  successfully write a client for data migration but unfortunelty during
> > loading on  VMWARE Debian 5 (3 laptops), suddenly disconnect from
> zookeeper
> > , session expired .
> >
> > I beleive this is issue in virtulization which same what you mentioned in
> > this thread .
> >
> >  i figured out that Java API is much faster than REST API :) .
> >
> > The question now is how we can do bulk delete ? or should we delete
> record
> > by record ?
> >
> > is there any integration with JMX to know the number of incoming requests
> > , I have checked JMX lily part but maninly i find timings max min etc...
> >
> > thanks,
> > Omar Atia
> >
> >
> >
> >
> >
> >
> > On Mon, Sep 24, 2012 at 1:53 PM, Bruno Dumon <[hidden email]> wrote:
> >
> >>  On Mon, Sep 24, 2012 at 11:12 AM, Omar Atia <[hidden email]
> >wrote:
> >>
> >>>  *Dears,*
> >>> **
> >>> *if i connect to lily with JAVA API using Zookeeper :*
> >>>
> >>> *LilyClient lilyClient = new LilyClient("localhost:2181", 20000);*
> >>> *I have one file of 10000 record CDRs for example .*
> >>> **
> >>> *should I connect to Zookeeper one connection per record means open and
> >>> close connection for each record or open once and load everything.*
> >>>
> >>
> >> No, absolutely not. Just create one LilyClient instance per JVM, so
> >> manage it as a singleton. It is thread-safe, thus you can call it from
> any
> >> number of threads without any need for synchronization.
> >>
> >>
> >>>  **
> >>> *I have downloaded artifacts using maven and i wanna start building
> >>> multithreaded client .*
> >>> *also if I have 3 Zookeeper , how could I write *
> >>> *LilyClient lilyClient = new LilyClient("localhost:2181", 20000); for
> >>> three of them.*
> >>>
> >>
> >>
> >> new LilyClient("host1,host2,host3", 20000);
> >>
> >> You don't need to specify the port number if it is the default 2181.
> >>
> >> --
> >>  Bruno Dumon
> >> NGDATA - Making Sense of Data
> >> http://www.ngdata.com/
> >>
> >>
> >
>



--