Problem with ETL example

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem with ETL example

JamesZhang
I have post the problem here.
http://cwiki.apache.org/confluence/display/CAMEL/ETL+Example

Maybe we should delete that comment.thank you.


When I run the example,the result is failed.

[nent.file.FileComponent@36d047] ScheduledPollConsumer          DEBUG Starting to poll
[nent.file.FileComponent@36d047] FileConsumer                   DEBUG Polling directory src\data
[nent.file.FileComponent@36d047] FileEndpoint                   DEBUG Endpoint[file:src/data?noop=true] using strategy: org.apache.camel.component.file.strategy.NoOpFileStrategy@98bbf6
[nent.file.FileComponent@36d047] FileConsumer                   DEBUG About to process file:  src\data\row1.xml using exchange: Exchange[FileMessage: src\data\row1.xml]
[onent.jpa.JpaComponent@1e16483] ScheduledPollConsumer          DEBUG Starting to poll
[nent.file.FileComponent@36d047] DeadLetterChannel              ERROR On delivery attempt: 0 caught: java.lang.ClassCastException: org.apache.camel.impl.DefaultExchange
java.lang.ClassCastException: org.apache.camel.impl.DefaultExchange
        at org.apache.camel.component.file.FileMessage.getExchange(FileMessage.java:51)
        at org.apache.camel.component.file.FileMessage.getExchange(FileMessage.java:33)
        at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:49)
        at org.apache.camel.component.jpa.JpaEndpoint$1.evaluate(JpaEndpoint.java:197)
        at org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:44)
        at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:69)
        at org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:77)
        at org.apache.camel.processor.Pipeline.process(Pipeline.java:57)
        at org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:77)
        at org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:50)
        at org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:57)
        at org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:44)
        at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:63)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
        at java.lang.Thread.run(Thread.java:595)

I have run the following testcase,and the result is failed too.

//------------------ START SNIPPET
package org.apache.camel.component.file;

import java.io.File;

import org.apache.camel.ContextTestSupport;
import org.apache.camel.Exchange;

public class FileExchangeTest extends ContextTestSupport {

         public void testCopy() {

                 File file = new File(FileExchangeTest.class.getResource("FileExchangeTest.class").getFile());
                 FileExchange fileExchange = new FileExchange(context,file);
                 Exchange exchange = fileExchange.copy();
                 assertTrue("the copy of a FileExchange is not a FileExchange instance",
                                                                                                         exchange instanceof FileExchange);
         }

}
//------------------ END SNIPPET

Thanks very much.
Reply | Threaded
Open this post in threaded view
|

Re: Problem with ETL example

jstrachan
On 8/8/07, jameszhang <[hidden email]> wrote:

> I have run the following testcase,and the result is failed too.
>
> //------------------ START SNIPPET
> package org.apache.camel.component.file;
>
> import java.io.File;
>
> import org.apache.camel.ContextTestSupport;
> import org.apache.camel.Exchange;
>
> public class FileExchangeTest extends ContextTestSupport {
>
>          public void testCopy() {
>
>                  File file = new
> File(FileExchangeTest.class.getResource("FileExchangeTest.class").getFile());
>                  FileExchange fileExchange = new FileExchange(context,file);
>                  Exchange exchange = fileExchange.copy();
>                  assertTrue("the copy of a FileExchange is not a
> FileExchange instance",
>
> exchange instanceof FileExchange);
>          }
>
> }
> //------------------ END SNIPPET
>
> Thanks very much.

Great catch James!

I've added this test case to camel-core along with a patch to fix it -
many thanks.
--
James
-------
http://macstrac.blogspot.com/
Reply | Threaded
Open this post in threaded view
|

Re: Problem with ETL example

James Zhang

James.Strachan wrote
On 8/8/07, jameszhang <jameszhang.james@gmail.com> wrote:
> I have run the following testcase,and the result is failed too.
>
> //------------------ START SNIPPET
> package org.apache.camel.component.file;
>
> import java.io.File;
>
> import org.apache.camel.ContextTestSupport;
> import org.apache.camel.Exchange;
>
> public class FileExchangeTest extends ContextTestSupport {
>
>          public void testCopy() {
>
>                  File file = new
> File(FileExchangeTest.class.getResource("FileExchangeTest.class").getFile());
>                  FileExchange fileExchange = new FileExchange(context,file);
>                  Exchange exchange = fileExchange.copy();
>                  assertTrue("the copy of a FileExchange is not a
> FileExchange instance",
>
> exchange instanceof FileExchange);
>          }
>
> }
> //------------------ END SNIPPET
>
> Thanks very much.

Great catch James!

I've added this test case to camel-core along with a patch to fix it -
many thanks.
--
James
-------
http://macstrac.blogspot.com/

OK.
I think I can write the spring example now.
Reply | Threaded
Open this post in threaded view
|

Re: Problem with ETL example

jstrachan
On 8/8/07, James Zhang <[hidden email]> wrote:

>
>
>
> James.Strachan wrote:
> >
> > On 8/8/07, jameszhang <[hidden email]> wrote:
> >> I have run the following testcase,and the result is failed too.
> >>
> >> //------------------ START SNIPPET
> >> package org.apache.camel.component.file;
> >>
> >> import java.io.File;
> >>
> >> import org.apache.camel.ContextTestSupport;
> >> import org.apache.camel.Exchange;
> >>
> >> public class FileExchangeTest extends ContextTestSupport {
> >>
> >>          public void testCopy() {
> >>
> >>                  File file = new
> >> File(FileExchangeTest.class.getResource("FileExchangeTest.class").getFile());
> >>                  FileExchange fileExchange = new
> >> FileExchange(context,file);
> >>                  Exchange exchange = fileExchange.copy();
> >>                  assertTrue("the copy of a FileExchange is not a
> >> FileExchange instance",
> >>
> >> exchange instanceof FileExchange);
> >>          }
> >>
> >> }
> >> //------------------ END SNIPPET
> >>
> >> Thanks very much.
> >
> > Great catch James!
> >
> > I've added this test case to camel-core along with a patch to fix it -
> > many thanks.
> > --
> > James
> > -------
> > http://macstrac.blogspot.com/
> >
> >
>
>
> OK.
> I think I can write the spring example now.

Great! :)
--
James
-------
http://macstrac.blogspot.com/
Reply | Threaded
Open this post in threaded view
|

Re: Problem with ETL example

James Zhang
James.Strachan wrote
On 8/8/07, James Zhang <jmszhang@gmail.com> wrote:
>
>
>
> James.Strachan wrote:
> >
> > On 8/8/07, jameszhang <jameszhang.james@gmail.com> wrote:
> >> I have run the following testcase,and the result is failed too.
> >>
> >> //------------------ START SNIPPET
> >> package org.apache.camel.component.file;
> >>
> >> import java.io.File;
> >>
> >> import org.apache.camel.ContextTestSupport;
> >> import org.apache.camel.Exchange;
> >>
> >> public class FileExchangeTest extends ContextTestSupport {
> >>
> >>          public void testCopy() {
> >>
> >>                  File file = new
> >> File(FileExchangeTest.class.getResource("FileExchangeTest.class").getFile());
> >>                  FileExchange fileExchange = new
> >> FileExchange(context,file);
> >>                  Exchange exchange = fileExchange.copy();
> >>                  assertTrue("the copy of a FileExchange is not a
> >> FileExchange instance",
> >>
> >> exchange instanceof FileExchange);
> >>          }
> >>
> >> }
> >> //------------------ END SNIPPET
> >>
> >> Thanks very much.
> >
> > Great catch James!
> >
> > I've added this test case to camel-core along with a patch to fix it -
> > many thanks.
> > --
> > James
> > -------
> > http://macstrac.blogspot.com/
> >
> >
>
>
> OK.
> I think I can write the spring example now.

Great! :)
--
James
-------
http://macstrac.blogspot.com/

hi:
I should modify the "org.apache.camel.processor.Pipeline" to make the example finally works.

    protected Exchange createNextExchange(Processor producer, Exchange previousExchange) {
        Exchange answer = copyExchangeStrategy(previousExchange);

        //-----JamesZhang ADD BEGIN
        Object input = previousExchange.getIn().getBody();
        if (input != null) {
            answer.getIn().setBody(input);
        }
        //-----JamesZhang ADD END
       
        // now lets set the input of the next exchange to the output of the
        // previous message if it is not null
        Object output = previousExchange.getOut().getBody();
        if (output != null) {
            answer.getIn().setBody(output);
        }
        return answer;
    }

If I do not do that,the body of copy FileExchange is null;and the router finally find to use org.apache.camel.converter.jaxb.FallbackTypeConverter at the time of "convertBodyTo(PersonDocument.class).to("jpa:org.apache.camel.example.etl.CustomerEntity");"
But we should expect to use org.apache.camel.example.etl.CustomerTransformer.

There should be a better scheme to fix it.Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: Problem with ETL example

jstrachan
On 8/8/07, James Zhang <[hidden email]> wrote:

> hi:
> I should modify the "org.apache.camel.processor.Pipeline" to make the
> example finally works.
>
>     protected Exchange createNextExchange(Processor producer, Exchange
> previousExchange) {
>         Exchange answer = copyExchangeStrategy(previousExchange);
>
>         //-----JamesZhang ADD BEGIN
>         Object input = previousExchange.getIn().getBody();
>         if (input != null) {
>             answer.getIn().setBody(input);
>         }
>         //-----JamesZhang ADD END
>
>         // now lets set the input of the next exchange to the output of the
>         // previous message if it is not null
>         Object output = previousExchange.getOut().getBody();
>         if (output != null) {
>             answer.getIn().setBody(output);
>         }
>         return answer;
>     }
>
> If I do not do that,the body of copy FileExchange is null;and the router

FWIW this bug indicates a real bug elsewhere - that an Exchange
implementation does not implement copy() properly. I've patched the
code to do as you ask, and have added a big warning to note which
Exchange implementations do not properly copy across the in-body so we
can fix them as we see the warning.

BTW any idea what the exchange implementation was that caused this
issue? FWIW I've added extra checks to the FileExchangeTest to ensure
its not the FileExchange.

It'd be good to add a test case to reproduce this bug you're seeing -
to be sure we've fixed it etc.

--
James
-------
http://macstrac.blogspot.com/
Reply | Threaded
Open this post in threaded view
|

Re: Problem with ETL example

jstrachan
On 8/9/07, James Zhang <[hidden email]> wrote:

> James.Strachan wrote:
> >
> > On 8/8/07, James Zhang <[hidden email]> wrote:
> >> hi:
> >> I should modify the "org.apache.camel.processor.Pipeline" to make the
> >> example finally works.
> >>
> >>     protected Exchange createNextExchange(Processor producer, Exchange
> >> previousExchange) {
> >>         Exchange answer = copyExchangeStrategy(previousExchange);
> >>
> >>         //-----JamesZhang ADD BEGIN
> >>         Object input = previousExchange.getIn().getBody();
> >>         if (input != null) {
> >>             answer.getIn().setBody(input);
> >>         }
> >>         //-----JamesZhang ADD END
> >>
> >>         // now lets set the input of the next exchange to the output of
> >> the
> >>         // previous message if it is not null
> >>         Object output = previousExchange.getOut().getBody();
> >>         if (output != null) {
> >>             answer.getIn().setBody(output);
> >>         }
> >>         return answer;
> >>     }
> >>
> >> If I do not do that,the body of copy FileExchange is null;and the router
> >
> > FWIW this bug indicates a real bug elsewhere - that an Exchange
> > implementation does not implement copy() properly. I've patched the
> > code to do as you ask, and have added a big warning to note which
> > Exchange implementations do not properly copy across the in-body so we
> > can fix them as we see the warning.
> >
> > BTW any idea what the exchange implementation was that caused this
> > issue? FWIW I've added extra checks to the FileExchangeTest to ensure
> > its not the FileExchange.
> >
> > It'd be good to add a test case to reproduce this bug you're seeing -
> > to be sure we've fixed it etc.
> >
> Maybe it's wrong what I wrote last night.
> The "getIn().getBody()"method of the copy fileExchange will return a
> instance of FileInputStream,not null.
> But It should return a instance of PersonDeocument.
>
> I will write the testcase tonight.
> And I will add a  testcase when I produce a bug aftertime.Thanks.

Ah I get it now :). A FileExchange has had its body changed, but when
its copyied, the body is not copied as well. DOH! It turned out to be
my mistake - I overloaded the wrong method (copy() instead of
newInstance()) - I've added a test case to FileExchangeTest for this
and patched it in trunk

Incidentlaly for CAMEL-90 I fixed the FileExchange.in.body being a
FileInputStream; its now a File, so you can re-parse it many times
etc.

--
James
-------
http://macstrac.blogspot.com/
Reply | Threaded
Open this post in threaded view
|

Re: Problem with ETL example

James Zhang
James.Strachan wrote
On 8/9/07, James Zhang <jmszhang@gmail.com> wrote:
> James.Strachan wrote:
> >
> > On 8/8/07, James Zhang <jmszhang@gmail.com> wrote:
> >> hi:
> >> I should modify the "org.apache.camel.processor.Pipeline" to make the
> >> example finally works.
> >>
> >>     protected Exchange createNextExchange(Processor producer, Exchange
> >> previousExchange) {
> >>         Exchange answer = copyExchangeStrategy(previousExchange);
> >>
> >>         //-----JamesZhang ADD BEGIN
> >>         Object input = previousExchange.getIn().getBody();
> >>         if (input != null) {
> >>             answer.getIn().setBody(input);
> >>         }
> >>         //-----JamesZhang ADD END
> >>
> >>         // now lets set the input of the next exchange to the output of
> >> the
> >>         // previous message if it is not null
> >>         Object output = previousExchange.getOut().getBody();
> >>         if (output != null) {
> >>             answer.getIn().setBody(output);
> >>         }
> >>         return answer;
> >>     }
> >>
> >> If I do not do that,the body of copy FileExchange is null;and the router
> >
> > FWIW this bug indicates a real bug elsewhere - that an Exchange
> > implementation does not implement copy() properly. I've patched the
> > code to do as you ask, and have added a big warning to note which
> > Exchange implementations do not properly copy across the in-body so we
> > can fix them as we see the warning.
> >
> > BTW any idea what the exchange implementation was that caused this
> > issue? FWIW I've added extra checks to the FileExchangeTest to ensure
> > its not the FileExchange.
> >
> > It'd be good to add a test case to reproduce this bug you're seeing -
> > to be sure we've fixed it etc.
> >
> Maybe it's wrong what I wrote last night.
> The "getIn().getBody()"method of the copy fileExchange will return a
> instance of FileInputStream,not null.
> But It should return a instance of PersonDeocument.
>
> I will write the testcase tonight.
> And I will add a  testcase when I produce a bug aftertime.Thanks.

Ah I get it now :). A FileExchange has had its body changed, but when
its copyied, the body is not copied as well. DOH! It turned out to be
my mistake - I overloaded the wrong method (copy() instead of
newInstance()) - I've added a test case to FileExchangeTest for this
and patched it in trunk

Incidentlaly for CAMEL-90 I fixed the FileExchange.in.body being a
FileInputStream; its now a File, so you can re-parse it many times
etc.

--
James
-------
http://macstrac.blogspot.com/
Next time ,I will

(1)Just go to bed at 11:59pm ,and tell you nothing what happend,
but report the bug and add a testcase next night.

or

(2)Report the bug,but tell you should never take care util I have write a testcase.

or

(3) Never report the bug beacause I can't understand the code thus can't add a test case .

:)
Just a joke ,I love testcase and refactoring.