stream caching not configurable for rest consumer?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

stream caching not configurable for rest consumer?

Vojtech Fried
Hi,

I am trying to fix upload of huge files via camel rest endpoint. The problem is that stream caching enables itself in this case (without being enabled in camel context or the route), but when camel asks a StreamCachingStrategy if the data should be moved from memory to file, it always says no, because it has "enabled" flag false (because no one enabled stream caching).

There is a Jira issue https://issues.apache.org/jira/browse/CAMEL-8821 that should have fixed it, but it actually did not, because as the last comment says, it calls a wrong method "readResponse..." instead of "readRequest...", the fix is only in "readRequest..." method. So I think it should be reopened.

Is there any other way how to disable stream caching?

My code is in configure method in a class that extends RouteBuilder:

restConfiguration()
            .contextPath(...)
            .enableCORS(true)
            .component("servlet")
rest("...")
            .id("...")
            .produces("application/json")
            .post("...")
            .to("direct:...");

Vojtech
Reply | Threaded
Open this post in threaded view
|

Re: stream caching not configurable for rest consumer?

Claus Ibsen-2
Hi

Ah yeah looks like the code should be identical for those 2 (or maybe
just one common method as its InputStream as input).
And then they should check the camel context for stream caching in use
or not. Then you can turn it on|off on context level and have it work.

Can you create a new JIRA ticket

On Wed, Apr 1, 2020 at 11:06 AM Vojtech Fried <[hidden email]> wrote:

>
> Hi,
>
> I am trying to fix upload of huge files via camel rest endpoint. The problem is that stream caching enables itself in this case (without being enabled in camel context or the route), but when camel asks a StreamCachingStrategy if the data should be moved from memory to file, it always says no, because it has "enabled" flag false (because no one enabled stream caching).
>
> There is a Jira issue https://issues.apache.org/jira/browse/CAMEL-8821 that should have fixed it, but it actually did not, because as the last comment says, it calls a wrong method "readResponse..." instead of "readRequest...", the fix is only in "readRequest..." method. So I think it should be reopened.
>
> Is there any other way how to disable stream caching?
>
> My code is in configure method in a class that extends RouteBuilder:
>
> restConfiguration()
>             .contextPath(...)
>             .enableCORS(true)
>             .component("servlet")
> rest("...")
>             .id("...")
>             .produces("application/json")
>             .post("...")
>             .to("direct:...");
>
> Vojtech



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: [EXTERNAL] - Re: stream caching not configurable for rest consumer?

Vojtech Fried
I have created https://issues.apache.org/jira/browse/CAMEL-14823
Thanks.

________________________________
From: Claus Ibsen <[hidden email]>
Sent: Wednesday, April 1, 2020 12:11
To: [hidden email] <[hidden email]>
Subject: [EXTERNAL] - Re: stream caching not configurable for rest consumer?

Hi

Ah yeah looks like the code should be identical for those 2 (or maybe
just one common method as its InputStream as input).
And then they should check the camel context for stream caching in use
or not. Then you can turn it on|off on context level and have it work.

Can you create a new JIRA ticket

On Wed, Apr 1, 2020 at 11:06 AM Vojtech Fried <[hidden email]> wrote:

>
> Hi,
>
> I am trying to fix upload of huge files via camel rest endpoint. The problem is that stream caching enables itself in this case (without being enabled in camel context or the route), but when camel asks a StreamCachingStrategy if the data should be moved from memory to file, it always says no, because it has "enabled" flag false (because no one enabled stream caching).
>
> There is a Jira issue https://urldefense.com/v3/__https://issues.apache.org/jira/browse/CAMEL-8821__;!!Obbck6kTJA!Mgt7NJHDilliMB-tmUd04UaESpJLL6kaGPRG0pn6alRqo2ZN25TD-AQTWe2DPxWy$  that should have fixed it, but it actually did not, because as the last comment says, it calls a wrong method "readResponse..." instead of "readRequest...", the fix is only in "readRequest..." method. So I think it should be reopened.
>
> Is there any other way how to disable stream caching?
>
> My code is in configure method in a class that extends RouteBuilder:
>
> restConfiguration()
>             .contextPath(...)
>             .enableCORS(true)
>             .component("servlet")
> rest("...")
>             .id("...")
>             .produces("application/json")
>             .post("...")
>             .to("direct:...");
>
> Vojtech



--
Claus Ibsen
-----------------
https://urldefense.com/v3/__http://davsclaus.com__;!!Obbck6kTJA!Mgt7NJHDilliMB-tmUd04UaESpJLL6kaGPRG0pn6alRqo2ZN25TD-AQTWU2V8rSp$  @davsclaus
Camel in Action 2: https://urldefense.com/v3/__https://www.manning.com/ibsen2__;!!Obbck6kTJA!Mgt7NJHDilliMB-tmUd04UaESpJLL6kaGPRG0pn6alRqo2ZN25TD-AQTWV_-2ajy$
Reply | Threaded
Open this post in threaded view
|

Re: stream caching not configurable for rest consumer?

Vojtech Fried
Still isn't it a bug that if stream caching is not enabled in the context, what happens in rest consumer is that it first uses the cache and second it uploads the whole input into memory? Shouldn't the default strategy be to spool to file after some limit?

BTW: I have found a way how to disable stream caching explicitly (without the fix):
restConfiguration()
    ...
    .endpointProperty("disableStreamCache", "true")

________________________________
From: Vojtech Fried <[hidden email]>
Sent: Wednesday, April 1, 2020 12:27
To: [hidden email] <[hidden email]>
Subject: Re: [EXTERNAL] - Re: stream caching not configurable for rest consumer?

I have created https://urldefense.com/v3/__https://issues.apache.org/jira/browse/CAMEL-14823__;!!Obbck6kTJA!PlMm8fZT6I7H24xabbHpPa5z-_TkYnV_X5Z33-L0c8PxlT2rwmPaoB3RCnc8KGlN$
Thanks.

________________________________
From: Claus Ibsen <[hidden email]>
Sent: Wednesday, April 1, 2020 12:11
To: [hidden email] <[hidden email]>
Subject: [EXTERNAL] - Re: stream caching not configurable for rest consumer?

Hi

Ah yeah looks like the code should be identical for those 2 (or maybe
just one common method as its InputStream as input).
And then they should check the camel context for stream caching in use
or not. Then you can turn it on|off on context level and have it work.

Can you create a new JIRA ticket

On Wed, Apr 1, 2020 at 11:06 AM Vojtech Fried <[hidden email]> wrote:

>
> Hi,
>
> I am trying to fix upload of huge files via camel rest endpoint. The problem is that stream caching enables itself in this case (without being enabled in camel context or the route), but when camel asks a StreamCachingStrategy if the data should be moved from memory to file, it always says no, because it has "enabled" flag false (because no one enabled stream caching).
>
> There is a Jira issue https://urldefense.com/v3/__https://issues.apache.org/jira/browse/CAMEL-8821__;!!Obbck6kTJA!Mgt7NJHDilliMB-tmUd04UaESpJLL6kaGPRG0pn6alRqo2ZN25TD-AQTWe2DPxWy$  that should have fixed it, but it actually did not, because as the last comment says, it calls a wrong method "readResponse..." instead of "readRequest...", the fix is only in "readRequest..." method. So I think it should be reopened.
>
> Is there any other way how to disable stream caching?
>
> My code is in configure method in a class that extends RouteBuilder:
>
> restConfiguration()
>             .contextPath(...)
>             .enableCORS(true)
>             .component("servlet")
> rest("...")
>             .id("...")
>             .produces("application/json")
>             .post("...")
>             .to("direct:...");
>
> Vojtech



--
Claus Ibsen
-----------------
https://urldefense.com/v3/__http://davsclaus.com__;!!Obbck6kTJA!Mgt7NJHDilliMB-tmUd04UaESpJLL6kaGPRG0pn6alRqo2ZN25TD-AQTWU2V8rSp$  @davsclaus
Camel in Action 2: https://urldefense.com/v3/__https://www.manning.com/ibsen2__;!!Obbck6kTJA!Mgt7NJHDilliMB-tmUd04UaESpJLL6kaGPRG0pn6alRqo2ZN25TD-AQTWV_-2ajy$