[GitHub] [camel-kafka-connector] songlingling0012 opened a new issue #1124: camel-minio-kafka-connector can't split minio file

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[GitHub] [camel-kafka-connector] songlingling0012 opened a new issue #1124: camel-minio-kafka-connector can't split minio file

GitBox

songlingling0012 opened a new issue #1124:
URL: https://github.com/apache/camel-kafka-connector/issues/1124


   no matter how big the minio file , it maked as one message to kafka topic.
   i want split minio file content by '\n',one splitted  as one kafka message.
   how could i do?


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [camel-kafka-connector] oscerd commented on issue #1124: camel-minio-kafka-connector can't split minio file

GitBox

oscerd commented on issue #1124:
URL: https://github.com/apache/camel-kafka-connector/issues/1124#issuecomment-805549538


   This is how the underline component works, so you can't split it at the moment.


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [camel-kafka-connector] oscerd edited a comment on issue #1124: camel-minio-kafka-connector can't split minio file

GitBox
In reply to this post by GitBox

oscerd edited a comment on issue #1124:
URL: https://github.com/apache/camel-kafka-connector/issues/1124#issuecomment-805549538


   This is how the under the hood component works, so you can't split it at the moment.


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [camel-kafka-connector] songlingling0012 commented on issue #1124: camel-minio-kafka-connector can't split minio file

GitBox
In reply to this post by GitBox

songlingling0012 commented on issue #1124:
URL: https://github.com/apache/camel-kafka-connector/issues/1124#issuecomment-815550655


   Thanks for your reply!
   I modified the source codes by overriding the poll() method , I got SourceRecord and splitted it's content.
   Is there a downside to this´╝č


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [camel-kafka-connector] oscerd commented on issue #1124: camel-minio-kafka-connector can't split minio file

GitBox
In reply to this post by GitBox

oscerd commented on issue #1124:
URL: https://github.com/apache/camel-kafka-connector/issues/1124#issuecomment-815552014


   No, the problem is it won't work if you use different connectors


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [camel-kafka-connector] songlingling0012 commented on issue #1124: camel-minio-kafka-connector can't split minio file

GitBox
In reply to this post by GitBox

songlingling0012 commented on issue #1124:
URL: https://github.com/apache/camel-kafka-connector/issues/1124#issuecomment-815585261


   I see...
   I'm just using the minio-connector for my project, so it doesn't matter.
   But I got a new problem,the  minio file that I already readed don't deleted or removed


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [camel-kafka-connector] valdar commented on issue #1124: camel-minio-kafka-connector can't split minio file

GitBox
In reply to this post by GitBox

valdar commented on issue #1124:
URL: https://github.com/apache/camel-kafka-connector/issues/1124#issuecomment-816691620


   @songlingling0012 what version of `camel-kafka-conenctor` are you using? and how have you modified the `poll()` method?
   
   I suspect you might have crippled the exchange completion upon commit mechanism.


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [camel-kafka-connector] valdar edited a comment on issue #1124: camel-minio-kafka-connector can't split minio file

GitBox
In reply to this post by GitBox

valdar edited a comment on issue #1124:
URL: https://github.com/apache/camel-kafka-connector/issues/1124#issuecomment-816691620


   @songlingling0012 what version of `camel-kafka-conenctor` are you using? how you have configured minio connector? and how have you modified the `poll()` method?
   
   I suspect you might have crippled the exchange completion upon commit mechanism.


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]