Sign Up

Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.

Have an account? Sign In

Have an account? Sign In Now

Sign In

Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.

Sign Up Here

Forgot Password?

Don't have account, Sign Up Here

Forgot Password

Lost your password? Please enter your email address. You will receive a link and will create a new password via email.

Have an account? Sign In Now

You must login to ask question.

Forgot Password?

Need An Account, Sign Up Here

Please briefly explain why you feel this question should be reported.

Please briefly explain why you feel this answer should be reported.

Please briefly explain why you feel this user should be reported.

Sign InSign Up

StackOverflow Point

StackOverflow Point Navigation

  • Web Stories
  • Badges
  • Tags
Search
Ask A Question

Mobile menu

Close
Ask a Question
  • Web Stories
  • Badges
  • Tags
Home/ Questions/Q 3388
Alex Hales
  • 0
Alex HalesTeacher
Asked: June 2, 20222022-06-02T12:15:11+00:00 2022-06-02T12:15:11+00:00

apache kafka – Processing CRUD events in event driven pipeline

  • 0

[ad_1]

I am converting a monolithic application which was handling CRUD events into multiple microservices. Each microservice being designed as kafka stream with specific function. Idea was to allow scaling each microservice individually.
The challenge is, there is a difference in the processing flow of Create and update events where Create event has to go through multiple functional microservices( filtering, enriching) in the pipeline where as update event need not go through these processors. But in order to process the create and update event in order, update events also have to flow through all the microservices( read/write from kafka topics)if not update event can get published even before create event is processed.

event publisher(create/update event) =>(kafka topic)=> filter =>(kafka topic)=> enrichment=>..=>...=>(output topic)

Is this solution ok to have dummy pipe/service for update event flow to achieve ordering?

Another approach is, instead of designing each microservice as kafka stream, i create each microservice with a synchronous rest interface and call it from a single kafka stream application which can then manage the order of processing create and update events.

event publisher(create/update event) =>(kafka topic)=> event processor=>(output topic)

where event processor invoked each functional microservice with rest interface.
In this case, although i still achieve separate microservice for each function which can scale individually, invoking the rest calls inside an async streaming application might impact its scalability and other potential problems.

Any comments/suggestion on both the approaches and any alternative approach recommendation would help.

[ad_2]

  • 0 0 Answers
  • 15 Views
  • 0 Followers
  • 0
Share
  • Facebook
  • Report
Leave an answer

Leave an answer
Cancel reply

Browse

Sidebar

Ask A Question

Related Questions

  • xcode - Can you build dynamic libraries for iOS and ...

    • 0 Answers
  • bash - How to check if a process id (PID) ...

    • 8055 Answers
  • database - Oracle: Changing VARCHAR2 column to CLOB

    • 1842 Answers
  • What's the difference between HEAD, working tree and index, in ...

    • 1924 Answers
  • Amazon EC2 Free tier - how many instances can I ...

    • 0 Answers

Stats

  • Questions : 43k

Subscribe

Login

Forgot Password?

Footer

Follow

© 2022 Stackoverflow Point. All Rights Reserved.

Insert/edit link

Enter the destination URL

Or link to existing content

    No search term specified. Showing recent items. Search or use up and down arrow keys to select an item.