How to develop scalable progress tracker for backend that can be consumed by frontend to show ETA?

Hi Everyone,

I've got a problem and have been seeking for its solution for quite sometime but didn't find any substantial information.

Problem:

There are multiple python based containerised backend applications that are being run on AWS batch and currently the progress of these application are being displayed on UI based on the start, fail, or success of the processes.

Now the problem is I have to develop a system/microservice that we can integrate with the current backend services that can provide the realtime progress and ETA updates on the frontend.

I've been searching for similar solutions but didn't find more of it.

​

The idea in my mind is, there can be two stages in this solution:

1. Get the progress of the backend running processes, that can vary application wise

2. Send the progress to some microservice that can be consumed by frontend

​

​

I am not sure though if this is a valid approach or not.

​

Please share your experience of developing scalable progress tracker for backend that can be consumed by frontend to show ETA.

​

Are there any open source solutions that already exist? I know of TQDM in python but not how can it be helpful here.

​

Any links of design pattens, technical blogs, video, etc. are highly appreciated.

​

Thanks

​

​

TLDR: How to build a system/microservice that can take the ETA updates of the progress in the backend script and can be consumed on the frontend?