Polling data from source (db) every 10 minute and checking for changes

We have a some external databases where we want to poll new data from in every 10 minutes. We want to apply this to some other data sources as well so it is not just limited to database, but you can think of db as 1 data source for this.

In every 10 minutes, we want to check if something happened (e.g. new entries were added).

Now, going through all database table for that is quite pointless as we can use a cursor (like a bookmark) that saves last id or timestamp from previous polling query and queries the newest data using that cursor as the last change pointer. This way we don't have to query whole table.

This is just an idea that spawned into my mind, are there any other efficient ways to do it (polling)?

submitted by /u/IamJustAWizard
[link] [comments]

from Software Development – methodologies, techniques, and tools. Covering Agile, RUP, Waterfall + more! https://ift.tt/2WHArqR

Leave a comment

Design a site like this with WordPress.com
Get started
search previous next tag category expand menu location phone mail time cart zoom edit close