Using per day calculation approach is leading to huge DB, is this valid system design approach ?

Hi, I have recently joined a Fintech startup which still at growing stage.The platform we manage is basically portfolio management.

We take account transactions from our users banks, exchange rates, asset prices (from 3rd party like Reuters) and calculate portfolio valuation and performance.

So the flow is can be summarized as

security transactions -> asset units -> prices -> exchange rates -> portfolio value 

My question is regarding an old and core micro service in this platform which has SOA. It has several performance issue and causes are several but primary bottle neck is DB.

Currently the DB size is 400 GB in production though start up is just 4 years old. When i checked, i feel a measure thing was missed out while designing this service or i might be wrong also.

The approach used in design is that for, at any stage of processing the service calculates per day values and stores them in DB.

What i mean by per day value is better to explain in examples.

The basic calculation flow is

Transaction - > Asset Units * Price * Exchange Rate = current value 

Now for asset units , there is a per day table i.e. every day for all users total units of each asset is calculated and inserted into DB irrespective of new transactions came in or not.

Same for exchange rates and prices, every day for each currency a new row is inserted even if it didn't change.

This looks very clean as you take per day values from each table for a date and multiple to get portfolio value on particular date.

But it leads to redundant data and huge DB. As you can observe here space complexity is not only factor of users & transactions but number of days passed so far which is infinite.

So during proper system design anyone would know it leads to redundancy, huge DB and will not scale. But question is, can it be some FinTech weird bureaucratic or compliance requirements to keep Per Day calculation Or is it some system design style which i am not aware of ? Off-course the original developer has left to ask him / her and rest are just making guesses or some arguments which don't still justify this approach.

submitted by /u/goro-7
[link] [comments]

from Software Development – methodologies, techniques, and tools. Covering Agile, RUP, Waterfall + more! https://ift.tt/ySecjPK

Leave a comment

Design a site like this with WordPress.com
Get started
search previous next tag category expand menu location phone mail time cart zoom edit close