A Real-Time Monitoring System for Financial Transactions. Easier with Spark Streaming – Databricks

A Real-Time Monitoring System for Financial Transactions. Easier with Spark Streaming

Money transactions are very delicate operations. Every single one must be accomplished and acknowledged. Every transaction undergoes hundreds of different steps from origin to destination in order to be complete. The amount of data issued on each step is considerable, so when we talk about five million transactions every day, this amount becomes enormous. Aiming to ease the monitoring process and accelerate error detection, we have developed a solution at Stratio. In order to provide better monitoring in real-time, we combined different Apache technologies. Transactions are classified by different categories and grouped in different levels within the same category. They must be parameterized from the outset, so the system know what is being inputted. The heart of the application is Spark Streaming, on which we rely for the ETL, CEP and backup processes. The new tool is capable of framing the system in a time window or a specific moment. Within this window it’s possible to drill down to the tiniest detail and browse around. With its incidences module, the system triggers user-defined alerts and warnings on different levels of gravity. These are conceived as a direct link to the error so you can isolate the transaction that is causing the problem and inspect it.



« back