Principal DWH/BI Engineer
From an hour to a few seconds: real-time stream data processing with fully managed AWS Kinesis services
To process data in real-time to derive insights and act upon it quickly, an enterprise needs to build an end-to-end real-time data processing system from data acquisition, data ingestion, data processing, and building platform to serving and sharing the results. In this talk, we will describe the use case we were trying to solve at Telesign. We will present the overall architecture of our scalable cloud solution and share how we leveraged AWS fully managed services Kinesis Data Stream and Kinesis Data Analytics (based on Apache Flink) to build real-time exactly-once stream data processing system.