{"id":14996,"date":"2021-05-28T09:00:00","date_gmt":"2021-05-28T07:00:00","guid":{"rendered":"https:\/\/www.codemotion.com\/magazine\/?p=14996"},"modified":"2022-01-05T20:02:44","modified_gmt":"2022-01-05T19:02:44","slug":"aws-kinesis-serverless-event-processing","status":"publish","type":"post","link":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/","title":{"rendered":"Serverless Event Processing on AWS Platform w\/ Kinesis"},"content":{"rendered":"\t\t\t\t<div class=\"wp-block-uagb-table-of-contents uagb-toc__align-left uagb-toc__columns-1  uagb-block-715f2742      \"\n\t\t\t\t\tdata-scroll= \"1\"\n\t\t\t\t\tdata-offset= \"30\"\n\t\t\t\t\tstyle=\"\"\n\t\t\t\t>\n\t\t\t\t<div class=\"uagb-toc__wrap\">\n\t\t\t\t\t\t<div class=\"uagb-toc__title\">\n\t\t\t\t\t\t\tTable Of Contents\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"uagb-toc__list-wrap \">\n\t\t\t\t\t\t<ol class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#the-project\" class=\"uagb-toc-link__trigger\">The project<\/a><li class=\"uagb-toc__list\"><a href=\"#introduction-to-serverless\" class=\"uagb-toc-link__trigger\">Introduction to serverless<\/a><li class=\"uagb-toc__list\"><a href=\"#general-overview-of-an-event-architecture\" class=\"uagb-toc-link__trigger\">General overview of an event architecture<\/a><ul class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#producers\" class=\"uagb-toc-link__trigger\">Producers<\/a><li class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#the-schema-registry\" class=\"uagb-toc-link__trigger\">The Schema Registry<\/a><li class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#the-aws-lambda\" class=\"uagb-toc-link__trigger\">The AWS Lambda<\/a><li class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#the-dynamodb\" class=\"uagb-toc-link__trigger\">The DynamoDB\u00a0<\/a><\/li><\/ul><\/li><li class=\"uagb-toc__list\"><a href=\"#kinesis-the-aws-fully-managed-streaming-service\" class=\"uagb-toc-link__trigger\">Kinesis: the AWS fully-managed streaming service\u00a0<\/a><ul class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#aws-kinesis-vs-kafka\" class=\"uagb-toc-link__trigger\">AWS Kinesis Vs. Kafka\u00a0<\/a><li class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#kinesis-data-analytics\" class=\"uagb-toc-link__trigger\">Kinesis data analytics<\/a><li class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#glue\" class=\"uagb-toc-link__trigger\">Glue<\/a><li class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#athena\" class=\"uagb-toc-link__trigger\">Athena<\/a><li class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#api-gateway\" class=\"uagb-toc-link__trigger\">API Gateway<\/a><\/li><\/ul><\/li><\/ul><\/li><li class=\"uagb-toc__list\"><a href=\"#case-study-feeding-read-apis\" class=\"uagb-toc-link__trigger\">Case Study: Feeding Read APIs<\/a><ul class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#producer-the-code\" class=\"uagb-toc-link__trigger\">Producer: the code<\/a><li class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#lambda-function-the-code\" class=\"uagb-toc-link__trigger\">Lambda function: the code<\/a><\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li class=\"uagb-toc__list\"><a href=\"#setting-up-serverless-on-aws\" class=\"uagb-toc-link__trigger\">Setting up serverless on AWS<\/a><ul class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#configuring-aws-kinesis\" class=\"uagb-toc-link__trigger\">Configuring AWS Kinesis<\/a><li class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#configuring-dynamodb\" class=\"uagb-toc-link__trigger\">Configuring DynamoDB<\/a><li class=\"uagb-toc__list\"><li class=\"uagb-toc__list\"><a href=\"#configuring-the-lambda-function\" class=\"uagb-toc-link__trigger\">Configuring the Lambda function<\/a><\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li class=\"uagb-toc__list\"><a href=\"#conclusions\" class=\"uagb-toc-link__trigger\">Conclusions<\/a><\/ul><\/ul><\/ul><\/ul><\/ol>\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\n\n\n<h2 class=\"wp-block-heading\" id=\"h-the-project\"><strong>The project<\/strong><\/h2>\n\n\n\n<p>Openbank is the 100% digital bank of the Santander Group, currently undergoing a technological transformation and international expansion. The work is organized in a startup-like format, using agile methodologies to take client experience to the next level.&nbsp;<\/p>\n\n\n\n<p>The Netherlands, Germany and Portugal were OpenBank\u2019s flagships in 2019, with&nbsp; Argentina as the next target, with others to follow. The microservice architecture runs in AWS and the languages and frameworks used include React, Java, Spring, Kotlin, Scala, Spark, Python, Flink, and more.<\/p>\n\n\n\n<p>This article offers an initial description of the tools used, followed by a real use case with AWS platform w\/ Kinesis.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-introduction-to-serverless\"><strong>Introduction to serverless<\/strong><\/h2>\n\n\n\n<p>\u2018Serverless\u2019 is a model of Cloud-based execution in which applications are built and run, but management of the infrastructure is delegated to the cloud provider.&nbsp;<\/p>\n\n\n\n<p>With this model, tasks such as provisioning, configuring, maintaining, operating or scaling the server can be forgotten about. All the related billing phases can be simply managed when working with single functions or microservices.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-general-overview-of-an-event-architecture\"><strong>General overview of an event architecture<\/strong><\/h2>\n\n\n\n<p>Openbank has developed an event-based architecture that allows the decoupling of applications from each other. Broadly speaking, this architecture contains a group of data-producing applications and a group of data-consuming applications. Individuals can belong to one or the other, or to both groups.<\/p>\n\n\n\n<p>Both producers and consumers make intensive use of serverless technologies. A simplified diagram of the proposed architecture would be as follows:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh4.googleusercontent.com\/4R3fCSS05JsWKWcyxUZ0J5-QcnOpSL84T-l9xQWaBKCNmwhTqQLeO7ACUiFH4tPxn1F-h5bRuibolWdOJY1dEOlICnS95JLcYB2LPKKLfquDTgubjA_66TRKUQKKI71Yu9YdjONn\" alt=\"Diagram: A simplified diagram of the architecture proposed by Openbank.\"\/><figcaption>Serverless technologies can be used by both data-producing and data-consuming applications.<\/figcaption><\/figure>\n\n\n\n<p>The main players in this diagram are producers, the schema registry, and AWS Lambdas, although many other components are also necessary to the architecture.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-producers\"><strong>Producers<\/strong><\/h3>\n\n\n\n<p>Producers use a common client that sends messages to the Kinesis streams. The messages, schemas, and metadata are sent in the <a href=\"https:\/\/avro.apache.org\/\" target=\"_blank\" aria-label=\" (opens in a new tab)\" rel=\"noreferrer noopener\" class=\"ek-link\">Avro serialization format<\/a>.<\/p>\n\n\n\n<p>Avro is a row-oriented remote procedure call and data serialization framework developed within Apache&#8217;s Hadoop project. It uses the JSON format for defining data types and protocols, serializing data in a compact binary format. The three heading bytes are the identifier of the schema in the schema registry.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-the-schema-registry\"><strong>The Schema Registry<\/strong><\/h3>\n\n\n\n<p>This registry provides a metadata serving layer with a RESTful interface for storing and retrieving Avro, JSON Schema, and Protobuf schemas. The registry stores a versioned history of all schemas based on a specified subject name strategy, provides multiple compatibility settings and allows the evolution of schemas. <\/p>\n\n\n\n<p>The schema registry provides serializers that plug into Apache Kafka clients that handle schema storage and retrieval for Kafka messages sent in any of the supported formats.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-the-aws-lambda\"><strong>The AWS Lambda<\/strong><\/h3>\n\n\n\n<p><a href=\"https:\/\/aws.amazon.com\/es\/lambda\/\" target=\"_blank\" aria-label=\" (opens in a new tab)\" rel=\"noreferrer noopener\" class=\"ek-link\">AWS Lambda is an AWS service<\/a> that allows code to be executed in various languages \u200b\u200bsuch as Python, node.js, Go, Java, Ruby or Powershell without worrying about managing infrastructure. The system has a multitude of triggers that go from API gateways to events in S3 or Kinesis messages.<\/p>\n\n\n\n<p>Tasks that range from simple scripts that execute based on events to <a href=\"https:\/\/www.codemotion.com\/magazine\/dev-hub\/backend-dev\/interview-eugenio-petulla-uses-of-wordpress-rest-apis-also-aimed-at-non-developers\/\" target=\"_blank\" rel=\"noopener\">REST APIs<\/a> through lambdas can be executed.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-the-dynamodb\"><strong>The DynamoDB&nbsp;<\/strong><\/h3>\n\n\n\n<p>It is a fully managed key\/value NOSQL database that offers throughputs below 10 milliseconds at any scale. It provides a flexible pricing model, a stateless connection model that works seamlessly with serverless, and consistent response time even as your database scales to enormous size. It can be interesting to match its characteristics with other NOSQL databases such as MongoDB.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-kinesis-the-aws-fully-managed-streaming-service\"><strong>Kinesis: the AWS fully-managed streaming service&nbsp;<\/strong><\/h2>\n\n\n\n<p>Real-time data come in an almost infinite variety of formats. All of these need to be treated in the same way in today&#8217;s information-based systems.<\/p>\n\n\n\n<p><a href=\"https:\/\/aws.amazon.com\/es\/kinesis\/\" target=\"_blank\" aria-label=\" (opens in a new tab)\" rel=\"noreferrer noopener\" class=\"ek-link\">Amazon Kinesis<\/a> makes it easy to collect, process, and analyze in real-time, streaming new data formats such as website clickstreams or IoT streams, together with classical application logs, texts, audios, and videos, without the need to manage the related infrastructure.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-aws-kinesis-vs-kafka\">AWS <strong>Kinesis Vs. Kafka&nbsp;<\/strong><\/h3>\n\n\n\n<p><a href=\"https:\/\/www.codemotion.com\/magazine\/dev-hub\/big-data-analyst\/interview-stefano-pampaloni-big-data-stream-analytics-without-code\/\" target=\"_blank\" rel=\"noopener\">Data streams<\/a> are often managed through Kafka Streams, a kafka-based library for building streaming applications that transform inputs into database calls, API calls, or Kafka items. The library sports a concise code structure, a distributed architecture, and a fault-tolerant approach.<\/p>\n\n\n\n<p>AWS offers Kinesis in place of Kafka Streams. It is interesting to take a look at the main differences between Kafka Streams and Kinesis.<\/p>\n\n\n\n<p>Kafka requires the organization to book <a href=\"https:\/\/www.codemotion.com\/magazine\/dev-hub\/devops-engineer\/software-operability-devops\/\" target=\"_blank\" rel=\"noopener\">DevOps<\/a> time to manage clusters, while Kinesis comes in a fully-managed version. This means Kafka looks more flexible, but that comes at a cost. Absolute performances depend heavily on the use case.<\/p>\n\n\n\n<p>AWS Kinesis is fully compliant with the AWS structure, allowing data to be analyzed by lambdas and processing to be paid for by use.<\/p>\n\n\n\n<p>Kafka Streaming allows functional aggregations and mutations to be performed.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-kinesis-data-analytics\"><strong>Kinesis data analytics<\/strong><\/h3>\n\n\n\n<p>AWS Kinesis Analytics allows for the performance of SQL-like queries on data. This module runs flink jobs without having to manage a Hadoop cluster and can be used to do window operations on streams inside the proposed project.<\/p>\n\n\n\n<p>Further components are shown in the functional diagram of this project: Glue, Athena, and the API Gateway.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-glue\"><strong>Glue<\/strong><\/h3>\n\n\n\n<p>Glue allows Spark jobs to be run in a serverless way, without the need to manage a Hadoop cluster. Glue also has a fully managed metastore and a crawler to retrieve data.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-athena\"><strong>Athena<\/strong><\/h3>\n\n\n\n<p>This is the AWS serverless version of Apache Presto. Among other things, Athena allows queries to be launched on files in S3 buckets; thanks to federated queries, tables from different databases can be joined.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-api-gateway\"><strong>API Gateway<\/strong><\/h3>\n\n\n\n<p>A frequent way to transform Http req\/res into events that a lambda can handle.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-case-study-feeding-read-apis\"><strong>Case Study: Feeding Read APIs<\/strong><\/h2>\n\n\n\n<p>A good way to illustrate part of the event-based architecture is to&nbsp; focus on the part of the architecture that powers the reading APIs.<\/p>\n\n\n\n<p>In the example below, the payment module will send a message after making a payment: it is saved in the DynamoDB, so the reading APIs can consult it.<\/p>\n\n\n\n<p>The process to power the DynamoDB responds to the following diagram.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh4.googleusercontent.com\/4NaWpxlpnm5o03zuVFmJZO8812a2p7zm7DMfiHXMTc9_3FoHxlxnoqg4ZEpdg1Q0cUyjAlKpmQCfcgbY_DB80nOYLJa_60lR4r2GJhJIwPKiRSpoW9pP0qcS3zIjCDn0gAwrVxqF\" alt=\"Diagram detailing the API-powering part of the architecture.\n\nDescripci\u00f3n generada autom\u00e1ticamente\"\/><figcaption>DynamoDB is central to lambda usage.<\/figcaption><\/figure>\n\n\n\n<p>The producer registers the message&#8217;s schema in the schema registry. The producer sends the message to the Kinesis stream. The lambda takes three steps:<\/p>\n\n\n\n<ul class=\"is-style-default wp-block-list\"><li>it receives the message<\/li><li>it retrieves the schema from the schema registry and parses the message<\/li><li>it saves it in DynamoDB optimized for reading.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-producer-the-code\"><strong>Producer: the code<\/strong><\/h3>\n\n\n\n<p>The data producer will take care of sending the messages in Avro format to Kinesis.<\/p>\n\n\n\n<p>First, the <a aria-label=\" (opens in a new tab)\" href=\"https:\/\/docs.confluent.io\/platform\/current\/schema-registry\/index.html\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"ek-link\">confluent<\/a> dependency is added in order to be able to first serialize the messages, then access the schemas of the schema registry and the dependency of the AWS Kinesis SDK to be able to send the messages.<\/p>\n\n\n\n<script src=\"https:\/\/pastebin.com\/embed_js\/h7Fs3HPf?theme=dark\"><\/script>\n\n\n\n<p>A simple POJO (Plain Old Java, or JavaScript, Object) is created to be the message that will be sent to Kinesis serialized in Avro.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh4.googleusercontent.com\/6tEEXVAOpDO3zLP8mtTZfo_2o--X_Bt5Vgp6YxINXfDJXueYJq1CQu4wemuA2oTGKqj5j430--e8Mx6GjUfMsSBStnpsndry5_QgQ4uVKwXyQFPrVxFRhOaTnSElqEbI_7ugW9rm\" alt=\"\"\/><\/figure>\n\n\n\n<p>The producer code is:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh4.googleusercontent.com\/5qN5CT4TENdM51MKJMBWsr42n2X45R4Br9UBur5pOJOGqSe64sAB1q1xxJaCEiBkwx1QzT8EScpCI0H5KCplNgXlBIDuB5xfq9R9okM8MKyXufj-NEvuwZrgnZMJsltZFk5Q-dmn\" alt=\"\"\/><\/figure>\n\n\n\n<p>The process continues step-by-step:&nbsp;<\/p>\n\n\n\n<p>The first thing to do is to initialize the serializer for the message in Avro format:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh6.googleusercontent.com\/ZH5G5twOPg0Un7VfTFWc4_3OJJCujLJS-5ugMZ7kfA1thlCc1e67b-SKwwMzX9B31LpOClMY37ksJ-uBvilLr3OaZ4taFIEPxNz9IKAKmrHzPTvNRTrN0sWyMZCoaSx2pG3qN2iM\" alt=\"\"\/><\/figure>\n\n\n\n<p>The producer can be configured so that at the time of serializing Avro it generates the schema in the schema registry, or the schemas can be created in advance in the schema registry.<\/p>\n\n\n\n<p>The message to Avro is serialized:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh3.googleusercontent.com\/5tuMhbDPzO71LuCJfHH1sV68niCW_8kqHSyRxHkrmj37Ylq7qjlhnE3H27oQSZwD_zzrpbzEpECAVKEYPDnYiAJELpIGUfX8gPuJNh99peANnQlhmxywL21UifyV7MT-VBjM_k8j\" alt=\"\"\/><\/figure>\n\n\n\n<p>Then initialize the Kinesis client:&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh5.googleusercontent.com\/2xMXpftBJ0WvBi7vpzwgE_rsn03vIftGGJdBJShpyyv3STIQPf8uzYdW-KkEY4mLo2MqoGiaHsmbBYPgxGt6iEsTwSRG7R0ROF2vIsz5mCx2BGWP73COp1JWuieE0SsjV0qw2ijg\" alt=\"\"\/><\/figure>\n\n\n\n<p>And send the Avro to AWS Kinesis.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh4.googleusercontent.com\/d3zyqSHmRl4zWNprzm0TVpR-qnr12Uy048rTeAODWN6VMipOnPA7C_9rgXiaiFPMl0S6e-RgYFmI6WJaij-1yr7wT_POPNv5wPBW5kpscQ0LoIJDLFCNWkIA29xtaK4vPEoOC163\" alt=\"\"\/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-lambda-function-the-code\"><strong>Lambda function: the code<\/strong><\/h3>\n\n\n\n<p>A <a href=\"https:\/\/aws.amazon.com\/es\/lambda\/\" target=\"_blank\" aria-label=\" (opens in a new tab)\" rel=\"noreferrer noopener\" class=\"ek-link\">lambda function<\/a> will take care of receiving the messages in Avro format, passing them to JSON and saving them into the DynamoDB. First, it will receive the message in Avro containing, in its first three bytes, the schema id in the schema registry.&nbsp;<\/p>\n\n\n\n<p>The confluent library will access the schema in the schema registry from the id and will parse the message that will be passed to JSON to save in DynamoDB.<\/p>\n\n\n\n<p>To be able to deserialize the data, the confluent dependencies must be added.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh5.googleusercontent.com\/v62ZM03KKV-UPJjrw2pd-BYZeIkpfpGy0hc6HeIkqYYUpv0bv0k8pvEJN3ouP4j0QG6b14-NervCvdtLeb24AHaZTdHOsd243dmoyFRfRmVAIqK5R9xGDMNXXQgECCn96iLGPGJh\" alt=\"\"\/><\/figure>\n\n\n\n<p>Next, it\u2019s time for the lambda function, Kinesis, and DynamoDB dependencies:&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh6.googleusercontent.com\/h67nNzi1i_oAtLjaLzJH1QQb5uNjpetAATOvaH-NuuFn7WOjU243AgMSueD_rPcK1sJpqy_88i0uxvdLEwu85BywktdNutjEdzbWC1v1aWhaVFHV1OvBz_1CLxuSc5CHugoQ_uWu\" alt=\"\"\/><\/figure>\n\n\n\n<p>Finally, the JSON dependency.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh6.googleusercontent.com\/VHbB8uvghBiLW0GOHdgqYV5_53sgfJRn01rzhNc-zfmw5rwOtYOH94-U0T5oc0NCFYGyY0q3LyBzvRqIxkTpNQAOBqJPPnQZzRcto4KQVVJKrZ5iJ6FpoiXRlUdwuoS5PU8iThNf\" alt=\"\"\/><\/figure>\n\n\n\n<p>Lambdas in AWS require the reception of an Uber-Jar with all the dependencies in it. Use the <a href=\"https:\/\/maven.apache.org\/plugins\/maven-shade-plugin\/\">maven-shade-plugin<\/a> for this task.<\/p>\n\n\n\n<p>The lambda code looks like this:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh4.googleusercontent.com\/Kjr-rFlmUi12JK3Ns_WKPDx3Ptvxee51MLBapMJTHhFPFqPZoWDaW3ra0oDm1FU4R1qAGSt0qvee8HuoCccFcbZi_0fSghUMFm8yttsq5-bf-KMSeb1QA5Rp--LyQz1kklNqhT1r\" alt=\"\"\/><\/figure>\n\n\n\n<p>Here is the code for the Lambda function:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh3.googleusercontent.com\/Gp2hmSmNm0owozpXGE77LFXYVaYE6dSgFKWSB7Vol7DWvuvDYAm0KAXIh4jBBoYiknJCBkdiXLOiGAuGJ6tgSnATmKeVOtuB0EtSAQOwb86vtCXnB54i7luKevt6Se8FWviFpskS\" alt=\"\"\/><\/figure>\n\n\n\n<p>Now, it\u2019s time to implement the lambda interface for Kinesis.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh5.googleusercontent.com\/b_d1wK6fveOzTkJv-6NqCtZuc96erQY4w7RmZDsxJc-z3o16KvPu6MJPchGTBEIwVqqxP9ecAuI9Ug-ztpiymgTBs2NtngXQFRcffCaDb95PzK3oyrHdmi-Uxoqe9k80hOKL2RJB\" alt=\"\"\/><\/figure>\n\n\n\n<p>That means implementing the method:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh5.googleusercontent.com\/dI7Wy86L6MvNXqePlmy0LB8AiUpB7Kv-YdnfaeyZs10A8_iV8U0aTJnFGRcLbWZ6phH1lhNJY34MNv7XXfKugXjckidvx-DyYFJxh0FXjYCPwjGwtM_q_Zv2n-Q9e92YOimW7CxF\" alt=\"\"\/><\/figure>\n\n\n\n<p>This method will be in charge of receiving the calls with the AWS Kinesis messages:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh6.googleusercontent.com\/ihM-C56ZThXRQXMhZwL3LSXyOuA5f2fqLVCsUZx9FxcxAYL3qwFF3NTo9VksT2Tg4TZlMao64EWnzBk_-XzRyE09cGp0KNGcgaiA8gxPgi1iwnaUyfxEfxMFY4aYpojRZITvD91a\" alt=\"\"\/><\/figure>\n\n\n\n<p>The code will simply deserialize the received messages and save them in a POJO object that is annotated with DynamoDB annotations and will save the messages in a table.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh6.googleusercontent.com\/HxVcZKVddJbNshRAxmJxjI9W6yxjd1v6NLEGChZMBlS2qqNVPEyAMV6WOzZLfLf1xSbrGvKRZp5MZBZ7vR33ICfeEzLsx7CZj4vhek4p5j5adiMF7LcJmebbbJVHW4LXh6uGCW3L\" alt=\"\"\/><\/figure>\n\n\n\n<p>The most important annotations are:<\/p>\n\n\n\n<ul class=\"is-style-default wp-block-list\"><li>DynamoDBTable: contains the name of the dynamo table.<\/li><li>DynamoDBHashKey: the partition key.<\/li><li>DynamoDBAttribute: the attributes.<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-setting-up-serverless-on-aws\"><strong>Setting up serverless on AWS<\/strong><\/h2>\n\n\n\n<p>It\u2019s better to define all our infrastructure as code with cloudformation, but in this case to keep it simple we will define the infrastructure with the AWS console.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-configuring-aws-kinesis\"><strong>Configuring AWS Kinesis<\/strong><\/h3>\n\n\n\n<p>After accessing the AWS account and going to the Kinesis section, click on the option \u2018Create data stream\u2019.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh6.googleusercontent.com\/_MDfjpAIBaGzzKid9SM_5r4_nzR33MEbOjr9OzMq6WCytwiu4RrpXKX297HLXTYQBvxaqXzj_ao-Qd1BO7Q67cOCB5gzTSZsap6h8did2of0yVtDwhsnaqRUn2MCYNMEYRK6TckX\" alt=\"User interface: Amazon Kinesis data stream creation\"\/><figcaption>AWS Kinesis has an easy configuration interface.<\/figcaption><\/figure>\n\n\n\n<p>The configuration screen of the stream appears; here the name and the number of shards desired are to be specified. Then click on &#8220;Create data stream&#8221; and the stream is created!<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh4.googleusercontent.com\/hl-MHQg5BlRruL75A7BbEUTm3Ns4D0VUjKC19VF_8OfCj53RHTpMb6nDu6ETYfaudHijkEtMZi-DYcuxRYMN2WTTFjklb8iHpLicmwHmw_ZsDZN2QTuKQMX0m9iIf-iSlwQDf9-G\" alt=\"User interface: Configuring Data Streams in Amazon Kinesis.\"\/><figcaption>Specify the name and the number of shards desired for the data stream.<\/figcaption><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-configuring-dynamodb\"><strong>Configuring DynamoDB<\/strong><\/h3>\n\n\n\n<p>The DynamoDB section allows for creating tables.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh5.googleusercontent.com\/-qaLbEQbgEtTIg2IMVqcr5_EN_ehjLHEIObSae54if_RICqBf5DiQOJK5JnwJ1ecWBe3KJXRsh6UKP4y9tkp2L2w2KWn5FiN0l07pBvFrAaHx1q9Gg6-PaaXFtkPy769MHJAUmgh\" alt=\"User interface of Amazon Kinesis: Creating a table in DynamoDB.\"\/><figcaption>Creating a table is a one-click process on DynamoDB.<\/figcaption><\/figure>\n\n\n\n<p>The table name and the primary key are required.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh5.googleusercontent.com\/Qb51Uy8r4TjsRfn_fJC7PF-xot6dYZUU_QVvUccFVZe75ThxgRxD5DYynUAbBNvP2RsHK6h0_aA8ydpf5lWUSSyNdMf917N0_UjDOLp-_D86cDVh2kqtj2vyT3f-F8Es9jU2-w2u\" alt=\"User interface of Amazon Kinesis: the \u201cCreate table\u201d module.\"\/><figcaption>Input the table name and the primary key.<\/figcaption><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-configuring-the-lambda-function\"><strong>Configuring the Lambda function<\/strong><\/h3>\n\n\n\n<p>Load the code into the lambda function in the lambda section of the AWS console &#8211; just click the \u2018Create Function\u2019 button.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh6.googleusercontent.com\/ZRSermM-NbB_caZmlMlcCQkZZPKG4GoGScSJfsdfycU4oj3e_Gdh6VDxPRMKA9BTHOGEg40tKmVTBgtnfZNz61gwHa9AuO6CJe_1pVyITlESs0fYS-UtSHEWN8OpJk5byjuRZ1tj\" alt=\"Application: Configuring a lambda function in AWS Kinesis.\"\/><figcaption>The AWS console offers a direct link to the lambda-function section.<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh3.googleusercontent.com\/ZXBmPRNC7FmgCfrgh20bPIptGeYrMMXXlHnMHSJaOW3RyucbREj00ZeKoDAONKu6ZSNMtcS6WJf3XloEI1DYIezEvRAlNnkNdeM79feRpx0fMkI36fZjKX0WK3vLFpCGFFf8vvgR\" alt=\"User interface of Amazon Kinesis: Creating a lambda function.\"\/><figcaption>Choose \u2018Author from scratch\u2019 if you want to provideinformation about the lambda function.<\/figcaption><\/figure>\n\n\n\n<p>Name the function in the \u2018Author from Scratch\u2019 option, choosing Java 11 as the language.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh3.googleusercontent.com\/qr5xzBPReBynrQx00IwDlKejYTfIr0SDqLfhyxPqYFSpcnfKXpd5NmV6boYZPczDS-0tsAeOJsgq84d7IjX84rI9TJO7KpL9K6zUpJHttIRCob8S81vrXxKu-7fg6UCrYWBqgkKJ\" alt=\"User interface Amazon Kinesi: Coosing the language for Lambda runtime.\"\/><figcaption>Specify the runtime the code is written for.<\/figcaption><\/figure>\n\n\n\n<p>The next section is the Security section, to be compiled with an <a aria-label=\" (opens in a new tab)\" href=\"https:\/\/aws.amazon.com\/es\/iam\/\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"ek-link\">IAM role<\/a>; with both read and write permissions on DynamoDB, permissions to consume the AWS Kinesis stream and a lambda execution role.\u00a0<\/p>\n\n\n\n<p>Then hit the create button. The main configuration screen of the lambda function shows up.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh4.googleusercontent.com\/qPdc2uB9u5IdNLFwTAGTdDRK65Gw2RtRjW24TlCPR0FVFNb6dAe3U8r37mnqhLx75vDlX-Jh-A2mgBtj4biNuJ7NP8arTFw8-7GWknXBWj_HBgJepOPb1_B4BTthiqf6OgsSMqWi\" alt=\"User interface of Amazon Kinesis: The main configuration screen of the lambda function.\"\/><figcaption>The created lambda function page can be accessed easily.<\/figcaption><\/figure>\n\n\n\n<p>By pressing the \u2018+ Add trigger button\u2019,&nbsp; all possible triggers for a function are listed. Choose Kinesis in this case. A new form appears in which to choose the stream that has been previously created.<\/p>\n\n\n\n<p>The uber-jar that can be built with a simple command &#8211; <strong>mvn clean package <\/strong>&#8211; is uploaded in the code section. Put the handler in the \u2018Runtime settings section\u2019 and save it: the lambda function is already configured to receive messages from the stream, then write data in DynamoDB.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-conclusions\"><strong>Conclusions<\/strong><\/h2>\n\n\n\n<p>A very simple Proof of Concept that illustrates part of how to create a fully serverless event-based architecture has been demonstrated above. The creation of an event-based architecture is often highly complex in terms of both the configuration and operation of the platform.&nbsp;<\/p>\n\n\n\n<p>Thanks to the AWS serverless approach, these complexities are greatly reduced and the developer can spend the largest part of his\/her time building functionality, thus providing real business value.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The project Openbank is the 100% digital bank of the Santander Group, currently undergoing a technological transformation and international expansion. The work is organized in a startup-like format, using agile methodologies to take client experience to the next level.&nbsp; The Netherlands, Germany and Portugal were OpenBank\u2019s flagships in 2019, with&nbsp; Argentina as the next target,&#8230; <a class=\"more-link\" href=\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/\">Read more<\/a><\/p>\n","protected":false},"author":8,"featured_media":15014,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_editorskit_title_hidden":false,"_editorskit_reading_time":9,"_editorskit_is_block_options_detached":false,"_editorskit_block_options_position":"{}","_uag_custom_page_level_css":"","_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[5244],"tags":[71,75],"collections":[],"class_list":{"0":"post-14996","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-cloud","8":"tag-aws","9":"tag-serverless","10":"entry"},"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v26.9) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Serverless Event Processing on AWS Platform w\/ Kinesis -<\/title>\n<meta name=\"description\" content=\"Learn how to developed an event-based architecture for the decoupling of applications with AWS Kinesis, in this case study by Openbank.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Serverless Event Processing on AWS Platform w\/ Kinesis\" \/>\n<meta property=\"og:description\" content=\"Learn how to developed an event-based architecture for the decoupling of applications with AWS Kinesis, in this case study by Openbank.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/\" \/>\n<meta property=\"og:site_name\" content=\"Codemotion Magazine\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Codemotion.Italy\/\" \/>\n<meta property=\"article:published_time\" content=\"2021-05-28T07:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-01-05T19:02:44+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Leo Sorge\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@CodemotionIT\" \/>\n<meta name=\"twitter:site\" content=\"@CodemotionIT\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Leo Sorge\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"14 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/\"},\"author\":{\"name\":\"Leo Sorge\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/99fe84861e17797989e16094aac727d5\"},\"headline\":\"Serverless Event Processing on AWS Platform w\/ Kinesis\",\"datePublished\":\"2021-05-28T07:00:00+00:00\",\"dateModified\":\"2022-01-05T19:02:44+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/\"},\"wordCount\":1925,\"publisher\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg\",\"keywords\":[\"AWS\",\"Serverless\"],\"articleSection\":[\"Cloud\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/\",\"name\":\"Serverless Event Processing on AWS Platform w\/ Kinesis -\",\"isPartOf\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg\",\"datePublished\":\"2021-05-28T07:00:00+00:00\",\"dateModified\":\"2022-01-05T19:02:44+00:00\",\"description\":\"Learn how to developed an event-based architecture for the decoupling of applications with AWS Kinesis, in this case study by Openbank.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#primaryimage\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg\",\"contentUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg\",\"width\":1200,\"height\":628,\"caption\":\"Serverless Event Processing on AWS Platform w Kinesis\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.codemotion.com\/magazine\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"DevOps\",\"item\":\"https:\/\/www.codemotion.com\/magazine\/devops\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Cloud\",\"item\":\"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Serverless Event Processing on AWS Platform w\/ Kinesis\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#website\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/\",\"name\":\"Codemotion Magazine\",\"description\":\"We code the future. Together\",\"publisher\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.codemotion.com\/magazine\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#organization\",\"name\":\"Codemotion\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png\",\"contentUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png\",\"width\":225,\"height\":225,\"caption\":\"Codemotion\"},\"image\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/Codemotion.Italy\/\",\"https:\/\/x.com\/CodemotionIT\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/99fe84861e17797989e16094aac727d5\",\"name\":\"Leo Sorge\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/60ef89012822fdaeb014dc2b17dcc78c3a835e1dfe86ede053fa46a91926fd29?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/60ef89012822fdaeb014dc2b17dcc78c3a835e1dfe86ede053fa46a91926fd29?s=96&d=mm&r=g\",\"caption\":\"Leo Sorge\"},\"description\":\"I hold a degree in electronics. I talk and write about science and technology in both real and close-to-real worlds since 1976. I frankly believe that business plan and singularity are excellent starts for science-fiction stories.\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/author\/leo-sorge\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Serverless Event Processing on AWS Platform w\/ Kinesis -","description":"Learn how to developed an event-based architecture for the decoupling of applications with AWS Kinesis, in this case study by Openbank.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/","og_locale":"en_US","og_type":"article","og_title":"Serverless Event Processing on AWS Platform w\/ Kinesis","og_description":"Learn how to developed an event-based architecture for the decoupling of applications with AWS Kinesis, in this case study by Openbank.","og_url":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/","og_site_name":"Codemotion Magazine","article_publisher":"https:\/\/www.facebook.com\/Codemotion.Italy\/","article_published_time":"2021-05-28T07:00:00+00:00","article_modified_time":"2022-01-05T19:02:44+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg","type":"image\/jpeg"}],"author":"Leo Sorge","twitter_card":"summary_large_image","twitter_creator":"@CodemotionIT","twitter_site":"@CodemotionIT","twitter_misc":{"Written by":"Leo Sorge","Est. reading time":"14 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#article","isPartOf":{"@id":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/"},"author":{"name":"Leo Sorge","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/99fe84861e17797989e16094aac727d5"},"headline":"Serverless Event Processing on AWS Platform w\/ Kinesis","datePublished":"2021-05-28T07:00:00+00:00","dateModified":"2022-01-05T19:02:44+00:00","mainEntityOfPage":{"@id":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/"},"wordCount":1925,"publisher":{"@id":"https:\/\/www.codemotion.com\/magazine\/#organization"},"image":{"@id":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#primaryimage"},"thumbnailUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg","keywords":["AWS","Serverless"],"articleSection":["Cloud"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/","url":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/","name":"Serverless Event Processing on AWS Platform w\/ Kinesis -","isPartOf":{"@id":"https:\/\/www.codemotion.com\/magazine\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#primaryimage"},"image":{"@id":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#primaryimage"},"thumbnailUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg","datePublished":"2021-05-28T07:00:00+00:00","dateModified":"2022-01-05T19:02:44+00:00","description":"Learn how to developed an event-based architecture for the decoupling of applications with AWS Kinesis, in this case study by Openbank.","breadcrumb":{"@id":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#primaryimage","url":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg","contentUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg","width":1200,"height":628,"caption":"Serverless Event Processing on AWS Platform w Kinesis"},{"@type":"BreadcrumbList","@id":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/aws-kinesis-serverless-event-processing\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.codemotion.com\/magazine\/"},{"@type":"ListItem","position":2,"name":"DevOps","item":"https:\/\/www.codemotion.com\/magazine\/devops\/"},{"@type":"ListItem","position":3,"name":"Cloud","item":"https:\/\/www.codemotion.com\/magazine\/devops\/cloud\/"},{"@type":"ListItem","position":4,"name":"Serverless Event Processing on AWS Platform w\/ Kinesis"}]},{"@type":"WebSite","@id":"https:\/\/www.codemotion.com\/magazine\/#website","url":"https:\/\/www.codemotion.com\/magazine\/","name":"Codemotion Magazine","description":"We code the future. Together","publisher":{"@id":"https:\/\/www.codemotion.com\/magazine\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.codemotion.com\/magazine\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.codemotion.com\/magazine\/#organization","name":"Codemotion","url":"https:\/\/www.codemotion.com\/magazine\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/","url":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png","contentUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png","width":225,"height":225,"caption":"Codemotion"},"image":{"@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/Codemotion.Italy\/","https:\/\/x.com\/CodemotionIT"]},{"@type":"Person","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/99fe84861e17797989e16094aac727d5","name":"Leo Sorge","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/60ef89012822fdaeb014dc2b17dcc78c3a835e1dfe86ede053fa46a91926fd29?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/60ef89012822fdaeb014dc2b17dcc78c3a835e1dfe86ede053fa46a91926fd29?s=96&d=mm&r=g","caption":"Leo Sorge"},"description":"I hold a degree in electronics. I talk and write about science and technology in both real and close-to-real worlds since 1976. I frankly believe that business plan and singularity are excellent starts for science-fiction stories.","url":"https:\/\/www.codemotion.com\/magazine\/author\/leo-sorge\/"}]}},"featured_image_src":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis-600x400.jpg","featured_image_src_square":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis-600x600.jpg","author_info":{"display_name":"Leo Sorge","author_link":"https:\/\/www.codemotion.com\/magazine\/author\/leo-sorge\/"},"uagb_featured_image_src":{"full":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg",1200,628,false],"thumbnail":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis-150x150.jpg",150,150,true],"medium":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis-300x157.jpg",300,157,true],"medium_large":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis-768x402.jpg",768,402,true],"large":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis-1024x536.jpg",1024,536,true],"1536x1536":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg",1200,628,false],"2048x2048":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg",1200,628,false],"small-home-featured":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis.jpg",100,52,false],"sidebar-featured":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis-180x128.jpg",180,128,true],"genesis-singular-images":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis-896x504.jpg",896,504,true],"archive-featured":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis-400x225.jpg",400,225,true],"gb-block-post-grid-landscape":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis-600x400.jpg",600,400,true],"gb-block-post-grid-square":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2021\/05\/Serverless-Event-Processing-on-AWS-Platform-w-Kinesis-600x600.jpg",600,600,true]},"uagb_author_info":{"display_name":"Leo Sorge","author_link":"https:\/\/www.codemotion.com\/magazine\/author\/leo-sorge\/"},"uagb_comment_info":0,"uagb_excerpt":"The project Openbank is the 100% digital bank of the Santander Group, currently undergoing a technological transformation and international expansion. The work is organized in a startup-like format, using agile methodologies to take client experience to the next level.&nbsp; The Netherlands, Germany and Portugal were OpenBank\u2019s flagships in 2019, with&nbsp; Argentina as the next target,&#8230;&hellip;","lang":"en","_links":{"self":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts\/14996","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/comments?post=14996"}],"version-history":[{"count":7,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts\/14996\/revisions"}],"predecessor-version":[{"id":15106,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts\/14996\/revisions\/15106"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/media\/15014"}],"wp:attachment":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/media?parent=14996"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/categories?post=14996"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/tags?post=14996"},{"taxonomy":"collections","embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/collections?post=14996"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}