{"id":19641,"date":"2022-12-12T09:00:00","date_gmt":"2022-12-12T08:00:00","guid":{"rendered":"https:\/\/www.codemotion.com\/magazine\/?p=19641"},"modified":"2022-12-09T15:27:28","modified_gmt":"2022-12-09T14:27:28","slug":"is-apache-kafka-still-relevant","status":"publish","type":"post","link":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/","title":{"rendered":"Is Apache Kafka Still Relevant?"},"content":{"rendered":"\n<p>Things change quickly in the IT industry \u2013 so quickly that what one day is relevant can be obsolete the next. So what can be expected from a system launched back in 2011, like Apache Kafka?<\/p>\n\n\n\n<p>In this article, we look at whether this platform remains relevant <strong>a decade after its development<\/strong> and we explore its key functions and typical applications.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-brief-history-of-apache-kafka\">Brief history of Apache Kafka<\/h2>\n\n\n\n<p>In 2011, Kafka was born as an OpenSource platform at LinkedIn and remained in the Apache incubator until 2012.<\/p>\n\n\n\n<p>Initially, Kafka was developed to be used within the LinkedIn site, which had <a href=\"https:\/\/www.forbes.com\/sites\/stevenli1\/2020\/05\/11\/confluent-jay-kreps-kafka-4-billion-2020\/?sh=1a82e619709d\" class=\"ek-link\">low latency issues when handling high volumes of event data<\/a>. In the early 2010s, real time processing technologies didn\u2019t exist, and instead organizations had to rely on enterprise messaging or batch-based solutions.<\/p>\n\n\n\n<p>Kafka changed this thanks to its <a href=\"https:\/\/techbeacon.com\/app-dev-testing\/what-apache-kafka-why-it-so-popular-should-you-use-it\">superb data ingestion capabilities<\/a> (trillions at the time of writing). Currently, <a href=\"https:\/\/kafka.apache.org\/powered-by\" class=\"ek-link\">tens of thousands of organisations have adopted Kafka<\/a>, among them very popular real-time event-driven experiences, such as Netflix, PayPal, AirBnB and Pinterest.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-how-apache-kafka-works\">How Apache Kafka works<\/h2>\n\n\n\n<p>A typical Kafka environment consists of:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>One or more Producers, which write or publish data.<\/li>\n\n\n\n<li>A Kafka cluster containing one or more Brokers. Connecting to any one Broker means being connected to the entire cluster.<\/li>\n\n\n\n<li>Topics or data streams (similar to database tables).<\/li>\n\n\n\n<li>Partitions, where topics are split as separate nodes.<\/li>\n\n\n\n<li>One or more Consumers, which read data contained in a topic.<\/li>\n<\/ul>\n\n\n\n<p>A <a href=\"https:\/\/kafka.apache.org\/20\/documentation.html#api\">Kafka environment<\/a> works through five APIs:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>An Admin client.<\/li>\n\n\n\n<li>Producers API, used by producer apps to write data into topics.<\/li>\n\n\n\n<li>Consumers API, used by consuming apps to read topics.<\/li>\n\n\n\n<li>Connect API, has an import-export function as it enables data streaming between Kafka and other systems.<\/li>\n\n\n\n<li>Streams API, which transforms input topics into output topics in the form or real-time applications and micro-services.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-where-is-kafka-used-for\">Where is Kafka used for?<\/h2>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"518\" src=\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2020\/08\/BigData_2267x1146_trasparent-1024x518.png\" alt=\"Big Data\" class=\"wp-image-7724\" srcset=\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2020\/08\/BigData_2267x1146_trasparent-1024x518.png 1024w, https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2020\/08\/BigData_2267x1146_trasparent-300x152.png 300w, https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2020\/08\/BigData_2267x1146_trasparent-768x388.png 768w, https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2020\/08\/BigData_2267x1146_trasparent.png 1200w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Apache Kafka is good for big data.<\/figcaption><\/figure>\n\n\n\n<p>As a system, Kafka serves three main functions:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Real-time data processing.<\/li>\n\n\n\n<li>Accurate, durable, and sequential data storage.<\/li>\n\n\n\n<li>As a message broker, it lets applications publish or subscribe one or multiple event streams.<\/li>\n<\/ul>\n\n\n\n<p>This means that Kafka isn\u2019t only used to process and analyse real-time data, but also to respond to data events as they happen. <strong>As such, it\u2019s well-suited to power data-rich interactive ecosystems<\/strong> that rely on multiple layers of databases, events, and applications and who therefore need to integrate these layers to make optimal use of data.<\/p>\n\n\n\n<p>Its characteristics also make it popular to build applications on event-driven architectures, offering distinct advantages over traditional counterparts, namely scalability, fault tolerance, speed, and reliability.<\/p>\n\n\n\n<p>Another common use case is <a href=\"https:\/\/www.globenewswire.com\/en\/news-release\/2022\/01\/27\/2373968\/0\/en\/Event-Stream-Processing-Market-to-Touch-USD-4-156-Million-by-2027-with-a-CAGR-of-21-Report-by-Market-Research-Future-MRFR.html\">event streaming<\/a>, a booming market following the widespread adoption of<strong> remote or hybrid work models<\/strong> and the growing implementation of machine learning, artificial intelligence and Internet of Things in corporate environments.<\/p>\n\n\n\n<p>In summary, Kafka is used in environments with complex data needs, especially in large organizations with multiple departments and different lines of business, such as e-commerce, banking, cybersecurity, manufacturing, and social media.<\/p>\n\n\n\n<p>Specific applications include online and offline analytics, anomaly detection, trend analysis, platforms that handle interactive queries, payment processing, infrastructure monitoring, and financial trading, to mention just a few.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-real-time-data-processing\">Real-time data processing<\/h3>\n\n\n\n<p>Kafka excels at ingesting large volumes of real-time data from multiple sources. As a result, typical case use scenarios would be:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>&#8211; To build or to operate distributed applications with real-time streaming capabilities.<\/li>\n\n\n\n<li>&#8211; To build or to operate applications that need to function as real-time data pipelines.<\/li>\n\n\n\n<li>&#8211; In Big Data applications.<\/li>\n<\/ul>\n\n\n\n<p>Kafka\u2019s data processing capabilities provide an interface between different systems and databases. For example, let\u2019s take a supermarket with nationwide operations that generates data on sale transactions. Here, the store would typically have some software in place that monitors sales data. This software publishes data to Kafka, which then sorts it into ordered Topics or data streams (for example, sales between 7 a.m. and 9 a.m., sales under $100, sales transactions for product XYZ, etc.)<\/p>\n\n\n\n<p>Topics are turned into data streams using the Producer function, after which they can be read and fed to processing pipelines (<strong>think Spark or Storm<\/strong>) or to data lakes, through the Consumer function. Using the above example, Kafka would serve as the interface between data streams and the different departments that underpin the store\u2019s operations, for instance by providing real-time data relevant to the company\u2019s sales strategy, to its warehousing practices, or to forecast trends.<\/p>\n\n\n\n<p>Netflix is a real-life example of Kafka\u2019s data processing capabilities, since it uses the platform to ingest up to 500 billion events per day, using a setup that includes Java and REST APIs, ElasticSearch, Spark, multiple analytics engines, and Kafka as the structuring backbone of it all.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-messaging\">Messaging<\/h3>\n\n\n\n<p>As a <a href=\"https:\/\/talks.codemotion.com\/rest-in-peace-apache-kafka-is-alive\">message broker<\/a>, Kafka enables both queueing and publish-subscribe functionalities, and acts as the intermediary or bridge that enables communication between applications.<\/p>\n\n\n\n<p>Typical Kafka messaging flow would look like this:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>A Producer publishes messages to a Topic.<\/li>\n\n\n\n<li>Brokers store the messages in the form of serialized bytes. This is done in order making use of the message log structure.<\/li>\n\n\n\n<li>Consuming applications can read and subscribe those messages, as well as de-serialize them.<\/li>\n<\/ul>\n\n\n\n<p>There are additional things to note about this process. First, the Producer can create an unlimited number of topics, as long as each topic has a unique name. These topics are not stored in a single location or cluster, but instead subdivided into Partitions, which contains multiple messages or records.<\/p>\n\n\n\n<p>Second, messages or records published to topic partitions don\u2019t have a message ID, but rather an offset. The offset is <strong>assigned in an incremental manner<\/strong>, which makes sense given that Kafka records cannot be deleted or altered (they&#8217;re append-only).<\/p>\n\n\n\n<p>Lastly, Kafka persists and replicates messages to one multiple brokers over a given period of time, which is set by default to 7 days but can be configured depending on needs. This contributes to high durability and high reliability, which make Kafka relevant as a message broker.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-log-aggregation\">Log Aggregation<\/h3>\n\n\n\n<p>Apache Kafka doesn\u2019t use application logs as such, and instead conceptualizes a log as the basic structure upon which a database can be built. In other words, log aggregation in Kafka is the process of appending data in a sequential manner.<\/p>\n\n\n\n<p>In this respect, Kafka displays offers some advantages over other log aggregation tools. Instead of simply collecting log files and placing them in a centralized location for further processing, Kafka is capable of abstracting file details and presenting event or log data as a clean stream of messages. As a result, users benefit from low latency, which facilitates the processing and consumption of distributed data.<\/p>\n\n\n\n<p>This last aspect is crucial, since centralized log aggregation systems (CLS) run into performance issues once they need to operate in complex environments, such as multi-cloud or distributed applications \u2013 think log bursts or the inability to meet uptime as defined in SLAs. This isn\u2019t the case with Kafka, which certainly adds to its relevance.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-when-kafka-shouldn-t-be-used-for\">When Kafka shouldn\u2019t be used for<\/h2>\n\n\n\n<p>Despite its remarkable capabilities, Kafka doesn\u2019t suit every possible scenario. The technology has limitations in:<\/p>\n\n\n\n<p>&#8211; IoT applications where hard real-time with sub-millisecond latency and no spikes are a must. When we say Kafka delivers real-time processing, we mean soft or near-real time.<\/p>\n\n\n\n<p>&#8211; Settings that process safety-critical data, such as vehicle-to-vehicle communication, robotics, industrial process controllers, or medical systems. The reason being that Kafka is not a deterministic system.<\/p>\n\n\n\n<p>&#8211; ETL pipelines in tasks other than supporting real-time analytics.<\/p>\n\n\n\n<p>&#8211; Kafka doesn\u2019t replace a <strong>traditional <\/strong><a href=\"https:\/\/www.codemotion.com\/magazine\/backend\/10-commonly-used-sql-commands-for-solid-databases\/\" target=\"_blank\" aria-label=\"database  (opens in a new tab)\" rel=\"noreferrer noopener\" class=\"ek-link\"><strong>database<\/strong> <\/a>used for persistent storage, since it lacks important elements like indexes and transactions.<\/p>\n\n\n\n<p>&#8211; Systems that only handle a few thousand messages per day. Kafka will still serve its purpose, but implementing it outside of Big Data environments is overkill.<\/p>\n\n\n\n<p>This last point brings us onto the issue of Kafka alternatives, such as RabbitMQ.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-apache-kafka-vs-rabbitmq\">Apache Kafka vs RabbitMQ<\/h2>\n\n\n\n<p>RabbitMQ is an open source message broker. As a messaging framework, its key function is to provide point-to-point message queueing to so that server resources aren\u2019t overwhelmed, causing delayed responses. Rabbit is also used to balance loads across different workers or to distribute messages to different consumers.<\/p>\n\n\n\n<p>The message flow is similar to Kafka, as data goes from Producer to Consumer via a Broker, but these systems can\u2019t be used interchangeably.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-main-differences\">Main differences<\/h3>\n\n\n\n<p>&#8211; In terms of architecture, Kafka uses a partitioned log model, which is more complex than the messaging queue architecture used by Rabbit.<\/p>\n\n\n\n<p>&#8211; Message retention isn\u2019t a feature in Rabbit, since messages are deleted upon consumption.<\/p>\n\n\n\n<p>&#8211; Kafka replays topics as per the specified retention policy, so multiple consumers can subscribe or read the same message, which isn\u2019t possible with Rabbit due to default message deletion.<\/p>\n\n\n\n<p>&#8211; You can assign high-priority messages in RabbitMQ, but not in Kafka, where message ordering is embedded in its architecture.<\/p>\n\n\n\n<p>&#8211; Scalability in Kafka means distributing partitions in multiple servers, whereas in Rabbit is achieved by increasing the number of consumers.<\/p>\n\n\n\n<p>&#8211; Lastly, although both use binary protocols, Kafka relies on TCP while Rabbit supports AMQP as well as STOMP and MQTT through plugins.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-which-one-should-you-choose\">Which one should you choose?<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"683\" src=\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/02\/Data-Lakehouse-2-1024x683.jpg\" alt=\"\" class=\"wp-image-17086\" srcset=\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/02\/Data-Lakehouse-2-1024x683.jpg 1024w, https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/02\/Data-Lakehouse-2-300x200.jpg 300w, https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/02\/Data-Lakehouse-2-768x512.jpg 768w, https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/02\/Data-Lakehouse-2-1536x1024.jpg 1536w, https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/02\/Data-Lakehouse-2-600x400.jpg 600w, https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/02\/Data-Lakehouse-2.jpg 1920w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>This depends on your needs, so take the following considerations into account:<\/p>\n\n\n\n<p>Kafka is the gold standard for <strong>event streaming and was designed to be used in stream processing scenarios.<\/strong> Therefore, consider Kafka if you require log aggregation, metrics, event sourcing, activity tracking, applications with a stream history, or multi-stage pipelines.<\/p>\n\n\n\n<p>On the other hand, RabbitMQ\u2019s main functions are delivering a quick response, sharing loads, and efficient message delivery. This makes Rabbit best suited to \u201cLittle Data\u201d scenarios, whereas Kafka delivers in Big Data applications.<\/p>\n\n\n\n<p>In terms of performance, they both do well in their respective ranges: one of the main Kafka advantages is that <strong>it can handle 1 million messages \/ second<\/strong>, whereas RabbitMQ\u2019s upper limit is around 10,000 \/ second.<\/p>\n\n\n\n<p>Undoubtedly, Kafka\u2019s highlight is that it\u2019s built to perform at scale. There are no partitions in Rabbit, so if you\u2019re looking for both scalability and redundancy, Kafka is the way to go.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-conclusions\">Conclusions<\/h2>\n\n\n\n<p>Kafka is a powerful technology used by 80% of Fortune 100 companies, and for a good reason. The platform offers reliability, quick scalability and constant high performance in stream processing, persistent storage, and messaging being the idea choice for soft real-time applications in enterprise environments with complex data needs.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Things change quickly in the IT industry \u2013 so quickly that what one day is relevant can be obsolete the next. So what can be expected from a system launched back in 2011, like Apache Kafka? In this article, we look at whether this platform remains relevant a decade after its development and we explore&#8230; <a class=\"more-link\" href=\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/\">Read more<\/a><\/p>\n","protected":false},"author":64,"featured_media":19645,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_editorskit_title_hidden":false,"_editorskit_reading_time":7,"_editorskit_is_block_options_detached":false,"_editorskit_block_options_position":"{}","_uag_custom_page_level_css":"","_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[16],"tags":[9980],"collections":[],"class_list":{"0":"post-19641","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-big-data","8":"tag-apache-kafka","9":"entry"},"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v26.9) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Is Apache Kafka Still Relevant? - Codemotion Magazine<\/title>\n<meta name=\"description\" content=\"Apache Kafka is a powerful technology used by many Fortune 100 companies. It offers reliability, quick scalability, and high performance.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Is Apache Kafka Still Relevant?\" \/>\n<meta property=\"og:description\" content=\"Apache Kafka is a powerful technology used by many Fortune 100 companies. It offers reliability, quick scalability, and high performance.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/\" \/>\n<meta property=\"og:site_name\" content=\"Codemotion Magazine\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Codemotion.Italy\/\" \/>\n<meta property=\"article:published_time\" content=\"2022-12-12T08:00:00+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1386\" \/>\n\t<meta property=\"og:image:height\" content=\"756\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Codemotion\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@CodemotionIT\" \/>\n<meta name=\"twitter:site\" content=\"@CodemotionIT\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Codemotion\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/\"},\"author\":{\"name\":\"Codemotion\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/201bb98b02412383686cced7521b861c\"},\"headline\":\"Is Apache Kafka Still Relevant?\",\"datePublished\":\"2022-12-12T08:00:00+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/\"},\"wordCount\":1723,\"publisher\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg\",\"keywords\":[\"apache kafka\"],\"articleSection\":[\"Big Data\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/\",\"name\":\"Is Apache Kafka Still Relevant? - Codemotion Magazine\",\"isPartOf\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg\",\"datePublished\":\"2022-12-12T08:00:00+00:00\",\"description\":\"Apache Kafka is a powerful technology used by many Fortune 100 companies. It offers reliability, quick scalability, and high performance.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#primaryimage\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg\",\"contentUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg\",\"width\":1386,\"height\":756,\"caption\":\"Data volume analysis and computer science industry.3d illustration\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.codemotion.com\/magazine\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AI\/ML\",\"item\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Big Data\",\"item\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Is Apache Kafka Still Relevant?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#website\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/\",\"name\":\"Codemotion Magazine\",\"description\":\"We code the future. Together\",\"publisher\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.codemotion.com\/magazine\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#organization\",\"name\":\"Codemotion\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png\",\"contentUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png\",\"width\":225,\"height\":225,\"caption\":\"Codemotion\"},\"image\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/Codemotion.Italy\/\",\"https:\/\/x.com\/CodemotionIT\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/201bb98b02412383686cced7521b861c\",\"name\":\"Codemotion\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/cropped-codemotionlogo-150x150.png\",\"contentUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/cropped-codemotionlogo-150x150.png\",\"caption\":\"Codemotion\"},\"description\":\"Articles wirtten by the Codemotion staff. Tech news, inspiration, latest treends in software development and more.\",\"sameAs\":[\"https:\/\/x.com\/CodemotionIT\"],\"url\":\"https:\/\/www.codemotion.com\/magazine\/author\/codemotion-2\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Is Apache Kafka Still Relevant? - Codemotion Magazine","description":"Apache Kafka is a powerful technology used by many Fortune 100 companies. It offers reliability, quick scalability, and high performance.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/","og_locale":"en_US","og_type":"article","og_title":"Is Apache Kafka Still Relevant?","og_description":"Apache Kafka is a powerful technology used by many Fortune 100 companies. It offers reliability, quick scalability, and high performance.","og_url":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/","og_site_name":"Codemotion Magazine","article_publisher":"https:\/\/www.facebook.com\/Codemotion.Italy\/","article_published_time":"2022-12-12T08:00:00+00:00","og_image":[{"width":1386,"height":756,"url":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg","type":"image\/jpeg"}],"author":"Codemotion","twitter_card":"summary_large_image","twitter_creator":"@CodemotionIT","twitter_site":"@CodemotionIT","twitter_misc":{"Written by":"Codemotion","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#article","isPartOf":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/"},"author":{"name":"Codemotion","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/201bb98b02412383686cced7521b861c"},"headline":"Is Apache Kafka Still Relevant?","datePublished":"2022-12-12T08:00:00+00:00","mainEntityOfPage":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/"},"wordCount":1723,"publisher":{"@id":"https:\/\/www.codemotion.com\/magazine\/#organization"},"image":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#primaryimage"},"thumbnailUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg","keywords":["apache kafka"],"articleSection":["Big Data"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/","url":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/","name":"Is Apache Kafka Still Relevant? - Codemotion Magazine","isPartOf":{"@id":"https:\/\/www.codemotion.com\/magazine\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#primaryimage"},"image":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#primaryimage"},"thumbnailUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg","datePublished":"2022-12-12T08:00:00+00:00","description":"Apache Kafka is a powerful technology used by many Fortune 100 companies. It offers reliability, quick scalability, and high performance.","breadcrumb":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#primaryimage","url":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg","contentUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg","width":1386,"height":756,"caption":"Data volume analysis and computer science industry.3d illustration"},{"@type":"BreadcrumbList","@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/is-apache-kafka-still-relevant\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.codemotion.com\/magazine\/"},{"@type":"ListItem","position":2,"name":"AI\/ML","item":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/"},{"@type":"ListItem","position":3,"name":"Big Data","item":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/big-data\/"},{"@type":"ListItem","position":4,"name":"Is Apache Kafka Still Relevant?"}]},{"@type":"WebSite","@id":"https:\/\/www.codemotion.com\/magazine\/#website","url":"https:\/\/www.codemotion.com\/magazine\/","name":"Codemotion Magazine","description":"We code the future. Together","publisher":{"@id":"https:\/\/www.codemotion.com\/magazine\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.codemotion.com\/magazine\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.codemotion.com\/magazine\/#organization","name":"Codemotion","url":"https:\/\/www.codemotion.com\/magazine\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/","url":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png","contentUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png","width":225,"height":225,"caption":"Codemotion"},"image":{"@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/Codemotion.Italy\/","https:\/\/x.com\/CodemotionIT"]},{"@type":"Person","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/201bb98b02412383686cced7521b861c","name":"Codemotion","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/image\/","url":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/cropped-codemotionlogo-150x150.png","contentUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/cropped-codemotionlogo-150x150.png","caption":"Codemotion"},"description":"Articles wirtten by the Codemotion staff. Tech news, inspiration, latest treends in software development and more.","sameAs":["https:\/\/x.com\/CodemotionIT"],"url":"https:\/\/www.codemotion.com\/magazine\/author\/codemotion-2\/"}]}},"featured_image_src":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339-600x400.jpg","featured_image_src_square":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339-600x600.jpg","author_info":{"display_name":"Codemotion","author_link":"https:\/\/www.codemotion.com\/magazine\/author\/codemotion-2\/"},"uagb_featured_image_src":{"full":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg",1386,756,false],"thumbnail":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339-150x150.jpg",150,150,true],"medium":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339-300x164.jpg",300,164,true],"medium_large":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339-768x419.jpg",768,419,true],"large":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339-1024x559.jpg",1024,559,true],"1536x1536":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg",1386,756,false],"2048x2048":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg",1386,756,false],"small-home-featured":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339.jpg",100,55,false],"sidebar-featured":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339-180x128.jpg",180,128,true],"genesis-singular-images":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339-896x504.jpg",896,504,true],"archive-featured":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339-400x225.jpg",400,225,true],"gb-block-post-grid-landscape":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339-600x400.jpg",600,400,true],"gb-block-post-grid-square":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2022\/12\/iStock-1148438339-600x600.jpg",600,600,true]},"uagb_author_info":{"display_name":"Codemotion","author_link":"https:\/\/www.codemotion.com\/magazine\/author\/codemotion-2\/"},"uagb_comment_info":0,"uagb_excerpt":"Things change quickly in the IT industry \u2013 so quickly that what one day is relevant can be obsolete the next. So what can be expected from a system launched back in 2011, like Apache Kafka? In this article, we look at whether this platform remains relevant a decade after its development and we explore&#8230;&hellip;","lang":"en","_links":{"self":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts\/19641","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/users\/64"}],"replies":[{"embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/comments?post=19641"}],"version-history":[{"count":9,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts\/19641\/revisions"}],"predecessor-version":[{"id":19656,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts\/19641\/revisions\/19656"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/media\/19645"}],"wp:attachment":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/media?parent=19641"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/categories?post=19641"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/tags?post=19641"},{"taxonomy":"collections","embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/collections?post=19641"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}