LJC Virtual Meetup: Robin Moffatt's Apache Kafka Series
Description changed:
*** This is a placeholder for the event which is being run on Eventbrite. Please sign up with your full name here - https://www.eventbrite.co.uk/e/ljc-virtual-meetup-robin-moffatts-apache-kafka-series-tickets-103321752142 ***
Join the LJC over the next 3 weeks for a series of Apache Kafka talks by Robin Moffatt. Robin is a Senior Developer Advocate at Confluent, the company founded by the original creators of Apache Kafka, as well as an Oracle ACE Director (Alumnus).
Talk 1: Apache Kafka and ksqlDB in Action: Let’s Build a Streaming Data Pipeline! - Wednesday 29th April at 7pm
Talk 2: From Zero to Hero with Kafka Connect - Wednesday 6th May at 7pm
Talk 3: Introduction to ksqlDB - Wednesday 13th May at 7pm
Apache Kafka and ksqlDB in Action : Let’s Build a Streaming Data Pipeline!
Kafka is a whole lot more than just a message bus! It includes stream processing and integration capabilities, and this talk will be mostly live coding to demonstrate what can be built around it, how to do it - and why to do it.
Have you ever thought that you needed to be a programmer to do stream processing and build streaming data pipelines? Think again! Apache Kafka is a distributed, scalable, and fault-tolerant streaming platform, providing low-latency pub-sub messaging coupled with native storage and stream processing capabilities. Integrating Kafka with RDBMS, NoSQL, and object stores is simple with Kafka Connect, which is part of Apache Kafka. ksqlDB is the source-available SQL streaming engine for Apache Kafka, and makes it possible to build stream processing applications at scale, written using a familiar SQL interface.
In this talk, we’ll explain the architectural reasoning for Apache Kafka and the benefits of real-time integration, and we’ll build a streaming data pipeline using nothing but our bare hands, Kafka Connect, and ksqlDB.
From Zero to Hero with Kafka Connect
Integrating Apache Kafka with other systems in a reliable and scalable way is often a key part of a streaming platform. Fortunately, Apache Kafka includes the Connect API that enables streaming integration both in and out of Kafka. Like any technology, understanding its architecture and deployment patterns is key to successful use, as is knowing where to go looking when things aren’t working.
This talk will discuss the key design concepts within Kafka Connect and the pros and cons of standalone vs distributed deployment modes. We’ll do a live demo of building pipelines with Kafka Connect for streaming data in from databases, and out to targets including Elasticsearch. With some gremlins along the way, we’ll go hands-on in methodically diagnosing and resolving common issues encountered with Kafka Connect. The talk will finish off by discussing more advanced topics including Single Message Transforms, and deployment of Kafka Connect in containers.
Introduction to ksqlDB
You’ve got streams of data that you want to process and store? You’ve got events from which you’d like to derive state or build aggregates? And you want to do all of this in a scalable and fault-tolerant manner? It’s just as well that Kafka and ksqlDB exist!
This talk will be built around a live demonstration of the concepts and capabilities of ksqlDB. We’ll see how you can apply transformations to a stream of events from one Kafka topic to another. We’ll use ksqlDB connectors to bring in data from other systems and use this to join and enrich streams—and we’ll serve the results up directly to an application, without even needing an external data store.
This event is organised by RecWorks on behalf of the London Java Community.
See our latest jobs here: https://recworks.co.uk/java-developer-jobs-london/.
See our privacy policy here: http://recworks.co.uk/privacy-policy
Continue the conversation at our Slack Group: https://londonjavacommunity.slack.com
Sign up here if you're not a member: https://bcrw.typeform.com/to/IIyQxd
