Stefano Pampaloni shared with us his opinion on Big Data Stream Analytics.
Hello Stefano. First of all, tell us a little bit about yourself!
I’m the sole administrator of Seacom, a company based in the technological centre of Navacchio near Pisa. I didn’t study computer science but mathematics, but this has still helped me a lot in my IT career. The technologies I’ve been using for many years are all Open Source, which is one of my passions both as tools and also for studying business models. In the company we began a long time ago with Linux, with PHP 20 years ago, and then continued with Zimbra, Elasticsearch, Elastic and Kafka. I’m part of the communities both in Milan and Rome and because of this we got to know Codemotion with the AperiTech in Rome. Apart from Open Source, my other passions are travel, football, the Fiorentina football team and sailing, even if there aren’t many occasions when I get to do it.
The pace of technological change is faster than ever. What do you do to keep up with the latest news? What is your point of reference for identifying the right Open Source program to use?
Of course the communities are very useful for keeping yourself up to date. Moreover, voluntary associations are also an important part, but perhaps they risk not being able to ensure sustainability, especially in enterprise contexts. Other ways I keep myself up-to-date are various events around Italy and the world, both on specific products and linked to Open Source, in Paris, New York and Italy with Codemotion.
During one of the AperiTech evenings, you talked about big data stream analytics without code. Can you tell us more about it?
In this talk, which was the first AperiTech in Rome, we wanted to show how it was possible through Apache Kafka to implement a large-scale streaming processing solution without a single line of code. In this specific case we took data from an SQL database and put it in the Kafka queue using Kafka Streams.
What type of user can make use of it?
Kafka can basically be used in various contexts where an extremely efficient mechanism is useful to manage events and data queues, even when you need to guarantee the service through spikes in use and traffic peaks. Kafka is a buffer.
Does Kafka have any competitors or is it one-of-a-kind?
There are certainly some competitors, but they have a slightly different purpose and, above all, they’re suitable for contexts where there’s no strong need for performance. The key thing that differentiates Kafka is that it manages a very large number of data connections per second on fairly contained structures. There are other competitors coming out with the same concept that are even less developed, but in perspective they may have similar characteristics and targets.
Could we ask you why, together with your colleague, did you decide to talk about this specific topic?
We wanted to show how much you can do with Kafka in a short time and the results you can obtain, because it’s very interesting, in our opinion, to show, approach and make accessible Kafka’s technology even to those who aren’t really experts at programming and development. This is because, although it requires a certain level of competence, it’s not necessary to be a skilled programmer to use this tool and the power it makes available.
We were wondering if you’re planning on delivering some other talks related to Kafka’s technology, or maybe you plan to present something brand new next time?
With Kafka we do various meetups so we study the technology in-depth; probably already by September we’ll do something related to integrating microservices. My colleague Dario gave a talk on how to write in Kafka Connect. We’ll definitely do other talks and they’ll involve other people as well.
Finally I would like to ask you what advice you would like to give anyone wanting to start using Kafka?
Kafka is a tool and as such when it’s put in capable hands it can give great satisfaction. It’s certainly a solution worth considering for those working in the field of data streaming. Participating in the community is one of the ways to acquire skills and understand from others how to make the most of these topics.