A Messaging Platform to Ease Development Around Streaming Data – InApps Technology 2022

In this demonstration, we do a walk-through of Streamlio Cloud, a hosted cloud-native messaging, processing and event storage service built on the open source Apache Pulsar distributed pub-sub messaging system. Our tour guide is Jon Bock, vice president of marketing for Streamlio.

In this model example, a hypothetical Java developer has created a program to sort messages (“Tweets”) coming from a Twitter feed into two separate feeds, those with positive sentiments and those with negatives sentiments (as determined by some imported natural language processing libraries). In the past you might have used some complex string-processing route to filter and redirect the messages. In this approach, using the Streamlio software development kit (SDK) with their favorite IDE, the programmer only needs to identify the input feeds and the output feeds. The resulting code can be compiled into a function, checked into a version control system, and kick off a build to deploy it into the messaging environment.

“It is relatively straight forward to write this code. You won’t see anything related to functional programming, maps or flat maps or concepts like that. What you see here is basically is something that focuses on the logic the developers are creating,” Bock said.

The service provides a frontend dashboard providing operational summaries of applications, which can be grouped into different namespaces, tenants or topics (individual streams). Thanks to its use of Prometheus, Streamlio reports on how often a function is called, how much data is being processed, how long a queue is. The developer only sees the topics they have access to, allowing them to see how their application is performing in a multitenant environment.

Bock demonstrated how Streamlio makes things easy for the operations teams as well. The operations team can use the software to allocate and cap resources to individual topics. For instance, the platform allows the operations team to spin up duplicate topics to increase the amount of data that can be processed. It also provides capabilities to store all the messaging data in the cloud, relieving that task of detailed storage management from the developer.

“One of the big challenges for developers is often that they get pulled into dealing with a lot of the production and operations. And certainly the ops team isn’t going to know the internals of the developers,” Bock said. “But we really want to decouple that so that the development team didn’t get burdened as much with operations concerns.”

TNS Managing Editor Joab Jackson contributed to this write-up.