Faust, a Python-Based Distributed Stream-Processing Library – InApps is an article under the topic Software Development Many of you are most interested in today !! Today, let’s InApps.net learn Faust, a Python-Based Distributed Stream-Processing Library – InApps in today’s post !

Read more about Faust, a Python-Based Distributed Stream-Processing Library – InApps at Wikipedia

You can find content about Faust, a Python-Based Distributed Stream-Processing Library – InApps from the Wikipedia website

Robinhood, a free trading platform for stocks and cryptocurrency, has open-sourced its Python-based stream processing library Faust.

Faust is a Python 3 library available on GitHub, and takes advantage of Python recent performance improvements and integrates with the new AsyncIO module for high-performance asynchronous I/O.

“Faust comes with the benefits of Python — it’s just very simple to get started with,” said data infrastructure engineer Vineet Goel, one of its developers. “Robinhood has a relatively small team building a lot of different systems, so the simplicity of getting started and ease of use are things that help you.”

The Menlo Park, Calif.-based company wanted it to be distributed to help with scaling challenges it was facing. It does a lot of batch processing, principal software engineer Ask Solem said, so it wanted something easy to use and in Python, as it is a Python shop.

Solem, the creator of the task queue Celery, has background with large-scale Python projects, and Goel with distributed systems, so with Goel’s research into Apache Kafka, they said they felt confident they could make this happen.

Faust provides both stream processing and event processing, similar to Kafka Streams, Apache Spark, Storm, Samza and Flink.

“While existing streaming systems use Python, Faust is the first to take a Python-first approach at streaming, making it easy for almost anyone who works with Python to build streaming architectures,” according to Goel.

Read More:   Mozilla and the ‘Planet-Incinerating Ponzi Grifters’ – InApps 2022

Faust takes heavy inspiration from Kafka Streams, yet takes a different approach, notably that it does not use a domain-specific language. As a Python library, it can be dropped into any existing Python code, with support for all the libraries and frameworks Python developers like to use, such as NumPy, PyTorch, Pandas, NLTK, Django, Flask and SQLAlchemy.

Faust supports any type of stream data: bytes, Unicode and serialized structures, but also comes with “Models” that use modern Python syntax to describe how keys and values in streams are serialized.

Its concept of “agents” comes from the actor model, which means the stream processor can execute concurrently on many CPU cores, and on hundreds of machines at the same time, according to its documentation.

It requires Python 3.6 or later to run multiple stream processors in the same process, along with web servers and other network services. It also does not require use of resource managers such as Yarn or Mesos.

The company uses Faust for streaming all the events on the Robinhood app, such as data analysis, risk, fraud and security in real time. It acts as the back end for the Robinhood feed, which is a chat, monitoring execution quality.

“It’s useful for writing back-end services, especially for companies that have iOS apps, Android apps and maybe a web app. They need a back end that does data analysis and can do all that with a single solution,” Solem said.

Jay Kreps, co-creator of Kafka, was among those who tweeted about the project:

Django co-creator Simon Willison was another:

It was trending as one of the top Python projects on GitHub after the announcement last Tuesday and has already received over 1500 stars, over 50 forks, and three pull requests from contributors outside Robinhood. Faust was number three on Hacker News on the day it launched.

Read More:   A Tentative Well-Wishing for 20 Years of .NET – InApps Technology 2022

List of Keywords users find our article on Google:

celery python
celery task
faust wikipedia
apache storm kafka
flask sqlalchemy
flink github
apache storm wikipedia
django ecommerce github
sqlalchemy asyncio
celery github
apache spark github
streamdata.io
flutter stream
flask sqlalchemy order by
storm kafka
monitoring kafka streams
ecommerce django github
python developers recruitment
trading python
django celery
whatsapp module in python
ask solem
“ksql” kafka or apache
asyncio github
robinhood kubecon
python stream processing
sqlalchemy github
robinhood faas
django template not equal
python github library
kafka streams github
stream chat flutter
qc library
kafka github
celery documentation
faust libraries
python sqlalchemy order by
python send message to teams
aiohttp sqlalchemy
flink python
python distributed
flask template github
kafka processor
python-celery
kafka streams
react native stream
python processing

Source: InApps.net

Rate this post
Content writer

Let’s create the next big thing together!

Coming together is a beginning. Keeping together is progress. Working together is success.

Let’s talk

Get a custom Proposal

Please fill in your information and your need to get a suitable solution.

    You need to enter your email to download

      Success. Downloading...