Data

In this article, I share my critical view on the current state of data engineering, dominated by heavyweight platforms like Spark and Databricks, and introduce Sail, an open-source engine built on top of Apache Arrow and DataFusion, written in Rust, that offers a new path: lightweight, efficient, and powerful.

Defining the serializer and transformation from Yaml to Core

Defining the model core from Scala to Spark

Exploring how to create Apache Spark ETL workflows with zero code.