transitioning from data analyst β€ data engineer | contributor at @TDataScience | #learninginpublic
Joined February 2022
- Tweets 262
- Following 150
- Followers 78
- Likes 318
21 Photos and videos
Pinned Tweet
Iβm learning Data Engineering from scratch as a Data Analyst.
So I mapped out a 12-month self-study roadmap to guide the transition.
I just published it here @TDataScience
You can read it here
towardsdatascience.com/from-β¦
2
3
21
3,197
I built my first ETL pipeline as a complete beginner.
No Airflow.
No Spark.
No cloud infrastructure.
Just Python, pandas, and a GitHub API.
Recently shared the full story in @TDataScience.
Here's exactly what I learned π§΅π
1
1
35
12/ If you're trying to break into data engineering:
Stop waiting for the perfect roadmap.
Pick a small project.
Build something ugly.
Finish it.
Your first pipeline won't be impressive.
But it will be the project that teaches you the most.
Read the full article below π
1
15
Ibrahim Salami (Ibby) retweeted
May 29
2026 Data Engineering Roadmap π
Stage 1: SQL
Stage 2: Python
Stage 3: Data Modeling
Stage 4: ETL Pipelines
Stage 5: PostgreSQL
Stage 6: Spark
Stage 7: Airflow
Stage 8: Data Warehousing
Stage 9: Cloud (AWS/GCP/Azure)
Stage 10: Streaming Systems
Stage 11: Data Quality Monitoring
Stage 12: Production Pipelines
Data scientists get the spotlight.
Data engineers move the data.
10
22
211
6,622
Data Engineering work is mostly ETL (Extract, Transform, Load).
So to learn about ETL, I decided to build a basic ETL pipeline that extracts data from GitHub repositories and saves it as CSV
Read about the entire process @TDataScience
Using the GitHub API, @ibbysalam shows us the steps he took to build an extract, transform, load data pipeline from scratch βΒ and as a complete beginner. towardsdatascience.com/i-buiβ¦
3
25
4,727
Ibrahim Salami (Ibby) retweeted
May 20
π Complete Data Engineering Roadmap
π§ STEP 1: Learn Programming Basics
β Start with Python
β Variables, Functions & Loops
β OOP Concepts
β APIs & JSON Handling
π STEP 2: Master SQL
β SELECT & WHERE
β JOINS & GROUP BY
β Window Functions
β CTEs & Subqueries
β Query Optimization
π Databases to Learn:
β MySQL
β PostgreSQL
β MongoDB
β‘ STEP 3: Learn Data Warehousing
β ETL vs ELT
β Data Pipelines
β Star & Snowflake Schema
β Batch Processing
β Data Modeling
π Tools to Learn:
β Snowflake
β Amazon Redshift
β BigQuery
βοΈ STEP 4: Learn Big Data Technologies
β Distributed Systems
β Parallel Processing
β Streaming Data
β Real-Time Analytics
π Technologies to Learn:
β Apache Spark
β Hadoop
β Apache Kafka
π STEP 5: Learn Data Pipelines & Orchestration*
β Workflow Scheduling
β Data Automation
β Monitoring Pipelines
β Error Handling
π Tools to Learn:
β Apache Airflow
β dbt
β Prefect
βοΈ STEP 6: Learn Cloud Platforms
β Cloud Storage
β Data Lakes
β Serverless Processing
β Cloud Security Basics
π Platforms to Learn:
β AWS
β Microsoft Azure
β Google Cloud Platform
π STEP 7: Learn DevOps for Data Engineering
β Version Control
β CI/CD Basics
β Containerization
β Deployment Automation
π Tools to Learn:
β Git
β Docker
β Kubernetes
π₯ STEP 8: Build Real Projects*
β ETL Pipeline Project
β Real-Time Data Streaming
β Sales Data Warehouse
β Data Lake Project
β Analytics Dashboard Backend
π‘ The best way to become a Data Engineer:
π Learn SQL β Build Pipelines β Work with Big Data β Deploy on Cloud
Data Engineering Resources: whatsapp.com/channel/0029Vaoβ¦
π¬ Tap β€οΈ if this helped you!
12
34
172
10,769
Happy to see a lot of people are resonating with this article. Can't wait to see where this journey takes me. If you're looking to break into or transition to data engineering. This is a good read
If you're considering a role change, or curious about the path to become a data engineer, don't miss @ibbysalam's new series on his own journey, covering the tools and resources he'll rely on and the many twists and turns he's bound to face. towardsdatascience.com/from-β¦
1
2
73
Ibrahim Salami (Ibby) retweeted
May 14
"It is tempting to treat hybrid search as something you can tune once: pick a merge algorithm, choose a lexical/semantic weight, and ship it - but there is no globally correct merge strategy"
hornet.dev/blog/100m-doc-seaβ¦
2
8
61
3,311