Day 5/14 - Databricks Al Challenge Diving into my journey into the Databricks. Worked on feature table and perform validation using test train split. Grateful to
@databricks
@sahibmusharraf_
@codebasicshub
@indiandataclub
#DatabricksWithIDC #Databricks
2
15
Feb 20
Day 1/14 – Databricks AI Challenge
Started my journey into the Databricks Lakehouse! Learned:
Why Databricks > Pandas & Hadoop
Set up Community Edition, built my first notebook & ran PySpark
Grateful to @databricks @codebasics @indiandataclub
#DatabricksWithIDC #Databricks
3
147
Day 14 challenge for @databricks 14 Days AI Challenge.
Let’s do it together!💪
#DatabricksWithIDC
2
73
Day 13 | 14 Days AI Challenge 🚀
Trained multiple models using Spark ML, tracked runs in MLflow, and compared RMSE to select the best model.
@indiandataclub
@databricks
#MLflow #Databricks #SparkML #14DaysAIChallenge #DatabricksWithIDC
2
40
Today's log ✅️
[Model Comparison & Feature Engineering]
-Trained multiple models
-Tuned hyperparameters, compared metrics in MLflow
-Explored feature importance
-Built a Spark ML pipeline to select the best-performing model
@databricks @indiandataclub #DatabricksWithIDC
Day 12 | MLflow Basics
-Learned MLflow components
-Experiment tracking & model logging
-Trained a simple regression model, logged parameters and metrics
-Viewed runs in the MLflow UI, and compared experiments
@databricks @indiandataclub
#DatabricksWithIDC
4
266
Day 12 | 14 Days AI Challenge 🚀
Learned how experiment tracking helps compare models and ensure reproducibility in real ML workflows.
@indiandataclub @databricks
#MLflow #Databricks #MachineLearning #DataAnalytics #14DaysAIChallenge
#DatabricksWithIDC
2
29
Day 12 challenge for @databricks 14 Days AI Challenge.
Let’s do it together!💪
#DatabricksWithIDC
2
55
Day 11 | 14 Days AI Challenge 🚀
Worked on statistical analysis and feature engineering.
Calculated descriptive stats, tested hypotheses, analyzed correlations, and created ML-ready features.
@indiandataclub @databricks
#Day10 #14DaysAIChallenge #AIChallenge #DatabricksWithIDC
2
38
Learned
-SQL Warehouses
-Writing complex analytical queries
-Dashboard creation & visualizations, filters & refresh
Did
-Created a SQL warehouse
-Wrote analytical queries
-Built dashboards for revenue trends, funnels & products
@databricks @indiandataclub #DatabricksWithIDC
Completed yesterday’s pending tasks
-Learned Unity Catalog governance, including the Catalog → Schema → Table hierarchy
-Access control using GRANT/REVOKE
-Data lineage, and the difference between managed and external tables
@databricks @indiandataclub #DatabricksWithIDC
4
184
Jan 17
Day 7/14 #DatabricksWithIDC: Workflow Orchestration! ⚙️🤖
Leveling up to automation:
✅ Multi-task Jobs: Bronze ➡️ Silver ➡️ Gold
✅ Dynamic Notebooks: Used parameter widgets 🛠️
✅ Scheduled: Automated daily runs! ⏰
@databricks @codebasicshub @indiandataclub
2
31
Day 9 | 14 Days AI Challenge 🚀
Built analytics using Databricks SQL.
Created top products and funnel metrics from Gold tables.
@indiandataclub @databricks
#Day9 #DatabricksSQL #Analytics #DatabricksWithIDC
PS: Couldn’t build the dashboard today due to time constraints.
2
20
Day 9 challenge for @databricks 14 Days AI Challenge.
Let’s do it together!💪
#DatabricksWithIDC
2
48
Day 8 | 14 Days AI Challenge 🚀
Built a Bronze → Silver → Gold pipeline using PySpark and Delta tables.
Cleaned raw events, enriched data in Silver, and created analytics-ready metrics in Gold.
@indiandataclub
@databricks
#Day8 #DeltaLake #LearningInPublic #DatabricksWithIDC
1
5
50
Jan 15
Day 07 of AI Challenge by
@indiandataclub ,@codebasicshub,@databricks
Moved from Databricks notebooks to Jobs by building a multi-task Bronze→Silver→Gold pipeline.
Here is my Linkedin Post: linkedin.com/feed/update/urn…
#DatabricksWithIDC #LakkiData #LearningSteps
2
27
Day 7 | 14 Days AI Challenge 🚀
Explored Databricks Workflows.
Built multi-task jobs for Bronze → Silver → Gold with parameters, dependencies, and scheduling.
@indiandataclub @databricks
#Day7 #14DaysAIChallenge #Databricks #LearningInPublic #DatabricksWithIDC
3
52
Jan 15
Day 7/14 — Databricks AI Challenge🌟
Today’s learnings💡
- Databricks Jobs vs notebooks
- Multi-task workflows
- Parameters & scheduling
- Error handling
Day 7 done — 7 more to go!
#DatabricksWithIDC
2
35
Jan 14
Day 6/14 — Databricks AI Challenge 🌟
Today’s learnings💡
- Bronze (raw) → Silver (cleaned) → Gold (aggregated)
- Best practices for each layer
- Incremental processing patterns
Day 6 done — 8 more to go!
#DatabricksWithIDC
4
40
Jan 13
Day 5/14 — Databricks AI Challenge 🌟
Today’s learnings💡
• Time travel (version history)
• MERGE operations (upserts)
• OPTIMIZE & ZORDER
• VACUUM for cleanup
Day 5 done — 9 more to go!
#DatabricksWithIDC
2
29
Jan 10
1/n) Day-2 Completed: @databricks
14 Days: AI Challenge #DatabricksWithIDC
Completed Day-2 of the challenging, kicking things off with Apache Spark Architecture
2
2
95
Taking the Databricks 14 Days AI Challenge 🚀
Ready to dive into hands-on AI, data engineering, and real-world learning.
Let’s build, break, and learn.
@indiandataclub @databricks
#Databricks #AIChallenge #DataAnalytics #DatabricksWithIDC
2
29