projects: expr() or selectExpr()? #DataEngineering #PySpark #SQL #SparkSQL #ApacheSpark #Databricks #BigData #DataTransformation #ETL #ELT #DataEngineer #LearnDataEngineering #AnalyticsEngineering #DataPipeline #DataAnalytics

transformations regardless of the tool being used. Which do you use more often in PySpark: .filter() or .where()? #DataEngineering #SQL #PySpark #SparkSQL #ApacheSpark #Databricks #BigData #DataTransformation #DataAnalytics #ETL #ELT #DataEngineer #LearnDataEngineering

The key lesson in Data Engineering: Understanding the difference between main and analytics use cases! Learn more about it in this video: bit.ly/4kb1ogp #dataengineering #usecases #learndataengineering

Follow these 4 key phases to make every #dataproject successful👇 Learn how to master them effectively with my new "Becoming a Better Data Engineer" course in my Academy, trusted by over 2,000 students 🚀 bit.ly/3CJN7qd #DataEngineering #LearnDataEngineering

The Key to Success In Learning Data Engineering 🚀 Out from my Plumbers of Data Science podcast episode "The One Thing All Successful Students Have". Watch the complete episode here: bit.ly/4cb2wM2 #dataengineer #dataengineering #datascience #learndataengineering

Yes, it can be very frustrating. BUT IT DOES NOT HAVE TO BE! Check out my Academy: bit.ly/3R13ao0 #dataengineering #dataengineer #bigdata #learndataengineering

You don't have to be a genius to succeed 🚀 Out from my Plumbers of Data Science podcast episode "The One Thing All Successful Students Have". Watch the complete episode here: bit.ly/4cb2wM2 #dataengineer #dataengineering #datascience #learndataengineering #datajobs

Learn about the main four things that will make you a better Data Engineer! Data Engineering Explained 🎓 For more content like this, visit my YouTube channel: bit.ly/3KIrsh4 #dataengineer #dataengineering #datascience #bigdata #learndataengineering

This is one of the key factors for a successful job application! 🔑 Out from my Livestream Hangout. Check out the complete video here: bit.ly/3xISeEP #dataengineer #dataengineering #datascience #bigdata #learndataengineering

Let me introduce to you our fresh Data Engineer Coaching Program! 🎉 I'm super excited to announce the launch of our updated Data Engineer Coaching Program, designed to help you transition into Data Engineering, advance your skills, or move up to a senior position 🚀 What We Offer: ✅ 6 Months of Personalized Coaching ✅ Lifetime Access to the Academy ✅ Custom Learning Plans ✅ Project Documentation Reviews & CV Assistance ✅ Weekly Live Sessions ✅ Private Discord Channels ⏰ Join Now: We're offering a 30% discount for July, so it's the perfect time to start your journey with us! 👉 Check out the full details here: bit.ly/4f67pss For current Academy members: You can upgrade to the Coaching program by only paying the difference. Get in touch with us for your special discount code! Join us and take your Data Engineering skills to the next level. Can't wait to see you in our Coaching sessions! #DataEngineering #Coaching #LearnDataEngineering #DataScience

🎥 Join my #dataengineering Hangout - live now on YouTube! 👉 bit.ly/3zSwjM0 I will do an insightful walk through my new Academy course: SQL for Data Engineers! Come and ask your burning questions throughout the session 🚀 #datascience #sql #learndataengineering

Learn why Data Scientists need Data Engineering, especially when it comes to data quality! Data Engineering Explained 🎓 #dataengineer #dataengineering #datascience #datascientist #learndataengineering

Ready to dive deep into the world of data engineering? 𝐓𝐨 𝐀𝐩𝐩𝐥𝐲: takeo.ai/data-engineer-bootc… #dataengineerbootcamp #learndataengineering #dataskills #bigdata #codingbootcamp #careeropportunity #techcareer #machinelearning #takeo #takeobootcamp #datacareer #data

What a mind-blowing milestone 🤯 We’ve just welcomed our 2,000th student to the Data Engineering Academy! To celebrate this, we’re offering a special 30% discount on the 12 months or unlimited access to our Academy! 🚀 Don't miss out on this exclusive offer, available only until July 1st! Join now: learndataengineering.com/p/a… Take advantage of the latest courses: ✅ Data Engineering on GCP ✅ Data Preparation & Cleaning for Machine Learning ✅ Relational Data Modeling Plus, get ready for upcoming content: ✅ Dimensional Data Modeling 2 ✅ Advanced SQL for Data Engineers ✅ Platform & Pipeline Design 2 Let's celebrate and become part of our Learn Data Engineering community! #DataEngineering #LearnDataEngineering #DataScience

Ever wondered how Docker works? Here you go! Data Engineering Explained 🎓 In my Data Engineering Academy I used Docker for several of my courses. Check them out 👉 learndataengineering.com/p/a… #dataengineer #dataengineering #datascience #bigdata #learndataengineering #docker

📢 NEW Substack is online! Becoming a successful #dataengineer is tough. It's not just about certifications or the job market. In my latest post, I reveal the true key to success in #dataengineering. Check it out! 👉 bit.ly/4avFgaD #datascience #learndataengineering

Mini roadmap for that one Data Engineering project that everyone can do no matter the platform! Start your own Data Engineering journey in my Academy trusted by almost 2,000 students 👉 learndataengineering.com/p/a… #DataEngineer #DataEngineering #LearnDataEngineering #DataScience

Data Engineers AI for the win! Thinking about the future, I think Generative AI is going to help us in our work as Data Engineers. It's going to make stuff easier. It's going to solve the easy things for us. Maybe it's going to create a boiler code where we could just add a few things or it could even write us whole codes for simple data integration. But at some point this is going to get more complicated as well as the requirements. So then you need an engineer to actually figure this out and actually solve this. So, Generative AI or AI ML in general is going to help us a lot in solving the easy issues. But it is just like with low code tools. You have your building blocks for the simple stuff and it's all good until it gets more complicated. Then you're going to run into the topic that you need an engineer. You need somebody who's doing the custom coding, who's doing the implementation for it because the standard blocks or the standard generated stuff from the AI is not enough. #dataengineer #dataengineering #datascience #bigdata #learndataengineering #AI

Don't lose yourself looking at every tool out there! My message to those who want to become a Data Engineer: don’t get into 20,000 things when you are at the beginning of your Data Engineering journey. Start from what you're good at! If you're coming from the Software Development side, try that stuff from the engineering part first. If you're coming from the Analyst side, start from the warehouse part and then stack on top of that. If you're new to Python, it might also make sense to start for instance with Snowflake and dbt. These are super simple and it’s not that difficult to develop stuff there, to do transformations. You can do a lot of stuff with SQL as well. It's easy to set up. So start with the low-hanging fruits. Start with the simple things and then build on top of that. I think that's very important. A great place to start with the basics, by the way, is my Learn Data Engineering Academy. Check it out: learndataengineering.com/p/a… #dataengineer #dataengineering #datascience #bigdata #learndataengineering

Looking for a Redshift alternative that just makes sense? Here you go! I once got asked if Spark can replace a data warehouse like Amazon Redshift? One alternative to Redshift that is out there and that makes sense to use is Databricks. Because Spark itself is just the processing, right? But what they did on Databricks, which I really like, is that you can actually build a warehouse yourself. You can start creating tables, you can ingest data for instance from files, run ETL jobs all within Spark. You can put the data into tables and then you can either make it accessible through the compute cluster to external clients or you can even generate a separate data warehouse with separate resources for clients to come in and actually create reports of the data. So, Databricks it is ;) Try it out! A great place to start is the Data Engineering on Databricks course in my Academy: learndataengineering.com/p/d… #dataengineer #dataengineering #datascience #bigdata #learndataengineering #databricks