It’s becoming difficult to keep track of all the new data warehouse solutions that are trying to challenge the current incumbents. Choosing the best data warehouse to meet the needs and objectives of your operation is a crucial component of your business strategy. Unfortunately, many organizations are still struggling with this decision. To add to…
Read more
Source Apache Airflow is a popular open-source tool that helps teams create, schedule, and monitor sequences of tasks, known as “workflows.” Managed Workflows for Apache Airflow, abbreviated as MWAA, is a managed service designed for orchestration in Apache Airflow. With MWAA, users can easily operate data pipelines at scale, setting up and managing them from…
Read more
Companies of all sizes are looking for faster and easier ways to take advantage of data through analytics and BI. In particular, knowing who your customers are, how they interact with your business, products and services is a modern necessity. In a customer-centric world where data can help improve the overall customer experience and relationships,…
Read more
Photo by Christina @ wocintechchat.com on Unsplash Why does it feel like so many more articles are discussing the data engineering profession? Perhaps it’s because Dice’s 2020 tech jobs report cites data engineering as the fastest-growing field in 2020, increasing by a staggering 50%, while data science roles only increased by 10%? Or maybe it’s just because the Medium…
Read more
Photo by Green Chameleon on Unsplash All signs point towards an auspicious future for data engineering. Dice’s 2020 tech jobs report cites Data Engineering as the fastest growing job in 2020. Increasing by a staggering 50%, while Data Science roles only increased by 10%. You can rest assured that the influx of data engineering will not regress anytime soon.…
Read more
Software engineer vs. data engineer. People outside these professions are often confused about the differences between these experts. They may think these are two different names for the same thing: a programming expert. Of course, the duties of each engineer type are different, but their roles are similar enough to confuse people who are in…
Read more
Photo by ThisisEngineering RAEng on Unsplash By Iris Huang I’ve been four years into a career leading to data engineering, and am very glad to see the wave has risen. Influencers (from YouTube to LinkedIn top voices) and companies have started to recognize the importance of onboarding a data engineer before a data scientist if…
Read more
Photo by Clem Onojeghuo on Unsplash Owning and operating a business in today’s tech-savvy world opens up a plethora of opportunities for improvement. Through innovative technologies, business intelligence and data science are two fields proliferating the level of insight that owners have into all aspects of their organizations. This is done by collecting, maintaining, and…
Read more
Photo by Windows on Unsplash It seems like it might be finally happening. Data engineering is finally getting a little bit of the data lime light. Which makes sense. The data being stored and analyzed is not only becoming more voluminous but its speed, complexity and variety are also increasing. Making it difficult to wrangle. With…
Read more
Photo by Guille Álvarez on Unsplash With an extensive background in data science, analytics, and cloud computing, I am consistently asked the same questions repeatedly. Besides wanting to know the difference between a data engineer and a data scientist, one of the most common questions is, what skills should I learn as a data engineer?…
Read more