Apache Airflow is an extremely popular tool that data engineers rely on. But why? Why do data engineers like Airflow? Also, what does the Apache Airflow event do? In this article, we will answer questions like: What is Airflow? What is a DAG? Why do people use Apache Airflow? Why do we like Airflow?…
Read more
Photo by Startaê Team on Unsplash 2022 is around the corner and it is time to start looking towards improving your data strategy. Our team has seen several trends in 2021 in terms of methods which can help improve your data analytics strategy. Whether it be optimizing your Snowflake table structures to save $20,000 a…
Read more
Data is an integral aspect of any business. It allows for solution development, metric tracking, and creates a structure for streamlined and integrated processes. Data empowers business decisions. I say that both from the fact that consulting firms like McKinsey have found that in their research companies that are using AI and analytics can attribute 20%…
Read more
Source Apache Airflow is a popular open-source tool that helps teams create, schedule, and monitor sequences of tasks, known as “workflows.” Managed Workflows for Apache Airflow, abbreviated as MWAA, is a managed service designed for orchestration in Apache Airflow. With MWAA, users can easily operate data pipelines at scale, setting up and managing them from…
Read more
Photo by Green Chameleon on Unsplash All signs point towards an auspicious future for data engineering. Dice’s 2020 tech jobs report cites Data Engineering as the fastest growing job in 2020. Increasing by a staggering 50%, while Data Science roles only increased by 10%. You can rest assured that the influx of data engineering will not regress anytime soon.…
Read more
Photo by Windows on Unsplash It seems like it might be finally happening. Data engineering is finally getting a little bit of the data lime light. Which makes sense. The data being stored and analyzed is not only becoming more voluminous but its speed, complexity and variety are also increasing. Making it difficult to wrangle. With…
Read more
Photo by C Dustin on Unsplash Data warehousing was introduced in 1988 by IBM researchers Barry Devlin and Paul Murphy. Since then the concept has evolved and taken on a life of its own. Increasing challenges and complexities of business have forced data warehousing to become a distinct discipline. Over the years this has led to best…
Read more
.Photo by Kelly Sikkema on Unsplash As 2020 has comes to a close, we wanted to take a moment to reflect on all the changes in technology as well as look to see where things are going. Whether you are looking at start-ups and their IPOs, improvements in technology or you paid attention to Amazon Re:Invent.…
Read more
Even small companies these days on average have 47.81 terabytes of data that they manage. Regardless if you’re a small company or a trillion-dollar behemoth, data is driving decisions. But as data ecosystems become more complex, having the right tools for the job is important. One modern data management tool that can help manage data at really…
Read more
Photo by Kaleidico on Unsplash Companies of all sizes have embraced using data to make decisions. However, according to a 2019 report from Goldman Sachs it’s actually quite difficult for businesses to use data to build a sustainable competitive advantage. Our team has worked with and for companies across industries. We have seen the good, the bad and the…
Read more