Databricks is a company that aims to make managing big data easy. By utilizing Apache Spark as its core processing engine, Databricks aims to make managing and processing big data easy. As one of the leading providers of big data solutions, Databricks has made a name for itself in the industry. What is Databricks? So,…
Read more
“Why do we still struggle to answer basic questions like how many customers are still active or what exactly is the company’s churn?” This was a question posed to a panel I was sitting on at the Snowflake Summit. Several panelists provided different answers and all come from very different perspectives. Truth be told I…
Read more
Photo by Nana Smirnova on Unsplash There is a lot of discussion on how you should implement your data stacks today. Should you bundle or unbundle, should you use a traditional data warehouse, lakes, lake houses, or mesh? Or a hybrid of all of them? With all these different ways you can put together your…
Read more
Photo by Pawel Czerwinski on Unsplash Big data is touted as the solution to many problems. However, the truth is, big data has caused a lot of big problems. Yes, big data can provide more context and provide insights we have never had before. It also makes queries slow, data expensive to manage, requires a lot of expensive…
Read more
Photo by Leohoho on Unsplash Migrating to the modern data stack means poising your organization to meet the varying demands of modern data. The modern data stack seeks to overcome the problems and challenges associated with modern data while enabling an organization to innovate and automate like never before. As more organizations migrate to the modern…
Read more
Source Apache Airflow is a popular open-source tool that helps teams create, schedule, and monitor sequences of tasks, known as “workflows.” Managed Workflows for Apache Airflow, abbreviated as MWAA, is a managed service designed for orchestration in Apache Airflow. With MWAA, users can easily operate data pipelines at scale, setting up and managing them from…
Read more
Photo by Clem Onojeghuo on Unsplash Owning and operating a business in today’s tech-savvy world opens up a plethora of opportunities for improvement. Through innovative technologies, business intelligence and data science are two fields proliferating the level of insight that owners have into all aspects of their organizations. This is done by collecting, maintaining, and…
Read more
Photo by Windows on Unsplash It seems like it might be finally happening. Data engineering is finally getting a little bit of the data lime light. Which makes sense. The data being stored and analyzed is not only becoming more voluminous but its speed, complexity and variety are also increasing. Making it difficult to wrangle. With…
Read more
Tips For Your Next Technical Interview Technical interviews don’t just test your understanding about data structures and algorithms for calculating the probability of events occurring. These interviews attempt to understand your level of responsibility in your previous projects, your communication abilities, as well as gauge your past experiences to see if you can make good decisions independently. This is why…
Read more
As the data industry evolves with new technology, so do data engineering challenges. What can new engineers expect? As data engineers, you play vital roles in your field by collecting and analyzing data. But necessary data engineer skills today aren’t the same as they were in years past, and the role is seeing some serious…
Read more