Databricks - State of Data + AI 2023

The 2023 State of Data + AI report provides valuable insights into the current landscape of data and AI, highlighting trends and patterns in adoption across various industries. The report reveals that machine learning adoption is rapidly increasing, with a particular focus on natural language processing (NLP) and large language models (LLMs). Open source technologies are dominating the data and AI markets, with 8 out of 10 widely-adopted products being based on open source. The use of the Lakehouse for data warehousing is on the rise, as evidenced by the popularity of Microsoft Power BI, the growth of data integration tools like dbt and Fivetran, and the accelerated adoption of Databricks SQL. The report also highlights the importance of business intelligence (BI) tools, which are widely adopted across organizations, with a 66% year-over-year (YoY) increase in adoption. This aligns with the trend of organizations performing data warehousing on a Lakehouse. As the volume of data explodes, an increasingly large proportion is in the form of semi-structured and unstructured data. This presents new challenges for organizations, but also new opportunities for innovation and success. By leveraging the power of AI, organizations can unlock new insights and drive growth in a rapidly-evolving landscape. Overall, the report provides valuable insights into the current state of data and AI adoption, highlighting the importance of machine learning, open source technologies, and the use of the Lakehouse for data warehousing.