The world of data is moving quickly and traditional relational database technology can be a limiting factor in responding to change. Teams want to move quicker, work with a wider array of data, handle massive datasets and augment their code with open-source libraries and projects. Data delivery and demand for immediate insights mean we no longer have the luxury to extract, transform and load datasets before we need to realise the value locked within our data. SQL Server 2019 has made radical architecture changes to meet these challenges, introducing in-built data lakes, Spark clusters, massive data ingestion engines and the ability to harness massively parallel processing architectures. These engines are all implemented behind a single, scalable interface that streamlines data acquisition, transparently and without costly movement operations.

In this talk we will outline the problems that can be tackled with the new SQL Server Big Data Clusters, provide an overview of how they have been implemented and discuss how SQL Server can now handle your Big Data problems. We will be drawing parallels to the Azure Data Platform and highlight where we can adopt similar patterns in our on-premises data platforms.