Big Data

Big Data refers to the technologies and practices for processing, storing, and analyzing datasets too large or complex for traditional tools. From distributed computing frameworks to real-time streaming architectures, Big Data enables organizations to derive insights from massive volumes of structured and unstructured data.

Related topics on this blog:

• Spark — Fast, in-memory data processing
• Lambda Architecture — Combining batch and real-time processing
• Kappa Architecture — Stream-first data architectures
• Data Mesh — Decentralized, domain-oriented data ownership

For sovereign, enterprise-grade analytics, see Scrydon’s Big Data Platform.

• Data Mesh using Microsoft Fabric at Cloud Scale Analytics '23

• Data Mesh in Azure at Microsoft Customer Success Day '22

• Dataminds Connect: The Synapse Keynote

• Configuring Apache Spark to oversubscribe the # of cpu cores

• ML in Production @ AZUG

• ML in Production @ dataminds.be

• Cloudera, Azure and Big Data at Cloudera Meetup '17

• Predicting Occupancy on the Belgian railroads, based on spitsgids/iRail data, using Azure ML.

• Microsoft Advanced Analytics @ Data Science Ghent '16 meetup

• Spark on Azure: a Gentle Introduction at Cloudbrew '15

• #48 on Grey's The 150 Most Influential People in Big Data & Hadoop

• Virdata: lessons learned from the Internet of Things and M2M Cloud Services @ IBM Big Data Developers Meetup, Silicon Valley

• A Lambda Architecture using Hadoop and Storm @ NoSQL Matters Cologne '14

• A Big Data Overview, from DataCrunchers @ Devoxx 2013

• A real time architecture using Hadoop and Storm @ Devoxx 2013

• A real time architecture using Hadoop and Storm @ JAX London 2013

• Microsoft Big Data @ SQLUG 2013

• A real time architecture using Hadoop and Storm @ FOSDEM 2013

• Microsoft Inspirience '12: Getting more out of your Big Data

• Hadoop Pig: Map Reduce the Easy Way @ FOSDEM 2011

Discover more posts and tags on the overview page.