# DataEngineering
Dispelling LTAP myths
tech_blogs_databricks

#DataEngineering
Dispelling LTAP myths
www.databricks.com
November 12, 2026 at 10:30 PM
The Missing Data Contract for Autonomous AI Systems
tech_blogs_acm | Author: Aditya Ranjan

#DataEngineering
The Missing Data Contract for Autonomous AI Systems
cacm.acm.org
November 15, 2026 at 4:39 AM
Jimmy Angelakos got 5 minutes & no slides at Community Over Code Glasgow for pgEdge ColdFront: #PostgreSQL data tiering to Apache Iceberg, cold rows still writable.
🎬 www.youtube.com/watch?v=W5bW...
📖 vyruss.org/blog/communi...

#Postgres #ApacheIceberg #DataEngineering #OpenSource
November 12, 2026 at 11:20 PM
Don't add a project to your portfolio just to show five logos.

Add it because you can explain why each tool is there, and what you'd use instead in a smaller setup.

#dataengineering #databs
November 12, 2026 at 12:00 PM
Your Athena bill isn’t a pricing problem.

It’s a file layout problem.

Parquet + partitions can turn “scan 1 TB” into “scan a few GB.”

Same query. Fraction of the cost. 🧵

#AWS #DataEngineering
November 16, 2026 at 1:30 AM
Good database design starts with proper planning.

Learn the basics of tables, relationships, keys, and normalization in this beginner's guide.

victorhugosolis.com/posts/databa...

#DatabaseDesign #SQL #DataEngineering
November 13, 2026 at 4:07 PM
A portfolio project idea from Andreas Kretz: a real-time data pipeline.

FastAPI takes in JSON events, Kafka holds them, Spark transforms them as they arrive, MongoDB stores the results and Streamlit shows them on a live dashboard. Docker Compose runs it all.

#dataengineering #databs
November 12, 2026 at 12:00 PM
NEW! Leanpub Book LAUNCH 🚀 Blockchain Data Engineering Guide: The hands-on guide for data and analytics engineers working with onchain data by Vincent Charles

youtu.be/gGMwcTrj4LQ

#books #leanpublishing #selfpublishing #blockchaindata #dataengineering #onchaindata #DeFiAnalytics
November 14, 2026 at 6:15 PM
❄️ Snowflake Semantic views can now do many-to-many. I spent the morning finding out what that means in practice 😎

#Snowflake #SnowflakeSquad #CortexAnalyst #SemanticViews #DataEngineering
November 13, 2026 at 10:57 AM
580 - Once an organization lands its data, it almost never wants to move it again. A big Databricks estate with tons of tables stays put. The win shows up when the environment is still new.

#databricks #datagravity #dataengineering #migration
November 16, 2026 at 9:07 PM
Challenge du lundi : LinkedIn en une seule image !

Recruteurs en France : cessez de payer un abonnement hors de prix quand vous pouvez avoir accès à France Travail ou a Apec - Association Pour l'Emploi des Cadres !

#Recrutement #LinkedIn #DataEngineering #Emploi #Travail ...
November 16, 2026 at 2:25 PM
Build a real-time next-best-offer recommender using Databricks Feature Views
tech_blogs_databricks

#DataEngineering
Build a real-time next-best-offer recommender using Databricks Feature Views
Build once, serve everywhere: Feature Views unify batch and streaming features for training and serving.
www.databricks.com
November 16, 2026 at 6:09 PM
Spark, MongoDB and Streamlit in one pipeline, each with one job:

- Spark reads from Kafka and transforms the data as it arrives
- MongoDB stores the results as JSON documents, so you don't need a strict schema upfront
- Streamlit shows the data on a simple live dashboard

#dataengineering #databs
November 12, 2026 at 12:00 PM
You know AWS Glue Redshift S3 and PySpark.

But can you answer this?

Why Glue instead of EMR?

AWS Data Engineer interviews aren't memory tests. They're problem-solving tests.

Stop memorizing services. Start understanding architecture, trade-offs and real scenarios.

#DataEngineering #AWSInterview
November 11, 2026 at 7:07 PM
Building data infrastructure is one thing. Making data reliable is another.

Clear ownership, shared definitions, and managed changes help teams work with information they can trust.

That's why data governance matters.

#DataGovernance #DataInfrastructure #DataManagement #DataEngineering
November 13, 2026 at 7:25 PM
What is Real-Time Mode in Apache Spark?
tech_blogs_databricks

#DataEngineering
What is Real-Time Mode in Apache Spark?
Real-Time Mode is a Spark Structured Streaming trigger for millisecond, end-to-end latency. Learn how it works, what it supports and when to use it.
www.databricks.com
November 16, 2026 at 6:09 PM
577 - Handing a table from one team to another gets messy fast. Someone still has to explain how a number moved from bronze to gold. That explanation has a cost leadership keeps forgetting to fund.

youtu.be/xx1Fn1YMKdc

#dataengineering #datagovernance #lakehouse #microsoftfabric
November 11, 2026 at 1:35 PM
578 - Microsoft wrote these Fabric APIs for agents to call. Tom's article makes that hard to miss, and if you work in that sandbox you should ask who they were built for.

youtu.be/zxBHFKGIWpo

#microsoftfabric #agents #dataengineering #apis
November 14, 2026 at 2:25 AM
Also check out more related #dataengineering architectures below.
Data Engineering Architecture
A good data engineering architecture is hard.
www.ssp.sh
November 11, 2026 at 11:12 AM
Talk to your clinical data: unlocking AWS HealthLake with OpenSharing and Genie One
tech_blogs_databricks

#DataEngineering
Talk to your clinical data: unlocking AWS HealthLake with OpenSharing and Genie One
Set up a custom OpenSharing integration that brings AWS HealthLake FHIR data into Databricks, with Unity Catalog governance and Genie One analytics.
www.databricks.com
November 16, 2026 at 6:39 PM
📌 Start here: free data, AI & cloud certification guides

Snowflake · Qlik · SAS · Google Cloud · Cloudera · Databricks · AWS AI · Nutanix

Exam topics, domain weights and study plans. A new guide every few days 👇 www.bigdatarise.com

#Certification #DataEngineering #AI
November 13, 2026 at 12:14 PM