Chat noir très attentif au moindre bruit, assez timide. Celui d'une voisine. Bon signe ?
#Recrutement #DataEngineering #JdR #Chat #ChatNoir ...
#Recrutement #DataEngineering #JdR #Chat #ChatNoir ...
November 7, 2026 at 1:51 PM
Chat noir très attentif au moindre bruit, assez timide. Celui d'une voisine. Bon signe ?
#Recrutement #DataEngineering #JdR #Chat #ChatNoir ...
#Recrutement #DataEngineering #JdR #Chat #ChatNoir ...
Never Restart a Failed AI Pipeline Again | Union.ai on Google Cloud
#MLOps #SovereignAI #AIInfrastructure #GKE #Kubernetes #GoogleCloud #GCP #Flyte #UnionAI #MachineLearning #AI #DataPipelines #DevOps #DataEngineering
#MLOps #SovereignAI #AIInfrastructure #GKE #Kubernetes #GoogleCloud #GCP #Flyte #UnionAI #MachineLearning #AI #DataPipelines #DevOps #DataEngineering
Never Restart a Failed AI Pipeline Again | Union.ai on Google Cloud
google.smh.re
November 9, 2026 at 7:44 AM
Never Restart a Failed AI Pipeline Again | Union.ai on Google Cloud
#MLOps #SovereignAI #AIInfrastructure #GKE #Kubernetes #GoogleCloud #GCP #Flyte #UnionAI #MachineLearning #AI #DataPipelines #DevOps #DataEngineering
#MLOps #SovereignAI #AIInfrastructure #GKE #Kubernetes #GoogleCloud #GCP #Flyte #UnionAI #MachineLearning #AI #DataPipelines #DevOps #DataEngineering
577 - The coding agent said the Spark was validated and the refresh looked fine. Those rows were still wrong. Project OSMOS starts from the business outcome, before anyone asks for code.
youtu.be/xx1Fn1YMKdc
#microsoftfabric #projectosmos #spark #dataengineering
youtu.be/xx1Fn1YMKdc
#microsoftfabric #projectosmos #spark #dataengineering
November 10, 2026 at 10:36 AM
577 - The coding agent said the Spark was validated and the refresh looked fine. Those rows were still wrong. Project OSMOS starts from the business outcome, before anyone asks for code.
youtu.be/xx1Fn1YMKdc
#microsoftfabric #projectosmos #spark #dataengineering
youtu.be/xx1Fn1YMKdc
#microsoftfabric #projectosmos #spark #dataengineering
Where do you actually store Bronze, Silver and Gold?
In a dbt and warehouse stack, Bronze lives as files in object storage, because it's rarely queried. Silver and Gold live in the warehouse, because that's where the SQL work and the dashboards are.
#dataengineering #databs
In a dbt and warehouse stack, Bronze lives as files in object storage, because it's rarely queried. Silver and Gold live in the warehouse, because that's where the SQL work and the dashboards are.
#dataengineering #databs
November 9, 2026 at 12:00 PM
Where do you actually store Bronze, Silver and Gold?
In a dbt and warehouse stack, Bronze lives as files in object storage, because it's rarely queried. Silver and Gold live in the warehouse, because that's where the SQL work and the dashboards are.
#dataengineering #databs
In a dbt and warehouse stack, Bronze lives as files in object storage, because it's rarely queried. Silver and Gold live in the warehouse, because that's where the SQL work and the dashboards are.
#dataengineering #databs
Against all odds, CSV remains the backbone of data ingestion, yet viewing or editing them still breaks things.
I built NanoCell: https://github.com/CedricBonjour/nanocell-csv
Coded with ❤️. Break it with your worst datasets and open an issue!
#OpenSource #DataEngineering
I built NanoCell: https://github.com/CedricBonjour/nanocell-csv
Coded with ❤️. Break it with your worst datasets and open an issue!
#OpenSource #DataEngineering
GitHub - CedricBonjour/nanocell-csv: A free csv file viewer & editor
Lightweight, cross-platform CSV viewer and editor built for developers and data engineers. 100% client-side and text-safe.
github.com
November 10, 2026 at 10:02 AM
Against all odds, CSV remains the backbone of data ingestion, yet viewing or editing them still breaks things.
I built NanoCell: https://github.com/CedricBonjour/nanocell-csv
Coded with ❤️. Break it with your worst datasets and open an issue!
#OpenSource #DataEngineering
I built NanoCell: https://github.com/CedricBonjour/nanocell-csv
Coded with ❤️. Break it with your worst datasets and open an issue!
#OpenSource #DataEngineering
❄️ Snowflake backup sets now cover 🧊 Iceberg tables (GA since Nov 4). That says something about the architecture 😎
#Snowflake #SnowflakeSquad #ApacheIceberg #DataEngineering #Lakehouse #DisasterRecovery
#Snowflake #SnowflakeSquad #ApacheIceberg #DataEngineering #Lakehouse #DisasterRecovery
November 9, 2026 at 7:49 AM
❄️ Snowflake backup sets now cover 🧊 Iceberg tables (GA since Nov 4). That says something about the architecture 😎
#Snowflake #SnowflakeSquad #ApacheIceberg #DataEngineering #Lakehouse #DisasterRecovery
#Snowflake #SnowflakeSquad #ApacheIceberg #DataEngineering #Lakehouse #DisasterRecovery
💼 Quantiphi is hiring a Data Engineer - Senior - USA
Location: 🇺🇸 New York, New York, United States
#DataEngineering #DataEngineeringjob #hiring #techjob #jobs #careers
https://datasciencejobs.com/jobs/data-engineer-senior-quantiphi-united-states-7/
Location: 🇺🇸 New York, New York, United States
#DataEngineering #DataEngineeringjob #hiring #techjob #jobs #careers
https://datasciencejobs.com/jobs/data-engineer-senior-quantiphi-united-states-7/
November 6, 2026 at 8:00 PM
💼 Quantiphi is hiring a Data Engineer - Senior - USA
Location: 🇺🇸 New York, New York, United States
#DataEngineering #DataEngineeringjob #hiring #techjob #jobs #careers
https://datasciencejobs.com/jobs/data-engineer-senior-quantiphi-united-states-7/
Location: 🇺🇸 New York, New York, United States
#DataEngineering #DataEngineeringjob #hiring #techjob #jobs #careers
https://datasciencejobs.com/jobs/data-engineer-senior-quantiphi-united-states-7/
Never Restart a Failed AI Pipeline Again | Union.ai on Google Cloud
#MLOps #SovereignAI #AIInfrastructure #GKE #Kubernetes #GoogleCloud #GCP #Flyte #UnionAI #MachineLearning #AI #DataPipelines #DevOps #DataEngineering
#MLOps #SovereignAI #AIInfrastructure #GKE #Kubernetes #GoogleCloud #GCP #Flyte #UnionAI #MachineLearning #AI #DataPipelines #DevOps #DataEngineering
Never Restart a Failed AI Pipeline Again | Union.ai on Google Cloud
google.smh.re
November 10, 2026 at 8:50 PM
Never Restart a Failed AI Pipeline Again | Union.ai on Google Cloud
#MLOps #SovereignAI #AIInfrastructure #GKE #Kubernetes #GoogleCloud #GCP #Flyte #UnionAI #MachineLearning #AI #DataPipelines #DevOps #DataEngineering
#MLOps #SovereignAI #AIInfrastructure #GKE #Kubernetes #GoogleCloud #GCP #Flyte #UnionAI #MachineLearning #AI #DataPipelines #DevOps #DataEngineering
Why the people who look after your data should be in the room when you talk about AI governance
mrneilbryan.substack.com/p/why-the-pe...
#AIGovernance #DataEngineering #DataGovernance #ResponsibleAI
mrneilbryan.substack.com/p/why-the-pe...
#AIGovernance #DataEngineering #DataGovernance #ResponsibleAI
Why the people who look after your data should be in the room when you talk about AI governance
I keep noticing who isn’t there: the data engineers, the DBAs and the infrastructure people.
mrneilbryan.substack.com
November 7, 2026 at 9:20 AM
Why the people who look after your data should be in the room when you talk about AI governance
mrneilbryan.substack.com/p/why-the-pe...
#AIGovernance #DataEngineering #DataGovernance #ResponsibleAI
mrneilbryan.substack.com/p/why-the-pe...
#AIGovernance #DataEngineering #DataGovernance #ResponsibleAI
577 - Five or six people used to sit on this kind of data engineering work. An agent can chew through it and hand experts a first pass. How decent is that pass supposed to be?
youtu.be/xx1Fn1YMKdc
#dataengineering #aiagents #microsoftfabric #projectosmos
youtu.be/xx1Fn1YMKdc
#dataengineering #aiagents #microsoftfabric #projectosmos
November 10, 2026 at 11:50 AM
577 - Five or six people used to sit on this kind of data engineering work. An agent can chew through it and hand experts a first pass. How decent is that pass supposed to be?
youtu.be/xx1Fn1YMKdc
#dataengineering #aiagents #microsoftfabric #projectosmos
youtu.be/xx1Fn1YMKdc
#dataengineering #aiagents #microsoftfabric #projectosmos
577 - You can disconnect from the coding agent and the OSMOS task still keeps running. Fabric holds the context and the execution history after you leave. Does that second brain live off your laptop?
youtu.be/xx1Fn1YMKdc
#projectosmos #microsoftfabric #aiagents #dataengineering
youtu.be/xx1Fn1YMKdc
#projectosmos #microsoftfabric #aiagents #dataengineering
November 10, 2026 at 11:08 AM
577 - You can disconnect from the coding agent and the OSMOS task still keeps running. Fabric holds the context and the execution history after you leave. Does that second brain live off your laptop?
youtu.be/xx1Fn1YMKdc
#projectosmos #microsoftfabric #aiagents #dataengineering
youtu.be/xx1Fn1YMKdc
#projectosmos #microsoftfabric #aiagents #dataengineering
Learn how a data engineer rebuilt mobile game attribution using SQL, unified install data, revenue cohorts, and organic uplift modeling. #dataengineering
How I Built a Mobile Game Attribution Model Using SQL and Organic Uplift
hackernoon.com
November 8, 2026 at 6:49 PM
Learn how a data engineer rebuilt mobile game attribution using SQL, unified install data, revenue cohorts, and organic uplift modeling. #dataengineering
A silent SQL exclusion.
One NULL in an exclusion list can make NOT IN filter out every row. Check the list for NULL and consider NOT EXISTS with a correlated key. How do you test exclusion logic?
#SQL #DataEngineering
One NULL in an exclusion list can make NOT IN filter out every row. Check the list for NULL and consider NOT EXISTS with a correlated key. How do you test exclusion logic?
#SQL #DataEngineering
November 11, 2026 at 3:24 AM
A silent SQL exclusion.
One NULL in an exclusion list can make NOT IN filter out every row. Check the list for NULL and consider NOT EXISTS with a correlated key. How do you test exclusion logic?
#SQL #DataEngineering
One NULL in an exclusion list can make NOT IN filter out every row. Check the list for NULL and consider NOT EXISTS with a correlated key. How do you test exclusion logic?
#SQL #DataEngineering
Learn how a data engineer rebuilt mobile game attribution using SQL, unified install data, revenue cohorts, and organic uplift modeling. #dataengineering
How I Built a Mobile Game Attribution Model Using SQL and Organic Uplift
hackernoon.com
November 8, 2026 at 6:49 PM
Learn how a data engineer rebuilt mobile game attribution using SQL, unified install data, revenue cohorts, and organic uplift modeling. #dataengineering
Committed, but not visible?
One developer asked why a committed order was missing from a later read. Compare the write and read endpoints for that order ID before assuming the write was lost. Have you hit this?
#databases #dataengineering
One developer asked why a committed order was missing from a later read. Compare the write and read endpoints for that order ID before assuming the write was lost. Have you hit this?
#databases #dataengineering
November 10, 2026 at 3:50 AM
Committed, but not visible?
One developer asked why a committed order was missing from a later read. Compare the write and read endpoints for that order ID before assuming the write was lost. Have you hit this?
#databases #dataengineering
One developer asked why a committed order was missing from a later read. Compare the write and read endpoints for that order ID before assuming the write was lost. Have you hit this?
#databases #dataengineering
No timestamp, no trail?
I saw 1 question about syncing without a change timestamp. Compare keys and row hashes across complete snapshots to spot net changes—not every event. How do you track updates?
#dataengineering #database
I saw 1 question about syncing without a change timestamp. Compare keys and row hashes across complete snapshots to spot net changes—not every event. How do you track updates?
#dataengineering #database
November 9, 2026 at 4:53 AM
No timestamp, no trail?
I saw 1 question about syncing without a change timestamp. Compare keys and row hashes across complete snapshots to spot net changes—not every event. How do you track updates?
#dataengineering #database
I saw 1 question about syncing without a change timestamp. Compare keys and row hashes across complete snapshots to spot net changes—not every event. How do you track updates?
#dataengineering #database
Seattle and NYC, I’m Coming!
Been awhile since I’ve been to these cities so make sure to come join us at these Open Lakehouse & AI meetups where I’ll discussing the latest on V3, V4 and the REST Spec. Jaz will be talking Ontologies.
#ApacheIceberg #OpenLakehouse #DataEngineering #AgenticAI
Been awhile since I’ve been to these cities so make sure to come join us at these Open Lakehouse & AI meetups where I’ll discussing the latest on V3, V4 and the REST Spec. Jaz will be talking Ontologies.
#ApacheIceberg #OpenLakehouse #DataEngineering #AgenticAI
November 6, 2026 at 11:16 PM
Seattle and NYC, I’m Coming!
Been awhile since I’ve been to these cities so make sure to come join us at these Open Lakehouse & AI meetups where I’ll discussing the latest on V3, V4 and the REST Spec. Jaz will be talking Ontologies.
#ApacheIceberg #OpenLakehouse #DataEngineering #AgenticAI
Been awhile since I’ve been to these cities so make sure to come join us at these Open Lakehouse & AI meetups where I’ll discussing the latest on V3, V4 and the REST Spec. Jaz will be talking Ontologies.
#ApacheIceberg #OpenLakehouse #DataEngineering #AgenticAI
Show HN: WolveKafka – Consume Kafka with a plain HTTP endpoint
Discussion | hackernews | Author: piscesdk
#DataEngineering
Discussion | hackernews | Author: piscesdk
#DataEngineering
Show HN: WolveKafka – Consume Kafka with a plain HTTP endpoint
Consume Kafka with an HTTP handler in any language. Retries, DLQ, batching, and offsets handled for you. - wolveflake/wolvekafka
github.com
November 10, 2026 at 7:23 PM
Show HN: WolveKafka – Consume Kafka with a plain HTTP endpoint
Discussion | hackernews | Author: piscesdk
#DataEngineering
Discussion | hackernews | Author: piscesdk
#DataEngineering
One detail matters more than the benchmark: PRISM avoids the graph-based approach behind pgvector's HNSW index. Graph indexes carry memory overhead that grows with the data, and partitioning plus compression attacks that cost directly. Whether it holds at scale is what alpha
#DataEngineering
#DataEngineering
November 10, 2026 at 3:25 PM
One detail matters more than the benchmark: PRISM avoids the graph-based approach behind pgvector's HNSW index. Graph indexes carry memory overhead that grows with the data, and partitioning plus compression attacks that cost directly. Whether it holds at scale is what alpha
#DataEngineering
#DataEngineering
S3 Tables now migrates Apache Hive to Iceberg as a managed API call. No pipelines to build, sync while production runs, pay only for data copied.
Weeks of migration work just became an afternoon.
#Iceberg #DataEngineering
ift.tt/TZ2hjLf
Weeks of migration work just became an afternoon.
#Iceberg #DataEngineering
ift.tt/TZ2hjLf
November 4, 2026 at 11:39 PM
S3 Tables now migrates Apache Hive to Iceberg as a managed API call. No pipelines to build, sync while production runs, pay only for data copied.
Weeks of migration work just became an afternoon.
#Iceberg #DataEngineering
ift.tt/TZ2hjLf
Weeks of migration work just became an afternoon.
#Iceberg #DataEngineering
ift.tt/TZ2hjLf
❄️ Master Snowflake Data Engineering with DBT & Airflow!
👉 Course highlights:
✅ Expert trainers (live & real-time)
✅ Job-focused training
✅ Hands-on live projects
✅ Daily recorded sessions
✅ One-on-one sessions available
📱 WhatsApp: +91 7032290546
#Visualpath #AgenticAI #DataEngineering
👉 Course highlights:
✅ Expert trainers (live & real-time)
✅ Job-focused training
✅ Hands-on live projects
✅ Daily recorded sessions
✅ One-on-one sessions available
📱 WhatsApp: +91 7032290546
#Visualpath #AgenticAI #DataEngineering
November 9, 2026 at 11:55 AM
❄️ Master Snowflake Data Engineering with DBT & Airflow!
👉 Course highlights:
✅ Expert trainers (live & real-time)
✅ Job-focused training
✅ Hands-on live projects
✅ Daily recorded sessions
✅ One-on-one sessions available
📱 WhatsApp: +91 7032290546
#Visualpath #AgenticAI #DataEngineering
👉 Course highlights:
✅ Expert trainers (live & real-time)
✅ Job-focused training
✅ Hands-on live projects
✅ Daily recorded sessions
✅ One-on-one sessions available
📱 WhatsApp: +91 7032290546
#Visualpath #AgenticAI #DataEngineering
November 4, 2026 at 8:58 PM
A green run can hide gaps.
1 reader suggested tying checks to each run. Compare expected IDs with confirmed writes for that run, then inspect missing IDs before retrying. What do you check first?
#dataengineering #datapipelines
1 reader suggested tying checks to each run. Compare expected IDs with confirmed writes for that run, then inspect missing IDs before retrying. What do you check first?
#dataengineering #datapipelines
November 8, 2026 at 12:23 AM
A green run can hide gaps.
1 reader suggested tying checks to each run. Compare expected IDs with confirmed writes for that run, then inspect missing IDs before retrying. What do you check first?
#dataengineering #datapipelines
1 reader suggested tying checks to each run. Compare expected IDs with confirmed writes for that run, then inspect missing IDs before retrying. What do you check first?
#dataengineering #datapipelines
What AI can already do in data engineering:
- write ingestion scripts and API calls
- scaffold DAGs and dbt models
- explain error logs and suggest fixes
If that's most of your working day, start learning the parts of the job it can't do.
#dataengineering #databs
- write ingestion scripts and API calls
- scaffold DAGs and dbt models
- explain error logs and suggest fixes
If that's most of your working day, start learning the parts of the job it can't do.
#dataengineering #databs
November 8, 2026 at 3:56 PM
What AI can already do in data engineering:
- write ingestion scripts and API calls
- scaffold DAGs and dbt models
- explain error logs and suggest fixes
If that's most of your working day, start learning the parts of the job it can't do.
#dataengineering #databs
- write ingestion scripts and API calls
- scaffold DAGs and dbt models
- explain error logs and suggest fixes
If that's most of your working day, start learning the parts of the job it can't do.
#dataengineering #databs