🍻 it's official! inaugural data debug sf happy hour is live
july 29 • 5:30-7:30pm • sf (venue tba)
for everyone who's ever said "the data looks weird" and meant it
casual networking • data war stories • good drinks (alc & non-alc) • great people
rsvp: lu.ma/g92ckftj
#datadebug #dataeng #sf #irl
july 29 • 5:30-7:30pm • sf (venue tba)
for everyone who's ever said "the data looks weird" and meant it
casual networking • data war stories • good drinks (alc & non-alc) • great people
rsvp: lu.ma/g92ckftj
#datadebug #dataeng #sf #irl
Data Debug SF: Inaugural Happy Hour · Luma
🍻 Join us for the inaugural Data Debug SF happy hour!
We're launching San Francisco's community for data engineers, analytics engineers, and data…
lu.ma
July 10, 2025 at 9:33 PM
🍻 it's official! inaugural data debug sf happy hour is live
july 29 • 5:30-7:30pm • sf (venue tba)
for everyone who's ever said "the data looks weird" and meant it
casual networking • data war stories • good drinks (alc & non-alc) • great people
rsvp: lu.ma/g92ckftj
#datadebug #dataeng #sf #irl
july 29 • 5:30-7:30pm • sf (venue tba)
for everyone who's ever said "the data looks weird" and meant it
casual networking • data war stories • good drinks (alc & non-alc) • great people
rsvp: lu.ma/g92ckftj
#datadebug #dataeng #sf #irl
WireScreen hires a Senior Software Engineer in New York, NY to build scalable Python/Django/FastAPI, gRPC, asyncio, Terraform & Spark data pipelines that power a global supply‑chain intelligence platform (knowledge graph). $170K‑$210K. #TechJobs #Python #DataEng #AI aihackerjobs.com/company/wire...
Senior Software Engineer - WireScreen · AI Hacker Jobs
Senior Software Engineer at WireScreen - Python, Django, FastAPI, gRPC, asyncio, Terraform, PySpark, Apache Spark, MapReduce, Databricks, Snowflake, Parquet, Backend, API, Data Pipelines, Knowledge Gr...
aihackerjobs.com
January 16, 2026 at 2:55 PM
WireScreen hires a Senior Software Engineer in New York, NY to build scalable Python/Django/FastAPI, gRPC, asyncio, Terraform & Spark data pipelines that power a global supply‑chain intelligence platform (knowledge graph). $170K‑$210K. #TechJobs #Python #DataEng #AI aihackerjobs.com/company/wire...
2025 is AI's defining moment. We're seeing incredible breakthroughs—medical research acceleration, unsolved math problems cracked, productivity gains—but also facing real challenges: energy demands, job disruption, security risks.
This is just the beginning.
#AI #DataEng #Tech
This is just the beginning.
#AI #DataEng #Tech
December 12, 2025 at 8:33 AM
It would be a treat to work with @chris.wensel.net so if you are #DataEng with flare for #Java #opensource & #devops you may want to look into this
Hey all, I'm hiring for a #DataEng (and #DevOps) role in the SF Bay Area. Reach out directly to me for more info.
#Java background is a strong want plus AWS and a desire to work on #OpenSource stuff.
#DataBS
#Java background is a strong want plus AWS and a desire to work on #OpenSource stuff.
#DataBS
March 27, 2025 at 7:24 PM
It would be a treat to work with @chris.wensel.net so if you are #DataEng with flare for #Java #opensource & #devops you may want to look into this
#RandomResourceDrop 🪂🎁
Three data-related subreddits:
www.reddit.com/r/dataeng…
www.reddit.com/r/datacur…
www.reddit.com/r/datahoa…
#Resource #Dataef="/hashtag/DataEngineering" class="hover:underline text-blue-600 dark:text-sky-400 no-card-link">#DataEngineering #DataManagement #Data
Three data-related subreddits:
www.reddit.com/r/dataeng…
www.reddit.com/r/datacur…
www.reddit.com/r/datahoa…
#Resource #Dataef="/hashtag/DataEngineering" class="hover:underline text-blue-600 dark:text-sky-400 no-card-link">#DataEngineering #DataManagement #Data
January 2, 2026 at 1:16 PM
#RandomResourceDrop 🪂🎁
Three data-related subreddits:
www.reddit.com/r/dataeng…
www.reddit.com/r/datacur…
www.reddit.com/r/datahoa…
#Resource #Dataef="/hashtag/DataEngineering" class="hover:underline text-blue-600 dark:text-sky-400 no-card-link">#DataEngineering #DataManagement #Data
Three data-related subreddits:
www.reddit.com/r/dataeng…
www.reddit.com/r/datacur…
www.reddit.com/r/datahoa…
#Resource #Dataef="/hashtag/DataEngineering" class="hover:underline text-blue-600 dark:text-sky-400 no-card-link">#DataEngineering #DataManagement #Data
Rust-powered streaming at its finest! Arkflow brings speed + extensibility to your data pipelines #rustlang #dataeng #streaming https://github.com/arkflow-rs/arkflow
GitHub - arkflow-rs/arkflow: High-performance Rust stream processing engine, providing powerful data stream processing capabilities, supporting multiple input/output sources and processors.
High-performance Rust stream processing engine, providing powerful data stream processing capabilities, supporting multiple input/output sources and processors. - arkflow-rs/arkflow
github.com
April 30, 2025 at 7:44 AM
Rust-powered streaming at its finest! Arkflow brings speed + extensibility to your data pipelines #rustlang #dataeng #streaming https://github.com/arkflow-rs/arkflow
Arkflow: A new Rust stream processor that combines performance with simplicity. Data pipelines just got better.
#rust #streaming #dataeng https://github.com/chenquan/arkflow
#rust #streaming #dataeng https://github.com/chenquan/arkflow
ArkFlow – High-performance Rust stream processing engine
Comments
github.com
March 14, 2025 at 10:22 AM
Arkflow: A new Rust stream processor that combines performance with simplicity. Data pipelines just got better.
#rust #streaming #dataeng https://github.com/chenquan/arkflow
#rust #streaming #dataeng https://github.com/chenquan/arkflow
🔥 Just dropped: GCP Data Engineer Certification!
⚡️ Inside scoop:
- Duke/Northwestern curriculum
- 101 hands-on lessons
- Production-ready skills
💪 From BigQuery to ML Ops in 4 weeks
🔗 ds500.paiml.com/learn/course...
#GCP #DataEng #BuildSzn
⚡️ Inside scoop:
- Duke/Northwestern curriculum
- 101 hands-on lessons
- Production-ready skills
💪 From BigQuery to ML Ops in 4 weeks
🔗 ds500.paiml.com/learn/course...
#GCP #DataEng #BuildSzn
Google Cloud Platform Certified Data Engineer
Master data engineering on Google Cloud Platform through hands-on learning. Develop expertise in storage solutions, data pipelines, ML operations, and security best practices while building production...
ds500.paiml.com
January 20, 2025 at 7:16 PM
🔥 Just dropped: GCP Data Engineer Certification!
⚡️ Inside scoop:
- Duke/Northwestern curriculum
- 101 hands-on lessons
- Production-ready skills
💪 From BigQuery to ML Ops in 4 weeks
🔗 ds500.paiml.com/learn/course...
#GCP #DataEng #BuildSzn
⚡️ Inside scoop:
- Duke/Northwestern curriculum
- 101 hands-on lessons
- Production-ready skills
💪 From BigQuery to ML Ops in 4 weeks
🔗 ds500.paiml.com/learn/course...
#GCP #DataEng #BuildSzn
🔥 Just dropped: Databricks Engineering Mastery!
⚡️ Inside scoop:
- Duke & Northwestern curriculum
- Hands-on Delta Lake mastery
- Complete cert prep
💪 From clusters to Unity Catalog in 4 weeks
🎯 85 lessons to level up!
🔗 ds500.paiml.com/learn/course...
#Databricks #DataEng #BuildSzn
⚡️ Inside scoop:
- Duke & Northwestern curriculum
- Hands-on Delta Lake mastery
- Complete cert prep
💪 From clusters to Unity Catalog in 4 weeks
🎯 85 lessons to level up!
🔗 ds500.paiml.com/learn/course...
#Databricks #DataEng #BuildSzn
Databricks Engineering Mastery
This course provides a comprehensive overview of Databricks platform capabilities for getting the Data Engineer certification. Learn to build data pipelines, use Delta Lake for data management, schedu...
ds500.paiml.com
January 18, 2025 at 5:50 PM
🔥 Just dropped: Databricks Engineering Mastery!
⚡️ Inside scoop:
- Duke & Northwestern curriculum
- Hands-on Delta Lake mastery
- Complete cert prep
💪 From clusters to Unity Catalog in 4 weeks
🎯 85 lessons to level up!
🔗 ds500.paiml.com/learn/course...
#Databricks #DataEng #BuildSzn
⚡️ Inside scoop:
- Duke & Northwestern curriculum
- Hands-on Delta Lake mastery
- Complete cert prep
💪 From clusters to Unity Catalog in 4 weeks
🎯 85 lessons to level up!
🔗 ds500.paiml.com/learn/course...
#Databricks #DataEng #BuildSzn
I see some discussion of folks diving into #ipeds data today. I noticed the Access Files are not available just yet. I have been cooking up an idea for a #dataeng project over winter break and have been waiting for the new files.
My question: Does anyone know if the Access files are close behind?
My question: Does anyone know if the Access files are close behind?
January 7, 2025 at 9:53 PM
I used Pinecone cloud's free tier to help with a course I ran this semester. Having a local option can help frame the development cycle for the students next year. I'm aware this is not the intended use case, but it certainly will help next year
www.pinecone.io/blog/pinecon...
#vectordb #dataeng
www.pinecone.io/blog/pinecon...
#vectordb #dataeng
Accelerate prototyping and development with Pinecone Local | Pinecone
With Pinecone Local, you can now prototype, test, and develop AI applications locally while seamlessly integrating workflow testing into CI/CD pipelines.
www.pinecone.io
December 7, 2024 at 2:38 PM
I used Pinecone cloud's free tier to help with a course I ran this semester. Having a local option can help frame the development cycle for the students next year. I'm aware this is not the intended use case, but it certainly will help next year
www.pinecone.io/blog/pinecon...
#vectordb #dataeng
www.pinecone.io/blog/pinecon...
#vectordb #dataeng
This looks very, very interesting. Looking forward to having some time after the semester to play around with the new S3 Tables bucket.
aws.amazon.com/blogs/aws/ne...
#dataeng #iceberg #aws
aws.amazon.com/blogs/aws/ne...
#dataeng #iceberg #aws
New Amazon S3 Tables: Storage optimized for analytics workloads | Amazon Web Services
Amazon S3 Tables optimize tabular data storage (like transactions and sensor readings) in Apache Iceberg, enabling high-performance, low-cost queries using Athena, EMR, and Spark.
aws.amazon.com
December 3, 2024 at 8:17 PM
This looks very, very interesting. Looking forward to having some time after the semester to play around with the new S3 Tables bucket.
aws.amazon.com/blogs/aws/ne...
#dataeng #iceberg #aws
aws.amazon.com/blogs/aws/ne...
#dataeng #iceberg #aws
Holy moly this looks amazing. I'm a big fan of SFN and excited to see where this service can go.
My scale has never been FAANG but it certainly addresses many data team requirements, at least in my view.
aws.amazon.com/blogs/comput...
#AWS #DataEng
My scale has never been FAANG but it certainly addresses many data team requirements, at least in my view.
aws.amazon.com/blogs/comput...
#AWS #DataEng
Simplifying developer experience with variables and JSONata in AWS Step Functions | Amazon Web Services
This post is written by Uma Ramadoss, Principal Specialist SA, Serverless and Dhiraj Mahapatro, Principal Specialist SA, Amazon Bedrock AWS Step Functions is introducing variables and JSONata data tra...
aws.amazon.com
November 27, 2024 at 1:54 AM
Holy moly this looks amazing. I'm a big fan of SFN and excited to see where this service can go.
My scale has never been FAANG but it certainly addresses many data team requirements, at least in my view.
aws.amazon.com/blogs/comput...
#AWS #DataEng
My scale has never been FAANG but it certainly addresses many data team requirements, at least in my view.
aws.amazon.com/blogs/comput...
#AWS #DataEng
DuckDB Quack
DuckLabs has recently announced Quack, a new remote protocol over HTTP that lets multiple DuckDB instances connect to and work with the same database over a network. The protocol introduces client-server capabilities to a database that was previously local and embedded.
#dataeng
DuckLabs has recently announced Quack, a new remote protocol over HTTP that lets multiple DuckDB instances connect to and work with the same database over a network. The protocol introduces client-server capabilities to a database that was previously local and embedded.
#dataeng
June 1, 2026 at 4:28 PM
DuckDB Quack
DuckLabs has recently announced Quack, a new remote protocol over HTTP that lets multiple DuckDB instances connect to and work with the same database over a network. The protocol introduces client-server capabilities to a database that was previously local and embedded.
#dataeng
DuckLabs has recently announced Quack, a new remote protocol over HTTP that lets multiple DuckDB instances connect to and work with the same database over a network. The protocol introduces client-server capabilities to a database that was previously local and embedded.
#dataeng
There are 3 Types of Data Engineers
Then there were three. The final three.
www.linkedin.com
October 2, 2024 at 5:44 PM
Vector databases, meanwhile, are all about unstructured data, built to support modern workloads like generative AI, machine learning inference, recommendations, and natural language processing.
#dataeng #vector #database
#dataeng #vector #database
Why vector databases aren’t just databases
Vector databases don’t just store your data. They find the most meaningful connections within it, driving insights and decisions at scale.
www.infoworld.com
September 30, 2024 at 11:49 PM
Tips and tricks for building resilient payment systems from a Staff Developer working on Shopify’s payment infrastructure.
shopify.engineering/building-res...
#dataeng #softeng
shopify.engineering/building-res...
#dataeng #softeng
10 Tips for Building Resilient Payment Systems - Shopify
Top ten tips and tricks for building resilient payment systems from a Staff Developer working on Shopify’s payment infrastructure.
shopify.engineering
September 20, 2024 at 8:45 PM
Tips and tricks for building resilient payment systems from a Staff Developer working on Shopify’s payment infrastructure.
shopify.engineering/building-res...
#dataeng #softeng
shopify.engineering/building-res...
#dataeng #softeng
🚨 FINAL CALL! 🚨
The clock is running out on our extended Boxing Day deals.
You have until THIS SUNDAY, Jan 11th to: 🎟️ Grab the cheapest conference tickets ($99 / $299) 🗣️ Submit your talk for the CFP
Get in quick! 🏃♂️💨
👉 dataengbytes.com/
#DataEng #LastChance #TechSale #CFP
The clock is running out on our extended Boxing Day deals.
You have until THIS SUNDAY, Jan 11th to: 🎟️ Grab the cheapest conference tickets ($99 / $299) 🗣️ Submit your talk for the CFP
Get in quick! 🏃♂️💨
👉 dataengbytes.com/
#DataEng #LastChance #TechSale #CFP
January 9, 2026 at 5:04 AM
🚨 FINAL CALL! 🚨
The clock is running out on our extended Boxing Day deals.
You have until THIS SUNDAY, Jan 11th to: 🎟️ Grab the cheapest conference tickets ($99 / $299) 🗣️ Submit your talk for the CFP
Get in quick! 🏃♂️💨
👉 dataengbytes.com/
#DataEng #LastChance #TechSale #CFP
The clock is running out on our extended Boxing Day deals.
You have until THIS SUNDAY, Jan 11th to: 🎟️ Grab the cheapest conference tickets ($99 / $299) 🗣️ Submit your talk for the CFP
Get in quick! 🏃♂️💨
👉 dataengbytes.com/
#DataEng #LastChance #TechSale #CFP
The Grinch has had ENOUGH! 🛑😤
He says you are all being far too cheerful about saving money.
Please, respect his wishes. Do NOT buy a ticket before the sale ends this Sunday, Jan 11th.
(Or do... and make him really mad). 😈
🎟️ Ruin his day here: dataengbytes.com/
#DataEng #Grinch
He says you are all being far too cheerful about saving money.
Please, respect his wishes. Do NOT buy a ticket before the sale ends this Sunday, Jan 11th.
(Or do... and make him really mad). 😈
🎟️ Ruin his day here: dataengbytes.com/
#DataEng #Grinch
January 5, 2026 at 10:23 PM
The Grinch has had ENOUGH! 🛑😤
He says you are all being far too cheerful about saving money.
Please, respect his wishes. Do NOT buy a ticket before the sale ends this Sunday, Jan 11th.
(Or do... and make him really mad). 😈
🎟️ Ruin his day here: dataengbytes.com/
#DataEng #Grinch
He says you are all being far too cheerful about saving money.
Please, respect his wishes. Do NOT buy a ticket before the sale ends this Sunday, Jan 11th.
(Or do... and make him really mad). 😈
🎟️ Ruin his day here: dataengbytes.com/
#DataEng #Grinch
⚡ SPONSOR FLASH SALE ⚡
Want to get your brand in front of Data Engineers in Auckland or Brisbane?
We are offering a Booth or Coffee Cart sponsorship for just $5k! (Single city). ☕🎪
Valid until Jan 31st only.
#DataEng #TechSponsorship #DevRel #Auckland #Brisbane
Want to get your brand in front of Data Engineers in Auckland or Brisbane?
We are offering a Booth or Coffee Cart sponsorship for just $5k! (Single city). ☕🎪
Valid until Jan 31st only.
#DataEng #TechSponsorship #DevRel #Auckland #Brisbane
January 3, 2026 at 11:57 PM
⚡ SPONSOR FLASH SALE ⚡
Want to get your brand in front of Data Engineers in Auckland or Brisbane?
We are offering a Booth or Coffee Cart sponsorship for just $5k! (Single city). ☕🎪
Valid until Jan 31st only.
#DataEng #TechSponsorship #DevRel #Auckland #Brisbane
Want to get your brand in front of Data Engineers in Auckland or Brisbane?
We are offering a Booth or Coffee Cart sponsorship for just $5k! (Single city). ☕🎪
Valid until Jan 31st only.
#DataEng #TechSponsorship #DevRel #Auckland #Brisbane
⚠️ THE COUNTDOWN IS ON! ⚠️
Don't miss the lowest prices for the 2026 Data Eng conference: 🇳🇿 AKL / 🇦🇺 BNE: $99 🇦🇺 SYD / 🇦🇺 MEL: $299
Sale ends Dec 31st. Run, don't walk! 🏃♂️💨
Grab tickets: www.tickettailor.com/events...
#DataEng #TechSale #DataEngineering #NYE
Don't miss the lowest prices for the 2026 Data Eng conference: 🇳🇿 AKL / 🇦🇺 BNE: $99 🇦🇺 SYD / 🇦🇺 MEL: $299
Sale ends Dec 31st. Run, don't walk! 🏃♂️💨
Grab tickets: www.tickettailor.com/events...
#DataEng #TechSale #DataEngineering #NYE
December 26, 2025 at 11:42 PM
⚠️ THE COUNTDOWN IS ON! ⚠️
Don't miss the lowest prices for the 2026 Data Eng conference: 🇳🇿 AKL / 🇦🇺 BNE: $99 🇦🇺 SYD / 🇦🇺 MEL: $299
Sale ends Dec 31st. Run, don't walk! 🏃♂️💨
Grab tickets: www.tickettailor.com/events...
#DataEng #TechSale #DataEngineering #NYE
Don't miss the lowest prices for the 2026 Data Eng conference: 🇳🇿 AKL / 🇦🇺 BNE: $99 🇦🇺 SYD / 🇦🇺 MEL: $299
Sale ends Dec 31st. Run, don't walk! 🏃♂️💨
Grab tickets: www.tickettailor.com/events...
#DataEng #TechSale #DataEngineering #NYE
🥊 BOXING DAY DATA DEALS ARE LIVE! 🥊
🇦🇺 Brisbane OR 🇳🇿 Auckland: $99* 🇦🇺 Sydney OR 🇦🇺 Melbourne: $299*
Don't wait—this sale ends Dec 31st or until sold out! 🏃💨
🎟️ grab yours: www.tickettailor.com/events...
#DataEng #BoxingDay #TechSale #DataEngineering
🇦🇺 Brisbane OR 🇳🇿 Auckland: $99* 🇦🇺 Sydney OR 🇦🇺 Melbourne: $299*
Don't wait—this sale ends Dec 31st or until sold out! 🏃💨
🎟️ grab yours: www.tickettailor.com/events...
#DataEng #BoxingDay #TechSale #DataEngineering
December 25, 2025 at 11:01 PM
🥊 BOXING DAY DATA DEALS ARE LIVE! 🥊
🇦🇺 Brisbane OR 🇳🇿 Auckland: $99* 🇦🇺 Sydney OR 🇦🇺 Melbourne: $299*
Don't wait—this sale ends Dec 31st or until sold out! 🏃💨
🎟️ grab yours: www.tickettailor.com/events...
#DataEng #BoxingDay #TechSale #DataEngineering
🇦🇺 Brisbane OR 🇳🇿 Auckland: $99* 🇦🇺 Sydney OR 🇦🇺 Melbourne: $299*
Don't wait—this sale ends Dec 31st or until sold out! 🏃💨
🎟️ grab yours: www.tickettailor.com/events...
#DataEng #BoxingDay #TechSale #DataEngineering
Our General Admission – DataEng Day Only Sydney tickets are available. If you’d like to attend the DataEng day only, book now before they’re all gone.
dataengbytes.com/sydney
dataengbytes.com/sydney
July 16, 2025 at 4:00 AM
Our General Admission – DataEng Day Only Sydney tickets are available. If you’d like to attend the DataEng day only, book now before they’re all gone.
dataengbytes.com/sydney
dataengbytes.com/sydney
Sydney people, as you may have already seen, Melbourne is sold out for General Admission – DataEng Day Only tickets, and we have only 14 left in Sydney. If you only want to attend Data Engineering Day, this is your chance to book a single-day ticket.
dataengbytes.com/sydney
dataengbytes.com/sydney
July 11, 2025 at 12:00 AM
Sydney people, as you may have already seen, Melbourne is sold out for General Admission – DataEng Day Only tickets, and we have only 14 left in Sydney. If you only want to attend Data Engineering Day, this is your chance to book a single-day ticket.
dataengbytes.com/sydney
dataengbytes.com/sydney
Melbourne, you’ve done it! We’re now officially sold out of the General Admission - DataEng Day Only tickets. There are only a few General Admission - 2 Day tickets left, and we’re confident they’ll be gone in no time.
Thank you for all your love and support.
Thank you for all your love and support.
July 10, 2025 at 4:00 AM
Melbourne, you’ve done it! We’re now officially sold out of the General Admission - DataEng Day Only tickets. There are only a few General Admission - 2 Day tickets left, and we’re confident they’ll be gone in no time.
Thank you for all your love and support.
Thank you for all your love and support.