The way one company can bring down half the fucking internet
October 20, 2025 at 2:02 PM
The way one company can bring down half the fucking internet
We are back with a new blog post: The 2026 AWS #Nextflow Survival Guide
My new guide covers a cost-conscious, scalable setup using AWS Batch, local NVMe scratch, S3, IAM least privileges and nf-probe testing. Without relying on the Seqera Platform or Fusion.
lpembleton.rbind.io/posts/the-20...
My new guide covers a cost-conscious, scalable setup using AWS Batch, local NVMe scratch, S3, IAM least privileges and nf-probe testing. Without relying on the Seqera Platform or Fusion.
lpembleton.rbind.io/posts/the-20...
The 2026 AWS Nextflow Survival Guide – Luke Pembleton
An updated guide to setting up Nextflow on AWS Batch in 2026 without fusion
lpembleton.rbind.io
September 24, 2026 at 4:58 PM
We are back with a new blog post: The 2026 AWS #Nextflow Survival Guide
My new guide covers a cost-conscious, scalable setup using AWS Batch, local NVMe scratch, S3, IAM least privileges and nf-probe testing. Without relying on the Seqera Platform or Fusion.
lpembleton.rbind.io/posts/the-20...
My new guide covers a cost-conscious, scalable setup using AWS Batch, local NVMe scratch, S3, IAM least privileges and nf-probe testing. Without relying on the Seqera Platform or Fusion.
lpembleton.rbind.io/posts/the-20...
I can't do batch content creation because AWS fucked with Buffer and Canva. *sigh*
October 20, 2025 at 8:48 PM
I can't do batch content creation because AWS fucked with Buffer and Canva. *sigh*
It’s been two years since I wrote my original blog post on setting up #Nextflow with #AWS Batch, and I’m excited to share a refreshed version! 🎉
The update includes the awesome Fusion file storage system, which is a great alternative to pricey EBS storage 👍
lpembleton.rbind.io/posts/nextfl...
The update includes the awesome Fusion file storage system, which is a great alternative to pricey EBS storage 👍
lpembleton.rbind.io/posts/nextfl...
Luke Pembleton - Nextflow on AWS Batch with Fusion
An updated guide to setting up Nextflow on AWS Batch with fusion file storage
lpembleton.rbind.io
November 18, 2024 at 11:24 PM
It’s been two years since I wrote my original blog post on setting up #Nextflow with #AWS Batch, and I’m excited to share a refreshed version! 🎉
The update includes the awesome Fusion file storage system, which is a great alternative to pricey EBS storage 👍
lpembleton.rbind.io/posts/nextfl...
The update includes the awesome Fusion file storage system, which is a great alternative to pricey EBS storage 👍
lpembleton.rbind.io/posts/nextfl...
medium.com/@hobballah.y...
Using an event-driven approach and dynamic resources for on demand processing makes more sense then having always-on capacity. One nice way to handle this on AWS is using AWS Batch. With Batch you can use containers for your business logic and only what is needed.(1/3)
🧵
Using an event-driven approach and dynamic resources for on demand processing makes more sense then having always-on capacity. One nice way to handle this on AWS is using AWS Batch. With Batch you can use containers for your business logic and only what is needed.(1/3)
🧵
Building a Scalable Batch Processing Serverless Pipeline with AWS Step Functions and Batch
In this article, we will explore an event-driven architecture on AWS that uses AWS Step Functions to trigger lambda functions in the…
medium.com
March 7, 2025 at 1:56 PM
medium.com/@hobballah.y...
Using an event-driven approach and dynamic resources for on demand processing makes more sense then having always-on capacity. One nice way to handle this on AWS is using AWS Batch. With Batch you can use containers for your business logic and only what is needed.(1/3)
🧵
Using an event-driven approach and dynamic resources for on demand processing makes more sense then having always-on capacity. One nice way to handle this on AWS is using AWS Batch. With Batch you can use containers for your business logic and only what is needed.(1/3)
🧵
tfw a mandatory Copilot PR reviews added to your repos, and they're patently stupid.
December 16, 2025 at 2:11 PM
tfw a mandatory Copilot PR reviews added to your repos, and they're patently stupid.
"From Raw Node.js Scripts to Production OTT: Building a Scalable AWS Video Transcoding Pipeline" by Hariprasad takle
#amazon-s3 #aws-lambda #aws-batch #amazon-ec2 #amazon-cloudfront
#amazon-s3 #aws-lambda #aws-batch #amazon-ec2 #amazon-cloudfront
From Raw Node.js Scripts to Production OTT: Building a Scalable AWS Video Transcoding Pipeline
Transition from fragile, blocking Node.js FFmpeg scripts to a production-ready, event-driven video transcoding pipeline on AWS. Learn how to build an auto-scaling OTT media infrastructure using Amazon S3, AWS Lambda, Docker, AWS Batch, and CloudFront to process HLS adaptive bitrate streams cleanly at scale.
community.aws
September 27, 2026 at 7:00 AM
"From Raw Node.js Scripts to Production OTT: Building a Scalable AWS Video Transcoding Pipeline" by Hariprasad takle
#amazon-s3 #aws-lambda #aws-batch #amazon-ec2 #amazon-cloudfront
#amazon-s3 #aws-lambda #aws-batch #amazon-ec2 #amazon-cloudfront
CloudFront invalidations force edge caches to fetch fresh objects before TTL expiry. Track them in batches with the AWS CLI. https://www.valtersit.com/vault/execute-targeted-cloudfront-cache-invalidation-with-batch-tr-fc115d/ #aws #cloudfront #cache
Execute Targeted CloudFront Cache Invalidation with Batch Tracking
www.valtersit.com
September 26, 2026 at 7:00 PM
CloudFront invalidations force edge caches to fetch fresh objects before TTL expiry. Track them in batches with the AWS CLI. https://www.valtersit.com/vault/execute-targeted-cloudfront-cache-invalidation-with-batch-tr-fc115d/ #aws #cloudfront #cache
darryl-ruggles.cloud/serverless-d...
Here is a blog I wrote a while back about an event-driven architecture using a serverless approach to dataset/batch processing on AWS. (1/5)
🧵
Here is a blog I wrote a while back about an event-driven architecture using a serverless approach to dataset/batch processing on AWS. (1/5)
🧵
AWS Lambda Serverless Processor: Rust, Fargate
Serverless data processing with AWS Lambda, Step Functions, Fargate, Rust; scalable large batch processing, real-time updates
darryl-ruggles.cloud
November 16, 2024 at 5:21 PM
darryl-ruggles.cloud/serverless-d...
Here is a blog I wrote a while back about an event-driven architecture using a serverless approach to dataset/batch processing on AWS. (1/5)
🧵
Here is a blog I wrote a while back about an event-driven architecture using a serverless approach to dataset/batch processing on AWS. (1/5)
🧵
medium.alejofig.com/job-orchestr...
Using an event-driven architecture and serverless components in your AWS apps can help with decoupling, demand scaling and more. Using AWS Step Functions to fan out work to other services works well in many cases.(1/4)
🧵
Using an event-driven architecture and serverless components in your AWS apps can help with decoupling, demand scaling and more. Using AWS Step Functions to fan out work to other services works well in many cases.(1/4)
🧵
Job Orchestration Architecture in AWS
Using StepFunctions to orchestate jobs with AWS batch.
medium.alejofig.com
November 21, 2024 at 3:27 AM
medium.alejofig.com/job-orchestr...
Using an event-driven architecture and serverless components in your AWS apps can help with decoupling, demand scaling and more. Using AWS Step Functions to fan out work to other services works well in many cases.(1/4)
🧵
Using an event-driven architecture and serverless components in your AWS apps can help with decoupling, demand scaling and more. Using AWS Step Functions to fan out work to other services works well in many cases.(1/4)
🧵
"Container in AWS Batch retrieves secret from AWS Secrets Manager" by Mehdi Yosofie
#startups #containers #aws-batch #aws-secrets-manager #vpc-endpoint
#startups #containers #aws-batch #aws-secrets-manager #vpc-endpoint
Container in AWS Batch retrieves secret from AWS Secrets Manager
Learn how to securely access secrets in AWS Secrets Manager from containerized AWS Batch jobs.
community.aws
February 9, 2026 at 1:30 PM
"Container in AWS Batch retrieves secret from AWS Secrets Manager" by Mehdi Yosofie
#startups #containers #aws-batch #aws-secrets-manager #vpc-endpoint
#startups #containers #aws-batch #aws-secrets-manager #vpc-endpoint
AWS still cares enough about Intel to order up a fresh batch of custom Xeons
AWS still cares enough about Intel to order up a fresh batch of custom Xeons
Memory bandwidth boost appears to be the secret sauce in chips used for new memory optimized instance types
Amazon Web Services has revealed it’s started running some custom cuts of Intel’s Xeon 6 processors.…
dlvr.it
August 20, 2025 at 2:49 AM
AWS still cares enough about Intel to order up a fresh batch of custom Xeons
October 2025 batch of Kraken 2 indexes, including core_nt and many others, available: benlangmead.github.io/aws-indexes/k2
Coming soon to K2: a feature for querying many K2 indexes as though they're a single index. Highly useful if the index you want to query is too big to build and/or fit in RAM.
Coming soon to K2: a feature for querying many K2 indexes as though they're a single index. Highly useful if the index you want to query is too big to build and/or fit in RAM.
Index zone by BenLangmead
benlangmead.github.io
October 30, 2025 at 6:11 PM
October 2025 batch of Kraken 2 indexes, including core_nt and many others, available: benlangmead.github.io/aws-indexes/k2
Coming soon to K2: a feature for querying many K2 indexes as though they're a single index. Highly useful if the index you want to query is too big to build and/or fit in RAM.
Coming soon to K2: a feature for querying many K2 indexes as though they're a single index. Highly useful if the index you want to query is too big to build and/or fit in RAM.
Hey! 👋 I'm a technical product owner, and I'm a first batch AWS Community Builder for Serverless with interest in data engineering / analytics.
I created a Starter pack for AWS Serverless:
go.bsky.app/DatZhiS
I created a Starter pack for AWS Serverless:
go.bsky.app/DatZhiS
November 1, 2024 at 1:01 PM
Hey! 👋 I'm a technical product owner, and I'm a first batch AWS Community Builder for Serverless with interest in data engineering / analytics.
I created a Starter pack for AWS Serverless:
go.bsky.app/DatZhiS
I created a Starter pack for AWS Serverless:
go.bsky.app/DatZhiS
Weekly news update - 25.9.2026: ☁️ Cloud
AWS, Microsoft, and Google are expanding multi-cloud observability, AI agent isolation, and batch pipeline state preservation to support large-scale enterprise workloads.
🔒 Cybersecurity
Rapidly surging malicious bot… #cloud #cybersecurity #ai
AWS, Microsoft, and Google are expanding multi-cloud observability, AI agent isolation, and batch pipeline state preservation to support large-scale enterprise workloads.
🔒 Cybersecurity
Rapidly surging malicious bot… #cloud #cybersecurity #ai
Weekly news update - 25.9.2026
☁️ Cloud
AWS, Microsoft, and Google are expanding multi-cloud observability, AI agent isolation, and batch pipeline state preservation to support large-scale enterprise workloads.
🔒 Cybersecurity
Rapidly surging malicious bot traffic and autonomous AI agents are rendering static IAM and perimeter security controls obsolete, forcing a shift toward runtime authorization and behavior-based telemetry.
🤖 AI/ML
Major providers are releasing specialized low-latency agent runtimes and cross-cluster routing tools, while unauthorized agent actions highlight the need for deterministic IAM boundaries and strict execution limits.
Source: https://www.linkedin.com/pulse/weekly-news-update-2592026-eyal-estrin--adlhf/
#Cloud #Cybersecurity #AI
dlvr.it
September 25, 2026 at 5:34 AM
Weekly news update - 25.9.2026: ☁️ Cloud
AWS, Microsoft, and Google are expanding multi-cloud observability, AI agent isolation, and batch pipeline state preservation to support large-scale enterprise workloads.
🔒 Cybersecurity
Rapidly surging malicious bot… #cloud #cybersecurity #ai
AWS, Microsoft, and Google are expanding multi-cloud observability, AI agent isolation, and batch pipeline state preservation to support large-scale enterprise workloads.
🔒 Cybersecurity
Rapidly surging malicious bot… #cloud #cybersecurity #ai
📰 New article by Angel Pizarro
Using the Terraform AWS Cloud Control provider for managing AWS Batch resources
#AWS #HPC
Using the Terraform AWS Cloud Control provider for managing AWS Batch resources
#AWS #HPC
Using the Terraform AWS Cloud Control provider for managing AWS Batch resources
The Terraform AWS Cloud Control (AWSCC) provider now supports AWS Batch job definitions, enabling you to leverage recent and future enhancements to AWS Batch. Learn more in our latest blog post.
aws.amazon.com
February 11, 2025 at 2:26 PM
Private AI coding agents are now practical: open weight models on Bedrock keep your code inside AWS with IAM and PrivateLink controls, cutting batch refactor costs by 50%. https://aws.amazon.com/blogs/machine-learning/use-open-weight-models-as-your-ai-coding-agent-with-amazon-bedrock
September 24, 2026 at 6:05 AM
Private AI coding agents are now practical: open weight models on Bedrock keep your code inside AWS with IAM and PrivateLink controls, cutting batch refactor costs by 50%. https://aws.amazon.com/blogs/machine-learning/use-open-weight-models-as-your-ai-coding-agent-with-amazon-bedrock
Exciting updates to AWS Batch on Amazon EKS! Configurable namespaces, persistent volume claims, and more. Check out our blog post to see how these features can help manage your complex containerized workloads. aws.amazon.com/blogs/hpc/ad...
Adding configurable namespaces, persistent volume claims, and other features for AWS Batch on Amazon EKS | Amazon Web Services
Exciting updates to AWS Batch on Amazon EKS! Configurable namespaces, persistent volume claims, and more. Check out our blog post to see how these features can help manage your complex containerized w...
aws.amazon.com
February 3, 2025 at 8:09 PM
Exciting updates to AWS Batch on Amazon EKS! Configurable namespaces, persistent volume claims, and more. Check out our blog post to see how these features can help manage your complex containerized workloads. aws.amazon.com/blogs/hpc/ad...
Just submitted my AWS Community Builders application for 2025. Time flies, I've been a member since the first batch in 2020...
Fingers are crossed 😀
Fingers are crossed 😀
March 16, 2025 at 12:28 PM
Just submitted my AWS Community Builders application for 2025. Time flies, I've been a member since the first batch in 2020...
Fingers are crossed 😀
Fingers are crossed 😀
AWS batchで組むかSageMaker Batch Transformで行くか www.wix.engineering/post/sagemak...
SageMaker Batch Transform Unleashed: My Journey at Wix to Achieve Scalable ML
Insights and Lessons Learned from Wix's Implementation of AWS SageMaker Batch Transform for Streamlined ML Deployment at scale. Photo by Lajos Szabo on UnsplashIntroductionHello, fellow data enthusiasts! Today, I'm excited to share a behind-the-scenes look at a project we did at Wix — we're implementing AWS SageMaker Batch Transform to run machine learning models at scale. As I started working with SageMaker Batch Transform, I realized that practical documentation and examples were lacking, part
www.wix.engineering
October 18, 2024 at 5:03 AM
AWS batchで組むかSageMaker Batch Transformで行くか www.wix.engineering/post/sagemak...
"Amazon S3 for Beginners: A Simple Guide to Cloud Storage
" by SRI REETHIKA S aids
#aws-academy #amazon-s3 #cloud-storage #aws-batch #aws-docs
" by SRI REETHIKA S aids
#aws-academy #amazon-s3 #cloud-storage #aws-batch #aws-docs
Amazon S3 for Beginners: A Simple Guide to Cloud Storage
Learn the basics of Amazon S3, including buckets, objects, storage classes, security, and common use cases. This beginner-friendly guide helps you understand how S3 can be used to store and manage data in the AWS Cloud.
community.aws
September 25, 2026 at 6:30 AM
"Amazon S3 for Beginners: A Simple Guide to Cloud Storage
" by SRI REETHIKA S aids
#aws-academy #amazon-s3 #cloud-storage #aws-batch #aws-docs
" by SRI REETHIKA S aids
#aws-academy #amazon-s3 #cloud-storage #aws-batch #aws-docs
AI agents are batch jobs. My talk at AWS re:Inforce is on threat modeling batch jobs. I started writing about security automation using batch jobs before the AI and AI Agent craze but it’s all relevant. I added some information specific to AI to my presentation based on research. Hope you can join!
June 8, 2025 at 1:45 PM
AI agents are batch jobs. My talk at AWS re:Inforce is on threat modeling batch jobs. I started writing about security automation using batch jobs before the AI and AI Agent craze but it’s all relevant. I added some information specific to AI to my presentation based on research. Hope you can join!
The AWS CDK 2.96.0 is out! Some highlights:
F: batch, ecs
BF: codedeploy, custom-resources, dynamodb, lambda
A: glue, integ-tests, synthetics
Read more:
github.com/aws/aws-cdk/...
F: batch, ecs
BF: codedeploy, custom-resources, dynamodb, lambda
A: glue, integ-tests, synthetics
Read more:
github.com/aws/aws-cdk/...
Release v2.96.0 · aws/aws-cdk
Features
update AWS Service Spec (#27095) (4942658)
batch: Stabilize Batch (#27059) (5fc707a), closes #27054
ecs: add Amazon Linux 2023 to EcsOptimizedImage (#26989) (0bac8a5), closes #26988
Bug ...
github.com
September 14, 2023 at 4:45 PM
The AWS CDK 2.96.0 is out! Some highlights:
F: batch, ecs
BF: codedeploy, custom-resources, dynamodb, lambda
A: glue, integ-tests, synthetics
Read more:
github.com/aws/aws-cdk/...
F: batch, ecs
BF: codedeploy, custom-resources, dynamodb, lambda
A: glue, integ-tests, synthetics
Read more:
github.com/aws/aws-cdk/...
And if you are computational.... sometimes it's made brutally clear to you 🤣
October 21, 2025 at 12:44 AM
And if you are computational.... sometimes it's made brutally clear to you 🤣
I just realized there’s a video out of my presentation at the AWS Security Community Day. Check it out if AWS security is of interest to you. Thanks John Varghese !
youtu.be/9WS2mnaQ7dM?...
youtu.be/9WS2mnaQ7dM?...
Threat Modeling a Batch Job System on AWS [Advanced] - Teri Radichel
YouTube video by AWS Community Day
youtu.be
May 2, 2025 at 3:18 PM
I just realized there’s a video out of my presentation at the AWS Security Community Day. Check it out if AWS security is of interest to you. Thanks John Varghese !
youtu.be/9WS2mnaQ7dM?...
youtu.be/9WS2mnaQ7dM?...