**Command-line Tools can be 235x Faster than your Hadoop Cluster**
"This find | xargs mawk | mawk pipeline gets us down to a runtime of about 12 seconds, or about 270MB/sec, which is around 235 times faster than the Hadoop implementation."
#complexity #ShellTools #RightToolForTheRightJob […]
"This find | xargs mawk | mawk pipeline gets us down to a runtime of about 12 seconds, or about 270MB/sec, which is around 235 times faster than the Hadoop implementation."
#complexity #ShellTools #RightToolForTheRightJob […]
Original post on bookmarks.kvibber.com
bookmarks.kvibber.com
May 23, 2025 at 6:12 AM
**Command-line Tools can be 235x Faster than your Hadoop Cluster**
"This find | xargs mawk | mawk pipeline gets us down to a runtime of about 12 seconds, or about 270MB/sec, which is around 235 times faster than the Hadoop implementation."
#complexity #ShellTools #RightToolForTheRightJob […]
"This find | xargs mawk | mawk pipeline gets us down to a runtime of about 12 seconds, or about 270MB/sec, which is around 235 times faster than the Hadoop implementation."
#complexity #ShellTools #RightToolForTheRightJob […]