This was prompted by an error I was running into with the AWS s3 service: I needed to tell the transfer utility the size of the data, in bytes, when transferring large files.<\/p>\n
In this case I am looking at files of characters<\/em>. Some of these methods should work equally well for binary files, and others don’t. In the following examples, I’ll use the full text of Moby-Dick<\/em> from Project Gutenberg, A couple commands to get size in bytes immediately came to mind: All those options work. But what if the input isn’t a file on disk, and instead is an input stream? This is to demonstrate counting the bytes in a character stream coming from any source, so forgive the “useless use of cat”:<\/p>\n The output from There are at least 5 methods to find the size of a file using common command-line tools:<\/p>\n Know of others? Leave a comment below.<\/p>\n","protected":false},"excerpt":{"rendered":" This was prompted by an error I was running into with the AWS s3 service: I needed to tell the transfer utility the size of the data, in bytes, when transferring large files. In this case I am looking at files of characters. Some of these methods should work equally well for binary files, and … Continue reading Size of data in bytes<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[232],"tags":[],"class_list":["post-3424","post","type-post","status-publish","format-standard","hentry","category-tips-tricks"],"_links":{"self":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts\/3424","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/comments?post=3424"}],"version-history":[{"count":16,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts\/3424\/revisions"}],"predecessor-version":[{"id":3443,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts\/3424\/revisions\/3443"}],"wp:attachment":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/media?parent=3424"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/categories?post=3424"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/tags?post=3424"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}2701-0.txt<\/code>, as the target file. I retrieved the file using the following command:<\/p>\ncurl -O http:\/\/www.gutenberg.org\/files\/2701\/2701-0.txt<\/code><\/pre>\nls<\/code>, stat<\/code>, and wc<\/code>. <\/p>\n$ ls -l 2701-0.txt | cut -d' ' -f5\r\n1276201\r\n\r\n$ stat --format %s 2701-0.txt \r\n1276201\r\n\r\n$ wc -c 2701-0.txt | cut -d' ' -f1\r\n1276201<\/code><\/pre>\n$ cat 2701-0.txt | wc -c\r\n1276201\r\n\r\n$ cat 2701-0.txt | cksum | cut -d' ' -f2\r\n1276201\r\n\r\n$ cat 2701-0.txt | dd of=\/dev\/null\r\n2492+1 records in\r\n2492+1 records out\r\n1276201 bytes (1.3 MB, 1.2 MiB) copied, 0.00997434 s, 128 MB\/s<\/code><\/pre>\ndd<\/code> above is not the simplest thing to parse. It’s multi-line and sent to stderr<\/code>, so I redirected it to stdout<\/code> and grepped for “bytes”:<\/p>\n$ cat 2701-0.txt | dd of=\/dev\/null 2>&1 | grep 'bytes' | cut -d' ' -f1\r\n1276201<\/code><\/pre>\n\n