python - Getting MD5 hash of the files is terribly slow -
i'm using following code md5 hash several files approx. total size of 1gb:
md5 = hashlib.md5() open(filename,'rb') f: chunk in iter(lambda: f.read(128*md5.block_size), b''): md5.update(chunk) filehash = md5.hexdigest()
for me, it's getting pretty fast takes 3 seconds complete. unfortunately users (having old pc's), method slow , observations may take 4 minutes user of file hashes. annoying process them, @ same think simplest & fastest way possible - right?
would possible speed-up hash collecting process somehow?
i have weak laptop well, , tried - can md5
1 gb in 4 seconds well. go several minutes, suspect it's not calculation reading file hard disk. try reading 1 mb blocks, i.e., f.read(2**20)
. should need far fewer reads , increase overall reading speed.
Comments
Post a Comment