file_util.c: refactor and fix checksum_hex2bin()