How To Solve Error “UnicodeDecodeError: ‘charmap’ codec can’t decode byte” In Python

UnicodeDecodeError: ‘charmap’ codec can’t decode byte

“UnicodeDecodeError: ‘charmap’ codec can’t decode byte” is a common error related to reading a text file. The below explanations can help you know more about the cause of this error and solutions.

How does the error “UnicodeDecodeError: ‘charmap’ codec can’t decode byte” happen?

The error is related to the file reader. You will get this error while requesting to read a text file containing some characters that are not encoded in the declared codec. 

This error indicates that the decoding process could not be completed successfully. It may be because you have used a codec that cannot decode the texts you are reading. For example, you write the string to a file using utf-8 codec and then read a file with a different codec:

f = open("demo", "a", encoding="utf-8")
f.write("𝘈𝘈")
f.close()
f = open("demo", "r", encoding="windows-1252")
txt = f.read()

Output:

UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 1023: character maps to <undefined>

In general, the error occurs because your codec given doesn’t support decoding one or some characters in your strings into bytes that can be written print to the terminal. In some cases, the error message may be different, for instance:

f = open("demo", "a", encoding="utf-8")
f.write("𝘈𝘈")
f.close()
f = open("demo", "r", encoding="utf-7")
txt = f.read()

Output:

UnicodeDecodeError: 'utf7' codec can't decode byte 0xc3 in position 0: unexpected special character

How to solve this error?

Use another type of codec to decode characters

You must specify the correct encoding (mostly utf-8) when encoding and decoding the string into bytes or reading/writing it from/to the file.

f = open("demo", "a", encoding="utf-8")
f.write("𝘈𝘈")
f.close()
f = open("demo", "r", encoding="utf-8")
txt = f.read()

Output:

𝘈𝘈

Replace the incorrect char at the error position

Another way to overcome this problem is to delete the character that caused the error at the position in your error message. If you don’t want to delete the incorrect character, you can change it to some character in the Latin alphabet that doesn’t have any accent:

f = open("demo", "a", encoding="utf-8")
f.write("AA")
f.close()
f = open("demo", "r", encoding="utf-8")
txt = f.read()
print(txt)

Output:

AA

Summary

“UnicodeDecodeError: ‘charmap’ codec can’t decode byte” error occurs when encountering a UTF-8 character which is not supported by the current encoding. When saving a text file or reading it, checking for the UTF-8 codec is always recommended so that the encoding can be propagated to avoid such a trivial problem, as it wastes a lot of time to fix.

Maybe you are interested:

Leave a Reply

Your email address will not be published. Required fields are marked *