Python Continue Decoding Despite Error
When using chardet for detecting character encodings, a low confidence mark can cause decoding issues. To mitigate this, specify ‘error’ with options like ‘ignore’, ‘replace’, or ‘xmlcharrefreplace’ to handle the decoding problems.
When using chardet you may get some confidence mark over a particular file. If that number is below one then you may face issues when decoding.
Specify error=<error_handle_strategy> can mitigate this issue.
The default is ‘strict’ meaning that encoding errors raise a UnicodeEncodeError. Other possible values are ‘ignore’, ‘replace’ and ‘xmlcharrefreplace’ as well as any other name registered with codecs.register_error that can handle UnicodeEncodeErrors.