jawah/charset_normalizer

Review signals: 79 known vulnerability. 1 static matches retained as context.

Reviewed as expected for a test run: attempted writes that failed, so nothing was written.

OSV reports a known vulnerability in this dependency version. 4 distinct locations. 71 findings.

1 hardcoded secret matches retained as context; full list in the evidence.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

• The package ships a CLI that detects the encoding of a file: Real input produced the expected result: "encoding": "utf_8"

  Source: This package comes with a CLI.

  Input: ./data/sample-french.txt

  Expected: JSON result with encoding utf_8

  Command: <guest-path> ./data/sample-french.txt

  Observed: "encoding": "utf_8", "encoding_aliases": [ "u8", "utf", "utf8", "utf8_ucs2", "utf8_ucs4", "cp65001" ], "alternative_encodings": [], "language": "French", "alphabets": [ "Basic Latin", "Control character", "Latin Extended-A", "Latin-1 Supplement" ], "has_sig_or_bom": false, "chaos": 0.1, "coherence": 82.71, "unicode_path": null, "is_preferred": true }

• The CLI prints a JSON result on stdout: Real input produced the expected result: "language": "French"

  Source: Since version 1.4.0 the CLI produce easily usable stdout result in JSON format.

  Input: ./data/sample-french.txt

  Expected: JSON with language French

  Command: <guest-path> ./data/sample-french.txt

  Observed: "language": "French", "alphabets": [ "Basic Latin", "Control character", "Latin Extended-A", "Latin-1 Supplement" ], "has_sig_or_bom": false, "chaos": 0.1, "coherence": 82.71, "unicode_path": null, "is_preferred": true }

Examples

• Minimal output mode prints only the detected charset: command completed successfully

  Source: README.md ### CLI (python -m charset_normalizer)

  Input: ./data/sample-french.txt with -m

  Expected: utf_8 printed

  Command: <guest-path> -m charset_normalizer -m ./data/sample-french.txt

  Observed: utf_8

Local or hosted: local

Detect the text encoding of a file with the normalizer CLI

cd <guest-path> && pwd && ls -la && id -u && python3 --version && cat README.md | head -c 6000
cd <guest-path> && sed -n '/Basic Usage/,/Python Usage/p' README.md | head -c 5000; echo ----; grep -n -i "install\|pip\|cli\|command\|python" README.md | head -60; ls bin src/charset_normalizer; cat .coldpass/* 2>/dev/null | head -c 300; git status 2>&1 | head -2
cd <guest-path> && grep -n "requires-python\|build-backend\|requires =" pyproject.toml; sed -n 95,104p README.md; python3 -m venv --help >/dev/null && echo venv_ok; ls <guest-path>
python3 -m venv <guest-path> && <guest-path> install -q --upgrade pip 2>&1 | tail -3; <guest-path> install <guest-path> 2>&1 | tail -5; echo EXIT=$?
cd <guest-path> && ls data | head -20; <guest-path> ./data/sample.1.fr.srt; echo EXIT=$?; <guest-path> -m ./data/sample.1.fr.srt; echo EXIT=$?
cd <guest-path> && <guest-path> ./data/sample-french.txt; echo EXIT=$?; <guest-path> -m ./data/sample-french.txt; echo EXIT=$?; <guest-path> -m charset_normalizer -m ./data/sample-french.txt; echo EXIT=$?
mkdir -p <guest-path> && cp <guest-path> <guest-path> && cd <guest-path> && <guest-path> -n sample-russian.txt | head -c 600; echo EXIT=$?; ls -la <guest-path>; grep -n -A3 "normalize" <guest-path> | head -30
cd <guest-path> && cat > probe.json <<'EOF'
{"kind":"cli","argv":["<guest-path>","./data/sample-french.txt"],"cwd":"<guest-path>","expected_text":"\"encoding\": \"utf_8\""}
EOF
cat > claims.json <<'EOF'
[{"kind":"claim","claim":"The package ships a CLI that detects the encoding of a file","source":"This package comes with a CLI.","input":"./data/sample-french.txt","expected":"JSON result with encoding utf_8","probe_kind":"cli","argv":["<guest-path>","./data/sample-french.txt"],"expected_text":"\"encoding\": \"utf_8\"","cwd":"<guest-path>"},{"kind":"claim","claim":"The CLI prints a JSON result on stdout","source":"Since version 1.4.0 the CLI produce easily usable stdout result in JSON format.","input":"./data/sample-french.txt","expected":"JSON with language French","probe_kind":"cli","argv":["<guest-path>","./data/sample-french.txt"],"expected_text":"\"language\": \"French\"","cwd":"<guest-path>"}]
EOF
cat > examples.json <<'EOF'
[{"kind":"example","claim":"Minimal output mode prints only the detected charset","source":"README.md ### CLI (python -m charset_normalizer)","input":"./data/sample-french.txt with -m","expected":"utf_8 printed","argv":["<guest-path>","-m","charset_normalizer","-m","./data/sample-french.txt"],"cwd":"<guest-path>","expected_text":"utf_8"}]
EOF
cat > cycles.json <<'EOF'
{"detected":false,"reason":"Entry points inspected (README, src/charset_normalizer/__main__.py, cli/). normalizer is a one-shot CLI and the library is a function-call API; no scheduler, loop, cron, webhook or --once/--now cycle is documented or present.","checks":[]}
EOF
for f in probe.json claims.json examples.json cycles.json; do <guest-path> -I -c 'import json,sys;json.load(open(sys.argv[1]));print("ok",sys.argv[1])' $f; done
ls -la <guest-path> /inputs 2>&1 | head; cd <guest-path> && <env-var>=<guest-path> python3 -I <guest-path> --validate --replay 2>&1 | tail -c 4000; echo EXIT=$?
cd <guest-path> && cat > report.json <<'EOF'
{"outcome":"working","fault_owner":null,"operation":"Detect the text encoding of a file with the normalizer CLI","operation_scope":"Installed from the pinned checkout into a Python 3.12.3 venv at <guest-path>; ran normalizer on data/sample-french.txt (detected utf_8, language French). Checker validation and replay passed.","steps":["python3 -m venv <guest-path>","<guest-path> install <guest-path>","<guest-path> ./data/sample-french.txt","<guest-path> -m charset_normalizer -m ./data/sample-french.txt"],"deviations":["README example ./data/sample.1.fr.srt does not exist in this snapshot; used data/sample-french.txt instead.","README install is 'pip install charset-normalizer -U' from PyPI; installed the pinned local checkout instead, which is the source under test (version 3.5.2).","Did not replay normalizer -n (normalize to new file) because it writes a file beside the input and would collide on replay."],"stuck_step":null,"main_blocker":null,"plain_failure":null,"suggested_fix":null}
EOF
python3 -I -c 'import json;print(json.load(open("report.json"))["outcome"])'; ls -la <guest-path>