How to convert a CSV to UTF-8 and fix broken characters

é where you expect é means a UTF-8 file was read as an 8-bit encoding such as Windows-1252 or Latin-1. In UTF-8, é takes 2 bytes, and an 8-bit reader shows each byte as its own character.

The reverse problem shows �, a question mark in a black diamond, or an empty box. That happens when a Latin-1 or Windows-1252 file is read as UTF-8 and the reader replaces the bytes that aren't valid UTF-8.

A CSV file is plain text with no header that says which encoding wrote it, so every program has to guess from the bytes or fall back to a system default. Four encodings cause almost all the trouble:

• UTF-8: the standard for web apps, databases and APIs. It uses 1 to 4 bytes per character and is identical to ASCII for plain English letters and digits.
• Latin-1 (ISO-8859-1): one byte per character for Western European languages. Bytes 0x00 to 0xFF map to the first 256 Unicode code points, so 0x80 to 0x9F are control codes, not letters.
• Windows-1252: Microsoft's variant of Latin-1. It matches Latin-1 except in 0x80 to 0x9F, where it puts the euro sign, curly quotes and long dashes.
• UTF-16: 2 or 4 bytes per character, usually behind a 2-byte byte order mark (BOM). A UTF-8 file can also start with a 3-byte BOM, EF BB BF; Excel needs it to spot UTF-8, while Python's csv module opened with encoding='utf-8' reads it as part of the first column name.

Drop a .csv, .tsv or .txt file onto rows.page and DuckDB-WASM reads it inside your browser tab, without uploading it. rows.page checks for a UTF-16 BOM, then for valid UTF-8, and reads anything else as Latin-1. A note under the query bar says which fallback it used: "Read as Latin-1 text." or "Read as UTF-16 text."

A Windows-1252 file that only uses letters such as é, ü or ñ has the same bytes as Latin-1, so it opens normally. The euro sign, curly quotes and long dashes are what stop it.

This works for UTF-8 files with a BOM, UTF-16 files and Latin-1 files:

1. Drop the .csv file onto rows.page.
2. Check the note ("Read as Latin-1 text." or "Read as UTF-16 text.") and scroll a few pages of the grid to confirm the accented text looks right.
3. Open the Export menu and choose CSV.

The export is UTF-8 without a BOM, comma-separated, with a header row, and there is no option to change the encoding, delimiter or BOM. It writes the current view, so a filter or a hidden column trims the file too. Cells that start with =, +, -, @, a tab or a carriage return get a leading ' so spreadsheets don't run them as formulas.

If rows.page shows "Couldn't open this data" for a CSV from an older Windows program, or you know it has euro signs, curly quotes or long dashes, convert it with iconv in a terminal:

iconv -f WINDOWS-1252 -t UTF-8 in.csv > out.csv

iconv maps every byte through the Windows-1252 table, so € becomes the 3-byte UTF-8 sequence E2 82 AC, and it leaves the commas and straight quotes as they are. Drop out.csv onto rows.page and it opens as UTF-8 with no note.

If iconv stops with an error such as illegal input sequence, the file has a byte that Windows-1252 doesn't define (0x81, 0x8D, 0x8F, 0x90 or 0x9D), so it probably isn't Windows-1252.

In Excel, choose File > Save As and pick CSV UTF-8 (Comma delimited). Like the other CSV types, it saves only the active sheet.

Opening a UTF-8 CSV in Excel is the harder part. Excel only assumes UTF-8 when the file starts with a UTF-8 BOM, so double-clicking a file without one, such as a rows.page export, can turn é into é. Import it instead:

1. On the Data tab, click From Text/CSV in the Get & Transform Data group.
2. Pick the CSV file.
3. In the preview, set File Origin to 65001: Unicode (UTF-8), then click Load.

On Windows, Notepad can convert a file too: open the CSV, choose File > Save As, set Encoding to UTF-8 and save.