Byte order mark (UTF-8 / UTF-16) file signature
Magic number at the start of the file
A byte order mark is the encoded form of the character U+FEFF placed at the very start of a text file. EF BB BF means UTF-8, FF FE means UTF-16 little-endian and FE FF means UTF-16 big-endian, so these three byte pairs identify the encoding rather than the file format.
What this magic number means
A magic number is a short, fixed run of bytes at a known position that tells a program what a file really is, no matter what the filename says. If a file starts with the bytes EF BB BF, the file is Unicode text encoded as UTF-8. The mark says how the text is stored, not which format it is.
In UTF-16 and UTF-32 the mark does real work: it tells a reader which way round the bytes of each character are stored. In UTF-8 there is only one byte order, so the mark carries no ordering information and simply signals that the text is UTF-8. The Unicode standard permits it there but does not require it.
That optional UTF-8 mark is behind a long list of everyday bugs. A shell script whose first line is EF BB BF #!/bin/sh will not run, a CSV opened by a parser that does not strip the mark shows a stray character in the first column header, and a PHP file with a BOM emits three bytes before any headers. Editors call the two options UTF-8 and UTF-8 with BOM.
When a text file carries a BOM, the mark sits in front of everything else, so a signature check on a .txt, .csv, .xml or .srt file may see EF BB BF before the content it expected. Skip the three bytes and then test the format.
Note that FF FE and FF FE 00 00 overlap: a UTF-32 little-endian file starts with the UTF-16 little-endian mark followed by two zero bytes, so a reader has to check four bytes rather than two before deciding.
What each byte of EF BB BF means
Here is the signature byte by byte, the way a hex editor shows it: the position in the file, the value in hex and in decimal, and the character that value stands for in ASCII. Bytes with no printable character show a dot.
| Byte offset | Hex | Decimal | ASCII |
|---|---|---|---|
| 0 | EF | 239 | . |
| 1 | BB | 187 | . |
| 2 | BF | 191 | . |
Signature variants
Byte order mark (UTF-8 / UTF-16) appears with more than one byte pattern. These all identify the same family:
| Hex signature | Offset | Note |
|---|---|---|
| EF BB BF | 0 | UTF-8 |
| FF FE | 0 | UTF-16 little-endian |
| FE FF | 0 | UTF-16 big-endian |
| FF FE 00 00 | 0 | UTF-32 little-endian |
| 00 00 FE FF | 0 | UTF-32 big-endian |
Formats that use the EF BB BF signature
These file types in our database carry this signature:
It is also seen in: CSV, XML, HTML, SRT, any Unicode text file.
How to check a file's signature
You can read the bytes of any file yourself. They are shown in hex, the same way this page lists them.
Linux & macOS
- xxd -l 16 example.txt
- hexdump -C -n 16 example.txt
- file example.txt
Windows (PowerShell)
- Format-Hex -Path example.txt -Count 16
Python
- open("example.txt","rb").read(3).hex()
In your browser
- Drop the file into the WhatFileType identifier, which reads the signature without uploading it.
Frequently asked questions
What is EF BB BF at the start of a file?
It is the UTF-8 byte order mark, the three bytes that encode the character U+FEFF. It marks the text as UTF-8 and is stripped by readers that understand it. The equivalents are FF FE for UTF-16 little-endian and FE FF for UTF-16 big-endian.
Should I save files with or without a BOM?
Without, in almost every case. UTF-8 has only one byte order, so the mark adds nothing and breaks shell scripts, CSV imports, JSON parsers and PHP output. Save with a BOM only when a specific tool asks for it, which on Windows is mostly older versions of Excel reading CSV.
How do I remove a byte order mark?
In VS Code, Notepad++ or Sublime Text, pick the encoding UTF-8 without BOM and save again. On the command line, sed -i '1s/^\xEF\xBB\xBF//' file.txt strips the three bytes from the first line.
Does a BOM mean the file is not plain text?
No. A file with a BOM is still plain text; the mark is simply the first three (or two) bytes of it. That is why a text editor shows nothing unusual while a program reading the raw bytes trips over an invisible character.
Related signatures
Sources
Reuse this signature
This entry is part of the WhatFileType file signature reference, published under a CC BY 4.0 licence with the full provenance for every entry, a copy-paste embed and a JSON export. Credit WhatFileType and the data is yours to republish.