Readwyn 0.1.0

What is a DjVu file, and why are so many scanned books in it?

Many of the DjVu books you find online are Internet Archive scans. The Archive made a DjVu copy of each book uploaded to it until it announced in February 2016 that it would stop, and Wikisource's help page on Internet Archive uploads still names it as one of the main sources of the DjVu files that Wikisource works from. The format itself is older. A research team at AT&T Labs started DjVu (said like the French déjà vu) in 1996, and its trick is to split each page into a sharp layer for the text and a soft layer for everything else and compress the two in different ways. A 300 dpi book scan then comes out several times smaller than the same pages stored as JPEG or TIFF images.

How DjVu shrinks a scanned page

The team's 1998 paper in the Journal of Electronic Imaging, High Quality Document Image Compression with DjVu, sets out the idea. Letters and line drawings need a high resolution to stay crisp. Photographs, illustrations and the tint of old paper change gently from pixel to pixel, and they survive being stored at a lower resolution. The encoder therefore pulls them apart.

The text becomes a one-bit mask coded with JB2, which is based on AT&T's proposal for the JBIG2 fax standard. LeCun's DjVu page describes how it works: JB2 finds shapes that repeat on a page, such as every lowercase "e" in one typeface, stores a prototype of each shape, and then stores only the list of places where it appears. The background goes through IW44, a wavelet coder, usually at a lower resolution than the text. Wavelet data is progressive, so a viewer can draw a rough page at once and sharpen it as more bytes arrive, which suited the aim that the 1998 paper states: sending document images quickly over slow connections.

In the 1998 paper, a colour magazine page scanned at 300 dpi compressed to between 40 and 60 KB, which the authors put at 5 to 10 times better than JPEG for similar visual quality. LeCun's page gives 5 to 30 KB for a black and white page at 300 dpi, 3 to 8 times smaller than TIFF Group 4, the usual format for such scans at the time. At those rates, a 400-page book of black and white pages comes to roughly 2 to 12 MB.

What is inside the file

A DjVu file ends in .djvu or .djv. Its media type, image/vnd.djvu, has an IANA registration created on 17 January 2002, with Léon Bottou of the AT&T team as the contact, and it cites the original specification that AT&T Labs dated 29 April 1999. The same record gives the signature that a program checks to recognise the file: the letters FORM at byte 4, then DJVU, DJVM, PM44 or BM44 at byte 12.

A page can also carry a hidden text layer made by OCR, stored with the image so that a reader can search the book and copy words out of it. Wikisource's help page describes the Archive's processing in those terms: it ran OCR on the scans and turned the result into a DjVu file with embedded text. The page itself stays a picture, so a DjVu book is always fixed layout. It can't rewrap its lines for a small screen the way an EPUB can (see fixed layout and reflowable ebooks).

Who made it, and what opens it

The credits page of DjVuLibre dates the start of the project to 1996 at AT&T Labs Research, where Yann LeCun began it, with Léon Bottou, Patrick Haffner, Paul Howard and Yoshua Bengio doing much of the research. LeCun's page says LizardTech, a Seattle company, bought the technology from AT&T in April 2000. In November 2000, according to the same credits, LizardTech published version 2.2 of the DjVu Reference Library under the GNU GPL, and later versions 3.0 and 3.5. DjVuLibre is Léon Bottou's clean-up of that released code, and the credits call him the original author of the library. On a computer, Okular, the KDE document viewer, opens DjVu through DjVuLibre.

E-readers are a thinner field. Kobo's list of supported file formats covers PDF, the CBZ and CBR comic archives, and plain image files such as TIFF, with no DjVu. Two makers build it into their own software. The user manual of the PocketBook Era Color lists DJVU among its book formats, with the caveat that for PDF and DjVu you can only scale the whole page, since its reflow mode, which rewraps the text, works for PDF alone. Onyx's product page for the Boox Go 7 (read in November 2026) has DJVU in its list of document formats and says its other Boox devices read the same formats. On a Kindle or a Kobo, install KOReader, a document reader for e-ink devices that lists DjVu among its formats and runs on Kindle, Kobo, PocketBook, Android and desktop Linux.

Why the Internet Archive stopped

The forum post of 26 February 2016, signed "brewster" as the Archive's digital librarian, gave three reasons: use of the format had declined, new files were coming out with errors, and the Java viewer was hard to support. The DjVu files already made would stay, unless a later change to an item made them obsolete. The Archive said it would go on producing its _djvu.xml files, the OCR output with the position of each word, so the DjVu name stayed in the file lists of new items after the format itself had stopped.

DjVu or PDF for a scanned book

LeCun's page carries no date, but its copyright line stops at 2004 and it still speaks of LizardTech as the company selling DjVu. It argued that PDF did not handle scanned documents well, colour scans above all. PDF gained the same tools afterwards. The ISO 32000 specification of PDF marks the JBIG2Decode filter for one-bit images as a PDF 1.4 feature and the JPXDecode filter for JPEG 2000 as PDF 1.5. With those, a PDF can lay a sharp text mask over a compressed background, the method known as mixed raster content (MRC).

The Internet Archive moved its own book PDFs to that method. Its open-source package, archive-pdf-tools, splits each page into a background, a foreground and a mask, and compresses the mask without loss, with JBIG2 or CCITT fax coding. In a November 2020 blog post, the Archive reported a compression factor of about ten for most of its books with this method.

None of that touched the files the Archive had already made. Wikisource's help page dates the change to March 2016 and warns that a book uploaded since then may have no DjVu file, but the older ones are still what Wikisource works from most: the same page says DjVu is the file type it uses more than any other.