It’s Postel’s law at the end of the day: “be conservative in what you do, be liberal in what you accept from others”. As a site owner I want my site to fail loudly and quickly before a user sees a problem; as a user I never want to see a problem.
ePub is in a nice place: the number of documents to check for errors is reasonable, and the resulting artefect is designed to be shipped and never (or rarely) amended. That means that we can shift the balance towards strict parsing. But for a web site of thousands (or millions) of documents that are being amended regularly, the balance shifts back to loose parsing as the best way of meeting user needs.
Isn't the developer always the first user? With strict parsing, testing a site before launch would show you the problem right there and allow you to fix it to launch a bug free site.
Postel's Law sounds nice but it can result in major problems. It results in a de facto spec that differs from the written spec, and disagreements about what a piece of data actually means can lead to bugs and even security vulnerabilities.
Having strictly parsed HTML from the start would be fine. You'd check it before you ship it and you'd make sure it's valid.
Requiring it now would be a disaster, of course. There's so much malformed HTML out there. But making HTML parsers accept garbage at the beginning was the wrong choice.
The widespread acceptance of Postel’s Law also encourages poor authorship, because if you know clients have to be liberal in what they accept, there is no incentive to be conservative in what you send.
Thanks for all the explanations. I always thought it was regular HTML, but now I know to watch out for the differences.
Can you say a few more words about the library https://github.com/standardebooks/tools ?
Can it generate ePub3 from markdown files or do I have to feed it HTML already. Any repo with usage examples of the `--white-label` option would be nice.
The tooling does two main things: create a valid epub3 skeleton for your content, and build your book into “compatible”, Kobo and Kindle versions. You need to supply the valid XHTML.
No, not really, and given the length of time ereaders stick around, you’re really probably best off assuming the worst. If there’s something complex layout-wise that I want to do then I often use @supports blocks. For example, in a recent book I worked on I had a layout that would be be accomplished with flexbox, but I started with floats instead and overrode that for better ereaders: https://github.com/standardebooks/harry-harrison_planet-of-t...
I was somewhat buying the article until I got to the monstrosity that is the “Special mention”, at which point I flipped to completely agreeing. That really is atrocious.
I’d love to poke at Homeworld again, but the remastered version seems to have failed to make the transition to ARM macOS. Anyone got an easy way to get it running?
I’m a contributor – I did Kafka’s The Castle, Agatha Christie’s Giant’s Bread, and Stella Benson’s The Faraway Bride for this launch – and I’m happy to answer any questions about Standard Ebooks.
I could not find out if there are outstanding todos that I could assign myself to as a newcomer. I'd like to contribute, but don't know where to start. Is there an issue board somewhere with missing books, or something else?
First-time contributors should select something from the appropriate section, because that gives you the greatest chance of succeeding and the least burden on our reviewers as you get started.
Yep! Have a look at our contribution page that lists a bunch of different options, including a list of good first books. https://standardebooks.org/contribute
Probably a dumb question, but how do you guys decide (and source) the book covers? I love how they look, but as a philistine can't put into words why.
Also thanks for doing this, I've read a bunch of stuff (GK Chesterton, Dorothy Sayers, Dashiell Hammett) that I wouldn't have otherwise if it weren't for this service.
The historical criteria is fine-art style oil painting. These days we’re starting to use first-edition cover art a bunch for more modern productions if it’s good quality. We also tend to use abstract oil paintings for sci-fi.[1] Obviously, all art is sourced from the public domain too. We’ve also started a database of confirmed-US-PD artwork that we can use for future productions.[2]
As Robin mentioned the typical style is "fine art oil painting", with some wiggle room allowed for exceptionally difficult cases (like Asian-themed books, as there just wasn't much fine art on that subject pre-1930).
We also require that the art have some kind of connection to the book itself, so it's not just some random fine art. Sometimes the connection is a little fuzzy, but we do the best we can given that art must be pre-1930 and also must have been previously published.
(My personal favorite artwork selection of the books I worked on is The Communist Manifesto[1]. That painting was actually made specifically for a different book by Willa Cather[2], but I thought the peasant laborer, holding a sickle in one hand, with a faraway look in her eyes as the red sun rises behind her was just too good to pass up for Marx!)
1920ish was when it started becoming much more common for books to have illustrated dust jackets, so now that more books from that era and onwards are entering the public domain, we opt to use the first edition dust jacket if it's in the appropriate style. Fortunately for us, that era also happens to be the so-called Golden Age of Illustration so it's not hard finding beautiful art to use!
Do you think the things that makes an edition special goes missing while converting to e.g. Standard Ebooks. I remember both the The Castle and Das Schloss like they had typesetting that helped me in perceiving the feel of the book. Is there anyway to preserve that feeling and still keep within the bounds of standardisation you adhere to? (I did a quick look through my copy and it does not seem to be much that makes it unique really, just the size of the book, and the chapter heading graphics..)
Do you know if the project try to look at other languages at all?
Nothing particularly in The Castle, from my production of it. As this was not previously PD there wasn’t any Gutenberg (or other) transcription available, so I did my own from the OCR of the original scans. A large part of the feel of the work, to me at least, comes from the extreme sentence / paragraph lengths though.
We do have a default typography across all our works (the “Standard” in “Standard Ebooks” refers to a standard imprint; think Penguin) but we usually retain specific famous things where possible in a reflowable format. For example, the Mouse’s Tail in Alice’s Adventures in Wonderland,[1] or the letter in E. A. Poe’s “Thou Art the Man”.[2]
We don’t take on other languages, no. Our tooling[3] and style guides[4] are tailored specifically to English. Absolutely nothing stopping another project from forking the codebase (it’s GPL-3) and giving it a go.
It seems you may be making assumptions that the formatting and typesetting of any particular edition were intentional or even deliberate on the part of the author, not any number of people, from editors to printers, who could and would have influenced those things for various reasons.
Something I am rather familiar with is brought out by your mention of the German edition/title; that the continental market seems to generally produce books that are far more densely formatted, i.e., smaller font and typesetting, thinner pages, and leading to overall tighter book formats. I actually appreciate it when, e.g., a book is 1/2 the size and weight, and usually also made far more durably; but it will invariably compromise any author intention related to the arrangement of the lettering.
Maybe you can confirm that based on what seems to be your English and German editions of the same novel.
Well that depends, there are obviously authors that care about these things. I have no idea what Brods intentions were with the book, and if he cared about layout.
The German and Swedish editions I read were similarly typeset, and the first scan I found in English felt similar. What I wanted to know was if there was some thought into it, because the website is nicely designed so striving for a unique typesetting strategy could be a goal.
I found it amusing, considering all those memes about German words with 35 letters each.
And, as I get older, I began to consider letter size relevant to choose a book edition. Gave up buying new books and went for used, older editions with bigger letters.
I seem to remember that they had some very opinionated rules at the beginning regarding allowed spelling and typography. Some of them felt distinctly American to me. I don't know if that's still the case.
It’s en-US typography (flavoured by the Chicago Manual of Style). Spelling is based on the original book, though some modernisations are made. Commits with these always start with [Editorial] for easy later reference, and are typically things like to-day -> today.
What’s the approach to embedding fonts in standard ebooks’ epubs? Curious whether there’s a set of fonts that producers are allowed to embed in finished books, or whether there’s project consciously avoids embedding fonts, and if so why.
I tried to find a policy page on this via a standardebooks.org site search but nothing looked relevant.
I’m asking after realizing that some of my favorite books were books where the ebook had intentional font choices, for example different fonts for chapter titles vs body text, fonts that matched the vibe of the book (historical, more modern, etc.) It would be nice if more ebook readers made it easy to import more than the ~8 fonts they include by default but the next best thing is when the book itself includes a great font.
The ebooks we produce are entirely in the US public domain, including metadata and any other files. Unfortunately there are basically no good fonts released under the CC0 license. (Most open fonts are released under the OFL license, which is not the same.) Therefore we don't embed any font files, except for Standard Blackletter[1] when necessary, which is a font we developed especially for our use based on public domain specimens, and released via the CC0 license.
Our projects are built with the broad assumption that the latest tech is supported across the board: the “advanced epub” is generated by zipping that up and renaming it to .epub. The build step then runs the book through a compatibility pass to remove some of the functionality that simply isn’t there in many readers. For example, it:
1. renders the cover as a JPG from the native SVG
2. converts more complicated selectors (e.g. :nth-of-type, :empty, etc.) to classes and class selectors
3. renders MathML as images
4. renders SVGs as images
…and a bunch more things to make it work better with older ereaders, which tend hang around for a long time.
It’s worth pointing out that the Kobo builds are closer to the “advanced” build as they have great native support for MathML and SVG. But the build step adds in a bunch of Kobo-specific markup to make it work better there.
In my experience, Apple Books does the best with the advanced builds, but it doesn’t support SVG covers so you end up with autogenerated ones, which seems a shame. I’ve sent a couple of feedback issues to them over the years, but if anyone else wants to also do this feel free.
How did you get these ready for release on Public Domain Day without breaking copyright law during the production process? I am not a lawyer, so I have only a surface understanding.
I’m not based in the US and have no intention of travelling there, so I don’t count myself in much danger for US copyright law.
Regardless, my understanding of copyright is that people broadly get annoyed when you infringe by distribution. In this case the distribution didn’t happen until the copyright had expired. People preparing these projects for later launch do it purely on their own machines, and nothing arrives at Standard Ebooks until the day of release.
It would be close to impossible to demonstrate loss, which is always highly relevant in copyright suits (not a lawyer, but I have scanned and digitized hundreds of thousands of pages at https://CatholicLibrary.org and read up on the topic over the years).
We do include illustrations, but in fairly defined categories: when they’re factually illustrative, when they’re graphs or charts, when they’re part of the plot (an illustration of a clue in a murder mystery, for example), or when they’re referenced from the text.
The last point is why the Beatrix Potter compilation I did ended up with illustrations: the text specifically references the illustrations in a couple of the books (“Can you see…?”) so they remained. It did mean writing 602 pieces of alt text though[1] so it was a fairly major undertaking to include them.
Have any public library systems ever tried to partner with Standard Ebooks to help improve the discoverability and accessibility of classics for ebook users?
It would be neat to see those editions show up in when browsing collections in apps like Libby or Overdrive (at least in the U.S.).
No, none have reached out yet. I've had some brief, high-level discussion along those lines with some people in the library industry, and the conclusion I drew is that public libraries in the US are highly fragmented in terms of technological capability. Instead of partnering with individual local library systems, it would make the most sense to - as you mentioned - partner with Overdrive. But there's been no movement in that direction. If anyone from Overdrive is reading, get in touch :)
I don't know their reasons but PDF is a rather problematic format so I suspect that's why.
You can run their EPUB through Pandoc to convert yourself, or put some effort in and setup your own Calibre instance which will do something similar when you ask it to.
>and I’m happy to answer any questions about Standard Ebooks.
When will they start assigning catalog numbers to each of their works in the way that Project Guteneberg does? I'd like a unique id per ebook and since you don't (for obvious reasons) use ISBNs, there's nothing really to be done. I can't use an OCLC id because Standard Ebooks aren't consistently listed in Worldcat, I can't use Bookbrainz or Open Library ids for the same reason.
It costs nothing and is a low-effort fix. It's been an industry (and library science) thing for decades or longer. Can Standard Ebooks finally stop the amateur hour crap?
I know you griped about this in a different thread, but we won't be doing that, sorry. You can uniquely identify an ebook and its version by using dc:identifier in combination with dcterms:modified in the metadata file. If you desperately need a filesystem-safe string then concatenate those two and sha it.
ePub is in a nice place: the number of documents to check for errors is reasonable, and the resulting artefect is designed to be shipped and never (or rarely) amended. That means that we can shift the balance towards strict parsing. But for a web site of thousands (or millions) of documents that are being amended regularly, the balance shifts back to loose parsing as the best way of meeting user needs.
reply