This whole thing is making Clojure seem more interesting to me. I'd be curious to see how close Node.js can come to extracting this much write performance from CouchDB.
For me, the problem has always been the indexed views taking too long to generate for a large amount of documents on first view. I know that some people do a bulk update, then hit the different indexed views to get them updated incrementally.
What can be done to increase the speed of the first view of new indexed views that you might have later on? Do you just have to wait for it?
What about ad-hoc views? Are there ways to speed that up?
The key is: don't experiment / develop on a giant database. Make a small subset of the database for testing views on. Then they are plenty fast.
As far as generation speed, what matters is that the indexer can keep up with the insert rate. Unless you are doing a big import from an existing dataset, you'll have to have A LOT of user activity to generate so much data that you are outrunning the indexer. In that case you probably have a big enough project that it makes sense to use a cluster solution like CouchDB-Lounge (which will divide the generation time by roughly the # of hosts you bring into the cluster).
Someday soon I hope we'll have an integrated clustering solution (Cloudant is working to contribute theirs back to the project) so you can just deploy to a set of nodes and get these benefits without much additional operational complexity.
i find this whole thing a lot more of an experiment in clojure performance than CouchDB throughput.
he doesn't attach a handler to the response on his writes and is queuing writes in memory and bulking them. he's not concerned with durability at pretty much any layer.
this kind of setup is great for a large initial import of data but i would be weary of running it in production.
This whole thing is making Clojure seem more interesting to me. I'd be curious to see how close Node.js can come to extracting this much write performance from CouchDB.