November 15, 2015
the Satoshi effect - Bitcoin paper success against the academic review system
One of the things that has clearly outlined the dilemma for the academic community is that papers that are self-published or "informally published" to borrow a slur from the inclusion market are making some headway, at least if the Bitcoin paper is a guide to go by.
Here's a quick straw poll checking a year's worth of papers. In the narrow field of financial cryptography, I trawled through FC conference proceedings in 2009, WEIS 2009. For Cryptology in general I added Crypto 2009. I used google scholar to report direct citations, and checked what I'd found against Citeseer (I also added the number of citations for the top citer in rightmost column, as an additional check. You can mostly ignore that number.) I came across Wang et al's paper from 2005 on SHA1, and a few others from the early 2000s and added them for comparison - I'm unsure what other crypto papers are as big in the 2000s.
|Conf||paper||Google Scholar||Citeseer||top derivative citations|
|jMLR 2003||Latent dirichlet allocation||12788||2634||26202|
|NIPS 2004||MapReduce: simplified data processing on large clusters||15444||2023||14179|
|CACM 1981||Untraceable electronic mail, return addresses, and digital pseudonyms||4521||1397||3734|
|self||Security without identification: transaction systems to make Big Brother obsolete||1780||470||2217|
|Crypto 2005||Finding collisions in the full SHA-1||1504||196||886|
|SIGKDD 2009||The WEKA data mining software: an update||9726||704||3099|
|STOC 2009||Fully homomorphic encryption using ideal lattices||1923||324||770|
|self||Bitcoin: A peer-to-peer electronic cash system||804||57||202|
|Crypto09||Dual System Encryption: Realizing Fully Secure IBE and HIBE under Simple Assumptions||445||59||549|
|Crypto09||Fast Cryptographic Primitives and Circular-Secure Encryption Based on Hard Learning Problems||223||42||485|
|Crypto09||Distinguisher and Related-Key Attack on the Full AES-256||232||29||278|
|FC09||Secure multiparty computation goes live||191||25||172|
|WEIS 2009||The privacy jungle: On the market for data protection in social networks||186||18||221|
|FC09||Private intersection of certified sets||84||24||180|
|FC09||Passwords: If We’re So Smart, Why Are We Still Using Them?||89||16||322|
|WEIS 2009||Nobody Sells Gold for the Price of Silver: Dishonesty, Uncertainty and the Underground Economy||82||24||275|
|FC09||Optimised to Fail: Card Readers for Online Banking||80||24||226|
What can we conclude? Within the general infosec/security/crypto field in 2009, the Bitcoin paper is the second paper after Fully homomorphic encryption (which is probably not even in use?). If one includes all CS papers in 2009, then it's likely pushed down a 100 or so slots according to citeseer although I didn't run that test.
If we go back in time there are many more influential papers by citations, but there's a clear need for time. There may well be others I've missed, but so far we're looking at one of a very small handful of very significant papers at least in the cryptocurrency world.
It would be curious if we could measure the impact of self-publication on citations - but I don't see a way to do that as yet.
Ledger - a journal for cryptocurrency papers
"Ledger" was recently announced as a journal for cryptocurrency papers, and the timing was rather spectacular. Everyone agrees this is a good idea.
Today I had a look, because I and some friends have some papers that might be published there. Several things reached out, so I thought I'd put them out here and see if they resonate.
1. The Ledger team seem to have taken on some criticism of the academic process and gone for more openness in several areas:
- Ledger has created a peer review system where reviews are publishable by authors. What Ledger have done is ensured that reviewers can be published and held accountable for their reviews. This should go some way to stopping academic cliques building up, a fate that I can attest to directly.
- Papers are CC-licensed so immediately and popularly available. Discourse is well served. I am not sure where the others are now, but I've had my arguments in the past with the proxies of Springer-Verlag wanting to own my mind. Those days are dead.
- Fast turn arounds promised.
2. Business-wise, Ledger is a direct competitor to existing forums Financial Cryptography (the conference) and to a lesser extent WEIS. Now, this is fine in my view as (a) the space has massively enlarged from the niche it once was, and we can easily support more forums, and (b) Ledger is oriented to the paper distribution process whereas others are primarily presentation-oriented and networking. Also (c) the founder and coiner of Financial Cryptography, Bob Hettinga, always made clear that this was a competitive market ;)
3. It is not immediately clear who the reviewers are. While the core might be its Editorial base, the asset of a peer-reviewed journal is its hardworking reviewers. Specifically, the asset can be attached.
4. And, immediately the attachment begins. If you look at the Editor's page, they have fallen into the same trap as the financial cryptography conference fell into in 1998 - academic control. Of the very long list of fine editors, only a tiny minority are outside the University system by either affiliation or title. Whatever you think of the academic world, it is very clear that it is a discriminatory system, and many fine contributions are squashed or stolen for it.
5. In which world, reputation and cites rule. Which leads to anonymous authorship:
Under extenuating circumstances, the journal may permit authors to publish under a pseudonym. Authors should include a statement describing why they wish to remain anonymous at the time of article submission. Only manuscripts where quality can be judged exclusively from the content presented in the paper, and where the scope of any conflict of interest problems would be limited (should they exist), will be considered for anonymous authorship.
Ledger are clearly skeptical of the notion of anonymous authorship because as academics they are so used to leaning on the reputation of the author. A bad paper by a leading author always trumps a good paper by an unknown, and it is practically the law that the profs must co-author the papers of the candidates so as to cross that barrier.
Ledger are thus clearly skeptical that the paper's words mean much independently of the author's reputation. Leaving it at odds with the Bitcoin community is as it is, as, under those rules, Satoshi's paper would not have been published, and we'd not be having these discussions. Now, it's fine for them to do this, but what I'd point out here is that this is further evidence of 4. above: academics setting themselves up to capture cites.
6. In not charging for papers, nor distribution & access, the Ledger has a clear financial business problem. It (probably) relies entirely on two sources: the volunteer time of reviewers, and the paid salary of academics.
The nature of scientific enquiry has moved on since the days of the controlled paper distribution. All papers from now on must be free of economic control, or we get the Satoshi effect - the most important paper in the field was never published in a forum, because under the rules of all the forums, it could not be published. The old forums out there had economic controls, and those controls were captured by the very people who could benefit from the controls - cites are promotions are money, and paper is trees is subscriptions.
Ledger presents the disturbing academic dilemma in a nutshell. The Internet has solved the paper-subscription economic barrier, but not the citation-peer-review circle. And, it leans very heavily on academics on salary, which is the other side of the same coin - what is the economic model that both sustains the machine, and rewards the quality?
If you're thinking I'm arguing both sides of this - you're right. I can see the problem. I don't have the answers - unless you want something superficial like "publish papers on the blockchain!" But we won't find the answers until we understand the problems.