I got the data by searching PubMed with the query
("[year]"[Publication Date])where [year] was each year from 1950-2009. Then I charted the results in R, and resized them in Photoshop.

The data, R code, and images  are all CC0 (public domain), and can be used wherever and for whatever you fancy.

small version of graphic

num-medline-articles-published-by-year.txt

1
2
3
4
5
6
7
8
9
10
11
12
13
14

# setup.
pub <- read.table("path_to_data_file", header=TRUE)
par(cex=2.2) # controls the relative size of the text
mainTitle <- "MEDLINE-indexed articles\npublished per year"
 
# make the plot.
# see http://www.harding.edu/fmccown/r/ for a nice intro on plot options
plot(pub, main=mainTitle, ylab='', xlab='', type="l", axes=F, 
     col='red', lwd=6, ylim=c(0,1000000))
 
# label the axes
axis(1, at=seq(1950, 2010, 10), lab=seq(1950, 2010, 10))
labs <- c('','','200k','','400k','','600k','','800k','','1M')# quick and dirty labels...
axis(2, at=seq(0, 1000000, 100000), lab=labs, las=2)  # "las" makes labeles display horiz.

So, I decided to lay it all out in a (relatively) simple timeline format; as far as I know, this doesn’t exist anywhere else.  You’re welcome, The Internet.  Below are my sources and some notes; where possible, links are to the original recommendations or RFCs:

SGML is an ISO standard from the 80′s.  Unlike the other standards on this list, it’s not open (the ISO sells copies for >$200).  HTML is an “SGML application“, and has been from the beginning. The Wikipedia article has a lot more information on its origins, as does the W3C.

HTML 2.0 and HTML 3.2 , the first two W3C specs, are both pretty straightforward. Also straightforward is XML, which dropped in February 1998. Like HTML, XML is “an application profile…of SGML.”

In December 1999, the HTML 4.01 recommendation came out, followed a month later by XHTML 1.0.  The important thing to note is that both of these are still HTML 4; however, XHTML is “a reformulation of HTML 4 as an XML 1.0 application,” while HTML 4.01 is still plain ol’ SGML.

No one knows yet exactly what HTML5 is going to look like, as it’s still several years off.  However, the W3C tells us that the HTML5, like HTML4, is going to have two different “serializations.”  One will be an XML syntax, and is currently being called XHTML 5 (wait, why not “XHTML 2?”  Hang on, we’ll get there). You might expect that the other serialization would be SGML a la HTML 4.01.  You’d be wrong.

Although HTML is technically SGML, most browsers and authoring tools couldn’t care less about the broader SGML standard; they just implement HTML.  So the W3C’s plan seems to be to ditch the SGML legecy and replace it with “html” (note the lowercase), an entirely new standard…which happens to look pretty much like HTML has always looked.

Whew, we’re almost done.  OK, what about XHTML2?  Despite the name, project was not a “next step;” it was a huge break with the whole HTML/XHTML tradition, an effort to completely remake web markup.  In July, the W3C decided to let it die on the vine and focus on HTML5.  So XHTML 5, with its HTML lineage, will be a more incremental change than XHTML 2 would’ve been.

There you have it.  If I missed anything or got something turned around, let me know.

“Take that, spammer scum!” I thought as I finished, basking in my newfound invulnerability to the v1@gr@-hawking vermin.  After all, if lots of people use address munging, it must work, right?

Right?

Darn it, now I’ve got to start reading about it.  So I did.  And after a few hours of reading blogs and writing code, I am now an Expert With Advice (hey, this is the internet).  And the advice is this:

Stop trying to obfuscate your email address.  Stop now.

I’ve got two reasons (and for a few more, some other folks have blogged about this, too).  First, the more theoretical one:

Spam is a problem for you–obfuscation makes it a problem for your users.

After all, they’re the ones who are going to have to do all the de-munging.  Are they always going to notice that they have to remove “.invalid” from the end?  Do they all know that the English “at” means “@”?   Do they have time to edit text in their address lines?   Address munging is fundamentally inelegant, because it intentionally works against clarity.

People have been making this argument for a very long time. It’s particularly relevant nowadays, though, because of the growing promise of the semantic web.  We want data to be machine readable, because then we can do cool stuff with it.  FOAF and the hCard microformat are pretty pointless if they don’t have real email addresses to work with.  “Hide the data from the machines” is a good strategy for fighting Skynet, but not for the future of the web.  Ok, reason two:

Address munging just doesn’t work.

It can’t.  It’s putting glasses on Superman.  Although in theory a valid email can be pretty hard to identify, in practice, emails addresses use a very limited vocabulary–and computers are good at identifying limited vocabularies.  Don’t forget, everyone has been using the same old [at] and “dot” tricks for decades–this is security through obscurity at its very worst.

But don’t take my word for it.  I took a couple hours and worked up a demo email obfuscation decoder that breaks the vast majority of text-based obfuscations; it’s also got an input field for you to test out your own munges (some other people have built similar demos, too).  It’s not perfect, but it correctly decodes most obfuscations–and remember that this is a novice programmer, working for an afternoon.  It’s that easy. Supporters of obfuscation argue that spammers will go after the low-hanging fruit; folks, text-based obfuscation is the low-hanging fruit.

Now, the Alert Reader has by this time noticed that I’ve limited my critique to text-based munging.  “What about more sophisticated methods,” the Alert Reader now asks?  “What about using an image, or CSS, or Javascript to hide addresses?”  Good questions, Alert Reader; you are very alert.  Alright, let’s take a quick look at these, too:

Images

There’s not really much I can say about this one, save this: making content completely opaque to visually-impaired users simply shouldn’t be an option. And of course, spammers still can OCR your images.

CSS

Obviously, something like  foo@bar<span style=”display:none”>NULL</span>.com is silly; the spambot can filter out “display:none” spans pretty easily, or even just discard everything in a span.  <span class=’a’>foo</span><span class=’b’>bar</span>@“<span class=’c’>foo</span><span class=’d’>bar</span>.com at least requires the bot to open your stylesheet to see which spans are hidden.  But remember, your server will happily dish out your easily-parsed css to anyone who asks for it; this is not a good place to hide secrets.

Javascript

There are too many js methods to cover in any detail here.  Some are better than others; a few try to degrade gracefully for users without Javascript support.  All of them, though, share the same weakness as CSS: everyone can read your Javascript.  And you certainly don’t need a browser to run it; there are lots of JS interpreters that are more than happy to run on a spammer’s server.

Sure, you can get pretty clever with this technique (I particularly like the idea of decoding not on the onload event, but on a click event), but you can’t change the fact that ultimately the bad guys can do everything with your code that a browser does–and eventually, they will.

Now, I’ll admit that images, CSS, and Javascript approaches are more effective than text-based ones.  All of them (when done properly) require the spammer to pay for more bandwidth and/or processor cycles.  But they all also inconvenience some or all of your users, and none of them are compatible with the sementic web.  They all give you false sense of security, and they’re ugly, hackish solutions. True, some obfuscations have performed well empirically–but keep in mind that these (pretty informal) experiments are years old.  As more people have adopted these measures, be sure that more spammers are spending the time to counter them, as well.

Now, I can’t go so far as to condemn anyone who obfuscates an address; I get that spam is a pain, and filters aren’t perfect.  Sometimes an ugly, hackish solution is the only way.  But I’m suggesting that you think twice before you give in to the spammers and obfuscate, especially given the relative ineffectiveness of many commonly-used methods.  The Web reaches its full promise when information is made easier to find, not harder.

Over the course of Dreaming, though,  we see this elite team gradually self-destruct. We see vague spec. We see unrealistic deadlines. We see huge mid-stream course changes.  As Rosenberg writes, “By now, I know, any software developer reading this volume has likely thrown it across the room in despair, thinking, ‘Stop the madness! They’re making every mistake in the book!’”  Dreaming finally ends four years into Chandler’s development—with version 1.0 still a distant vision (it was finally released, mostly to yawns, last August ).

Rosenberg, though, is savvy enough to turn the Chandler team’s failure into his own success.  Not only does he use the story to anchor an excellent (if basic) introduction into the practices and quirks of the industry as a whole, he weaves an engrossing and deeply human narrative.

Aristotle said tragedy should evoke fear and pity in the viewer, and Rosenberg deftly supplies us with both. On the one hand, Dreaming reads like watching a horror movie: “No! Why are you splitting up to explore the house!? Why do you keep changing the UI every 6 months!? Noooo!!!!” At the same, Rosenberg does a pretty good job of making us really like many of the characters. Kapor, in particular, comes off as both an intelligent visionary and genuinely good guy. Watching Chandler implode, I feel bad for him.

In interviews, Rosenberg shows again and again how the characters, all experienced programmers, understand the Classic Mistakes. Then he describes with agonizing clarity how they turn right around and proceed to make just those mistakes. I think it’s this quality that put me so in mind of classical tragedy, where the noble hero is undone by just these sorts of tragic flaws or mistakes.

Rosenberg resist the temptation to write another Lessons From Software Failure manual.  Instead he shows how smart, capable programmers working in an ideal environment can reenact the same fatal mistakes programmers were cataloging decades ago. Like Greek drama, Dreaming confronts the ineluctability of failure head-on.  Rosenberg’s ultimate thesis is nothing more or less than the classic words of  Donald Knuth, with which he opens the book: Software is hard. Sophocles would be proud.

Other reviews I liked:

• Amazon
• Joel Spolsky: discusses the technical aspects more; doesn’t think Chandler was a very good idea to begin with.  Has some good points, here.
  
• Adam Barr: discusses the individual parts of the book more.
  

My FeedVis project–the interactive tagcloud for a group of feeds–has been out for a week now, I’ve been thrilled at the positive response I’ve gotten so far.  One rather glaring problem with the program, though, was that you could only look at the top 50 edublogs.

Not anymore.  After a few late nights, I’ve got a beta system for uploading and analyzing your own sets of feeds.  You just upload your opml, wait a few minutes, and you’re set: FeedVis gives you a custom page that you can bookmark and return to anytime you like; it’ll continue to update every time you visit.  You can also browse visualizations of other people’s feeds.

It’s pretty untested, and I’m sure use will uncover some bugs.  But it’s got potential; I’m excited to see what people think.

Tagclouds have value, but, as I’ve written before, they’ve a number of shortfalls as well.  I’ve just finished my attempt to remedy some of these problems: FeedVis.  It’s an animated tagcloud that lets you compare word frequencies accross different time periods and authors, then check out the posts that used the words.  The demo is using the feeds for Scott McLeod’s Technorati-compiled list of top 50 edublogs, since that’s what got me started about feeds and tagclouds in the first place (although the program will work with any set of feeds).  More details about how it works are on the demo page.

I think what I’m really most excited about is the way this uses animation to let you actually see the words changing from one sample to the next.    Motion is such an important part of the way we see the world, and it’s been underemployed in information visualization, I think (although this changing; Hans Rosling’s TED talks have gotten a lot of buzz, for instance).

The project has been really fun, and a great learning experience; it’s gotten me really pumped about inofVis for learning about online interaction.  I think there is a lot of potential there for ed tech research.  I’m also pretty excited about programming; I started learning in February (with php), and then started javascript a couple months ago.  It’s been a really mind-expanding experience, and I’m looking foward to my next project, probably once I get done with grad school apps.

One difficulty I (and others) have had with Zotero, though, is in generating reports for a group of articles. Particularly, there’s no way to customize the categories you display in the report. This can be a real problem if you’re trying to share your sources with a co-author; at best, there’s a lot of unneeded metadata cluttering up the document (at worst, your email says you’ve been working on this for weeks, while your articles’ Date Added data tells a different tale…).

Now, I’m told this will be corrected in a later version of Zotero. However, I turned to PHP and a bit o’ regular expression magic to do it now. It turned out to be a good learning project, and I’ve been pleased to see that a few hundred other people (if Google Analytics is to be believed) have gotten some use out of it, too. The tool’s listed in the Zotero documentation, and–by far the most important of all–I got a free Zotero t-shirt out of the deal, which is now my favoritist garment ever.

I’ve also gotten quite a few feature requests from folks, including a request to help localize the script for German (you can find that German-language version here). Since my PHP skills have broadened in the last several months (I’m all the way to “novice” now!), I figured it was time to do an update. So, here is Zotero Report Customizer 2.0. New features include javascript form validation, a bunch of new categories, and the option to specify your own categories to delete if I don’t list ‘em. The script is also a ton easier to modify if you want to customize it to a different language, and can be set up to work in multiple languages at once. (I added a little German support for an example).

Have fun, and if you think of anything else you’d like in this, just let me know.