My Reign as the King of Pings

I've been running Weblogs.Com since June for Dave Winer, who wanted to see if service performance could be improved as he began to receive seven-digit inquiries about selling it.

Weblogs.Com ran on Frontier for six years from its founding in 1999, handling the load reasonably well until the number of pings topped one million per day within the last year.

In a frenzied weekend, I recoded the site as an Apache/MySQL/PHP web application running on a Linux server, writing all of the code from scratch except for XML-Simple, an XML parsing library I adapted from code by Jim Winstead. Hosting was provided by ServerMatrix, which charges around $80-$140/month for a dedicated server running Red Hat Enterprise Linux 3 with a 1,200-gigabyte monthly bandwidth limit.

On an average day, my application served 34.65 gigabytes of data, took 1.1 million pings and sent 11,000 downloads of changes.xml, a file larger than 1 megabyte. The LAMP platform is ideal for running a high-demand web application for as little money as possible.

When Dave rerouted Weblogs.Com to my new server and it instantly deluged the box with more than a dozen pings per second, I felt like Lucy Ricardo pulling chocolates off the conveyer belt.

The server ran well, crashing only a few times over four months because of a spammer sending thousands of junk pings per minute. Every few days, I used the iptables firewall to block requests from the IP addresses of the worst abusers.

Business reporter Tom Foremski and others have suggested that the Weblogs.Com sale might reveal a lack of faith in blogging as a business.

I think the sale was motivated by the realization that the demands of running Weblogs.Com had become much too large for Dave's one-man company. He could either hire people and start pursuing revenue opportunities or sell the service.

VeriSign got a good deal acquiring it for a reported $2 million. The company's now at the center of the blogosphere, a giant web application and information network with more than 15 million users, and ought to be able to leverage those pings into new services built on XML, XML-RPC and RSS.

One thing I'd like to see is a real-time search engine built only on the last several hours of pings, which could be a terrific current news service if compiled intelligently. While I was running Weblogs.Com, I wanted to use my brief moment as the king of pings to extend the API, which VeriSign appears to be considering, but Dave didn't want to mess with things while companies were loading a truck with money and asking for directions to his house.

I want to pursue these ideas, either independently or in concert with VeriSign and Yahoo Blo.gs. No knock intended, but big companies tend to sit on purchases like this rather than implementing new features. Blogger still lacks category support two years after being purchased by Google, an omission so basic you have to wonder whether it's serious about fending off competition from Six Apart, UserLand, and WordPress.

Php · Mysql · Linux · Frontier · 2005/10/12 · 27 COMMENTS · Link

Comments

Are or were you employed by Dave Winer?

#1 · Robert Sayre · 2005/10/12

I was paid for this project on a freelance basis. I've never been one of his employees, either at Scripting News Inc. or UserLand.

#2 · Rogers Cadenhead · 2005/10/12

I wish yahoo had just sat on its purchase of blo.gs - instead they seem to have made it much worse. Subscription lists being altered, tons of erroneous pings being recorded, search function being removed, and lots of new spam pings.

#3 · Dave G · 2005/10/12

I think that's hilarious about switching it to LAMP. I heart LAMP. especailly PHP5 and its great XML stuff

#4 · kosso · 2005/10/12

Don't be disingenuous, please answer the question. Who wrote the checks that paid for your freelance work?

#5 · Charles · 2005/10/12

I thought my last comment was clear, Charles -- Dave paid me to do the work. Are either of you going somewhere interesting with this line of questioning?

#6 · Rogers Cadenhead · 2005/10/12

Jeez, some commenters are so demanding. What does it even matter who exactly paid him?

I enjoyed reading his story. =)

#7 · Dustin · 2005/10/12

Roger,

Thanks for your work on Weblogs.Com.

Thanks to Dave for paying for it. I'm happy to see he was compensated through the recent purchase.

Don

#8 · Donald Larson · 2005/10/12

Agree with Dustin. What a weirdo. Who cares who paid for Rogers' work, and what right do they even have to ask?

#9 · Jake · 2005/10/12

No, it wasn't clear, you made it sound like Dave did NOT pay you.

Anyway, I just wanted to establish for the record that you are one of Winer's PAID sycophants.

#10 · Charles · 2005/10/12

I, for one, welcome our Paid-Winer-sycophant overlords!

#11 · Joe · 2005/10/12

Charles,
You are WAY out of line here. Rogers is a good guy. If you dislike Dave take up your beef with him. If Rogers is guilty of anything, it's of being diplomatic, which last time I checked was a virtue, not a crime.
Joel

#12 · Joe1 Norve11 · 2005/10/12

I just wanted to establish for the record that you are one of Winer's PAID sycophants.

So noted. I'm all about the money. The truth is, I hated RSS 2.0 until Dave's friend Benjamin Franklin changed my mind. And I'm still on the fence about OPML. I could go either way on the subject. Know what I mean?

#13 · Rogers Cadenhead · 2005/10/12

Charles, fyi I also did a freelance work, Frontier IPC driver, for Dave eons ago. Does that make me one of the Paid-Winer-sycophants? What kind of shit filled thinking is that?

#14 · Don Park · 2005/10/12

Interesting story. It's a shame that Frontier was not up to the task. Where was the bottleneck in the process?

#15 · Todd Higgins · 2005/10/12

My god, the Winer haters are everywhere now. Did Dave confess to being the anthrax mailer or something and nobody posted it to a blog?

As for Google, Blogger and categories, hell I remember that coming up at a staff meeting while I was contracting for Ev three and a half years ago. But I think the situation's about to change since they've added tags to the bookmarks you can make on your Personal Search History. Where you can leave notes too. And which seems to have a new search operator (label:) as well. Not too far to jump that over to Blogger, one hopes.

#16 · BillSaysThis · 2005/10/12

Rogers - thanks for the story and insights.

#17 · Bill Riski · 2005/10/12

Hi Rogers

I love the power of LAMP. Thanks for providing a great real-world example

I love the power of one person to help plumb the web. Thanks Dave.

Given the history of VeriSign, their purchase of weblogs.com might be an indicator that they're less stupid than they used to be. Hopefully so.

-- stan

#18 · Stan Krute · 2005/10/13

Charles,

What part of Trollville do you live in? What is the name of your employer? How much do you get paid?

Gary

#19 · Gary Secondino · 2005/10/13

His name is Mark Pilgrim and he works for IBM in North Carolina.

#20 · Portal van Nederland · 2005/10/13

I would like to become a Paid-Winer-sycophants. Is there a special handshake? If not, I charge more.

#21 · Bubba the less rightious · 2005/10/13

Roger,

Ingore the knuckleheads - you were upfront about your involvement and how much you were paid is your business and yours to share if you decide. Interesting writeup - even though I'm a Perl guy, I enjoyed the technical stuff.

Best Wishes.

#22 · alek · 2005/10/13

No knock intended, but big companies tend to sit on purchases like this rather than implementing new features

Or, like Yahoo with blo.gs, they just take something that's incredibly useful and slowly break it. Like disabling search, the main means I have of adding new blogs to my favorites list. No can do. Search is temporarily disabled and has been for over a month, now.

Ask for information or news re: temporarily disabled search (maybe they *are* doing something with it)?

Silence.

(sigh)

#23 · Susan Kitchens · 2005/10/13

Holy crap! Simple question, simple answer. I guess it sounds a little snippy if you're looking for trouble. Jesus.

#24 · Robert Sayre · 2005/10/13

As I am completely flummoxed by the language here, my only defense is to cry "nerd." And then cry some more.

If Rogers is guilty of anything, it's criminal sexiness.

#25 · Uncle Mikey · 2005/10/14

Congressman McCarthy: "Are you now or have you ever been a friend of Dave Winer?"

Rogers:"On advice of counsel, I decline to answer the question since it violates my 5th amendment rights."

------------------

But seriously... I'm fascinated that by this more complete back story to the weblogs.com meltdown of 2004: "a frantic weekend". Dave Winer mentioned at the time that it was an outtage but he didn't disclose the nature of the problem and he was attacked for turning off individuals blogs, which you also offered to re-host for a fee, as I recall. It was a public display of friendship and good business skills.

You are a good friend to Dave Winer and I seriously hope that he shares some of the proceeds for the weblogs.com $2M sale to Verisign with you for saving his bacon. The service would have been worth a lot less if it ceased to function for a few weeks or months. I wouldn't go so far as to say that you have any rights to the proceeds but it's pretty clear that you might deserve a small share of the bounty.

The real value of weblogs.com was built through tireless evangilizing and standards building by Dave himself and I don't have any problems seeing him gain some measure of financial independence for that contribution to the net. He's a seriously misunderstood philanthropist in his contributions to the public domain through his software efforts with
RSS, XMLRPC, OPML and the message that everyone should become a publisher...

So, keep the real stories coming because they are so much more interesting than the superficial dramas of the blog-o-loid memes. We want some truth occasionally... and we want to know what
it's like to have a small role in the nets history.

It ain't a bubble for the folks that sold when the value was high. It's a bubble for those that bought in and held until it was shown to be worthless.

What will Verisign do with weblogs.com ping service? Maybe they'll need some help. They would be lucky to get your interest in any problems they might encounter.

#26 · visitor · 2005/10/15

i love my LAMP server

#27 · ostsee · 2005/10/18