Planet Beagle

Murder your darlings

2009-05-17T21:00:07+00:00

Lately I’ve been spending time getting rid of things that I don’t use or need. Books, toys, electronics, tools, clothing, pictures, gadgets, trinkets.

My theory is that the things that you own that you’re not using or don’t need aren’t just a waste of money and space: they’re draining you of your energy. Every time you walk past that cookbook that you never opened, or that model airplane kit that you meant to assemble, or the oscilloscope that you haven’t turned on in a year, a little neural pattern fires that says “Someday I should..” or “I always meant to…” or “God, I really ought to take care of that.”

Each of these tiny feelings of obligation or regret is almost imperceptible on its own, but their accumulation throughout the day is a burden that you may not even know you’re bearing until it’s gone.

I’m a pack-rat by nature. I love to own things that I think are cool or interesting. So I’m naturally collecting all kinds of junk that is neat at first but that I don’t truly need. And it’s hard to get rid of these things. The crappy video eyeglasses that plug into an iPod, the mountain bike headlamp with an enormous battery that fits into your water-bottle holder, a couple of old monitors, a broken GPS, a pile of bad books.

There’s a saying among writers about the process of copy-editing: murder your darlings. You may have crafted a beautiful phrase or metaphor in this paragraph, but if it’s not serving the whole piece, it’s got to come out. You have to murder your darlings.

So as Stephanie and I go around the house spring-cleaning, we’ll hold up this or that item, and ask “murder?” Sometimes you need to use a vicious word to make a hard decision.

It’s hard to do, but the result is a house that gives you room for the things that really matter.

Some early Avro benchmarks

2009-05-12T20:00:54+00:00

Avro is my current project. It’s a slightly different take on data serialization. Most data serialization systems, like Thrift and Protocol Buffers, rely on code generation, which can be awkward with dynamic languages and datasets. For example, many folks write MapReduce programs in languages like Pig and Python, and generate datasets whose schema is [...]

Cupcake!

2009-05-08T17:14:49+00:00

For users: Top 10 features you'll love about Android 1.5

For developers: Download SDK 1.5

Twitter norming: do you believe in God?

2009-05-07T12:41:45+00:00

Lately I’ve been using Twitter and Google Docs to find out how I compare to my presumptive peer group with little spot polls. A few weeks ago I found out that I do, indeed, wake up later than most of you. On Tuesday I asked the Twitterati if they believe in God. Here are the results, based on 172 replies:

Surprised? I was. I doubt that this poll would have had the same results three years ago, before the atheist “coming out” movement had so much momentum. Three years ago, I think there would have been a lot more agnostics.

I was also surprised that there were so few animists. How many people say they are “spiritual, but not really religious?” I think a lot of that is acting out a primitive animism. A genetic ur-religion, rooted in our instinct for anthropomorphization. Scientists who refer to subatomic particles as “guys,” programmers who say their code is “unhappy,” the urge to describe a car’s “personality.”

Crickets mourn -
sing out of genetic code.

Here’s a random selection of the comments people left on the poll:

I wish there were a god, but there probably isn’t and we can’t know and oh my god this dog is so beautiful with such a shiny coat and such well-formed hears and attractive feathering on its rump, clearly there is a god

A monotheistic form of Hinduism! [ I didn't even know this existed! --Nat]

Active Atheist. Working to start my own church celebrating science and discovery.

Anyone who thinks they can argue rationally for the existence of god(s) needs to read Kant. Anyone who thinks belief has any place near things we can know needs to read Saint(sic!) Augustine’s “de utiltate credendi”.

In America, Most of the people are atheistic about gods like Thor, some daring individuals go one god (Jesus) further. — Dawkins

I believe in one God but don’t judge those that don’t

I was raised a Catholic and still consider myself a Christian. I do, however, respect everyone’s choice (or non-choice) of religion, don’t try to push my beliefs on anyone.

I was raised Jewish but became estranged when I realized that Judaism and Zionism are inextricable. Today I consider myself a sort of vaguely proto-American Buddhist, except without the theological aspects.

I’m a born again Christian although I’ve also studied Messianic Judaism. Both believe that Jesus (Yeshua) is the son of God.

I’m a reverend in Church of Sweden (lutheran, but think episcopalian).

I’m sick of being tolerant and respectful to believers. Religion has way to much power and influence in our societies. Based on crazy people hearing voices. Watch George Carlin: Religion is Bullshit

I’m too lazy to pick a side. You can’t prove that there are or are not gods, so refusing to answer seems the more prudent solution. Does it really matter, regardless? Even if there is a God, we still need to solve our own problems. We’re not children.

No god, but I do enjoy many aspects of religious culture - which is to say community culture based on groupings by religion.

You can call God in different ways but he’s always the same

if a burning bush told you to kill your son today, people would rightly think you were high…

Thanks for playing!

The LiveJournal to WordPress migration

2009-04-26T16:56:22+00:00

Thought I’d outline a bit of what I did to get all my posts and tags migrated from LiveJournal to WordPress 2.7.1. Note that this information will be redundant soon enough — there’s much better LJ import support in the latest WordPress trunk. It’ll even pull in your ‘Current Music’ and ‘Current Mood’ fields, which I couldn’t do. :-(

Some background first. LiveJournal lets you export your blog posts one month at a time. You can feed these files to the WordPress LiveJournal importer. I’ve been blogging there since December 2003, so that was definitely not an option. Some digging around eventually brought me to ljdump. This is a really nifty tool, even if you just want to back up all your posts. It dumps your data into a large set of XML files, which you can collate with the convertdump.py script for uploading to the WordPress LiveJournal importer.

There was one hiccup here — a lot of the XML files corresponding to my earlier posts (at least) had an extraneous ASCII character 4 at the end of some lines. I had to use a simple for i in <lj-user>/*xml; do sed -i -e s:$'\004':: before using convertdump.py, and things were back on track (sed ftw!). I used the script to make one big XML file with all my posts, and fed it to the LJ importer, and all my posts were in.

But my tags, unfortunately, were not. ljdump happily pulls the tags from LiveJournal, but the importer just ignores them. I found a sort-of patch to fix this, but it seems to be quite antiquated. Based on this and the WordPress importer (that’s the importer that allows WordPress to import from another WordPress blog’s exported output), I wrote my own patch to import LJ tags (against WordPress 2.7.1). Just cd into your blog directory and do a patch -p0 < wp-livejournal-import-tags.patch to use it.

That’s it — I dropped all the old posts (requires a plugin to do it all at one shot), and then imported the big XML file again, and voila!

Trivial as it was, it was great to see how easy hacking the WordPress code was. There’s more to come in days ahead. I hope it remains this easy. :D

Update: Just noticed that the imported comments are not threaded. This kind of blows, because there have been some really long threads on some posts. I guess I’ll wait till the new WordPress goes stable and do a re-import. (file under #suckage)

What can you do to change the world?

2009-04-20T19:14:48+00:00

My friend Alex just spent 2 months in India and Nepal, where he hiked the 300km Annapurna Circuit. During a fifteen-hour layover in Munich yesterday, we had some beers and schnitzel together at a Biergarten, and Alex asked me, “Nat, you and I and all our friends are pretty smart, capable people. Why aren’t we working on something great that could save the world or be worthy of a Nobel prize?”

I love my work in the Linux world, and hope it has had some positive impact on the world. But what Alex said hit home. Right now, could I be doing something bigger? Something better? Could you? Even if you are passionate about your work, it’s a good call-out and an important thing to reflect about. If you’re not trying to do your very best, why not? Do you have a good reason?

Alex is pounding this drum on twitter.

This theme of Alex’s reminded me of Tim O’Reilly’s “work on stuff that matters” post from January. Tim had some nice guidelines for “stuff that matters,” but I liked the gut-instinct feeling of Alex’s question. What’s your Nobel-prize project?

I finally got to use my soldering iron for something

2009-04-20T07:25:31+00:00

Last month while he was visiting, my friend Rony and I built a picture frame that can display three images on a single piece of paper. Two of the images are mapped to the red and blue channels and linearly combined, and the third image (the word MARCH in the video above) is projected onto the paper from behind using a stencil.

Creating the stencil.

A microcontroller controls a set of red, blue, and white LEDs that light the picture, selecting each image in sequence by turning on one set of LEDs at a time. Rony built the frame itself out of the black foam-core that architects use to make models, and it is really gorgeous.

The entire package slides out neatly.

One of the challenges was calibrating the red and blue levels in the printed image such that under blue light the blue-printed image disappears completely, and the red image shows with good contrast, and vice versa under red light. This required a lot of different test printouts, which after the project was over I taped above my desk at home. I think they look pretty cool on their own.

Calibration images

The images were generated with a tiny opencv-based program that you can find here. If you want to use it yourself, you’ll probably have to recalibrate the WHITE_POINT macros for your printer/paper. We printed the final image on acid-free paper so the colors don’t change over time.

By the way, opencv is a great library for doing real-time computer vision. We used it for a very trivial operation, but the samples that ship with the library do things like real-time face detection, and there’s even an eye tracker that uses commercial USB webcams that some people are working on.

The light from the white LEDs is diffused a bit before it passes through the stencil.

This was my first project using an Arduino and I was completely blown away by the platform. The Arduino is an Italian-made open-source electronics prototyping platform. Ours was a very simple Arduino project, just fading in and out some LEDs (you can get the code here), but the platform can do a lot more. We used a Duemilanove (”2009″) board (pictured below) which has many digital input/output and analog input pins.

The board comes with a very simple IDE based on Java, Processing, and avr-gcc. You code for the device in C and a single click reprograms the onboard Atmel microcontroller over a USB cable. The documentation is excellent and the platform is extremely easy to code for; it only took us about 15 minutes to get the basic functionality working for this project. There’s a great serial interface you can use for printf-style debugging; just use Serial.println to send some output to your PC while your code is running. And Arduinos are extensible via a series of pluggable shields that can provide additional functionality like GPS, WiFi, and touch-screen support.

It really is the perfect starter platform for hardware hacking, and if you have any interest in this sort of thing at all, I strongly urge you to go buy the Arduino starter pack from adafruit industries right now.

Overall, this was an awesome way to spend a couple of days, and it was also great to work on a project with Rony, who deserves full credit for this idea (which he had while we were jogging around the Nymphenburger Palace) and for the majority of the work. You can see more photos of the project on Rony’s flickr photostream or mine.

Twitter Sleep Survey

2009-04-04T20:57:42+00:00

After waking up at 1pm this “morning,” I posted a survey to figure out when my fellow Twitterers go to sleep and wake up.

I used a Google Spreadsheet for the poll, which generates a nifty little summary. Here’s a screenshot of the scientific results:

Sleep Poll Results

The average sleep time of my twitter-peers seems to be around 7 hours and 15 minutes per night, which sounds about right.

Thanks to everyone who responded! You guys are a lot more normal than I expected.

Where I've Been

2009-03-31T12:18:16+00:00

I have been silent, I know. Both work and life keep me busy. At work, we released Android 1.1, which added voice search, Latitude, and paid apps. We continue to advance the platform, with exciting upcoming releases including the anticipated cupcake milestone. And of course there will be more phones.

In life, I spend most of my blogging cycles on my food blog (feed), knocking out several posts a week—that is not just a lot of blogging, but quite a bit of braising, infusing, roasting, and even foam making.

I find myself again with pen to paper—nothing anytime soon—and am happy to announce two new translations of Linux Kernel Development: Korean and Simplified Chinese. These new, updated, translations reflect the latest printing of the second edition. Find them at your local bookseller.

Also, I got engaged.

Outside Bankie Banx's Dune Preserve, Rendevzous Bay, Anguilla, Inauguration Day

But the largest reason for the radio silence is that a lot of my blogging is on economics and I do not have any insight into our current situation. It is disingenuous to blog otherwise. I don't have a great hold on what is going on, and neither do most commentators, including many economists. If the top macroeconomists are without agreement, I am not sure what a trade economist can add, let alone I.

That said, I did say this, seven months before AIG's liquidity crisis:

The problem: As CDS contracts are not collateralized or otherwise guaranteed, their real value depends on the creditworthiness of the involved parties. The CDS contracts are being marked to market as sizable profit, but if a series of defaults hit, can the reinsuring parties pay the hedgers?

Six months ago, I wrote that while the societal and economic situation is not as bad as during The Great Depression, the financial situation is worse. I believe that continues to be true. But therein lies our problem: This is largely a financial, not an economic, mess, thus our tonic is financial, not economic. Few outside of Wall Street fully understand the esotericism that led us here. Yet few inside of Wall Street are trusted by the public. The Obama administration, led by Secretary Geithner, continues to balance that opposition with a "fix" rather than "replace" Wall Street approach. That is my policy prescription, too.

Let's hope it works.

Flintstoning

2009-03-24T19:22:53+00:00

A few years ago, I broke my wrist in a snowboarding accident and hired an assistant to help me type, in a setup which gave me a simulation of near-perfect speech recognition:

If I don’t look up from the screen, I can pretend he’s not there and that I have the world’s most powerful voice recognition engine. So I have a sneak peek into what computers will be like when voice recognition works really well. … It is fun to try technology years before it exists. I wonder if there are other things we can simulate like this?

Today I learned that in the field of human-computer interaction, this type of simulation is known as a Wizard of Oz experiment.

Some people apparently call it “Flintstoning.”

So, the question stands.

What else can we simulate with a man behind the curtain?

there is only one correct way to skin a cat

2009-03-18T21:20:07+00:00

I have ranted about the iPhone’s horrible iPod interface in the past, and any improvement they can make is certainly welcome. But the improvements in the iPhone OS 3.0 update seem more half-assed than a true solution. Yes, the ability to skip back 30 seconds will be nice, but it’s still a ham-fisted solution to the problem of exact scrolling inside a 70 minute podcast. And the “scrubber” interface seems complicated and error-prone.

The frustrating thing is that Apple already has the One True User Interface for playing audio: the click-wheel. With its handling of acceleration you can both seek through hours of audio extremely quickly while still giving you the one-second resolution to seek to the exact point you want, and I don’t understand why it isn’t emulated on the iPhone and iPod touch. At this point, I have to believe that there is some limitation of the touchscreen hardware which prevents it. Sigh.

python daemon threads considered harmful

2009-02-25T17:50:30+00:00

The other day at work we encountered an unusual exception in our nightly pounder test run after landing some new code to expose some internal state via a monitoring API. The problem occurred on shutdown. The new monitoring code was trying to log some information, but was encountering an exception. Our logging code was built on top of Python’s logging module, and we thought perhaps that something was shutting down the logging system without us knowing. We ourselves never explicitly shut it down, since we wanted it to live until the process exited.

The monitoring was done inside a daemon thread. The Python docs say only:

A thread can be flagged as a “daemon thread”. The significance of this flag is that the entire Python program exits when only daemon threads are left. “

Which sounds pretty good, right? This thread is just occasionally grabbing some data, and we don’t need to do anything special when the program shuts down. Yeah, I remember when I used to believe in things too.

Despite a global interpreter lock that prevents Python from being truly concurrent anyway, there is a very real possibility that the daemon threads can still execute after the Python runtime has started its own tear-down process. One step of this process appears to be to set the values inside globals() to None, meaning that any module resolution results in an AttributeError attempting to dereference NoneType. Other variations on this cause TypeError to be thrown.

The code which triggered this looked something like this, although with more abstraction layers which made hunting it down a little harder:
try: log.info("Some thread started!") try: do_something_every_so_often_in_a_loop_and_sleep() except somemodule.SomeException: pass else: pass finally: log.info("Some thread exiting!")

The exception we were seeing was an AttributeError on the last line, the log.info() call. But that wasn’t even the original exception. It was actually another AttributeError caused by the somemodule.SomeException dereference. Because all the modules had been reset, somemodule was None too.

Unfortunately the docs are completely devoid of this information, at least in the threading sections which you would actually reference. The best information I was able to find was this email to python-list a few years back, and a few other emails which don’t really put the issue front and center.

In the end the solution for us was simply to make them non-daemon threads, notice when the app is being shut down and join them to the main thread. Another possibility for us was to catch AttributeError in our thread wrapper class — which is what the author of the aforementioned email does — but that seems like papering over a real bug and a real error. Because of this misbehavior, daemon threads lose almost all of their appeal, but oddly I can’t find people really publicly saying “don’t use them” except in scattered emails. It seems like it’s underground information known only to the Python cabal. (There is no cabal.)

So, I am going to say it. When I went searching there weren’t any helpful hints in a Google search of “python daemon threads considered harmful”. So, I am staking claim to that phrase. People of The Future: You’re welcome.

another beagle innovation stolen

2009-02-24T16:18:33+00:00

From the Safari 4 beta release:

* Full History Search, where users search through titles, web addresses and the complete text of recently viewed pages to easily return to sites they’ve seen before;

If you’ve been a Beagle user in the last 3 years this has been supported for Firefox and Epiphany users. But I wouldn’t mind seeing Firefox have this sort of indexing and search built-in either… the AwesomeBar was a great first step in that direction.

FOSSKriti ‘09 is here

2009-02-10T22:10:00+00:00

Been a hectic few months, but I could hardly miss posting about this. Some of you might remember the little F/OSS miniconf, we did last year at Techkriti, IIT Kanpur’s technical festival. FOSSKriti ‘08 sparked off a number of great F/OSS events in colleges across the country. FOSSKriti ‘09 is now here, bigger and badder than ever (for small values of ever :P)!

Last year, we started planning the event sometime in mid-Jan, and we did the best we could in about a month. This year, Shashank (better known as Chintal), Zakir, Surya, and the rest of team had more time, and you can tell that they’ve been busy. The theme for this year is "The Open Web", and we have an awesome line-up of talks, workshops, and hackfests around this theme. We’ve got folks from Mozilla, Drupal, Yahoo, and Sahana and more. It’s going to be four butt-kickingly amazing days!

Bottom line: If you’re in the vicinity, be there. It’s happening from Feb 12th to 15th, at IIT Kanpur.

p.s.: It blows that I can’t make it. :(

Better than my best

2009-02-04T16:16:00+00:00

Wow! "Toshiba handheld hits 1GHz"
http://news.cnet.com/8301-13924_3-10155730-64.html

The news isn't that tomorrow's phones will beat my current best computer. Technology, like leaking water, always finds a way to go down ... err... advance. I like it or not.

What surprises me is the abusive intent of folks behind the technology. Like it or not, designing is more art than science. The kind of rationality and precision that goes into using the technology to create something more than a crapware is nothing short of the artistic choices a painter makes when rendering the perfect sunset over an island where he never went.

The minds that control the coding hands do not seem to realise this ... no matter what better hardware I come across, I also come across a newer version of the same software which makes it crawl. Even the upcoming dual-core 1.5GHz is not going to impress me much, not any more. Perfection is the key.

We have an interval

2009-01-22T12:55:04+00:00

A beautiful quote that a friend sent me several years ago.

We are all condamnés . . .: we are all under sentence of death but with a sort of indefinite reprieve . . .: we have an interval, and then our place knows us no more. Some spend this interval in listlessness, some in high passions, the wisest, at least among “the children of this world,” in art and song. For our one chance lies in expanding that interval, in getting as many pulsations as possible into the given time (….)

– Walter Pater, “Conclusion,” The Renaissance: Studies in Art and Poetry, 1868(1)

giticular cancer

2009-01-06T03:36:14+00:00

I haven’t been on d-d-l for months now, but when someone mentioned how comically entertaining the whole DVCS survey thread was, I just had to catch up.

From my cursory understanding, it seems like the data would be stored as bzr repositories and then new code would be developed to export those repositories over the git protocol, so that git users could use their own tools.

While it seems like a neat hack, and probably a worthwhile proof-of-concept, the idea that GNOME would switch to it seems completely insane to me. There are lots of reasons why, many addressed in the thread:

Why develop this code only for GNOME, instead of developing it with the support and blessing of upstream bzr and/or git? There the collective experience of these communities could guide and influence the development.
There is basically only one person developing this software, resulting in a critical piece of GNOME infrastructure with a bus factor of 1. This is very bad, and when you consider that the results of the survey strongly support Git, the vast majority of developers will be using it and would be inconvenienced if the system failed.
GNOME has a terrible track record of abandoned software — maybe it’s not actually any worse than other 10 year old large-scale open source projects — but it is very common. I don’t have any data to back this up, but I feel that in a lot of ways this is even more true for infrastructure that most developers never see.
The git-over-bzr option was never an option in the DVCS survey, but if it were it seems like it would have rated extremely lowly. There seems to be a vocal opposition to it in the thread.
This is an abstraction, and abstractions are always leaky. wxWindows kinda sucks because it can’t emulate all windowing systems equally. There is no way a git compatibility layer on top of a bzr repository will ever be as good as a native solution, just like git-on-svn isn’t as good as a pure git solution.
What happens when (not if, when) the git protocol changes in an incompatible way? Will we be at the mercy of someone to hack and fix the compatibility layer? Will the original author still be around and interested enough to do it, possibly years down the road?

The last point, combined with the abandonware point earlier, are what concern me the most. In the thread, David Zeuthen asked Olav Vitters:

Then what happens when a new version of git with a new feature, incompatible with the git-serve kludge, is released? Then we’re screwed, right? And who gets to pay? We do. We’re stuck with an old version of git. Us. The very same people who very clearly said “git”, not “bzr”.

Not the most politic way of saying it, but I think the point is valid. When I read that, I had deja vu, because I had just read this thread from early December about Bugzilla, initiated by Olav:

Subject: Reduced Bugzilla functionality for 6+ months — acceptable?

The GNOME Bugzilla is still using 2.20. Current stable upstream is at 3.2.
[...]
For that the proposal is that the following is not part of the initial
upgraded bgo:
* The points system
* index.cgi UI mods
* Making a new favicon
* The infomessages on show_bug.cgi
* Layout modifications for attachment table and the login box
* duplicates.cgi modifications
* Fixing the comment headers
* Patch and keyword emblems
* delete-keyword.pl, mass-reassign-bugs.pl, and year-end-stats.pl
* describeuser.cgi

In other words, upgrade the GNOME Bugzilla installation to a new version of the upstream software, and break all of GNOME’s current customizations. Is this not exactly what will happen eventually with the git kludge? I can foresee history repeating itself here. Bugzilla is pretty essential to GNOME, and degradation of service is undesirable. But a degraded, unavailable or fractured source control system is unconscionable.

Happy New Year

2008-12-31T14:08:51+00:00

This New Year's Eve, while you drink and dine and dance, as sapping as this roller coaster of a year has been, never take for granted what you have and who you are.

I'm luckier than most.

The Big Picture, The Globe's beautiful photography blog, features an amazingly well composed photo tour of Israel and Hamas's deterioration after six months of relative but illusive calm.

Even they are luckier than some.

best. christmas present. ever.

2008-12-25T20:40:50+00:00

On 20 December 2008 at 1:41 am, Brette and I welcomed our son Elliot into the world. Both mom and baby are doing very well, and we’ve loved the last several days getting to know each other.

This is his “black power” pose. Fight the man, little guy.

The previous day Brette, my mom and I were joking around that the baby would almost certainly come the following day — still a few days before his due date — during the snowstorm that was expected to drop up to 15 inches of snow on Boston. He didn’t disappoint. We found ourselves walking to the birth center about half a mile away in the 4 or 5 inches that had fallen at that point on nearly deserted roads during Friday rush hour, stopping briefly whenever Brette had a contraction.

The birth went amazingly well and I am so proud of Brette. All of the nurses and midwives at the birth center were talking about what a wonderful job she did, and I couldn’t agree more.

A few things I wanted to mention:

If you prefer to give birth in a hospital, or want a pre-scheduled c-section, go for it. Do whatever is most comfortable and appropriate for you. But Brette and I wanted a natural childbirth without surgical intervention unless medically necessary, and for us the Cambridge Birth Center was the perfect place. Situated inside an old Victorian house, it sits across the street from the Cambridge Hospital. The rooms are homey and comforting. The only thing that annoyed us was the constant snow plowing going on just outside our window… but once things got down to business we weren’t paying attention to any of that. The nurses and midwives were so wonderful and accommodating. We were allowed to walk around, bring whatever food we wanted with us, got to sleep after the birth and go home the following afternoon. Just a great experience.

Ok, this is important: get a doula. I think I was as helpful and comforting as I could be for Brette, but look, I don’t have a uterus. I never have and probably never will. But a doula does. I don’t know physically what it’s like to give birth. A doula very well may have gone through it herself. But most importantly, they have training on ways to make the laboring woman more comfortable, and her entire job is to support and comfort. Ours was fantastic, and I can’t imagine going through the birth without her.

Also many thanks to my mom, who has been with us for the past week. She has helped us tremendously with keeping the house clean and orderly, making us food all the time, giving us a relief when we’ve been overwhelmed with E. at all hours, and sharing her experience with us.

Merry Christmas.

your personal rate of return is -50.2%

2008-12-13T18:15:04+00:00

Market got you down?

Feel like the Google Finance favicon doesn’t accurately reflect the year you’ve had? Well, I can help! This Greasemonkey script will correctly replace the favicon with one more representative of the ongoing collapse of the entire financial system.

Enjoy it while you can! Pretty soon you’ll have to sell your computer and move to the poor house.

Update: Unlike the lame duck administration, I welcome oversight on my activities! Aaron suggests that the icon should be red instead of green. I agree with him. The red represents two things: (1) the financial value of your dwindling retirement nest egg and (2) the rage you will feel when a callous loan servicer refuses to renegotiate the terms of your mortgage and forecloses on your home. At least you could probably move back in with your parents.

The new change is now live.

I can't believe this is Massachusetts

2008-12-10T06:43:25+00:00

Crane Beach, Ipswich, MA

A Beautiful Day

2008-12-10T06:43:25+00:00

Spring arrives in Boston (cf. winter)

In yesterday's Financial Times, Larry Summers on tax competition and cooperation:

First, the US should take the lead in promoting global cooperation in the international tax arena. There has been a race to the bottom in the taxation of corporate income. Closely related is the problem of tax havens that seek to lure wealthy citizens with promises that they can avoid paying taxes altogether on large parts of their fortunes. It might be inevitable that globalization leads to some increases in inequality; it is not necessary that it also compromise the possibility of progressive taxation.

Agreeing or disagreeing with Secretary Summers' point is largely a question of the role of government as much as it is one of international economics. I generally view tax competition as a healthy restraint on the tax burden and thus a bridle on the size of the state. Here, Larry is taking the view that without cooperation, you will have nanny states without nannies and thus nothing to transfer.

You're gonna be so proud. Proud? Proud.

2008-12-10T06:43:25+00:00

From the make-Edward-Tufte-proud department, another stellar graphic in today's Sunday Times, this one visualizing the basket of goods making up the CPI and both the relative size of those goods within the consumption bundle and the year-on-year change in that size:

All of Inflation's Little Parts by Amanda Cox

You always glean points from a good visualization that you don't from the tabular data. For example, consumers spend the same amount (about 1% of total) on cable service as on doctor visits. The portion of consumer spending allotted to "computers" has declined 12% year-on-year. Rising import prices, particularly oil (which, although denominated in dollars, experiences the same exchange rate pressure as other world market goods), and growing food costs account for the bulk of inflationary pressures. I am happy to note if you rent your home, don't own a car, and spend most of your money on clothes and bacon, your purchasing power has actually increased year-over-year.

Note that, while a proxy, the change in spending on a category is not the same as inflation. For example, the share of spending on citrus dropped 9.5% year-over-year. That could be due to deflation, but the spending drop could also be caused by a decrease in demand—perhaps consumers are substituting oranges with apples, which grew 7.5% year-on-year. Alternatively, note that while the cost of most health-related expenses went up, so did the science advancing the field, ushering in new drugs and improved procedures. If you aren't comparing, say, apples to apples year-on-year, you are measuring more than monetary inflation. These are just two of a myriad of problems with computing inflation.

A page earlier, Alan Blinder argues for greater regulation of the financial industry. Unfortunately, Prof Blinder notes:

It will, for example, substantially reduce the profitability of investment houses and, therefore, reduce their scale. But that’s the price you pay for access to a publicly financed safety net.

No doubt increased regulation, particularly in the area of margin requirements curbing excess leverage, will lower short-term profits. But I don't see why the goal of any changes in regulation shouldn't include maintaining or even improving longer term profits. After all, you'd have to take substantial bites out of Goldman's earnings to equal the loss in a single implosion such as Bear's.

EDGE puts me on Edge

2008-12-10T06:43:24+00:00

The line of 3G-hopefuls outside of Boston's Apple Store:

The fervor Apple instills in their customers, particularly compared to their competitor, is impressive.

Incidentally, down the street, the line outside of the decidedly-less cool AT&T Store was only ten or so folks deep.

The Business of Elections

2008-12-10T06:43:24+00:00

To date, clearly poised to surpass $1 billion before the cycle is over, the campaigns have spent a whopping $900 million. The New York Times, again proving that their core competency is in producing remarkably-informative graphics, has this rad little interactive visualization:

$4.3m to Verizon for cell phones!

See also the related article, Cashing In on Obama and McCain .

Sur la Table

2008-12-10T06:43:23+00:00

If you are not reading my food blog, Food Tastes Good, you are missing out on recipes such as …

Spaghetti alla Carbonara con Lobster Mushroom

Slow-Braised Carnitas

Pesto alla Genovese

Wild Mushroom Risotto with Green Peas

Red Wine Braised Beef Short Ribs

If not the actual dishes, at least the pictures are ambrosial.

Copy — right?

2008-12-04T16:41:00+00:00

So were chatting about copyrights and I stumbled upon the website of the Government of India&aposs Copyright Office, and some clickety clicking later, came upon The Handbook of Copyright Law. Wanted to chronicle interesting bits for posterity.

Fair use: Includes standard stuff like research, private study, criticism/review, reporting current events, judicial proceeding, amateur performance to a non-paying audience and some more ambiguous stuff (”the making of sound recordings of literary, dramatic or musical works under certain conditions”)
You own copyright to all photos of yourself (caveat: see fair use): “In the case of a photograph taken, or a painting or portrait drawn, or an engraving or a cinematograph film made, for valuable consideration at the instance of any person, such person shall, in the absence of any agreement to the contrary, be the first owner of the copyright therein.”
Computer programs are abou the same as literary works: With the exception that you can “sell or give on hire or offer for sale or hire, regardless of whether such a copy has been sold or given on hire on earlier occasion.”
Translations: Are protected by your copyright
Registering copyright: By default, you own the copyright to work that you have created. “However, certificate of registration of copyright and the entries made therein serve as prima facie evidence in a court of law with reference to dispute relating to ownership of copyright.”
Term of copyright: 60 years after death of the author for most things. 60 years from date of publication cinematograph films, photographs, posthumous publications, anonymous and pseudonymous publications, and some other stuff.

Phew! Certainly learned some new stuff today.

Whither an Automotive Industry

2008-12-03T10:56:15+00:00

I have long argued, partly in jest, mostly serious, that America should just get out of the car industry altogether and focus our capital on things we are good at, such as the service sector, software, or torts. The cost issue was secondary, I would say, if the cars themselves aren't competitive. (Does everyone at GM have huge fingers? Why are the interiors filled with over-sized plastic buttons?)

My bafflement continues with this latest bailout—unlike the financial package, I am against bailing out the US auto industry—and these comments from Ford's chief on the necessity of action:

Ford said in its plan that it could survive through 2009 with its current cash levels and by tapping its credit line with private banks, and that it could return to profitability by 2011. Even though it is better prepared for the downturn, Ford said it wanted $9 billion in loans to draw upon if necessary.

Ford’s chief executive, Alan R. Mulally, said the prospect of a failure of G.M. would cascade through the entire domestic auto industry and put millions of jobs at risk.

"We are very, very concerned, and that’s why we went with G.M. and Chrysler to Congress even though we think we have sufficient liquidity," he said in an interview.

Mulally is saying Ford is financially secure and does not need the bailout to meet payments, but he is worried about the second stage effects from a GM or Chrysler failure.

That seems backwards to me. Ford—and every other car manufacturer—would assuredly benefit from two of its competitors going under. There would be a small drop in demand as supply falls and prices rises, but that drop would be significantly smaller than the decrease in supply. Moreover, the substitution from GM to other manufacturers would overly favor Ford, in contrast to Mulally's statements, as "buy American" types swap one Detroit icon for another. The converse has the government funding a broken GM, propping up supply to the detriment of Ford. I don't get it.

I suppose Mulally could be bluffing, hoping to look good in the eyes of Wall Street and Ford's creditors but still get government help—to have his cake and eat it too. But somehow I doubt his posturing is worth the risk.

It starts …

2008-11-25T12:12:00+00:00

So this is the proverbial it. FOSS.IN/2008 starts today. We took some hard decisions to come upon the current format. We have an amazing lineup of stuff that’s happening through every day (don’t believe me? See the schedule). All systems are, in fact, go (I feel more redundant with every passing year). I think this poster sums it up the best (shoutout to Hari for the awesome artwork!):

In the unlikely event that you’re still wondering what FOSS.IN is all about, and whether you should come, just head on over and check out the little video we’ve made. It should answer any questions that you have about what the 2008 edition of FOSS.IN is all about.

Time to head to the venue now, see you there!

p.s.: I like this one too :-) …

One small step for student-kind

2008-11-19T16:52:00+00:00

Today, the VTU (the university that granted me my bachelor’s degree) did something incredibly smart. In one fell swoop, they have achieved what Kerala and Andhra Pradesh have been trying to do for years, in vain.

That’s right — the VTU has done the one thing that will ensure that no student of theirs will ever learn a Microsoft-related technology — a ton of Microsoft software is now part of the official curriculum.

Thank you, VTU!

Aside …

Reminds me of the “Basic Computer Skills” Lab in 3rd semester, where we had to create a document in Word and a presentation in PowerPoint. The external examiner expected you to remember exactly under which menu each random feature lay. It took her about 10 minutes to figure out that I was searching through the menus blindly after every question. :)

Not to mention 5th semester, where our DBMS lecturer tried to strong-arm me into learning Visual Basic for a project on databases. This one I managed to hold out on, and did my work in PHP+MySQL.

Happy Saturday

2008-11-08T12:18:23+00:00

Thursday night at dinner, one of Stephanie’s colleagues gave me a puzzle to play with.

The idea is to twist and turn the little blocks until they form a 3×3x3 cube. This morning, after messing with it for a few minutes, I decided I didn’t want to brute-force it manually. So I wrote a little script.

And that, my friends, is just about a perfect Saturday morning.

(The script is here.)

I love this city tonight, I love this city always

2008-11-04T09:09:01+00:00

I will be live-twittering tonight's election results.

Boston, mid autumn

As I wrote four years ago: If you are informed and have an opinion, please vote. Laziness is not an excuse. Although voting for Ralph Nader is.

i called it

2008-10-31T19:22:19+00:00

I predict the rays-phillies world series to be the least watched since the invention of television.

– Me, on October 20.

It was the least-viewed Series by a significant amount.

– NY Times, on October 30.

I’ve said it before and I’ll say it again: they should not play summer or fall baseball in Florida.

The Economist Endorses Senator Obama

2008-10-30T17:12:04+00:00

The Economist endorses Senator Obama for President.

The why is summed up in part by the endorsement—"if only the real John McCain had been running"—and in part by last week's Conservatives for Obama, so-called Obamacon:

The biggest brigade in the Obamacon army consists of libertarians, furious with Mr Bush’s big-government conservatism, worried about his commitment to an open-ended "war on terror," and disgusted by his cavalier way with civil rights.

Of course, the endorsement should be no surprise. The Economist has a history of endorsing the other party: Governor Reagan in 1980, Governor Clinton in 1992, Senator Dole in 1996, Governor Bush in 2000, and Senator Kerry in 2004.

With this heady endorsement, the Illinois senator might just win.

FOSS.IN/2008: Delegate registrations are open

2008-10-30T01:04:00+00:00

FOSS.IN/2008 delegate registration is now open — what are you waiting for!

welcome to hope springs

2008-10-28T17:11:37+00:00

Karl,

There is no code for the Association Browser, unless that’s changed in the year since I’ve been gone. I remember the idea was first devised during a series of brainstorming meetings among the desktop hackers, probably in 2004, and Jimmac (I think? Maybe Garrett) created this first-pass mockup one day:

There may have been an internal wiki page about it, but certainly no code.

The idea with the association browser was that you’d have a “focal object” (in this case, “Brand Grub”) and you’d see items directly related to that object in a way that was appropriate for that data. This was somewhat similar to how Dashboard worked — each piece of data looked different and specific to its type: emails looked different than documents and addressbook contacts, for example. But unlike Dashboard, this would be a fully-fledged application you could use to navigate your data. If you wanted to shift your focal object, say from “Brand Grub” to “Bilbo Baggins”, you could do that easily… I think we called that “rotating”.

The purpose of showing this in my old GUADEC talk was to illustrate the kinds of interesting applications people could build on top of Beagle beyond the obvious and boring all-encompassing search tools. Unfortunately I don’t think that’s happened. From my perspective, both Beagle and Tracker ended up focusing way too much on the backend (storage and retrieval of metadata just aren’t that interesting to most people) and not enough on making users’ lives easier.

So there you have it. The “code” for the association browser. Next time, feel free to email me about it first.

Android is Now Open Source

2008-10-21T12:31:00+00:00

Via my coworkers at the Android Developers Blog: Android is now open source.

Android is the first free, open source, and fully customizable mobile platform. Android offers a full stack: an operating system, middleware, and key mobile applications. It also contains a rich set of APIs that allows third-party developers to develop great applications.

Across my career, I am most proud of Android—as a platform, as a family of phones, and as a catalyst for change in an otherwise closed industry. But the most exciting part is what's next. Download the SDK and start hacking.

Geeks Fighting Corruption

2008-10-17T21:42:52+00:00

One of the things I’m really hopeful about is technology that can improve the transparency of government. Money is a corrupting influence in politics, but websites that track every campaign contribution, contract bid, earmark author, and the passage of every bill through its development give corrupt politicians and self-interested lobbyists nowhere to hide. And that’s a good thing. Sunlight, as Supreme Court Justice Louis Brandeis said, is the best disinfectant.

When Larry Lessig announced his campaign to end corruption in politics, he started encouraging hackers to attack this problem, to build websites and databases that lay bare the innerworkings of our politics. There are a few well-known projects in this area, like The Sunlight Foundation, Open Secrets, and followthemoney.org. There’s also USASpending.gov, the website “where Americans can see where their money goes,” the product of a law co-sponsored by Barack Obama.

Recently on twitter I learned that Charlottesville provocateur and long-time friend Waldo Jaquith created and runs Richmond Sunlight, a one-man sunlight site to track Virginia state politics. On it, you can track the progress of bills and the activities of the legislators in the Virginia state congress. The site has no ads and no for-pay section and Waldo runs it purely because he believes in what he’s doing.

Running the site takes a huge amount of work, and a little money. One of Richmond Sunlight’s most important features is the collection of videos of the Virginia General Assembly. To get these videos online, Waldo has to manually convert DVDs provided by the state into a format suitable for posting online. It’s a lengthy process that involves OCRing parts of the video to extract bill numbers and legislator names to index and tag the videos properly. He’s doing all this on his only computer — an aging Mac Mini. And unbelievably, Waldo actually has to purchase these videos from the General Assembly. And he has ambitious plans for the site. Last night Waldo posted an appeal for resources to help him grow the project:

While hundreds of thousands of people have found the site very useful, I look at it and see unfulfilled promise. I want to rewrite Richmond Sunlight and give it away to a nonpartisan political group in every state in the union. I want to complete the API so that anybody can write software to interact with the Capital Sunlight in their own state (and let even more newspapers integrate it into their own websites). I want a Facebook application, I want daily podcasts, I want people recording secret subcommittee votes, I want to mash up the daily floor calendar with campaign finance data with minutes with video and create the most radical transparency a state legislature has ever seen.

Waldo is doing exactly what Larry Lessig is encouraging us all to do. He’s dreaming big about open government. But he needs help today to do the daily business of running Richmond Sunlight. So, what do you say we pitch in to help a corruption-fighting geek in need? If you’d like to help buy Waldo a new computer so that he can put these videos online faster without spending his whole weekend swapping discs and waiting for codecs to convert, visit his blog and drop him a line.

separated at birth?

2008-10-16T20:03:35+00:00

FOSS.IN/2008: Taking it to the next level

2008-10-10T01:29:00+00:00

Finally, after ages, “soon” is here, and my loyal readers can ascertain that I am, in fact, still alive. A wider, life update will come later (heh), but for now …

Preparation for FOSS.IN/2008 is well on way, and this year is going to be different. The Call for Participation is out. The newest thing in there is that there aren’t going to be nearly as many talks as before. You’ll see the term FOSS WorkOuts rather prominently displayed, and this is where the action is going to be. We’re going to be seeing a lot more doing than in years gone by. Head on over to the CfP to learn more.

Atul’s post on the new format has caused some furore in the community, in addition to some pockets of encouragement (links abound and the topic is hackneyed, so no linky). All I have to add is this — a lot of people who are working on distros and doing packaging seem to be gravely offended. Well, I’m a packager too (erm, did I mention that I am now a Gentoo developer?), and there is no reason to take offense. What we’re trying to say is that we can be achieving more at the event to increase both the number of contributors as well as the depth of contribution, and the latter especially is the focus. I can expound on about this, but there’s been enough talk.

The cool folks over at IndLinux have already started plotting, and we’ve been trying to get some traction on some GNOME performance work. Hope we can get some more folks to run with it. I, for one, am looking eagerly forward to the proposals we get this year.

Android!

2008-10-02T17:19:31+00:00

Download the Android SDK 1.0 release 1.

Walt Mossberg: "The first real competitor to the iPhone ... the software is slick ... the G1 is a powerful, versatile device."

But the sui generis: Android is open to developers, open to consumers, and open to handset manufacturers. Cannot wait to see what's next.

Just Do Whatever Bernanke Says

2008-09-29T13:16:14+00:00

The Journal has a great primer on this morning's compromise bailout bill.

Do read the article, but the gist is this: The Treasury will initially have $250b, and up to $700b, to buy either directly or via auction bad loans and assets from financial institutions, in return for warrants for equity. Compromise language includes disincentives for high CEO pay, additional congressional oversight, and a surprising requirement for the president "to submit a legislative proposal to seek reimbursement from the financial institutions that participated" if the value of the purchased assets yields a net loss.

The best analogy I can come up with to describe the crisis is the lemon problem, exasperated by mark to market accounting: Balance sheets are full of mortgage-backed or otherwise related assets, the popping of the housing bubble resulted in a revaluation of these assets, and capitalization requirements are driving banks to liquidate the assets. Enter the lemon market. Is the bank selling the assets because it needs cashflow, or because the assets are full of subprime contagion? Is this the firm's best or worst assets? The information asymmetry has snowballed to the point of credit market implosion. Thus the government's first solution, improving lending opportunities. When that was found insufficient, as the last few weeks have witnessed, we enter this second round, where the government actually buys the troubled assets.

It is hard to comprehend how dire this situation is as the economy still "feels" okay. Gas prices might be high, but unemployment is not at 30%. Yet while the societal ramifications are not as bad, the financial conditions are worse than those that kicked off The Great Depression.

auuuuuuugh

2008-09-26T00:30:23+00:00

As Putin rears his head and comes into the air space of the United States of America, where do they go? It’s Alaska. It’s just right over the border. It is from Alaska that we send those out to make sure that an eye is being kept on this very powerful nation, Russia, because they are right next to, they are right next to our state.

(CBS News)

Love (and blogging)

2008-09-07T23:00:19+00:00

It’s been a while since I wrote regularly on this blog, and people have been asking, with decreasing regularity, why my blog posts sputtered out.

At first, I wanted to take a break to shake off the “I can’t wait to blog about this” impulse that was starting to spring up in the middle of almost everything I was doing, and was threatening the in-the-moment joy of life’s little adventures by making them into a kind of low-grade performance literature.

Then, in early 2007, I moved to Munich and got busy learning about another culture. Moving to a new place has a way of disrupting all your old habits, so I stopped going to yoga and I stopped posting here, but I started programming a lot again (yay!), and running a few times a week.

And then twitter erupted into my social group like an invasive species and I found that my public-writing energy nibbled away bit by bit, never building past whatever critical threshold is required for something to be (dare I say) bloggable.

And then, unexpectedly, at a conference in Paris, I met the most dazzling girl. Smart and kind-hearted, and with an incredible appetite for life, she lived in Munich. When I moved here, she helped me find an apartment and get settled. Somewhere in there, she completely stole my heart. And so, earlier this year, on a hill in San Francisco, I asked her to be my wife, and she said “why the hell not” (I’m paraphrasing here).

We both love to travel, and she’s amused by my sense of whimsy. Over the last year we’ve had a lot of fun running around Europe. (More on that later.) And so I’ve found someone I want to share life’s adventures with, and you guys have recently taken second priority. Sorry about that, but I’m sure you can understand .

We do plan to have an actual wedding sometime next year, though we’re not sure exactly when or where. So, stay tuned for future episodes, now featuring Stephanie (introductory glam shot below).

Some sort of Cat

2008-08-22T15:14:43+00:00

The greatest Wondermark ever, if not the greatest thing, ever:

Slightly switching gears, David Leonhardt on Obamonics in the Times.

And, in case you missed it, we released version 0.9 of the Android SDK. Its dreamy.

Broken Elevators

2008-08-09T11:31:51+00:00

Via Google's Open Source Blog, this Google-sponsored project to study Linux I/O scheduler behavior is quite interesting, yielding unexpected results—for example, deadline is actually best for some workloads and CFS, while ideal for others, has awful worst-case performance.

Curious about I/O schedulers? Check out chapter 13 in my favorite kernel book. Want to optimize your code's file I/O and understand scheduling from the perspective of user-space? Read chapter 4 in my favorite system programming book.

Hummers, Cristal, and Cambodian Children: Hello, Nouveau Riche!

2008-08-08T11:13:36+00:00

Hat tip to loyal reader for making me a billionaire:

Expires six months after issue

Somewhat unrelated, Amazon is running a special this month wherein you can try Amazon Prime Free for One Month—in other words, get a month of gratis two-day shipping. US only, unfortunately.

< Insert Your Favourite > Desktop Search Hackfest

2008-07-23T20:55:00+00:00

Good news - a (read: the first ever) Desktop Search Hackfest is being
planned after the Maemo Summit, in Berlin.
http://wiki.maemo.org/Desktop_Search_Hackfest

Not so good news - I will not be able to attend. It's not easy to
sneak out as a grad student from an US university (bonus if you are an
_alien_). Joe is not going either. Our in-house Xesam guru is about a
join a real job, the money-paying kind. It might be hard for him to
attend either. I dearly wish Beagle could somehow benefit from this
meeting. Good luck to the other projects. Get some work done and make
the users happy. It is nice to see that Strigi/Nepomuk devs are
attending. And thanks to Nokia for making this happen.

On a side note, look at the number of participants from the Tracker
project ! Wow, they're booming. Double kudos to them.

There is always money in the banana stand

2008-07-19T16:20:58+00:00

Zimbabwe introduces $100 billion banknotes, each valued at one US dollar.

In all seriousness, for posterity, I would love to get one or two of these new bills. If anyone can help, I will pay handsomely.

Creating new backend - help wanted

2008-07-17T03:47:00+00:00

Three years ago this month, the above was the subject of my first post to the
dashboard-hackers mailing list. I was writing a backend for Akregator. Its
immaterial what Akregator is; I was mostly pattern matching back then and
that is how I got familiar with C#.

I remember submitting a patch to beagle bugzilla a few days before that. It
was to filter JPEG JFIF comments because that is where digikam stored all the
descriptions. Now it uses IPTC tags. Does not really matter since beagle
indexes both.

Backtracking further, I got interested in beagle several months before that. I
had this incredible urge to add /author/, /title/, /subject/ tags to the PDF
research papers that I download and use beagle to search among them. The
first time I visited the beagle website and downloaded the tarball, the huge
list of dependencies really really scared me off. I used to allow more Gnome
on my desktop back then than I do now but nevertheless building beagle seemed
like a daunting task. I later freed beagle from many of its dependencies; I
feel that is my biggest contribution to this project.

I made a second attempt later; soon realized that even though I can get beagle
going, there was (is ?) no easy way to modify PDF document properties (in
linux). Then I switched my goal to index my pictures, which I have started
tagging and adding comments to, using my new found love digikam. Thankfully
the one-liner JFIF patch did not require much C# knowledge. It would not have
mattered anyway, C# is darn easy to pattern match.

After all these months, I still haven't managed to build my repository of well
tagged PDF papers (if I wait for several more months and stop wasting time
then the need would be gone forever). I do keep a static index for my
pictures but rarely search them. What a waste of time :-). Silly me!

As I am about to sign off this blog entry, I noticed that I was siging off
as "Bera" in my initial few emails. I wonder when did I switch to
my "signature sign" dBera.

Did I mention that beagle 0.3.8 was released some 48 hours ago. Go go get it.

talk about a “very narrow viewpoint”

2008-07-13T18:35:56+00:00

I just came across this boneheaded blog post about Google’s newly open-sourced Protocol Buffers:

They claim they could not use XML because ‘it isn’t going to be efficient enough for this scale’. WTF??? If this statement came from someone else, I would understand, but these guys are supposed to KNOW markup.

Speed in a system in [sic] NOT just optimizing loops in code! It is the architecture: messaging, storage, and re-use. Yes, XML can be fat, but so can any other language. And if they took the time to improve the processing libraries instead of creating their own special methods, we would ALL benefit on projects that used XML, not just this so-called ‘protocol’.

And he had the audacity to title the post Google hates XML.

While the author is right that any attention paid to improving the performance of widespread XML libraries would be widely beneficial, he completely ignores protobuf’s strengths, aims, and specific use cases. I suspect he didn’t bother to read any of the documentation. After all, when you’re dealing with XML, everything looks like a nail. Or something like that.

Protobufs aren’t aimed at replacing the widespread utility of using XML for publishing data widely or providing human readable document formats. I don’t think anyone is suggesting that we replace HTML with this. They’re for (mostly) well-defined interfaces and serializing data in a compact and low-latency way. The author at one point suggests:

I bet I could make XML run circles around their system just by simplifying their schema. I once invented a technique called ‘XmlZip’ that would transform long element names and attributes to smaller symbols for faster transfer - why not try that?

But he obviously didn’t read the section on encoding:

Let’s say you have the following very simple message definition:

message Test1 { required int32 a = 1; }

In an application, you create a Test1 message and set a to 150. You then serialize the message to an output stream. If you were able to examine the encoded message, you’d see three bytes:

08 96 01

Three bytes! You can’t do anything in XML in three bytes. The simplest XML document you can have, which conveys no information, is 4 bytes: <a/>. That same message definition would look something like this in XML:

<?xml version="1.0"?> <Test1> <int32 value="150" /> </Test1>

That’s 61 bytes by my count, and even if you did condense it down to a tight, humanly-unreadable XML, you won’t get anywhere near 3 bytes. And if your messages really are that small, the gzip compression overhead is counterproductive and actually results in a bigger file. (82 bytes, from my testing.) If the messages were large enough that gzip compression did buy you size, you’d suffer additional latency because of the CPU time used to decompress.

When you’re talking about pushing huge amounts of data on a near-saturated gigabit ethernet link, an order of magnitude makes a big deal.

Protocol buffers aren’t going to replace XML — they’re not even really aimed at the same problem — but they are a better solution for certain use cases. Such a huge part of software development is using the right tool for the job, and just because XML can solve the problem doesn’t mean it should by default. Do the research and weigh the pros and cons. Otherwise, like using an O(n^2) sorting algorithm when there are vastly better alternatives, it’s just lazy programming.

i’ve done my part — complaining — and it worked

2008-07-13T18:35:05+00:00

Toward the end of my time at Novell, I was looking into a browser sync system for the GNOME Online Desktop. As I am a lazy hacker, the ideal solution at the time for me would have been for Google to open source its nice browser sync extension and then adapt it to the online desktop myself. I tapped my contacts inside Google to see if open sourcing it was in the cards. It wasn’t.

When I saw that the extension was being discontinued (and slowly-but-surely being replaced by Mozilla Labs’ Weave) and that it wasn’t immediately open sourced, I was furious! I planned a blog entry raking them over the coals, how could they abandon a perfectly useful piece of code, blah blah blah. It never happened, because I suck at blogging (remember, me=lazy).

Imagine my pleasure today when I came across their announcement open sourcing the extension. I hope people can take the code — mainly the Mozilla folks for Weave and the GNOME folks for online desktop — and more quickly build a high quality system. I am still looking for a way to sync my extensions between browsers!

And because false hubris is a cornerstone of this blog in addition to self-deprecation, I’d just like to say that it’s wonderful to see that the “additional pressure” I alluded to in my email has finally succeeded in ending the 11 month closed-source tyranny that began when I first heard of this extension last July. While I can’t prove that I single-handedly open sourced the extension, I know it to be true deep down in my heart. Hooray!

Task #1 is to add some tabs to it.

clearly nyt writers read my blog

2008-07-13T18:34:07+00:00

Consumerist (a must-read blog) has a link today to a New York Times article about CSAs, which I mentioned a couple of weeks ago.

this is going to get worse before it gets better

2008-07-13T18:33:37+00:00

Tell my wife I love her.

promotional consideration provided by

2008-07-13T18:33:14+00:00

Like Havoc, Brette and I are members of a CSA in which we pick up our veggies once a week from a truck in a parking lot in Central Square. I just picked up this weeks share, and we have some great beets, carrots, mustard greens, and the best strawberries I’ve ever had.

In addition to the veggie CSA, we’re also members of a meat CSA, Chestnut Farms. The meat is local — raised in western Massachusetts and slaughtered nearby — and I pick it up once a month. It comes frozen, but I think it’s done so quickly after slaughter that the meat is still incredibly fresh when thawed… it never tastes like a freezer, it never has freezer burn, and it’s never tough.

They offer different cuts of grass-fed beef, pork, lamb, and free range chicken every month and often have eggs for sale at an additional cost. We’re not big lamb eaters so we don’t get any, but the rest of the food is some of the best stuff I’ve ever tasted. The pork in particular has a much richer flavor than anything you can get in the store (including Whole Foods)… those factory farms just can’t make tasty pork.

We get 10 lbs every month for $70. A quick survey of the freezer shows me that have: ground beef, ground pork, country-style spare ribs, pork chops, pork tenderloin, pork breakfast sausage, chicken legs and thighs, chicken breast, smoked bacon, and beef round eye roast.

If you’re a meat lover and live in Massachusetts, I suggest signing up even though there’s now a waiting list. If you’re elsewhere, meat CSAs are becoming increasingly popular (Results 1 - 10 of about 643,000 for meat csa.) and well worth it.

i left the living room window open

2008-07-13T18:32:10+00:00

The weather here this week has been awesome.

It’s too bad Jacob moved away just as it was getting good. :(

These kinds of storms remind me of my childhood. When I was six, I remember sitting on the front porch of my grandparents’ house with them and listening to a battery-powered radio. The power had gone out, and a tornado had touched down about halfway between where my parents and grandparents lived — about 30 miles apart. Good times.

Martian Skies

2008-06-22T13:00:47+00:00

Collected by the Boston Globe, these photos of Martian skies are without peer. There is a romance to exploring the unexplored, about going somewhere new simply because that's what's next.

It reminds me of President Reagan's speech, quoting from the poem High Flight, later cribbed by The West Wing, on the night of the "Challenger" disaster. Scheduled to give his state of the union, he spoke in lieu from the West Wing:

For the families of the seven, we cannot bear, as you do, the full impact of this tragedy. But we feel the loss, and we're thinking about you so very much. Your loved ones were daring and brave, and they had that special grace, that special spirit that says, "Give me a challenge and I'll meet it with joy." They had a hunger to explore the universe and discover its truths. They wished to serve, and they did. They served all of us.

And I want to say something to the school children of America who were watching the live coverage of the shuttle's takeoff. I know it is hard to understand, but sometimes painful things like this happen. It's all part of the process of exploration and discovery. It's all part of taking a chance and expanding man's horizons. The future doesn't belong to the fainthearted; it belongs to the brave. The Challenger crew was pulling us into the future, and we'll continue to follow them.

The crew of the space shuttle Challenger honored us by the manner in which they lived their lives. We will never forget them, nor the last time we saw them, this morning, as they prepared for their journey and waved good-bye and "slipped the surly bonds of earth" to "touch the face of God."

Slipped the surly bonds of earth to touch the face of God.

Anyhow, beautiful pictures.

Food Blog

2008-06-17T11:21:21+00:00

I am keeping a food blog, Food Tastes Good. It is mostly recipes, such as,

Do check it out, if that sort of thing interests you.

N810 - we wantee!

2008-06-17T05:10:00+00:00

This article by Ted T’so is an excellent commentary on the controversy around Nokia’s Dr. Ari Jaaksi (one of the bigshots behind the amazing Nokia N770/800/810 internet tablets) recent comments (1, 2) on the need for open source developers to understand business constraints. Extremely well-balanced. Bruce Perens also wrote an interesting piece on it.

i would like to lick that sandwich

2008-06-11T15:45:08+00:00

From the Tri-State Observer:

Our concern is not that we are using the remainder of our strategic grain reserves for humanitarian relief. AAM fully supports the action and all humanitarian food relief. Our concern is that the U.S. has nothing else in our emergency food pantry. There is no cheese, no butter, no dry milk powder, no grains or anything else left in reserve. The only thing left in the entire CCC inventory will be 2.7 million bushels of wheat, which is about enough wheat to make 1⁄2 of a loaf of bread for each of the 300 million people in America.

Wait. We had a strategic cheese reserve and nobody told me about this? Because my strategy is to eat as much cheese as I can when my wife is not around. I have a cheese-eating strategy.

On an unrelated note, congratulations to Bockover, Gabriel, and the rest of the Banshee team for their 1.0 release. These guys are amazing hackers, and Banshee has really matured into a fantastic piece of software. And having a new website up and packages for several distros available on release day? These guys have their shit together.

Planet Beagle

Murder your darlings

Some early Avro benchmarks

Cupcake!

Twitter norming: do you believe in God?

The LiveJournal to WordPress migration

What can you do to change the world?

I finally got to use my soldering iron for something

Twitter Sleep Survey

Where I've Been

Flintstoning

there is only one correct way to skin a cat

python daemon threads considered harmful

another beagle innovation stolen

FOSSKriti ‘09 is *here*

Better than my best

We have an interval

giticular cancer

Happy New Year

best. christmas present. ever.

your personal rate of return is -50.2%

I can't believe this is Massachusetts

A Beautiful Day

You're gonna be so proud. Proud? Proud.

EDGE puts me on Edge

The Business of Elections

Sur la Table

Copy — right?

Whither an Automotive Industry

It starts …

One small step for student-kind

Happy Saturday

I love this city tonight, I love this city always

i called it

The Economist Endorses Senator Obama

FOSS.IN/2008: Delegate registrations are *open*

welcome to hope springs

Android is Now Open Source

Geeks Fighting Corruption

separated at birth?

FOSS.IN/2008: Taking it to the next level

Android!

Just Do Whatever Bernanke Says

auuuuuuugh

Love (and blogging)

Some sort of Cat

Broken Elevators

Hummers, Cristal, and Cambodian Children: Hello, Nouveau Riche!

< Insert Your Favourite > Desktop Search Hackfest

There is always money in the banana stand

Creating new backend - help wanted

talk about a “very narrow viewpoint”

i’ve done my part — complaining — and it worked

clearly nyt writers read my blog

this is going to get worse before it gets better

promotional consideration provided by

i left the living room window open

Martian Skies

Food Blog

N810 - we wantee!

i would like to lick that sandwich

FOSSKriti ‘09 is here

FOSS.IN/2008: Delegate registrations are open