Bug Blogger

One hundred thousand new operating systems.

I have been a huge fan of Gentoo Linux for several years. Why? Is it the elitist appeal of compiling your operating system from scratch? Could it be performance gains achieved by compiler and package optimizations for a particular piece of hardware? The excellent forums? Nope, none of the above. The Gentoo community instills a culture of looking under the hood, of customizing and tweaking, of not being afraid of greasy hands. I get laughed at in the office sometimes because of this and I have to spend more time installing, but I think it's worth it. Deep customization seems to be the wave of the future, and here is why I think so.

Enterprise Application Servers: great idea! Provide a uniform platform optimized for a specific type of application. Abstract the hardware layer with some operating system and a JVM. Serve up Ebay! But why stop there? Why can't the app server run on bare metal? What is the use of all that OS overhead/complexity/variance underneath? In software design, we constantly make compromises between optimization and generality. What if we didn't have to? What if we could develop and deploy an entire operating system for a specific application or job position or organization? Well of course you could but it would be incredibly expensive and take forever. And who's gonna write all those device drivers?

At SCALE in LA a few months back I had the opportunity to see a presentation by and briefly talk with Bruno Gonçalves de Albuquerque of the Haiku project. The concepts behind the operating system are fantastic. The level of optimization at the user experience level when you control the entire OS (read: filesystem) is really amazing, and clearly blows the doors off any desktop OS I've ever used. Why am I not using Haiku right now? Well apart from it not being quite done yet, hardware support. Developing device drivers is a constant and thankless battle, typically undertaken by Microsofts and OEMs. Where is the incentive for a Taiwanese OEM to provide a device driver for Haiku? While listening to Bruno Gonçalves de Albuquerque it occurred to me that this problem may just go away if Haiku had binary compatibility with the Linux driver model. All of a sudden my graphics card works, and my wifi (gasp) too! Linux, besides being an operating system, provides support for a very large number of devices and systems. Fresh off the presss, 2.6.25 supports a new hardware platform. The fact that all of this work is open source presents an opportunity for other OS producers to leverage this work. It lowers the cost and increases the potential user base by supporting a greater variety of hardware. But who cares? We have OS X and Ubuntu and XP. Do we really need another operating system?

On the hardware front things are changing radically. Eight cores today, 64 tomorrow. Specialized (audio, physics, credit card validators) cores on the same chip. There is talk of doing away with the graphics card altogether and putting that in the CPU too. The list goes on... But where is the software? Where are the operating systems and languages and runtime environments to fully take advantage of this latest round of hardware innovation? And who decides where virtualization occurs? Is it KVM...or perhaps some JVMs? Do I have to run one master OS that virtualizes all the others? Can I partition, version, and manage OSs per core? Dynamically? Can I have a custom OS for Crysis III that gives me a few more frames per second because my machine isn't looking for security patches? Do I really have one machine or many?

There is a shift happening in software caused by new hardware platforms. Component based design, concurrency, functional programming, and virtualization are all, in part, results of trying to utilize new hardware designs. This change is similar, but hopefully more fundamental to the shift from single-task operating systems to multitasking ones. From TSRs to Windows. It took so much time for that earlier change to manifest*. What's the difference between then and now? How long do we have to wait for MS Windows 95 part II?

Well I think the difference is Linux and open source. The difference is an open driver model that allows OS developers to focus on new ideas rather than supporting 3000 network cards. The ability to see solutions in the open, and the ability to collaborate. No more silos, no more ivory towers. The demand for radical OS customization will increase as divergence grows between hardware potential and what our existing operating systems, languages, and runtimes are capable of. And as this demand increases I hope to see new, radical, highly-custom new operating systems and runtimes emerge. Exciting times!

* And yes, I had an Amiga too back in 1989. And yes, it was sweet.

SDK now available for download

Before anything else, I just wanted to say thanks to everyone who met up with us at BUG+NY last night! We had a great time, good conversations, and some pix are online now.  Now onto the stuff you really care about.  I will start with a little note from Ken, the head of software for Bug Labs:

Many moons ago Peter and I met in a crowded starbox on a dark and stormy evening.  He had some wood blocks and visions of a computer program that made creating user-designed consumer electronics easy. There were some pictures and various notes.  Well, I'm proud to announce that the software that resulted from those early conversations has been publicly released!

Called the Dragonfly SDK, it is an Eclipse-based software development kit for the BUG.  It's designed as an easy-to-use embedded development environment with a built-in web application sharing website.  While we have not released the BUG hardware yet, a virtual one is included to play around with.  Install directions are here  or if you're an Eclipse pro here is the update site URL.  Also, we've included a minimal getting started guide here.

(thanks Ken!)

We consider the SDK in an open beta now.  Accompanying it are also discussion forums, a wiki, and BUGnet is now live (click here to register).  If you are wondering what these things have in common, it's the beginnings of our online community.  We are building out numerous components to really enhance the community experience in the future, including a BUGmodule recommendation/voting system and more.  Most important to us is that every aspect of our communications become two-way.

So, please comment on the blog, post on the forums, contribute to the wiki, and build applications for BUGnet!  We want to hear from you, tell us what you think, what you want, and what else we can do to make the BUG experience as good as possible.

Sony Goes Open Source

I bought a Sony Bravia LCD television last week and finally got around to looking at some of the paperwork that came with it.  One document caught my eye because it looked like a lot of fine print.  When I unfolded it and checked out the title I was stunned.  The title read "Required Public Statement for GPL/LGPL Licensed Software Used in This Television".  My curiosity piqued I continued to read.  Turns out Sony uses the following GPL executables in this TV:
- Monta Vista Linux Kernel
- busybox
- insmod

...and these LGPL libraries:
- libuClibc.so
- ld.so
- libc.so
- libm.so
- libpthread.so
- libgcc_s.so
- libstdc++.so

Source code for all the above is available at www.sony.com/linux (a link that right now, is not loading...)

The fact that Sony uses Linux surprised me.  Maybe it shouldn't, I don't know.  But it seems strange that the master of closed architectures is dipping into the OSS pool.  Are they giving back?  From the  looks of it they're not really touting their participation nor do I recall ever seeing their contributions mentioned anywhere.  The document was "required".  There is no indication anywhere in the doc regarding what they're doing with Linux or how it works.

I can't put my finger on exactly why this bugs me, but somehow it does.

Where's the Humanity?

If you are a Windows user then you might not believe me, but I have found the human factor in software.  The first inkling of this discovery was when someone plainly said to me, "well, you work with people, don't you?"  I had just said that I thought software development wasn't human enough.  I don't really even know what I meant by that.  It was a hard-to-describe feeling, but my friend saw right through it.  He reminded me that I do work with people, and it's one of the best things about my job.

As soon as I saw one byte of humanity, I started seeing more.  It occurred to me that the purpose of most software is to interface humans with very powerful thinking machines.  The best software for humans is the kind where the interface is complimentary and not adversarial.  When it comes down to it, making machines more human is the end-game for software engineers. Then there is the communicative nature of the internet which has fed the social nature of the web.  When it comes to the web, it's all about connecting humans--to their media, to the things they want to buy, and more and more, to each other.  Human software is social software.  It creates communities.

If we're bound for the Age of Spiritual Machines, it seems that human software is the road that will get us there.  Recently my friend's brother bought him a World of Warcraft character for about $100 on eBay.  My friend and his brother wanted to spend time together, and considering they live in different parts of the country, World of Warcraft was a great venue.  It also gave the brothers a chance to relive their childhood relationship as adults.  The big brother was happy for a new chance to show his little brother the ropes of this virtual world. Now, when my friend shares his online experiences with me, he often talks about the things he and his brother can do and not the things their characters do.  It's almost like a new version of the Turing test: Can the virtual world fool you into thinking your avatar is you?

It's impossible to talk about this without mentioning the Nintendo Wii.  The more social "party games" that have long been Nintendo's focus plus the Wii's brand new way of playing are an auspicious mix.  Accelerometers and motion sensors have been around for a while, but the real magic of the Wii is in the software.  How does some motion of your hand affect what happens in the game and does it make sense?  Thinking about World of Warcraft and the Wii, it is no surprise that games are leading the way in bringing the human factor into software.

Another development that has had a big impact on me as a software engineer is the popularity of Ruby.  Ruby claims to be the programmer's programming language.  Like the Wii, it is fun and intuitive to use.  Its flexibility and elegant syntax is, well, very human.  If you agree with the viability of a bottom up approach, then you can argue that writing software in a more human language will lead to more human software.  I think, with the many new web applications written in Ruby on Rails sprouting up lately, Ruby has helped make software more human.

I can't predict what comes after World of Warcraft, Ruby, and the Wii, but it will inevitably be more natural, more inuitive, and more human.

Become a Software Engineer

Last year, Money Magazine rated software engineer the best job in America.  When I read that, I was incredulous.  I'm a software engineer, I thought, and my job cannot be the best job in America.  My first question was, where do they come up with this crap? Luckily the information architects predicted that it would be my first question and left a link dangling right there in front of me.

It seems one of the most important factors for them is job growth.  This is why chief executive is further down the list than physician assistant.  They also don't include very rare jobs, regardless of how sweet. Superhero and rock-god are nowhere on the list.  Compensation is important, but the human factor also plays prominently in their analysis.  They judged factors such as "stress levels, flexibility in hours and working environment, creativity, and how easy it is to enter and advance in the field."

When viewing the career of software engineering through this lens, it really does make sense.  I was happy to see that the number of software engineering jobs in America is predicted to increase almost 45% over the next 10 years.  Just 5 years ago, outsourcing was predicted to destroy my career.  I guess that was wrong, but those predictions came during a bit of a dark time.  Software engineers and just about everyone else in the industry were suffering from disillusionment after the dot-com crash.  Luckily, the industry and the career has matured since then.  A real understanding in the value of creativity is coming from engineers and their employers.  That brings us to the human factor.  We're beyond the days of thinking that it's all about lines of code.  Because of that, the work has gotten better and so has the job.

At the time I read the Money article, I was working for a company that didn't understand the human factor.  I started working for them because they did travel and I love travelling (This is another point the Money article brings up, that as software engineers we can conflate other interests with out careers--travel, music, you name it, they need software).  But they viewed programming as lines of code that you could farm off to foreign lands for a fraction of the price.  For much of what they do, they can.  But because they treat all of their engineers as hot-swappable resources, they're never going to get much more out of what they are, and what they are is dying.

Of course, if you're in a situation that you think could be better, you should try to find the better, which is what I soon did.  Now I can talk about job growth and compensation in my estimation, but more importantly for me, there's the human factor.  My job has to be creative and interesting and the environment I am in has to facilitate that.  On a good team working on an exciting project for a company that's not trying to institutionalize the creative process, I start to think that Money magazine might just be right.