*scottstuff*

This was one of those weekends where I practically lived with a camera in my hand; I took around 500 halloween pictures yesterday and then moved on to Christmas-card pictures of the kids today. Everything was going well enough until late in the day, when I was trying to get a nice black and white shot of my son. In the middle of shooting I took a quick peek at my camera’s LCD display and noticed that the last shot had been completely underexposed. So I scrolled back a few shots and discovered that I’d been shooting nothing but black frames for about 30 seconds. One second it worked, the next it didn’t. All of the camera settings were the same–same aperture, ISO, and shutter speed. Same light. But no picture.

I double-checked things, but I was still getting nothing but black. With a sinking feeling, I popped the lens off and took a multi-second exposure while staring into the camera. I could see the mirror flip up, but instead of seeing the sensor, I was left staring at a closed shutter, which strongly suggests that my shutter has died. The shutter on my D60 is rated for 30,000 exposures, and I think I’m around 25,000 right now, so it’s a bit earlier then normal, but not utterly unexpected.

So, I guess I’ll be packing it up and shipping it off to Canon for service this week. I’m not sure how long it’ll take to get back, but I was hoping to use it at Mind Camp next weekend. It looks like I’m going to have to make alternate plans.

Flickr has finally added photo printing. As of today, US Flickr members can get 4x6 prints for $0.15. They also sell other sizes (5x7, 8x10, wallet, 5x5, [458]xD, and 20x30), but the other prices aren’t quite as enticing.

By default, you’re the only one allowed to print your pictures; it’s a safe default for Flickr, but I don’t really care who prints my pictures any more then I care who looks at them. If I wanted them to be private, I wouldn’t have put them on Flickr. You can change the setting via your flickr preference page; I changed mine so any Flickr member can order prints.

Er, well, some Flickr members can order prints. For now, the printing service is US-only. Considering that Flickr was a Canadian company (until Yahoo snapped them up), I find the US-centric printing kind of funny. They claim that they’re working on adding more countries.

I’ll probably order a few prints to test it out, but I doubt I’ll use Flickr’s printing service much, for the same reason that I’ve never been willing to use any of the online photo-printing places: they don’t do color management, so there’s no guarantee that your prints will look anything like the images on your screen. Instead, I use the profiles from Dry Creek Photo, burn a CD, and take it to my local Costco. I’ve had very good luck this way–I’ve churned out batches of 300 images without any problems or rejects before. The only problem is that I need to burn a CD and then make a couple trips to Costco; one to drop off the CD and another to pick up the prints. Most of the time, I’d rather just click “print” and wait a few days for a package to show up in the mail. There are a number of professional photo finishers that will accept color-managed images via FTP, but none of them are even close to being price-competitive with Flickr or Costco, and for big batches of 4x6 or 5x7 prints, price matters.

Which brings me back around to Apple’s Aperture again. One of the minor features that they’re touting is color-managed printing from within Aperture. I’d love that. Unfortunately, I’m not about to run out and buy a PowerMac and Aperture just to make photo printing easier, but it’s definitely a step in the right direction.

On the other hand, while Flickr’s non-color-managed prints may not be quite what I’m looking for, they’ll almost certainly save me a lot of hassle–I get a lot of requests for prints from friends and family, and I hate doing one-off prints for people. Now I can just point them to Flickr and let them do it themselves.

A number of users complained this weekend that Typo was using way too much memory, with reports of 100+ MB per FastCGI dispatcher. Typo usually uses around 20 MB, and even that’s too much; 100 MB is enough to cause big problems with hosting providers like TextDrive.

The first step that I took was to verify that the problem actually exists outside of TextDrive. I set up a test Apache/FastCGI/Typo server, disabled caching, and then pounded on it using curl:

# while true; do curl http://typo1/ > /dev/null; done

I let that run for a few seconds and watched while my dispatch.fcgi processes grew from 22 MB to 80 MB. I then did a bit of experimenting:

• The main index page leaked
• RSS feeds didn’t leak
• Individual article pages leak
• Static pages, like /pages/about leak
• Error pages even leak

Disabling the layout for a leaking page and then re-testing it showed that the leak followed the layout. Turning layouts back on and removing the sidebar block fixed the leak.

Entertainingly enough, disabling the sidebar from inside of the sidebar infrastructure didn’t fix the leak. The mere act of calling render_component to generate the sidebars seemed to be causing the memory leak. Since Typo is one of the very few users of Rails components, this suggests that render_component may have a leak that no one else has noticed, so I created a new test Rails app with only two files. First, app/controllers/foo_controller.rb:

class FooController < ApplicationController
  def bar
    render_component :layout => false, 
      :controller => 'sidebars/sidebar', :action => 'index'
  end
end

Then components/sidebars/sidebar_controller.rb:

module Sidebars
  class SidebarController < ApplicationController
    def index
      render :text => 'test', :layout => false
    end
  end
end

This is about as minimal as a Rails app can get. Then I set up a FastCGI server running this project, and ran curl against /foo/bar, and watched the process size climb. So the leak is part of Rails, not really part of Typo.

Unfortunately, I’m not sure where the leak is coming from. I read component.rb and made a few small changes, but the leak hasn’t stopped. So I’m going to file this as a Rails bug and see if we can get it fixed before 1.0.

Update: Rails bug 2589.

Update: Thanks to Scott Barron, the bug has been fixed. Users with memory problems should probably install the patch, although a bit of testing would obviously be recommended first. The next release of Rails (either 1.0rc3 or 1.0; I’m not sure what they’re planning) should include this fix.

I just uploaded version 0.2.0 of my Rails Schema Generator to Rubyforge. This is a minor update, but it was needed to make the schema generator work with Rails 1.0rc2 (AKA 0.14.1).

The schema generator is sort of the flip side of the new schema code in Rails 1.0. It takes a set of Rails migrations, aggregates them all together, and spits out a SQL file that describes the DB that you’d get if you ran all of the migrations. Or, viewed in a more useful light, it gives you a SQL file that you can use to create a new DB from scratch. The current version actually produces three different schema files, one for PostgreSQL, one for MySQL, and one for SQLite, each with DB-appropriate syntax and types.

This is an outgrowth of Typo; we’re up to 25 migrations now, and we actively support 3 different DBs. It was getting really painful to maintain 3 distinct schema files in addition to the collection of migrations, so I wrote this schema generator. Now we’re back to DRY-land–we create new migrations and let the schema generator do all of the hard work.

It looks like the long-fabled Iron Chef America episode featuring Tom Douglas is finally going to air.

And he’s throwing a party. From the Seattle P-I:

The “Battle Salmon” pitting Masaharu Morimoto against our own Tom Douglas will air on Nov. 6, and Douglas is hosting a dinner-and-viewing that night, starting at 6:30 p.m. at the Palace Ballroom, 2100 Fifth Ave.

Douglas promises slightly kitschy festivities featuring the same five-course meal he and colleagues Mark Fuller and Eric Tanaka presented in the Food Network show, plus a Japanese hachimaki headband, two cocktails, and a chance to watch the show ($95/person). For reservations call 206-448-2001.

That’s a bit steep for me, but I’d still be tempted if it wasn’t the same day as Mind Camp.

Apparently I need to track more local blogs, because I missed the original announcement of the Seattle MindCamp. Fortunately, Ted Leung pointed it out to me, or I probably would have missed it entirely. This should be interesting–it’s essentially a local version of Foo Camp (or Bar Camp), held in an office building over a 26-hour span during the first weekend in November.

From the MindCamp website:

Seattle Mind Camp is a self-organizing, digitally minded, entrepreneur-driven, overnight Seattle confab. What happens when you put 150 of Seattle’s smartest geeks in an empty office building for 24 hours? We’re not sure either, but we’d like to find out. It’s time to meet and connect with those involved in the interesting projects going on in Seattle in a relaxed environment.

What: A weekend, 24-hour, multi-track event. Think huge space with breakout rooms, broadband Wi-Fi, projectors, white boards - and you.

Who: 150 of Seattle’s forward thinkers: techies, entrepreneurs, executives, gamers, musicians, and anyone else with a great idea.

When: Mind Camp will take place on November 5-6

Why?: You know all those hallway conversations that never get to flourish during a “normal” conference? Now they will.

Seattle Mind Camp is completely free of charge, and registration will begin very soon. In the meantime, check out the About page for a little more information.

It looks like there are still a few spots left, but I doubt they’ll last very long.

Update: In the spirit of information spreading, I should probably mention Seattle Code Camp, which is happening this weekend. From looking at the code camp website, it looks a bit more organized, with pre-scheduled talks, and there seems to be a big Microsoft/C#/.NET focus on a lot of the events. There are a couple Perl/Linux sessions and an introduction to Ruby, as well as a pair of Cocoa/Objective C talks, but most of the content seems to be “cool new stuff in C#.” Which is fine, but it’s not really my sort of conference. MindCamp, on the other hand, seems to be drawing a more diverse crowd, with a number of open source people on the roster.

Dave Thomas has a nice little announcement, complete with screenshot: Agile Web Development with Rails is the best-selling programming book on Amazon.com. Even better, Programming Ruby has the number two spot. Corgratulations.

As far as I can see, Typo 2.5.6 (the most recently released stable version) should work fine with Rails 1.0. I just did a brief round of testing with 1.0rc2, and all of the tests pass. Er, except for one test that had a stupid typo that somehow still worked with Rails 0.13.1; the bug is in the test itself, though, so it’s not worth releasing Typo 2.5.7 just for that. If we ever release Typo 2.5.7, then I’ll make sure that the fixed test is included.

Also, Typo 2.5.6 should work just fine with Ruby 1.8.3, too, as long as you’re using Rails 1.0. I haven’t actually tested this yet, but I’d be surprised if it doesn’t work perfectly.

Surprisingly enough, the current Typo trunk (r683 or so) doesn’t work with Rails 1.0. All of the filtering code is broken; I’ll fix it shortly and check in the fix. Fortunately, the current Typo trunk is pinned to Rails 0.13.1 for now, so it should be safe to upgrade the version of Rails on the box; Typo will just ignore the new Rails for now.

Update: the Typo trunk r685 or later should be compatible with Rails 1.0rc2. I’ll probably break Rails 0.13.1 compatibility soon, so it’s time to upgrade.

Just for the record, current versions of Typo (either 2.5.6 or the current Subversion trunk) don’t work with Ruby 1.8.3. There are two problems–the Logger bug that keeps Rails 0.13.1 from working with Ruby 1.8.3 (this is easy to fix), and a second bug that I haven’t read about anywhere else–apparently YAML serialization is broken with Ruby 1.8.3 and Rails 0.13.1. This keeps Typo’s sidebar from working properly.

I’m going to see what it’ll take to get the Typo trunk working with Rails 1.0 (rc1 or rc2, if it’s out today), and then see if it works properly with Ruby 1.8.3. Once that’s done, the trunk will probably shift from 0.13.1-only to 1.0-only.

Update: That was quick. ChrisNolan on IRC pointed out that Rails bug #2304 contains a patch to fix this. The patch is already a part of the current Rails trunk, but you’ll need to patch 0.13.1 manually if you want to use it with Ruby 1.8.3.

I finally had a bit of time to do some Typo benchmarking over the weekend and (as usual) found that my instincts were all wrong.

I was specifically interested in the performance difference between the page cache and the action cache–my guess was that the action cache was a 10x performance hit.

So I set up a test environment under Xen, running Typo r683, PostgreSQL, Apache 2, FastCGI, Ruby 1.8.2, and Rails 0.13.1. I didn’t do any Apache or Postgres tuning–I just ran them out of the box.

Then I ran ab against a snapshot of scottstuff.net from a couple weeks ago. I used the index page for my testing, as it’s a fairly large page and I wanted to give Typo a real workout.

Here’s what I found:

That really wasn’t what I’d expected. The action cache underperformed my expectations by a factor of 20.

I then did a bit of experimentation. I created a new uncached action in my test Typo setup that did nothing but render :text => 'foo', :layout => false, just to see if the caching system was slowing things down. Result? 10 requests/second. Then I created a new Rails project from scratch and added a new controller with the same action, and saw the same results. Still 10 requests/second.

However, in Rails’s logs, it said that it handled the request in 2 ms, and I should be seeing 500 hits/sec for the “foo” page. So something is adding an extra 98 ms to each request. I’m still hunting for this–I don’t know if it’s something Xen-related on my system, an artifact of my Apache config, or what.

I’ve tried upgrading to Rails 0.14.0, but that’s a whole other article.

Conclusions:

1. The page cache is really, really fast.
2. The action cache is a substantial improvement over the uncached case–about 10x on this system–but can’t touch the performance of the page cache.
3. Changing the concurrency settings on ab and/or the number of FastCGI backends in use didn’t make a substantial performance difference. Settings 2-15 gave roughly the same results.
4. My caches_action_with_params is slightly faster then the stock action cache.
5. Moving the action/fragment cache from the FileStore (Typo default) to MemoryStore gives no real performance boost. Moving to the MemCacheStore is a substantial performance hit (~2 hits/sec vs 10 hits/sec).
6. Adding a new uncached action in ArticlesController that simply returns a fixed string (render :layout => false, :text => 'foo') is no faster then the action cache.
7. Moving the new action from the previous step to a Controller of its own doesn’t help.
8. Removing all of the routes except the default :controller/:action/:id route doesn’t help, either.

Frankly, on this hardware, I don’t seem to be able to get more then 10 requests/sec out of Rails no matter what I do. I’m pretty sure that this is a mistake, so I’ll post a followup when I figure out what’s wrong.

Update 1: Another datapoint. Running an example Ruby FCGI on this box gives me 832 hits per second. So whatever the problem is, it’s not fundamental to Ruby FCGI on this box. So I need to look into Rails and see what’s happening.

Update 2: Making some progress. Apparently I screwed up when I tested a new, standalone Rails FCGI app before (forgot to restart FCGI?). This time, I got 130 req/sec, which is a vast improvement over the 10 req/sec that I was seeing before. Most of that speed hit seems to come from using Postgres for session storage. Unfortunately, even after making that change, my null controller is still only getting 40 req/sec with Typo. Transporting the same controller to a blank Rails project gives me 130 req/sec with the same code. Watching strace, it looks like something is forcing Typo to reload the digest/md5 module for every hit. Unfortunately, I can’t figure out how that’s happening–my environment.rb is identical between the two trees, as is my database.yml. I’ll get back to this later; I have other things that I need to finish today.

Nokia announced a new block of phones today: the E60, E61, and E70. Unlike the earlier (and still pending) N-series, these are aimed towards corporate users. All three run Series 60 3rd edition and include 802.11g and Bluetooth. They also come with a SIP client so they can interact with business VoIP systems. The low-end model, the E60 is yet another Series 60 candybar phone, like the 6682 and N70, but with WiFi. The E61 is a Blackberry-like model with a QWERTY keyboard and landscape display. Finally, the E70 is a “handlebar” style phone that flips open to reveal a keyboard, half on the left side of the display and half on the right.

After playing with the Nokia 6682 and 9300 yesterday, I’m starting to think that a keyboard would be really nice. I might actually be more interested in the E70 then the N91. Here’s how they compare:

Same:

• 802.11g, bluetooth, 3G
• Series 60, 3rd edition
• 2 MP camera
• Scheduled for 1Q 2006 release
• Similar sizes: (N91: 113.1x55.2x22, E70: 117x53x22)

N91’s favor:

• 4 GB hard drive
• 3.5mm headphone jack

E70’s favor:

• built-in SIP client (might be on N91 also)
• high-res display (352x416, 4x the pixel count)
• QWERTY keyboard
• miniSD slot (1 GB for $75, partially counters the HD on the N91)
• lighter (127g vs 160g)

I’m not sure if the E70 includes the video player that comes on the N91. It’s possible that it includes a 3.5mm headphone jack–the specs list a MP3/AAC player application. The pricing rumors that I’ve seen put the whole E-series around €500 or so, which means that they may actually be cheaper then the N91.

Of course, availability is the key. The N91 was announced months ago, while the E70 is new. However, the E70 isn’t quite as innovative as the N91, so there’s a chance that they’ll arrive on the market (in Europe at least) in a similar timeframe. I have no idea which one will make it through the FCC first. I’m not particularly concerned about US carriers carrying the phone, now that Nokia is selling directly to US consumers. With any luck, our local Nokia demo kiosk will have both and I’ll be able to compare them in person.

While I really like all of the things that Asterisk allows me to do with my phone system, I’m really not very fond of its configuration language. The language provided in Asterisk 1.0 is slightly better then sendmail.cf, but it’s still a lousy language. They made a few small improvements early in the Asterisk 1.2 development process that helped a bit, but I assumed that we’d have to wait for another year or two before someone broke down and wrote a decent language for Asterisk.

It looks like I may have been overly pessimestic. I discovered the Asterisk extension language on a list of new features for Asterisk 1.2 today. Somehow I’d missed this when it first went into Asterisk.

Here’s a chunk out of my old config file. It’s not perfect, but it’s what you get when you’re stuck dealing with line numbers:

[macro-diallocal]
  exten => s,1,AbsoluteTimeout(7200)
  exten => s,n,SetAMAFlags(default)
  exten => s,n(analog),Dial(${TRUNK}/${ARG1})
  exten => s,n,Congestion
  exten => s,analog+101,Macro(condsetcid)
  exten => s,n,SetCIDName(LAIRD SCOTT)
  exten => s,n,SetAMAFlags(billing)
  exten => s,n(nufone),Dial(${NUFONE}/1${ARG1})
  exten => s,n,Congestion
  exten => s,nufone+101,Busy

And here’s the equivalent using the new configuration language:

macro diallocal( number ) {
  AbsoluteTimeout(7200);
  SetAMAFlags(default);
  Dial(${TRUNK}/${number});
  if(${DIALSTATUS} = "CONGESTION" || ${DIALSTATUS} = "CHANUNAVAILABLE") {
    &condsetcid();
    SetCIDName("LAIRD SCOTT");
    SetAMAFlags(billing);
    Dial(${NUFONE}/1${number});
    switch(${DIALSTATUS}) {
      case BUSY:
        Busy;
      default:
        Congestion;
    }
  }
}

It’s still not ideal–${VAR} is ugly–but it’s vastly better then the old syntax. This plus a bit of Rails integration should give us a really nice phone environment. I’m rapidly approaching the “tear it down and rebuild it” point with my Asterisk system–there’s a lot of stuff that I’d like to add that just doesn’t fit in with my current configuration files–so I’ll have a set of articles on integrating Asterisk, AEL, and Rails in a few weeks.

Geoffrey Grosenbach just announced the start of the official Typo theme contest. There are a number of fantastic prizes (including a 4 GB iPod nano) available, so take a look at the rules and fire up your editor now.

I got dragged to Alderwood Mall (just north of Seattle) today and discovered that they have a Nokia Kiosk just southwest of the Apple store. This seems to be run by Nokia itself, not a carrier or reseller, and they had a wide range of phones available, including the 6682 and 9300–the highest-end phones that Nokia is currently shipping in the US.

I didn’t ask, but I hope they’ll have the N90 and N70 later this year. I don’t expect to see the N91 in the US for another 6 months or so, but maybe they’ll have it eventually; it’d be really nice to be able to look one over before I have to decide if I’m going to order one or not.

After playing with the 6682 for a couple minutes, I’d love to see the N90’s display. The 6682’s display was much higher-resolution then I’d expected, and the the N90 has four times as many pixels in about the same space (416x352, 2.1” diagonal). I’m really curious what a 260 DPI LCD looks like. Not that I’d really like the phone, but I’d love to see the screen.

One side effect of running my own business is that I’ve been spending a lot of time on the phone. Unfortunately, the $10 analog phone that I was using was hard to use for more then 15 minutes at a time, and it wasn’t always very easy to hear what the other party was saying. That’s not a great mix for a business phone.

So, I’ve been looking around to find a cheap way to get a good phone with a headset on my desk. None of the analog headsets phones that I’ve looked at have been very appealing, so I’ve been looking at cheap VoIP phones with 2.5mm phone jacks. Amazingly enough, VoIPSupply.com decided to clear out a bunch of old phones via eBay at just the right time, so I was able to pick up a Pingtel Xpressa for a song.

The Xpressa was a first-generation VoIP phone, and it was orphaned by Pingtel over a year ago. However, in many ways it’s still the highest-end SIP phone on the market. It comes with a Palm-like 160x160 grayscale display, supports (pre-standard) Power-over-Ethernet, and runs Java applets natively on the phone. The SDK is still available from Pingtel, but you have to hunt for it a bit. I paid about the same for the Xpressa as I did for my Sipura/Linksys SPA-841, but the Pingtel is clearly better in nearly every area–it’s easier to use, it’s more solid, it’s more attractive, it has more features, and sounds better. It’s lacking a few NAT features and I can’t find a way to use different ringtones for different lines, but other then that it does everything that I need. It comes with a standard 2.5mm headphone jack, so I picked up a cheap Plantronics headset; I’ve spent nearly two hours on the phone so far today, and everything has been perfect.

Er, well, mostly everything. I actually ordered the mango colored model from VoIPSupply, but somehow ended up with a charcoal-colored phone instead. Hopefully they’ll be able to fix that soon. Also, since Pingtel discontinued the phone (and actually sold it off to an unnamed vendor, alleged to be 3com), they’ve pulled all of the add-on software packages off of their website; that means that I haven’t been able to find their LDAP-based phonebook. I’ve been fishing around and I’m sure that I can find a copy somewhere. For now, I’ve been using Jon’s Phone Tool to dial from Quicksilver. JPT is swimming in features, 95% of which are useless to me, but it seems to do a decent job dialing my phone for me, so I’ll probably pay the $12 fee for it if I can’t get the Pingtel Phonebook to work.

Update: As expected, VoIPSupply is fixing the mango-colored problem. I should have a mango phone on the way shortly.