*scottstuff*

I’ve updated this blog again, using my latest Typo patches. It’s currently running Rails 0.13 (less then 2 days old), Typo SVN r280 (the most recent), and every change that I’ve submitted to the Typo Trac system. This includes:

• Per-article RSS feeds, so you can track comments and article changes via RSS.
• Fixed ‘comment’ link on the main index page; it now points to the #comment anchor instead of the head of the article.
• Configurable number of articles included in index pages and RSS feeds.
• Improved comments. This includes comment threading, per-comment subject lines, and optional email addresses.
• Better text filtering, so I can use SmartyPants-style quote fixing and use different filtering styles for comments and articles.

I just upgraded this blog. It was running the latest released version of Typo (version 2.0.6). It’s now running a development version of Typo from Typo’s Subversion tree, plus the patches that I posted here earlier.

One upshot of this is that threaded comments should now work. Feel free to play with them here. If you have any problems with them, then please send me mail. Thanks.

One other change is that comments are now formatted with Markdown, so it’s easy to use bold, italics, lists, and links. See the Markdown page for details; eventually I’ll add a blurb to the comment form explaining the simple stuff.

As of about 5 minutes ago, the Typo version of this blog is now live, and the old Movable Type version is disabled. All old URLs should still work–I spent quite a bit of time generating a list of redirects to send each old page to the correct URL in Typo.

I still have a lot of template work to do–this site looks really plain to me, and I’ve been meaning to redesign the site for a while now, anyway. That should happen over the next week or so. I’d also like to implement threaded comments in Typo. The display end of threaded comments looks trivial to implement, but I’m not so sure about the other end of things.

Update–Wow, Typo publishes new articles way faster then MT did. From clicking the ‘Publish’ button in Ecto to seeing the new article appear in NetNewsWire is only a couple seconds. MT 2.6 took around a minute before the article was ready. On the downside, categories in Ecto seem to be broken.

If anyone sees problems with the new site, either leave a comment here or send me mail. Thanks.

I’ve spent most of the evening playing with Typo, and I believe I’ve fixed all of the problems that I was seeing this morning. Most of the problems that I saw were caused by the import script; adding a bunch of ORDER BY clauses suddenly made it import items in the same order that they were originally created.

At this point, I’m nearly ready to make it live. I just have three issues left:

1. Typo doesn’t ask for email addresses for comments, even though there’s a spot in the DB for them. I’ve found that it’s a lot easier to respond to people when the can provide me with an email address, even if it’s never published.

2. Similarly, there’s no way for it to display *my* email address. This should just be a quick template hack.

3. Finally, I really dislike Typo’s default date format. Instead of providing an exact date, it says things like “Posted by Scott 672 days ago.” It looks cool for recent entries, but it’s pointless for anything more then a couple weeks old.

That’s pretty much it. Once those are done, I’ll make Typo my primary blog and start turning Movable Type off. If anyone’s interested, they can see my work-in-progress Typo blog, although I don’t guarantee that it’ll always be running.

My test installation of Typo is up and running. First impressions:

1. It’s easy to install, once you have Rails working. I’d never actually installed Rails on my home web server, so I had to do the whole install gems, install rails, hunt down dependancies bit. I had an early version of Active Record installed, and it was being loaded instead of the current version that comes with rails, so I had to hunt it down and kill it.

2. The MT-to-typo conversion script worked okay, once I fixed the MySQL-isms in it. Specifically, it assumed that it could access the MT database by using databasename.tablename; that syntax gets you into different namespaces in Postgres, not different databases. I worked around it by copying the MT tables into the Typo database and then removing the databasename. part of the queries. It probably would have been easier to just pass public as the database name, though.

3. The script also assumes that all articles are formatted as HTML. Mine are (almost) all Markdown. To fix this, I had to install BlueCloth (gem install bluecloth) and then rebuild all of the cached versions of each of my articles. There’s no obvious way to do this in the Typo UI, but Typo’s internals makes it easy. All I had to do was run the Rails script/console command and then run Article.find_all.each { |article| article.text_filter='markdown'; article.save }. For the Ruby-phobic, this iterates through each Article in Typo’s DB, changing the formatting type and then saving the results back to the DB. Typo’s Article model hooks the save method to verify that text format conversions are handled correctly, so that’s all I really had to do. Nice and simple.

4. It’s kind of slow. I’m running Typo in debugging mode right now, and Rails is always slow in Debugging mode. I suspect that Typo will still be a bit pokey, even in production mode, though. I need to look into how it handles caching.

5. For some reason, not everything is sorted correctly. The ‘latest comments’ sidebar seems to be sorted randomly. This is probably an artifact of the conversion script–it looks like entity IDs aren’t in the same order as they were in MT, and that screws stuff up. That’s just a one-line fix to the conversion script.

6. Comments are listed newest-first. I’d rather have them appear in the order that they were written. This should be a one-line change.

7. The admin interface is sort of sparse. Rails makes it really easy to make calls directly into Typo’s API from the command-line, so I’m not too concerned with the lack of things to click on.

8. Somehow, some of the Article-Category associations were lost during the conversion. Most of them seem okay, but not all of them. This might take a bit of work to fix.

9. The categories list on the sidebar seems to be in ID order; it should be sorted by category name. This is easy to fix.

Nothing here looks fatally broken. I need to spend 15 minutes updating the conversion script and another few minutes fixing little issues with Typo, but the system seems to work well enough, at least on initial inspection. I’ll re-import the MT data again later today using the fixed conversion script, and then see what it takes to get it working with Apache in production mode. If that works out okay, then I’ll spend some time tweaking the CSS to make it look a bit more like my current site and then make it live.

Oh–one more thing–I need to have the conversion script save the old URLs from each entry to a file so I can write a bit redirect list that will point each MT article’s URL to the new Typo URL. I get too much traffic from Google to lose all of the links.

Last night’s upgrade to Movable Type 3.17 went fairly well. The upgrade from 2.6 to 3.17 was trivial–it installed and ran without problems. I didn’t really expect that, but I wasn’t about to complain. Once I verified that 3.17 was working, I started the process of moving Movable Type from using the Berkeley DB onto PostgreSQL, and then things started breaking.

Movable Type, like most open-source software, was really designed to use MySQL, with Postgres support added later. I’ve found a lot of software that’s supposed to work with Postgres, but actually fails in a number of fundamental ways, simply because there are a number of MySQL-related assumptions lurking in the code, and no one actually tested the system against Postgres.

In Movable Type’s case, the BDB-to-SQL conversion script is able to create the right set of tables in Postgres, and it’s even able to populate the tables with all of the right data. Unfortunately, it fails to update the sequence counters for any of the tables, which causes new articles and comments to show up as ID #1. If you already have a #1, then posting new articles and comments will silently fail.

It took me most of an hour to track that back and figure out why new comments weren’t being posted. Then it took about 15 seconds to fix–just do a select max(comment_id) from mt_comment to determine the maximum comment ID, then do select setval('mt_comment_id',9999), replacing 9999 with the number from the previous select. Then repeat for every table in the DB.

So, once that was done, everything worked, right? Ha. I was now able to post new comments and articles, but they didn’t show up on my site. Hitting ‘rebuild’ didn’t actually rebuild anything. So what happened this time?

Ah, the conversion script failed to populate the template table, so MT didn’t know which pages to build. Fortunately, I keep all of my templates in text files, not in the DB, so it only took 15 minutes to re-enter all of the template data; once that was done I was able to add comments and articles without problems, and hitting ‘rebuild’ correctly rebuilt the whole site.

So, everything appears to be working now. On the plus side, comment posting is way faster–it used to take around 30 seconds from the time users hit ‘publish’ until the browser reloaded. Now it’s down to 5 seconds or so. Hopefully that’ll cut down on the spate of duplicate comments that I’ve seen, because people won’t be so tempted to keep hitting ‘publish’ every ten seconds until it works.

Now that that’s done, I’m going to move on to testing out Typo. I noticed that the MT-to-Typo conversion script has a couple obvious MySQL-isms in it, but the script is so short that it shouldn’t be hard to work around.

I’ve been promising to upgrade this blog to something more modern and interesting for over a year, and still haven’t done anything about it. I looked at Drupal for a while, but it just isn’t what I’m looking for.

So, for now, I’m going to upgrade from MT 2.6 (using BDB) to MT 3.17 (on Postgres). Then, once I have a bit of time, I’ll probably move to Typo, largely because it’s written in Ruby and looks easy to hack. It’s lacking a few things that I really wish I had (like threaded comments), but the code is so clean and simple that it shouldn’t take long to add them.

I’d jump straight to Typo, except that the MT-to-Typo conversion script requires MT 3.x to work. So I’m going to upgrade MT first, and then later this week play with Typo and see how it goes.

Once I start the MT upgrade, I expect everything here to break for a few hours. I don’t see an easy way around that, though.

I was looking at my web logs last night, and noticed that (as usual) I was seeing a lot of hits coming from Google searches, but a lot of them were from country-specific Googles like google.co.uk, not just google.com. So, out of curiosity, I spent a few minutes writing a couple little scripts so see just how many different Googles have sent traffic to my site over the last year or so. The answer?

*90*.

Yep, 90 different countries worth of Google, counting .com as a country. The full list is below the break.

I’ve never really figured out why, but Google really likes me. Or, rather, it likes this blog. I keep showing up amazingly highly-ranked in common Google searches. Today’s example is treo wifi. In order, here are the top 10 results, out of 2.2 million possible matches:

1. TreoCentral: No Treo WiFi
2. TreoCentral: Treo 600 and WiFi?
3. .:UNEASYsilence: Treo 650 WiFi
4. PDA News: Treo 650 WiFi, Verizon announces XV6600, PalmOne…
5. scottstuff: another WiFi solution for the Treo 650
6. Slashdot: Enthusiast Hacks WiFi Into Treo 650
7. CNet: Treo 650 Update WiFi-less
8. Engadget: Add WiFi to your Treo 650! SD WiFi card drivers hacked
9. Engadget: some random search page
10. PalmInfocenter: HOWTO: Make that palmOne Treo 650 Even Better!

So, as I see this, Google sees me as a better source of information then Slashdot, CNet, Engadget, and PalmInfocenter? It can’t just be PageRank–from what I can see, I’m just a lowly PR5 this month, while Slashdot and CNet are PR9s, and Engadget is a PR6. It can’t be the number of links in Google’s database, because no one links to my Treo WiFi page. Can anyone explain how this works?

I received my first three pieces of trackback spam overnight. By blog receives tons of comment spam, most of which are blocked by MT-Blacklist, but I haven’t seen any trackback spam before, even though I’ve been expecting it for over a year. Since trackback is designed to be automated, it seems like it’d be easier to abuse then comment spam. Fortunately, a few relatively simple steps should help stop trackback spam. A quick verification step would probably stop most of it–if you hand me a trackback URL, and I go fetch it, is there a link to my page anywhere on the trackback page?

One quick-and-easy suggestion for stopping comment spam–add </a> to your blacklist. MT 2.6 allows HTML in comments, even though it’s not supposed to, and comment spammers always try to stick links in there. Since my comment page isn’t supposed to allow HTML anyway, this does a great job of blocking spam.