XyzzyB

Tumblr?

Blogging is hard.

It’s very easy to just not write anything. Good blog software should provide as little friction as possible.

I’ve always held back writing on my blog because I’m always thinking about tweaking the site design, the code, the server, the framework, etc. Content, even after I tried to just set it and forget, has fallen by the wayside for quite a while now.

Tumblr is really striving to be that blog software that just gets out of the way.

• You want to write a post on the site? Sure.
• You want to email a post with an attached image? No problem!
• You want to tie into twitter? Absolutely.
• You want to change themes with a click? Of course.
• Want to cohesively share different kinds of content? Out of the box.

I’ve gone through blog reorganizations before. I’ve switched from straight HTML to wordpress to generated HTML to perl to Rails. Nothing has ever really felt simple, easy, and polished. Tumblr does.

I feel really good about this setup. Let’s see if I stick with it.

UPDATE: there’s even a ruby wrapper and command line tool for interacting with the tumblr api. Awesome.

Simple Desktops

Got a computer? Got a window manager GUI? Want nice, clean, simple pictures to put up behind your windows? “Simple Desktops”:http://simpledesktops.com/

Some gems:

• Sine by Kyle
• Circuit by Saif Sajid
• Good Morning by Axel Valdez
• Tron Light Cycle Blue by Katie McCormack
• Tron Light Cycle Orange by Katie McCormack

A/Bingo in Rails 3.1

To add A/Bingo to a Rails 3.1 you can mostly follow the (Rails 2) install docs, but you’ll might run into some slight differences.

For a full set of instructions to get up and running, head to Andy Atkinson’s post Getting started with A/Bingo and Rails 3. I’ll just cover some extra tidbits here.

Using Redis as the data store

A/Bingo uses a cache layer for mapping sessions to experiment participation. Out of the box it will just use whatever Rails is already using for caching. That’s probably ok unless your app is hosted on multiple web servers, then you’ll need to bring up some persistent, shared cache like memcache.

In our setup, we already had redis for another application so I just piggybacked on that.

Gemfile

To setup A/Bingo to use redis, first edit your Gemfile and run bundle install:

gem 'redis'
gem 'redis-store', '1.0.0.rc1'

setup A/Bingo per environment

Edit your environment files and setup the redis configuration to point to your servers.

# Abingo options
# expires_in: prevents flooding redis and "consuming all memory"
redis_server = 'redis://production.redis.server:6379/1/broadband_com'
Abingo.options[:expires_in] = 1.hour
Abingo.cache = ActiveSupport::Cache::RedisStore.new redis_server

And there you go, A/Bingo in Rails 3.1 using redis for the caching storage. In actual use you’ll want to move the redis server specification to a yml file keyed by rails environment.

Bonus Round: authenticating A/Bingo access using CMS

If you’re using Comfortable Mexican Sofa then you’re 98% of the way there. Just change your dashboard controller to inherit from CmsAdmin::BaseController.

class AbingoDashboardController < CmsAdmin::BaseController

Done!

Putting the focus back on content

UPDATE (September 14, 2011): This blog is now on tumblr which is even better than a redesign. I can choose from thousands of designs with a click and I don’t have to think about the hosting or backend at all. Awesome.

For months now I’ve been meaning to design a new look for this site, but I’ve only ever made partial and half-hearted attempts at actually doing so. I’ve used the “it needs a new design” excuse to stop writing any content because I reasoned that time should be spent on making the new design. Trouble is, that’s silly. Blog content is limited to the web medium and, in fact, I hope you’re reading this via your favorite RSS reader. (Mine is Reeder on the iPad and I would use it for every site on the web if I could.)

So I’ve decided to stop procrastinating a redesign of this blog-o-sphere, use this very nice theme from the talented Jayj and be done with it. From now on, I’m just going to blog. If I find a theme I like then I’ll incorporate it, but I’m not going to waste time on the aspects of the site I don’t care as much about.

What’s been going on?

at work: broadband.com

After much work, we’ve pushed through a broadband.com re-re-re-launch. The new site is much more streamlined, there are less cooks, and it’s a good place to start iterating from. Is it a work in progress? Yes! A website that isn’t is dying. I’ve added a/bingo to the application and we’re getting some good buy-in to the idea of data driven development.

at home: home theater

What? Yes! Yes! My long-time dream of having a real home theater has been realized. An Epson 8700UB projector, a 120” screen (yes, 10 feet on the diagonal), sweet 7.1 sound, a dedicated room with no windows and dark color scheme…my wife and I are gleefully enjoying its entertainment potential.

Yeah, a dark color scheme. I didn’t think it’d make a big difference either and thought that all the folks on the home theater forums were fooling themselves. But think about it: a white wall reflects a ton of the light that hits it. When you’re projecting onto a screen then that screen gets washed out by additional light. Going from white to dark walls had as much of an effect as turning off the lights.

It’s such a fun experience to watch movies on the “big” screen that we’ve never been able to either because they were released before we were born or we just missed in the theater. The new Lord of the Rings blu-ray release is so far the most visually impressive. We can really see the loving attention to detail that Peter Jackson’s team etched into every prop, every set, every shot. Little details like the titles on the books in Bilbo’s study really just sell the entire experience.

True Grit (2010) was the first full movie we watched on it. We are perfectly happy to wait for the disc releases of movies now.

Videogames on the thing…I don’t think I’ve ever really played videogames before. The vistas in Red Dead Redemption or Uncharted 2 are simply stunning.

Of course, I don’t have as much time for videogames as I used to.

at home: our little boy

Being a parent is the best thing ever, and it just keeps getting better. Our little man is just so amazing, I think he’s at least ten times smarter and more insightful than I am. We taught him the word “opaque” and he came up with “o-seen” and “o-clear” as antonyms. He is also quite taken with anything that makes light.

at work: rails

Rails continues to be just a fantastic web technology to work with. Especially Rails 3.1 with its asset pipeline. PHP and Django are but troubling memories.

blurbs

That wraps it up for all the things I can think of, except now I’ve thought of some blurbs.

Google+ is great, grouping people is intuitive and that’s what I really wanted. You can easily find me just by going to [“http://xyzzyb/+”:http://xyzzyb.com/+]

Justified is a really great TV show, watch it.

The Hunger Games series is an addictive read, read it. It peaks about 75% of the way through the first book, but it’s worth it.

If you’re tired of messing with your home wireless network and want it to Just Work, then pick up an Apple Airport Extreme. Worth it, so worth it. I was a doubter, but then I tried it.

Launching the new broadband.com

I pulled my first late-late night at work Thursday night and Friday morning this week to launch the new broadband.com. We actually had a lot of fun coding pepped up on pizza and caffeine. I stayed until 4:30am and was the last of the team to leave, but our (awesome) boss kept working on content and ended up sleeping on a couch at work. I drove home and slept for about three hours then headed back in for the final push.

Friday was a day of fun and games (and launch t-shirts!) celebrating the site launch which was scheduled for 4pm. We were still heads down working on getting all the pieces together and, whew!, got completely deployed at 3:45pm. Just in time for the official launch but after the launch party down the street had started. :-)

Launch party? Yeah, I’d never had one of those either. The company rented out the party room at Rally Point and our whole company group (about 40 people) was in full attendance. I gotta say, it was a lot of fun to be cheered and applauded while drinking free beer and grooving to the awesome music of Mystery Sushi.

It was definitely a crunch for the team, and happily we came together perfectly. We have two primarily backend developers and I’m one of two frontend devs. Our backend guys really turned it all out this week too. At the start of the week we had no full production servers, and now we have some kind of crazy auto load balancing cluster of web and database servers all out on Amazon. It was pretty amazing to catch glimpses of that while we coded out the rails site.

IE almost ruined my whole day on Friday, but I got it at least to the point of functionality. (Yeah, we are explicitly allowed to treat IE as a second-class browser.)

Special shoutout to comfortable mexican sofa. The best CMS I’ve ever used. We need to refactor huge swaths of our site, but the system itself is pure gold.

Yeah, the site isn’t perfect (yet), but it’s out there.

Free Website Load Testing

UPDATE (September 14, 2011): This site is no longer an EnkiBlog but hosted on tumblr.

Load Impact is a freemium web service for stress testing websites. The free test gets you a test suite of 10, 20, 30, 40, and 50 concurrent clients with an average user load time for each. Going up to the paid service yields some cool features like viewing individual objects, more states, higher load testing, etc.

I ran XyzzyB (running EnkiBlog) through the free service and am pleased with the results.

DuckDuckGo has replaced Google as my default search engine

If you haven’t tried the new web search engine DuckDuckGo, you should. I never thought I’d say it, but it actually has replaced Google as my default search engine.

Replaced Google? How?

Literally? Just go to DuckDuckGo and click the “Add to Whatever Browser You Are Using” link. Google doesn’t have to be ubiquitous!

Replaced Google? Why?

At first I just decided to give DuckDuckGo a whirl, but after playing with the goodies and !bang searches, I got hooked.

• Great results, fast responses. It feels like my searches are just better in DuckDuckGo.
• Bang Syntax keywords to search specific websites. I can just search for “!wa weather” to do a Wolfram|Alpha search. I used to configure a bunch of these in any of my browsers, now I don’t have to.
• Zero-click results! DuckDuckGo will try to put actually relevant information (as opposed to ads) at the top of the search.
• Goodies!
• There’s a great iPad/iPhone app for searching/browsing. The zero click results allow this to be genuinely useful.
• Actual settings that set relevant things like whether your searches should display in the address bar (i.e. are bookmarkable) or not.
• Privacy: DuckDuckGo doesn’t track you. This one isn’t that important to me, but for many out there it’s DuckDuckGo’s critical feature.

I’ve emailed the dev (yes, it’s written by one guy) and gotten actual answers that same day. (He’s going to add in a !d20 search for the d20srd at my request.)

Give it a try, you can always get back to Google by searching for “!g whatever”.

mongodb: units in db.printCollectionStats()

This took far, far too long to find. Seriously. I was even digging into the source code.

The avgObjSize (and other data fields in the stats) returns the size in bytes.

Dynamically loading thousands of markers in Google Maps

UPDATE (September 17, 2011): If you want to effectively handle many thousands of points in a Google Map then the only real solution is using Google Fusion Tables. I hit on that as a solution a few months ago and it’s been working perfectly for The Broadband Map and it’s over 400,000 loadable points. Blog post with details coming soon.

When you’re making a Google Map with more than about a hundred markers, there are two established solutions: MarkerManager and MarkerClusterer.

MarkerManager:

• Loads/unloads markers in a map as you pan the viewport
• Allows adding sets of markers to specific zoom ranges

The restriction of markers to zoom levels means that you can use MarkerManager to roll your own clustering. That’s great for the end user because you serve up a completely customized experience. The downside is that you have to completely customize the experience. That means you’ll have to have some metric by which to split out your points into zoom levels. Say you have five thousand points: you’ll need to split them into zones so that no more than a few hundred (at the most) are visible at once.

MarkerClusterer

• Automatically clusters markers

That point is a big deal. Clustering isn’t cake, and marker cluster performs the tricky task with aplomb. The icons and styling is a little eh out of the box, but it gets you to the point where you actually care about icons and styling immediately.

The problem with both

Both solutions really want to work off a fixed array of markers that you’ve already pulled in. Feed either solution up to a couple thousand points of data and they’ll handle it (either automatically or with some work on your part) and you can move on.

But neither solution really deals well with loading almost a hundred thousand points of data. Obviously we won’t feed every map every point all at once, and neither MarkerManager nor MarkerClusterer handles dynamically adding and removing points that well since they are both doing a fair bit of calculation that requires knowing about every point.

My solution

My solution, so far, is to:

• dynamically load markers until we reach a threshold
• keep a hashtable of markers that have already been added
• after the threshold has been reached, remove markers that aren’t currently within the viewport
• remove all markers from the map when the user has zoomed out, and don’t load any markers until the user zooms back to a reasonable level

In my case, the point data is really only useful at the city level. This allows me to completely ignore the higher zoom levels.

Here’s the essentials from the code I have so far:

function requestMarkers() {
  // no loading markers when zoomed way out
  if (map.getZoom() < 11) { return };
  // start dropping markers before the browser bogs down
  if (markers.length > 50) {
    dropSuperfluousMarkers();
  }
  $.ajax({
    // [snip typical ajax data request]
    success: populateMarkers,
  });
}

// add the markers from pointsData if they haven't already been placed
function populateMarkers(pointsData, status, xhr) {
  for (var i = 0, ii = pointsData.length; i < ii; i++)  {
    var lat = pointData.geom.x;
    var lng = pointData.geom.y;
    var marker = new google.maps.Marker({
      position: new google.maps.LatLng(lat, lng),
      draggable: false,
      animation: google.maps.Animation.DROP, // whee!
      flat: true, // disables some styling DOM elements for a faster marker
    });

    // hash the marker position
    coordHash = calculateCoordinateHash(marker);

    /*
    if we haven't seen this hash, add the marker and mark as seen

    without this, the markers array quickly grows unwieldy as duplicate points
    are loaded
    */
    if(seenCoordinates[coordHash] == null) {
      seenCoordinates[coordHash] = 1;
      markers.push(marker);
      marker.setMap(map);
    }
  }
}

// turn marker coordinates into a hash key
function calculateCoordinateHash(marker) {
  var coordinatesHash = [ marker.getPosition().lat(),
                          marker.getPosition().lng() ].join('');
  return coordinatesHash.replace(".","").replace(",", "").replace("-","");
}

// remove markers that aren't currently visible
function dropSuperfluousMarkers() {
  mapBounds = map.getBounds();
  for (var i = 0, ii = markers.length; i < ii; i++)  {
    if (!markers[i]) {continue};
    if (!mapBounds.contains(markers[i].getPosition())) {
      // remove from the map
      markers[i].setMap(null);

      // remove from the record of seen markers
      coordHash = calculateCoordinateHash(markers[i]);
      if(seenCoordinates[coordHash]) {
        seenCoordinates[coordHash] = null;
      }

      // remove from the markers array
      markers.splice(i, 1);
    }
  }
}

// clear all markers from the map, empty the markers array and seen markers
function clearMarkers() {
  for (var i = 0, ii = markers.length; i < ii; i++)  {
    markers[i].setMap(null);
  }
  markers = [];
  seenCoordinates = {};
}