News You Can Bruise for 2006 July 25 (entry 0)

Tue Jul 25 2006 22:00 Beloved Ruby Cookbook Recipes #1: "Getting book information with Ruby/Amazon": You know those old-time radio shows, where Doris Day always guest-starred, and Bob Hope ended up wandering onto the field at Muroc and getting shot? Well, that particular one never aired, but you know the kind of show I mean. The shows that were sponsored by random household objects like soap, or a broom, where the straight man would set up a line and the funny man would always turn it into a little ad for the sponsor.

"Lou, I think we ought to stick around a while longer." "You know, Acme Wallpaper is the only thing today's handyman needs to 'stick around' the house. It's inexpensive, durable, and when properly applied it forms a Penrose tiling! Ask your dealer to see a catalog today." Thus allowing the straight man to set up the exact same line again: "I say, Lou, that we ought to stick around a while longer and watch after Doris Day."

This is the model for this series of weblog entries, where I show you how to do cool things with Ruby but it turns out all to be a scheme to plug the Ruby Cookbook, bad boy of the O'Reilly cookbook family. In the first three entries of this series, I'll show you the system I rigged up to track the sales of my books. I don't really have a plan after that, but I can work from a list I made of the coolest recipes in the Cookbook.

The Book Sales Trilogy:

1. Getting book information with Ruby/Amazon
2. Generating graphs with Gruff.
3. Generating sparklines with the sparklines gem.

The problem

When you write books but don't publish them, it's difficult to know how many you've sold. The publisher gives you a total maybe quarterly. So a lot of authors (including me) use their Amazon sales rank as a proxy for books sold.

There are numerous problems with this, stemming from two facts: 1) Amazon sales rank only covers books sold through Amazon, and 2) it only conveys how well your book is selling _compared to all other books_. Booms and crashes in the book market as a whole won't make any difference to your sales rank. But you can't beat something with nothing, and sales rank is the only way I know of to get anything like real-time sales figures for a book. I talked with Mike Loukides, my O'Reilly editor, today, and he said that giving authors access to sales information is a low-priority project. And we all know about low-priority projects.

So, whenever a book of mine goes up on Amazon (first Learning Python and now the Ruby Cookbook), I register its ISBN with a script that runs once an hour and uses Amazon Web Services to collect the sales rank numbers. My goal is to encapsulate this logic into a class which maps an ISBN to a text file full of timestamped sales ranks. It will also work with Amazon-specific ASINs: for books, the ASIN is just the ISBN.

I use the Ruby/Amazon library, which is covered in Recipe 16.1, "Searching for Books on Amazon". The recipe focuses on keyword searches: finding books or other objects to match criteria. Here, we search by ASIN.

The script is based around a SalesReport class that encapsulates data about one product's sales rank over time. Here's the first draft of SalesReport.

#!/usr/bin/ruby
require 'amazon/search'
class SalesReport < Array
  attr_accessor :name, :asin
  def initialize(asin, name)
    @asin, @name = asin, name
    open(self.class.filename(@asin)).each do |line| 
      self << line.split.collect { |x| x.to_f }
    end
  end

  def self.filename(asin)
    asin + '.txt'
  end

When you create a SalesReport object for a product, it opens up a file named after that product's ASIN and reads in sales data.

Where does that file come from? The update! method is in charge of creating and maintaining it. This is also where we make the actual AWS request (in bold).

  # Updates sales rank for a number of products, and yields a
  # SalesReport object for each one.
  def self.update!(aws_key, *asins)
    now = Time.now.to_i
    req = Amazon::Search::Request.new(aws_key)
    req.asin_search(asins) do |product|
      asin, name = product.asin, product.product_name

      # Update the file with the new sales rank.
      open(self.filename(asin), 'a') do |f| 
        f << now << ' ' << product.sales_rank << "\n"
      end

      # Parse the file into an SalesReport object and yield it.
      yield self.new(asin, name)
    end
  end
end

Three things about this method's signature. First, like filename this is a class method: it's a factory for SalesReport objects rather than something you run on a particular SalesReport. For more on this, see recipe 8.18, "Creating Class and Singleton Methods". The self here is the SalesReport class itself.

Second, this method takes a variable number of ASIN arguments, for which see Recipe 8.11, "Accepting or Passing a Variable Number of Arguments".

Third, as I've mentioned in every book I've ever written, to use AWS you need to sign up for a unique string called a "developer token". This method uses that string to create an Amazon::Search::Request object, which will be our interface to Amazon Web Services.

You can reuse a Request object to do multiple searches, but we're only doing one. To save time and bandwidth, we pass all of our ASINs in a single search: "give me info on these products". The query returns a big XML document, which Ruby/Amazon parses into struct-like Ruby objects. It yields each object to a code block.

Inside the code block, I have access to a variable product which represents what Amazon knows about one particular book. I take its sales rank reading and stick it along with a timestamp into that book's file. Within the code block I also build a SalesReport object, which I yield to another code block somewhere outside this function. The code that calls SalesReport.update! gets yielded a series of SalesReport objects to do with what they will. In this case, my will is to generate cool-looking graphs.

If code blocks confuse you (they confused me when I was new to Ruby), the first few recipes of chapter 7 will help. If you're not familiar with file access, the basics (including appending to a file) are covered in Recipe 6.7, "Writing to A File".

Okay, that's all the SalesReport code for today. Here's how you'd call SalesReport.update! with a code block:

key = 'my AWS developer token whatever'
SalesReport.update!(key, "0596523696", "0764596543") do |product|
  puts "I updated #{product.name} (Rank: #{product[-1][1].to_i})"
end
# I updated Ruby Cookbook (Cookbooks (O'Reilly)) (Rank: 1412)
# I updated Beginning Python (Programmer to Programmer) (Rank: 219370)

So now I've got two files on Crummy's hard disk full of sales rank readings. What to do with that data? That code block doesn't really do anything., Well, I'd love to compare the sales ranks to hard sales numbers, but I've never gotten any for the Python book, and the one data point I got from Michael today doesn't mean much. So I've been comparing these numbers to their earlier selves by graphing them versus time. This leads into tomorrow's episode, Graphic Violence, featuring Chico Marx.