Allen Day’s Blog

I’m using a T-mobile Android G1 on AT&T network. I don’t have a T-mobile SIM at all, and I wasn’t getting the Cupcake upgrade through the system menu, either when using AT&T EDGE or using WiFi.

So I followed this guide: http://www.androidandme.com/2009/05/guides/beginners-guide-for-rooting-your-android-g1-to-install-cupcake/

Worked great and easy/no-hassle to follow for an Android n00b like myself. Best and most comprehensive one I could find.

I’m writing some scripts to let me automate the downloading and seeding of torrents. The idea is to have torrents pulled in from RSS or a screenscrape (much as Azureus does this, but I want to script everything with Python/Mainline BitTorrent, bash, and Perl), then to sit on the torrents for a day or so after their last mtime, then checksum them and if they’re good move them elsewhere for watching, etc.

Part of this requires checksumming the files, and Mainline doesn’t ship with a standalone utility to do this. So I wrote one in Perl, see below. This only handles single-file torrents for now (i.e. no directories of files).

#!/usr/bin/perl
$|++;
use strict;
use Convert::Bencode_XS qw(bdecode);
use Data::Dumper;
use Digest::SHA1 qw(sha1);
use URI::Escape qw(uri_escape);
 
my $base = shift @ARGV;
my $torrent = "$base.torrent";
 
open( T, $torrent) or die $!;
my $torrent_data = join '', <T>;
close( T );
 
my $metainfo = bdecode( $torrent_data );
 
my $file_name = "$base/" . $metainfo->{'info'}->{'name'};
my $file_length = $metainfo->{'info'}->{'length'};
my $piece_length = $metainfo->{'info'}->{'piece length'};
 
my $pieces = $metainfo->{'info'}->{'pieces'};
my @pieces = ();
my $offset = 0;
while ( $offset < length( $pieces ) ) {
  my $p = substr( $pieces, $offset, 20 );
  $offset += 20;
  push @pieces, $p;
}
 
open( F, $file_name ) or die;
my $seek = 0;
foreach my $p ( @pieces ) {
  my $buf = '';
  seek( F, $seek, 0 );
  read( F, $buf, $piece_length );
  if ( $p eq sha1( $buf ) ) {
    print '.';
  }
  else {
    print 'x';
  }
  $seek += $piece_length;
}
close( F );

I deployed a Gbrowse/Chado installation last week at Dow Agrosciences.  It got me thinking about how slow and basic the searches are with the Bio::DB::Das::Chado* adaptor, and wouldn’t it be nice to use SOLR here?

I made up a test dataset of gene/mRNA/exon 3-tiered feature groups by permuting some gene model data from the knownGene annotation set of the Hg18 build of the human genome.  You can grab the data set and script used to generate it here.  There are several files mRNA.EN.txt.gz that contain gzipped gene models, where N=3..7 indicates there are 10^N models in the file, uniformly distributed across a 500-megabase reference sequence.

I’m planning to load these data into a couple of different systems and then compare performance on some of the typical Bio::DB::GFF API calls.  I can personally test on:

• Chado
• The default Bio::DB::GFF schema (does it have a name?)
• The SOLR backend I’m about to implement

I know there are other feature DBs out there.  It would be good to include them as well in a later pass or to have someone else contribute the data once I get the benchmarking script written.

I threw together a Mahout/Taste based item-item based recommender last night.

	public static void itemItemRecommendations(String path, String file) {
		File f = new File(path, file);
	    try {
			DataModel model = new FileDataModel(f);
			model.refresh(null);
		    ItemSimilarity itemSimilarity = new LogLikelihoodSimilarity(model);
		    ItemBasedRecommender itemRecommender = new GenericItemBasedRecommender(model, itemSimilarity);
		    for ( Item i : model.getItems() )
			    for ( RecommendedItem j : itemRecommender.mostSimilarItems(i.getID(), 50) )
			    	if ( j.getValue() >= 0.7 )
			    		System.out.println(i.getID() + "\t" + j.getItem().getID() + "\t" + String.format("%.3f", j.getValue()));
		} catch (FileNotFoundException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (TasteException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}

This outputs item1 –recommends–>item2 pairs with a weight. I’m taking this and putting it into a solr document so I can display related item2s alongside item1 when it’s viewed.

Input data are comma-delimited tuples like so:

1fe7401b81eed49353d0cbeba5383848,5212,0.6
3c1832954a6e8781836fed670bb37b24,5212,1
70273e4c7c77700ee97acb8d0306c405,5213,0.8
1f057ccde135acbc881008bbf466e7e1,5213,1
51d44c7baca65ad39d11ba87bf2d438b,5213,1
adc924559b37114cd97d1f5cf7c71419,5213,1
78e254b4a11e61d76ff63cea02de4de8,5213,1
5c373ec7d9ad4a6f392c291d8ccba5ce,5213,0.2
fab8537564094fa8885f6214e6b682e1,5213,1
127f46aabcdbc2d2d04da8398a996c75,5213,1

Works great. Thanks Sean.

I got an unlocked T-Mobile G1 today. Woo. There is a bunch of mis-information out there on blogs and forum about how to get the phone set up. Here’s the real deal, I found these settings on Piaw’s Blog.

Name: whatever_you_want_the_name_to_be
APN: wap.cingular
Username: wap@cingulargprs.com
Password: cingular1
MMSC: http://mmsc.cingular.com
MMS Proxy: wireless.cingular.com
MMS port: 80
MCC: 310
MNC: 410

Now, on to my first impressions of the phone.

Works:

• Calling works.
• Google contact import works.
• Google Chat works.
• EDGE data works.
• WiFi data works.
• Keyboard works. It rocks

Doesn’t work:

• AOL Chat does not work. Complains it can’t read my mobile number from my SIM card.
• 3G data does not work. I read that the phone doesn’t support the 3G band used by AT&T.

Works, but not well:

• The browser works, but it sucks compared to the iPhone. Feels very slow. I was expecting a lot more given that it’s using MobileSafari/WebKit.
• Video download/playback works. The player was branded with the YouTube logo, so I’m guessing it only supports YouTube out of the box.
• The UI feels clunky at first. Sort of feels like Nokia S60, too many menus and inconsistency in how different tasks are done

http://www.everydns.com

Found this today, and it works as advertised. Need to look more closely, but with this I think I can stop paying dyndns.com $30/year/domain for custom DNS.

1. Noticed a new Tesla Motors showroom in Los Angeles at Sepulveda/405 on Santa Monica Blvd last night on my drive to my guitar lesson. Made a mental note to stop by and check it out, looks like they have some museum-style exhibits –fuel cell cutaways, etc of the car.

2. Was talking it up to some friends/coworkers earlier today.

3. Was planning to go to the Hadoop Meetup tonight at Mahalo, but skipped it and worked late.

4. Saw an orange Lotus pass me on the way home… but wait… it has a TESLA logo! I pursued, thinking it was kind of late for a test drive.

5. Pulled up at a red light to listen to the silence / congratulate the driver on his nice ride. Honked my horn.

6. Driver rolls down the window and turns to me, and it’s none other than the CEO of Mahalo, Jason Calacanis!

A (partial) list I found today. Doesn’t include NIPS, so I’m not sure how exhaustive it is, but it has a bunch I haven’t seen before.

http://www.kmining.com/info_conferences.html

I’ve recently and in the past had several requests for my newsfeed reading list.

I don’t really use a proper newsreader. I use the “Web Clips” option in Gmail. It’s like a poor man’s newsreader. You can add RSS feeds under Settings > Web Clips, and article titles will appear in the bar above your mailbox, interleaved among the advertising that Google places there. You can only have up to 40 (shame on Google!) but it works well enough.

Anyway, I dumped out the links today into my blogroll. You can see them in the sidebar.

Here’s what the pricing looks like. Learn more about Amazon CloudFront at http://aws.amazon.com/cloudfront.

United States Edge Locations

Data Transfer

$0.170 per GB - first 10 TB / month data transfer out
$0.120 per GB - next 40 TB / month data transfer out
$0.100 per GB - next 100 TB / month data transfer out
$0.090 per GB - data transfer out / month over 150 TB

Requests
$0.010 per 10,000 GET requests

I evaluated AWS for hosting a while back and concluded that the bandwidth and storage costs were just too expensive if you have even a modest amount of storage traffic needs. Here’s the breakdown:

A dedicated 100Mbit line can xfer 30TB/month. Costs $1000/mo, or $10/Mbit/mo. Source: CalPOP. (I host here).

From AWS @ $0.120/GB that’s $3600/mo. If you’re pushing sizable volumes of bits, it seems like it will only make sense to do this under 2 scenarios:

1. you can benefit from having a >100Mbit/s cap b/c you have *very* spiky traffic. you xfer well over 400Mbit/s for a few hours/day (and 0Mbit/s the rest), and
2. you need lower latency than a 1-2 datacenter network can give you

I suspect most for most of their target clients it’s [2], or clients that are really in it for the whole S3/EC2/SQS/EBS bundle. Being able to rent cores at $0.10/hour can be really attractive for some types of services.

So no, it’s not right for me. YMMV.