One more robot learns to see

tl;dr I averaged every episode of That 70’s Show into a single video

A couple of weeks ago I was hanging out with my buddy and we were going to watch some TV. I was making a pitch for my new favorite show Rick & Morty which we couldn’t find using the PS3 internet browser. We did however find excellent video syncing Rick and Morty clips with Eminem’s Rap God.

One good YouTube find is never enough though so as soon as the video finished we eagerly checked the user’s other videos and thats when we stumbled upon this gem:

After watching a bit too much of ‘Every episode of Friends simultaneously’ I went looking for two things of which I found neither. Maybe I live in too much of a software bubble but I was genuinely surprised when I couldn’t find a link to the code used to make the video. I was equally surprised that this was the only video of its kind that I could find.

And thats how this project got started - I set out to create a script that would let me average all the episodes of any show into a new video. I wrote a script in python and opencv to do the work and then I picked That 70’s Show as my first test (I would have preferred Seinfeld but I don’t have it). That 70’s Show is a pretty good candidate - my main hypothesis was that shows that re-use the same locations frequently will be more interesting.

I am pretty pleased with the results! It’s pretty fun to watch and you can definitely pick out distinct patterns in how they frame shots and recognize scenes like the car from the opening credits (at ~2 minutes in). If I’d ever taken a course in film I’m sure I’d have some more intelligent things to say.

I also made this video where I ramp up to the average adding a new episode every 30 frames or around 1 second. So for the first second you’re only watching 1 episode then 2, 3 etc:

This is a post about developing my blog on my blog - take a second to appreciate how meta that is.

I recently transitioned my personal site from Wordpress to a static site on Github pages built using Middleman. Overall it was an awesome change and I feel much better having my personal site and assets etc in plain text and source control. The transition was pretty smooth however there were a few things that I had with Wordpress that took a bit of finesse to achieve with Middleman. Blog search was one of those things and I eventually came up with a pretty elegant solution that I would like to share.

The reason search is interesting is because Middleman generates static sites which means any search solution needs to be entirely client side. If you google for Middleman search you’ll find some other projects and gems but I think my solution is simpler.

I built my blog search using this nice little autocomplete library called Awesomplete. To start I added the following markup to the sidebar partial which is rendered on all the blog sections of my website:

<input id="search" class="form-control" list="articleList" placeholder="Search...">
<datalist id="articleList">
  <% blog.articles.each do |article| %>
    <option><%= article.title %></option>
  <% end %>
</datalist>

This creates the base list of all my blog articles. Now we need to bring it to life with Awesomplete:

  var input = $("#search")[0];
  var awesomplete = new Awesomplete(input, {
    minChars: 1,
    maxItems: 5,
    autoFirst: true
  });

  var searchIdx = {
    <% blog.articles.each do |article| %>
      <%= article.title.to_json %>: "<%= article.url %>",
    <% end %>
  }

  window.addEventListener("awesomplete-selectcomplete", function(e) {
    var articleTitle = $(e.target).val();
    var articleUrl = searchIdx[articleTitle];
    Turbolinks.visit(articleUrl);
  });

This code does a couple of things - first it initializes the Awesomplete widget. Next it defines a searchIdx variable which just maps article title to url and stores it in javascript land so we can access it later. The last bit of code listens to the awesomplete-selectcomplete event then looks up the url for the selected article and then loads the page with Turbolinks (if you’re not familiar this is just a slicker way of doing window.location = ).

I was really happy with how this turned out, super fast, simple and the code is nicely contained in my sidebar partial. Give it a try in the search box to the right!

I’ve been intrigued by D3.js and the impressive visualizations I’ve seen people make for quite some time and I finally got to check it out for myself. I wanted to write this blog post sharing my experiences and revelations from the perspective of someone who at one point spent a fair amount of time with Matlab/Octave and my personal fave matplotlib.

The first major point I want to make is that D3.js is not a plotting library - it should not be compared directly to Matlab/Octave or matplotlib because they serve different purposes. D3.js is a visualization library, sure this visualization may be a graph but you shouldn’t be playing with D3.js until you already understand your data and simply want to show it off. While exploring and figuring out your data matplotlib is going to be way faster and more effective (note there are some wrappers building standard plotting functions ontop of D3.js like NVD3 if javascript is really your thing).

Once you’ve figured out your data and it’s time to show-off D3.js really shines and here is the main reason why in my opinion. It’s due to a fundamental shift in a way of thinking that makes D3.js so powerful: rather than calling plot with an array of x and y values I am binding an array of javascript objects to my plot and then telling it which attributes to use for different things. Essentially you are pulling more data into the plot and you can write code at each level - let me show you the snippet that made me realize how powerful this shift in thinking is:

var chart = d3.select(".chart");

var tip = d3.tip()
.attr('class', 'd3-tip')
.offset([-10, 0])
.html(function(d) { return "<span>" + d.text + "</span>"; });

chart.call(tip);

var bar = chart.selectAll("g")
.data(data)
.enter().append("g")
.attr("transform", function(d, i) { return "translate(" + i * barWidth + ",0)"; });

bar.append("rect")
.attr("y", function(d) { return y(d.value); })
.attr("height", function(d) { return height - y(d.value); })
.attr("width", barWidth - 1)
.on('mouseover', tip.show)
.on('mouseout', tip.hide);

The important parts to note are this: I grab the value for my y axis of the rect from d.value where d is one of the objects in my data array that I am plotting. So far this is pretty standard but look at the tip function that gets called on mouseover - it shows a span which contains d.text! I’ve attached another piece of data here called text that can be used to display more information! This was the wow moment for me because this kind of thing isn’t possible in matplotlib and showing a simple tooltip is really only the beginning of what you can do with all this added context.

Also something about plots with a :hover effect is kind of awesome!

If you haven’t played with D3 yet and have some cool data I definetly recommend it, I used it to make some pretty sweet visualizations/tools for our Parity Ultimate Frisbee League here in Ottawa, check’em out:

https://github.com/kevinhughes27/ocua-parity-league