Short URLs, Big Data: Rundown of Hilary Mason at #ghc12

The purpose of computing is insight, not numbers – Hamming, 1961

Mason introduces data science and technology as ways to improve our understanding of the world and make better decisions. There is so much data out there, particularly in the social context, so how do we understand it?

An interesting question the presentation answers is ‘why now?’  as related to the rise of data science. Computing capacity, the evolution of algorithms, and the expansion of the data itself contribute to the growth of Big Data and Data Science.

Random app tip — Dark Sky for iOS gives you information about weather near wherever you are. Mason also mentioned some classics of data analysis online –ex. Netflix, Amazon recommendations, OkCupid, and DataKind (opens data to all!).  I hadn’t heard of patientslikeme, which does some interesting work with patient data (self-reported).

After the broad strokes of data science in the field, Mason went on to describe how and what data science looks like. She described it as lying at the intersection of engineering, computer science, mathematics, and curiosity. She definitely makes data science sound like a fun pursuit.

Following that, she discussed various work done by Bitly, ex. visualizations of news sources that are popular geographically, the distribution of traffic for different sorts of media (cute kitten vs. an earthquake). While this the data for sharing is very interesting,  the question “what do people read” is different than “what do people share”. This is where the science part seems to come in … Ask the interesting questions.

But what about process? Start with investigation and research, do some math and find shortcuts to the solution, design the infrastructure, and then prototype and redesign to improve weaknesses. (Aside: Bitly runs in Python :))

Bitly has also released a new realtime search product. A big difference is that ranking is done at query time. They do some if the tracking with a sort of moving average (aka fancy math).The service lives at http://rt.ly.

Once following phrases in realtime, the next task is to try to follow stories. (Sounds like an interesting question)

The session was very interesting. I got to say hi before the session, and Mason was very friendly and open. Her talk reputation is deserved, very well prepared and with a healthy dose of humor in the presentation.

This blog post is coverage of a talk at Grace Hopper 2012.

Tags: , , ,

No comments yet.

Leave a Reply