End of Year Top Albums 2010 – Raw Data

A few people have contacted me asking for a access to the raw data for my End of Year Top Albums 2010 interactive here.

I was planning to build functionality into the interactive so that people can download results from the search table view as I thought it would be good to be able to download charts based on your favorite genres. One of the main points of this interactive was being able to take everyone’s recommendations and than apply your own personality to them by filtering/sorting them based on the genres/charts you like.

Also I thought being able to look at the screen and have visual clues to the musical nature of each album chart would be really cool. I have always loved music charts and to me this was a way to find out which charts matched my music tastes. Who’s Indie Rock – pretty much all of them – but musicOMH tops out on 97% (actually 19.95% – see comments), who has the most Electronica? Rough Trade on 58% (12.31% – once again see comments) etc.

Not 100% sure when I will get the chance to add this functionality so in the meantime I have exported the three raw data source tables out as CSV and you can download them from the links below.

http://lab.zoho.co.uk/lab/data-downloads/eoy_albums.csv
http://lab.zoho.co.uk/lab/data-downloads/eoy_sources.csv
http://lab.zoho.co.uk/lab/data-downloads/eoy_tables.csv

Quite excited actually. When I started messing around with data interactions/visualisation I only really imagined I would ever be working on other peoples data. No it looks like people will be creating/designing etc from the data that I have spent too many late night hours putting together.  Quite unexpected and I can’t wait to see what other people do with it.

In case anyone wants to integrate the original reviews or any of the other content that was originally publish than once again here are all the source links for the original locations of the charts. Some of these sites have lots of additional resources, reviews, youTube videos, etc. Although you obviously need to take note of any copyright issues.

A.V. Club, Amazon, American Songwriter, Billboard, Bowlegs, Clash, Consequence of Sound, Daily Telegraph, Drowned in Sound, Faster Louder, Gorilla vs. Bear, last.fm, Mojo, MusicOMH, NME, No Ripcord, Paste, Pitchfork, PopMatters, Prefix, Q Magazine, Rave Magazine, Rhapsody SoundBoard, Rolling Stone, Rough Trade, Slant Magazine, Spin, Spinner, State, Stereogum, The Fly, The Guardian, The Line Of Best Fit, The Quietus, The Skinny, The Tidal Wave of Indifference, The Times, Tiny Mix Tapes and Uncut.

Related Posts:

This entry was posted in Data Visualisation, Interactive Data, Lab Examples and tagged , , , , , , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

17 Comments

  1. Posted January 12, 2011 at 3:10 am | Permalink

    You might want to combine these accidental-looking genre variations:

    Minimalist, Minimalism, Experimental, Experimental Music, Dance Pop, Dance-Pop, Blues-Rock, Blues Rock, Garage Rock, Garage, Post-Industrial, Post-Industrial Music, Psychedelic, Psychedelia, Psychedlica, Neo-Prog, New Prog

    • phodgeszoho
      Posted January 12, 2011 at 7:07 am | Permalink

      Cheers for pointing this out Glenn.

      I was cleaning these up, on the first run I had so many variations on genres that the Flash interactive was unworkable, especially on the NME version which has a lot more albums.

      I think towards the end I started to get a bit lazy and forgot to do this after I added Last.FM

      I will get these cleaned up.

  2. Posted January 12, 2011 at 5:10 am | Permalink

    Also, I’m trying to understand what your Rank % means. You say that musicOMH is 97% indie, but I see only 22 of its 50 albums tagged with any form of Indie, and your own link in that paragraph points to just 17 of 50 tagged “Indie Rock” specifically…

    • phodgeszoho
      Posted January 12, 2011 at 7:28 am | Permalink

      With the genre “ranking” I wanted to represent more than just the number of appearances/chart size as this would not distinguish the difference between a top 50 album chart with an indie album at number one from a chart with an indie album at number 50.

      So the genre ranking also factors in the albums position in the chart. This position applies an exponential weighting is so the 97% for musicOMH is (hopefully) due to the Indie Rock albums are all in towards the top of the chart.

      There is also some math added in to factor in the size of the charts so that the ranking is proportional regardless of the size.

      The final percentage calculation is based on a test ranking that is calculated to determine what the highest possible ranking score could be if you had a chart where every single album was the same genre.

      Hope that helps.

      • Posted January 12, 2011 at 6:26 pm | Permalink

        Hmm. If you’re willing to share your actual scoring math, we could scrutinize it together. I’m pretty sure it’s not working how you mean it to. The musicOMH poll has non-indie-rock albums in slots 3, 5, 6 and 9-15, just for starters, so calling it 97% indie rock seems like a sign of error.

  3. Posted January 12, 2011 at 5:13 am | Permalink

    Also: Minimalist, Minimalism, Minimal, Dance, Dance Music, World, World Music, Folk, Folk Music, Shoegaze, Shoegazing, Ambient, Ambient Music

    • Posted January 12, 2011 at 1:22 pm | Permalink

      Hi Glenn

      Hopefully it should all be sorted now. The eoy-albums.csv file has also been updated.

      The final genre list should now be:

      acoustic, afro-beat, alternative, alternative blues, alternative country, alternative dance, alternative hip hop, alternative metal, alternative rock, ambient, americana, art punk, art rock, avant-garde, avant-garde metal, balearic beat, baroque pop, bazombo trance, bluegrass, blues, blues rock, chamber pop, chicano rock, chillwave, country, country and western, country blues, country pop, country rock, cowpunk, crunkcore, dance, dance pop, dance punk, dark ambient, dark cabaret, darkgaze, delta blues, doom metal, downbeat, drag, dream pop, drone, drum and bass, dubstep, electro, electro clash, electro funk, electro house, electro pop, electronic, electronic dance, electronic rock, electronica, ethereal, ethio-jazz, europop, experimental, experimental electronic, experimental metal, experimental pop, experimental rock, folk, folk blues, folk pop, folk punk, folk rock, folktronica, freak folk, funk, funk rock, fusion, future shock, garage, gospel, goth, grime, gypsy punk, hard rock, heartland rock, heavy metal, hip hop, house, idm, indie, indie electronic, indie folk, indie pop, indie rock, industrial metal, instrumental rock, jam band, jazz, jazz fusion, jazz rap, latin, lo-fi, mash-up, math rock, metalcore, minimal, minimal techno, new age, new folk, new fusion, new grunge, new orleans, new prog, new psychedelic, new soul, new wave, no wave, no-fi, noise, noise pop, noise punk, noise rock, opera, orchestral, piano house, pop, pop punk, pop rap, pop rock, post-electro, post-grunge, post-industrial, post-punk, post-punk revival, post-rock, post-wave dance, power pop, progressive bluegrass, progressive folk, progressive metal, progressive rock, psychedelic, psychedelic pop, psychedelic rock, psychobilly, pub rock, punk rock, quiet storm, r&b, rap rock, reggae, rock, rockabilly, roots, roots rock, shoegazing, sludge metal, soft rock, sophisti-pop, soul, sound collage, soundtrack, southern rap, southern rock, space disco, space rock, spoken word, stoner metal, stoner rock, surf pop, surf rock, synthpop, synthrock, techno, trip hop, tropical, twee pop and worldbeat

      • Posted January 12, 2011 at 6:31 pm | Permalink

        For the Needle version I tweaked some of these a little further, so that I could then do some analysis based on genre words. This seems a little more useful than the actual verbatim genres, of which there are simply too many, at too many different levels of granularity. As it is, 2/3 of your genres have 3 or fewer albums, and 7 of your genres account for half the albums…

        • Posted January 12, 2011 at 9:59 pm | Permalink

          The first time I loaded the raw data into Flash I had around 300 genres. I spent a fair bit of time to get them down to the current 174 doing very much what you outlined above, i.e. removing minor and/or accidental variations, but generally I have tried to remain as true as possible to the source data. I don’t think I will be merging any more genres.

  4. Posted January 12, 2011 at 9:42 pm | Permalink

    Hi Glen

    OK, I think the confusion here is that the calculation for the ranking was being displayed as the adjusted value used to position the bubble between the circumference and the centre. Thus the 97% for Indie Rock was it’s position on the circle.

    It’s actual ranking is 19.59%. This number was generated as per the above description. As I was primarily trying to create an interactive visualisation that was engaging and visual pleasing I adjusted these rankings so that they display proportional across the radius.

    I think I decided to show the percentage of the bubble position instead of the actual underlying value as I thought this would be less confusion. I have changed this to now display the actual value.

    • Posted January 13, 2011 at 1:45 am | Permalink

      That makes much more sense now.

      The visualization is very cool, but I admit that the more I peer at it the less I like the showpiece bubble thing in the middle. The alphabetization of genres, combined with the profusion of subtly different categories, means that the most prominent visual features (color and polar position) are actually not meaningful at all. Plus the labels are extremely small, and half of them are upside-down! And of the two things that are meaningful, bubble-size is hard to compare visually, and radial position is essentially rendered useless by the fact that Indie Rock’s dominance squashes everything else into more or less the same orbit.

      The Indie Rock thing also speaks volumes about the aggregation’s selection bias. This isn’t necessarily a problem: no reason you can’t do a massive Indie Rock album-list aggregation. But if you really want a stylstic cross-section, you’ll need to seek out some less-indie sources. Decibel, Terrorizer, Metal Hammer and Kerrang all publish metal/hard-rock lists, for example. And if want your head to explode, check out Large-Hearted Boy’s master-list of lists!

  5. MingSu
    Posted April 5, 2011 at 9:54 am | Permalink

    Does anyone now how I can find data on streaming and sales of these albums also?

  6. Posted April 5, 2011 at 10:26 am | Permalink

    No worries, I would be interested to hear how you get on.

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>