NMA 2010 – Tweet History CMS

You can view the final interactive Flash version here:

Not completely sure why I am posting about this, as it is password protected so you will not be able to play with it anyway, other than to satisfy my compulsive need for structure and completism. One day I will get a life, but not today.

So, I have taken the off-line scripts I created to pull down and store the tweet time-lines and built them into a web interface. It basically loops through the interactive agencies stored in the two New Media Age lists I set up earlier and checks to see if I have tweets already stored, if it finds a match than it takes the most recent tweet status id and passes that to the Twitter API as the “since_id” argument.


If the script finds an agency twitter account listed that I do not hold any previous tweets for than it will loop through the tweet history using the page IDs argument to get the time-line back to a total of 3,200 tweets. This “cap” is imposed by Twitter and there does not seem to be anyway around it that I can find.


In either case any tweets returned in the resulting XML are than looped through, parsed into the format I desire and than stored in the server’s MySQL database. It than moves onto the next agency. If it errors for any reason than it pauses and tries again, if it errors three times in a row it logs it and moves onto the next agency.

Another thing to mention is that each Twitter API call cycles through different PHP proxies set up on different servers. As Twitter’s API limits each application’s API calls to 150 an hour per IP address than setting up a series of proxies on different servers allows the three Flash interactives to have increased access before they hit Twitter’s rate limit. This is not a viable commercial solution but considering not many people actually look at this stuff it will be proficient for my needs.

Hopefully in the future Twitter will reconsider it’s crossdomain.xml strategy so than Flash apps will not require to circumvent the cross domain sand box security by using PHP proxies. It works but the downside is that as all the API calls are filtered through the proxy servers’ IP address rather than coming directly from the users’ browser IP address than the cap kicks in after 150 calls from the app, rather than kicking in from 150 calls from an individual user.

One of the reasons I migrated this code into a browser interface was I was planning to create an interactive where users could point it to their own Twitter accounts, Twitter lists etc but I than came across Damon Cortesi aka (@dacort) excellent TweetStats and decided that I would move onto something else.

The other two New Media Age Twitter clients can be viewed at these links:

New Media Age 2010 Top 100 Interactive Agencies

New Media Age 2010 Interactive Agencies to Watch

Related Posts:

This entry was posted in Data Visualisation, Interactive Data, Lab Examples and tagged , , , , , , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

One Trackback

  1. By 1displease on January 12, 2022 at 11:17 pm


Post a Comment

Your email is never published nor shared. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>