Follow Me On Twitter

Showing posts with label data mining. Show all posts
Showing posts with label data mining. Show all posts

Wednesday, November 07, 2007

Further Exploitation via Data Mining on Twitter

Exploit: 1. To employ to the greatest possible advantage.

Being the sole internet marketer (slash blogger) with the specific aim of spreading good cheer and controversy simultaneously, I was very tempted to title this article, "Raping Twitter". But not for the mere shock value that certain sensitive individuals might assume. You'll see why, in just a second..

You see, in a previous post titled, "A Quick Jaunt Into Data Mining On Twitter", I shared my observation that Twitter is completely ripe for some serious data mining. I then included a specific recipe for a cursory exploration into data mining Twitter via the combination of three 3rd party applications found through the Twitter Fan Wiki site. The three were as follows:

  • Twitterverse, which showed a "word cloud" of the most popular phrases over the past 1, 5 and 10 hour periods.
  • TwitterSearch, which allows a user to search any term and see all the various ways and times it was being used.
  • TwitterVision, connected to TwitterSearch, it allows a user to see where a Twitterer lives.

To give an example of the power of Twitter, a Twitter friend named Shahi B twittered me to let me know that alongside TwitterSearch was another application called, "Terra Minds". It was that quick, too. Terra Minds, essentially, does the same thing as TwitterSearch except it allows you to see results beyond the first ten. One search, for example, responded with multiple pages and said,

data mining twitter sam freedom controversy blog

"found 645 twitter updates from the last 2 months
Subscribe to these results: rss"

While it's nice that Terra Minds totals search results and offers you RSS feeds of any search term results you wish, those are not the main "exploits" or "data mining" about which I'm writing. But, just the same, let's entertain them for a moment because they can STILL be very powerful.

So Then, Why Are You Using Twitter?

If you're using Twitter, then you want something.
Or, you're not using Twitter, but you STILL want something....

Once you figure out what it is, you set up an RSS feed at Terra Minds with the search terms that will help you get it. For instance, if you're looking to maintain a friendly Twitter conversation with "in-the-know" persons whom you might not have otherwise had any connection, then missing their Tweet for you, about you, or on a topic you were following could mean a lost advantage.

The obvious thing one might think to do is to grab a feed of THEIR (friends) tweets, or just follow what they are saying all the time. But what if there's more than one person who might speak on something about your target need? Do you really want to be sifting through every single Tweet of a dozen people who might also sometimes be tweeting gibberish? Instead, data mine by setting up the RSS feed at Terra Minds on search terms including things like a) your name, b) your area of interest, c) your area of expertise, d) anything else that might be of interest to you..

It's important to be as targeted in your search terms as possible since you WILL be getting those terms Tweeted from EVERYONE twitting. But it's not as scary as it sounds. Rather than sifting through 1000s of tweets from a handful of well known people who could be talking bunk or about any number of a million topics including their favorite movies, most of these targeted "search result" tweets will, at least be in the target area of your interest and it will be easy to scan them for interesting, targeted information and/or trends.

It can also alert you when ANYONE is talking about you if you ever reach the point where someone might.

Here, This One's Really Embarrassing!
And This, I Must Confess, I Enjoyed...


B
e careful what you twit about, it's ALL indexed and searchable!

As you can see, her editor doesn't need to BE on Twitter. And yes, I wrote and told her. But even my research for this article, leading to her site, just led to a new discovery...

TWIT THIS!

TwitThis is another simple, neat little 3rd party application that allows users on your site to Twit about your site INSTANTLY. It sends visitors to a TwitThis form which will auto-append your site link as a TinyURL to your Twitter tweet. When your visitor uses TwitThis via your icon, all your visitor's Twitter friends will get an instant link to your site AND, get this... that twitted information will end up INDEXED. That means both you and others can find it on Twitter AND Google, for starters.

twit this twitter 3r party application for instant site twitting on the fly!

So what you can do is go to TinyURL, create a TinyUrl for your main page and then create a Terra Mind's RSS feed (by initiating a search) for any instance of your site's TinyURL and you'll get regular notices of what visitors will have said about your site. There's also a plugin for WordPress users that allows their TwitThis button to send "specific pages" to be Twitted rather than just the main site url, so tracking these "page-based" TwitThis Tweets via Terra Mind's RSS could be a little more challenging.

Mine, (non-WordPress) for example, is
http://tinyurl.com/33febb

Personally, I'll be content to suffer my Non-WordPress main-page-address-only TwitThis solution as visitors can still talk about the specific pages in their Tweets and I can then easily track such Tweets by the TinyUrl . But both newbies AND advanced users need to take note:

  • newbies - TinyURL creates the SAME shortened url for the SAME url every time. So for MY BLOG, my TinyURL will ALWAYS be http://tinyurl.com/33febb

  • advanced users: set up your feeds for the last part of the TinyURL only. In my case, 33febb. It seems Terra Minds does not recognize the http:// part.

But That Also Means...
...You Can Set Up An RSS Search Feed for ANY Main Site Address Mentioned in Any TWEET.

Since we just went over that TinyURL makes the SAME TinyURL for any given URL, then you can go check (or create!) the TinyURL for ANY website URL that you want to monitor and your Terra Mind RSS search feed will alert you whenever that TinyURL is mentioned in a Tweet.

This is barely even scratching the surface, folks..

So Why "Raping" Twitter?

With all the usual disclaimers acknowledging that rape is a very disturbing topic, the fact is that it exists. I was curious just how many Twitterers had mentioned the word in their Tweets and it was quite startling, actually (see how many times). While many of us, initially, signed up for Twitter either because we saw it as a means to a money-generating end, a way to meet new friends, or as just plain cool (or all!), there will soon dawn on us many alternative uses for which we had previously never imagined - for example, a rape counselor would have a completely new channel from which to learn about peoples' real-time attitudes in their use of the word AND, also, a plethora of new eyes and ears reporting on new articles and sites related to the topic. And the same goes for anyone in any field, whether a professional or a hobbyist.

With the cursory look into the tools and techniques I've only begun to mention in these first two Twitter-based articles, one can already see how possible and easy it is to recruite a whole new ARMY of info-gathers who are completely unaware that they are helping you... and so far, for free.

Stay Tuned, I'm Going to Run a Test And Get Deeper Into This for You In One My Very Next Articles...


ps. Thanks to Andy Beard for Sphinning the Previous Part.


Sam Freedom"s Internet Marketing Controversy Blog

AddThis Social Bookmark ButtonAddThis Social Bookmark ButtonHome

Sunday, November 04, 2007

A Quick Jaunt Into Data Mining on Twitter

Holy Twitter, Batman!  It's Boris Groys!

So just who the hell is Boris Groys and why might any of us who Twitter care about him?  Other than the fact that he might be a teacher of art history and philosophy in Germany, or a professor of Slavic studies in New York, what other purposes might Boris Groys serve us?

"Boris Groys is a philosopher, essayist, art critic, media theorist, and an internationally acclaimed expert on late-Soviet postmodern art and literature, as well as on the Russian avant-garde. Dr. Groys’s writing engages the wildly disparate traditions of French post structuralism and modern Russian philosophy."

For those of you who recall the movie, "A Beautiful Mind", you'll recall John Nash was a schizophrenic who was able to discern all kinds of complex patterns in what most people would perceive as garbled gibberish.

twitter keyword activity explored at twitterversetwitter keyword activity explored at twitterverse
Wait, that's not Nash.That was Maximus!

Twitterverse

Enter Twitterverse... and Mr Nash is nowhere to be found.

Twitterverse is one of the many cool little play things that Twitterholics can find and play with at the Twitter Fan Wiki.

And that, ultimately, led me to Professor Groys...

twitter keyword activity explored at twitterverse
(Do ya really need me to highlight it in yellow for ya?  Didn't think so...)

So what's the big fuss about a bunch of words surrounding Boris Groys (except that one is a swear)?

Well, the cool thing about Twitterverse is that it allows you to see the words about which people are twittering AND to what degree.  The larger words are words that are being mentioned more often.  As I discovered though, a larger word doesn't necessarily mean a specific number of instances.  As far as I can tell, the word sizes only represent how many times the words are said in relation to each other.  So if all other words were mentioned once, a large word could be one that was mentioned, hypothetically, a mere 5 times.  But there are ways to determine that a little better (see below).

And Twitterverse also allows you to take 1, 5 and 10 hour snapshots which was pretty cool because it got me wondering why Boris Groys was so popular in the last hour and not the last 5 hours.  I'll show you why and how I found out.

TwitterSearch

TwitterSearch is another cool app found via the Twitter Fan Wiki page. It allows you to search all instances of a given keyword phrase on Twitter's Public Timeline.  The public time line is where all unprotected twitting gets dumped in real time.  So if you go to TwitterSearch and type in something like, oh say, BORISGROYS, you get something like this...

research keyword popularity on twitter with twittersearch
(there were about 10 more by the same guy over a 2hr period)

So our excitement dies down a bit because we see that it's one guy who is enamored of Boris Groys.  In fact, after looking at all of his Twitters, they struck me as having been auto-submitted.  Just like the following...

twitter user keywords explored through twittersearch
(there were a bunch more over a 6 hour period)

That came up during a "two-word" phrase search at Twitterverse resulting in this:

explore behavior patterns on Twitter with Twitterverse
(you can see the same twitterer triggered the term "music feed", as well)

And it was probably done from a feed submission app, ALSO found on the Twitter Fans Wiki page, such as Rss2Twitter:

submit blog feeds to twitter with rss2twitter
(I took the plunge and am auto-submitting my blog feed to Twitter now)

TwitterVision will then allow you to explore a little further to see if a specific term is location-popular.  But wait!  If you go directly to the homepage, you'll get the absolutely addictive world view showing from WHERE all the twitters from the public timeline are coming.  That can be interesting; and will definitely lead to addiction, but we are looking at it for another reason.

When in TwitterSearch, having searched for the term "BorisGroys" and getting the results shown above, you can click on the picture of any/all twitters and it will show you where they live on the map.  Here is one example:

exploring twitter users keywords and behavior on twittervision
(found by adding the Twitter username to the end of the domain:
http://twittervision.com/philosoful - looks like Brazil to me!)

Anyways, you see what this is getting at?  If someone were to create a Twitter data mining application that could highlight patterns, siphon out the refuse and forecast trends, I think it could potentially be worth a lot of money.  After all, what Twitter affords us is a HAIR-DOWN look into the minds of a great concentration of some very successful people...in about as close to "real time" as possible.

Sure we could throw some big bucks at people/companies who already do some very good research by other means but this is free, and it's right now.  And I can think of a lot worse ways to be spending your time.... like TWITTERING!

Now get to work! ;-)

Make sure to read Part 2:
Further Exploitation via Data Mining on Twitter


Sam Freedom"s Internet Marketing Controversy Blog

AddThis Social Bookmark ButtonAddThis Social Bookmark ButtonHome