Follow Me On Twitter

Wednesday, November 07, 2007

Further Exploitation via Data Mining on Twitter

Exploit: 1. To employ to the greatest possible advantage.

Being the sole internet marketer (slash blogger) with the specific aim of spreading good cheer and controversy simultaneously, I was very tempted to title this article, "Raping Twitter". But not for the mere shock value that certain sensitive individuals might assume. You'll see why, in just a second..

You see, in a previous post titled, "A Quick Jaunt Into Data Mining On Twitter", I shared my observation that Twitter is completely ripe for some serious data mining. I then included a specific recipe for a cursory exploration into data mining Twitter via the combination of three 3rd party applications found through the Twitter Fan Wiki site. The three were as follows:

  • Twitterverse, which showed a "word cloud" of the most popular phrases over the past 1, 5 and 10 hour periods.
  • TwitterSearch, which allows a user to search any term and see all the various ways and times it was being used.
  • TwitterVision, connected to TwitterSearch, it allows a user to see where a Twitterer lives.

To give an example of the power of Twitter, a Twitter friend named Shahi B twittered me to let me know that alongside TwitterSearch was another application called, "Terra Minds". It was that quick, too. Terra Minds, essentially, does the same thing as TwitterSearch except it allows you to see results beyond the first ten. One search, for example, responded with multiple pages and said,

data mining twitter sam freedom controversy blog

"found 645 twitter updates from the last 2 months
Subscribe to these results: rss"

While it's nice that Terra Minds totals search results and offers you RSS feeds of any search term results you wish, those are not the main "exploits" or "data mining" about which I'm writing. But, just the same, let's entertain them for a moment because they can STILL be very powerful.

So Then, Why Are You Using Twitter?

If you're using Twitter, then you want something.
Or, you're not using Twitter, but you STILL want something....

Once you figure out what it is, you set up an RSS feed at Terra Minds with the search terms that will help you get it. For instance, if you're looking to maintain a friendly Twitter conversation with "in-the-know" persons whom you might not have otherwise had any connection, then missing their Tweet for you, about you, or on a topic you were following could mean a lost advantage.

The obvious thing one might think to do is to grab a feed of THEIR (friends) tweets, or just follow what they are saying all the time. But what if there's more than one person who might speak on something about your target need? Do you really want to be sifting through every single Tweet of a dozen people who might also sometimes be tweeting gibberish? Instead, data mine by setting up the RSS feed at Terra Minds on search terms including things like a) your name, b) your area of interest, c) your area of expertise, d) anything else that might be of interest to you..

It's important to be as targeted in your search terms as possible since you WILL be getting those terms Tweeted from EVERYONE twitting. But it's not as scary as it sounds. Rather than sifting through 1000s of tweets from a handful of well known people who could be talking bunk or about any number of a million topics including their favorite movies, most of these targeted "search result" tweets will, at least be in the target area of your interest and it will be easy to scan them for interesting, targeted information and/or trends.

It can also alert you when ANYONE is talking about you if you ever reach the point where someone might.

Here, This One's Really Embarrassing!
And This, I Must Confess, I Enjoyed...

e careful what you twit about, it's ALL indexed and searchable!

As you can see, her editor doesn't need to BE on Twitter. And yes, I wrote and told her. But even my research for this article, leading to her site, just led to a new discovery...


TwitThis is another simple, neat little 3rd party application that allows users on your site to Twit about your site INSTANTLY. It sends visitors to a TwitThis form which will auto-append your site link as a TinyURL to your Twitter tweet. When your visitor uses TwitThis via your icon, all your visitor's Twitter friends will get an instant link to your site AND, get this... that twitted information will end up INDEXED. That means both you and others can find it on Twitter AND Google, for starters.

twit this twitter 3r party application for instant site twitting on the fly!

So what you can do is go to TinyURL, create a TinyUrl for your main page and then create a Terra Mind's RSS feed (by initiating a search) for any instance of your site's TinyURL and you'll get regular notices of what visitors will have said about your site. There's also a plugin for WordPress users that allows their TwitThis button to send "specific pages" to be Twitted rather than just the main site url, so tracking these "page-based" TwitThis Tweets via Terra Mind's RSS could be a little more challenging.

Mine, (non-WordPress) for example, is

Personally, I'll be content to suffer my Non-WordPress main-page-address-only TwitThis solution as visitors can still talk about the specific pages in their Tweets and I can then easily track such Tweets by the TinyUrl . But both newbies AND advanced users need to take note:

  • newbies - TinyURL creates the SAME shortened url for the SAME url every time. So for MY BLOG, my TinyURL will ALWAYS be

  • advanced users: set up your feeds for the last part of the TinyURL only. In my case, 33febb. It seems Terra Minds does not recognize the http:// part.

But That Also Means...
...You Can Set Up An RSS Search Feed for ANY Main Site Address Mentioned in Any TWEET.

Since we just went over that TinyURL makes the SAME TinyURL for any given URL, then you can go check (or create!) the TinyURL for ANY website URL that you want to monitor and your Terra Mind RSS search feed will alert you whenever that TinyURL is mentioned in a Tweet.

This is barely even scratching the surface, folks..

So Why "Raping" Twitter?

With all the usual disclaimers acknowledging that rape is a very disturbing topic, the fact is that it exists. I was curious just how many Twitterers had mentioned the word in their Tweets and it was quite startling, actually (see how many times). While many of us, initially, signed up for Twitter either because we saw it as a means to a money-generating end, a way to meet new friends, or as just plain cool (or all!), there will soon dawn on us many alternative uses for which we had previously never imagined - for example, a rape counselor would have a completely new channel from which to learn about peoples' real-time attitudes in their use of the word AND, also, a plethora of new eyes and ears reporting on new articles and sites related to the topic. And the same goes for anyone in any field, whether a professional or a hobbyist.

With the cursory look into the tools and techniques I've only begun to mention in these first two Twitter-based articles, one can already see how possible and easy it is to recruite a whole new ARMY of info-gathers who are completely unaware that they are helping you... and so far, for free.

Stay Tuned, I'm Going to Run a Test And Get Deeper Into This for You In One My Very Next Articles...

ps. Thanks to Andy Beard for Sphinning the Previous Part.

Sam Freedom"s Internet Marketing Controversy Blog

AddThis Social Bookmark ButtonAddThis Social Bookmark ButtonHome


Lucie Bellemare said...

Wow, this is getting very hard to follow! Most of the times, I like your controversial ideas and comments, but sometimes, they get a little bit overwhelming (or too technical) for me!

I had never heard of Twitter before. I will surely give it a try now!

Sam Freedom said...

Hi Lucie,

I've never really thought to RATE my articles in terms of learning levels but you've given me pause for thought.

Data mining Twitter is a kind of advanced thing for someone to be doing. I'm sorry if it overwhelmed you; it's certainly not intended for that but, in the very least, I would suggest you read it not really to fully understand it but to just try to see and grasp the concept in general.

The real point behind it all is that there are ways of doing things that are not what a given opportunities creators had intended. That's neither good nor bad; it's just that necessity is the mother of invention and people will always find a way to use things differently than expected when it provides a greater than usual boost.

Thank you for your comment and if you have any more questions, let me know.


Dan Mosqueda said...

Sam, what about those of us with "protected" streams. Are they still exploitable?



Sam Freedom said...

Hi Dan, good question but that leads me to clarify that when I use the term "exploit" it's not the insidious definition whereby you take advantage of someone selfishly. I mean it as a way of getting the greatest possible advantage.

For purposes of a stalker, or a malicious person who dislikes you, in particular, exploitation would mean the former.

For the purposes of using this article to data mine, patterns can still be discerned despite that there will be a number of protected threads.

So a DATA MINER can still exploit Twitter for all kinds of trends and patterns while your stalkers would not be able to specifically exploit you.... so easily.... because the one thing of which you might not have thought is that while you're tweets are protected, tweets to you in reply are not.

In fact, I made a friendly joke one to you already... you can find it using TerraMinds and searching for Mosqueda... and then, of course, Google.

Btw, sorry I missed your birthday. Libra, right? ;-)


Dan Mosqueda said...


Wow, you are right. The protected stream is only as good as the other half of the twitter conversation.

Thanks for the birthday wishes! Very interesting.

Cheers - Dan

Real Exam said...

Thanks for making me think of what I was NOT doing with twitter :)

Testking 642-812 said...

its really very nice and impressive posting thanks for sharing this with us..

Love sms said...

its great.Most of the times, I like your controversial ideas.I appreciate you.

vinyl sticker printing said...

Really good and i will be follow this information.

certificate of Deposit Rates said...

Great Blog. I really enjoyed reading it and i am looking forward for more posts. thanks for sharing this information.

storage trailers said...

That is a pretty interesting. Thanks for the info. such a very great post.

Crossbow said...

Good points in your post, you have a great blog here. Thanks for this information.

Cotton Yarn said...

I truly enjoyed this. It has been extremely informative as well as useful.thanks for sharing the information.

cheap shakira tickets said...

Thank you for your comment and if you have any more questions, let me know.

Famous Poems said...

Great Blog. I really enjoyed reading it and i am looking forward for more posts. thanks for sharing this information.

Testking 70-662 said...

I really impressed with your approach which is excellent.