Reading Twitter Stream using Twitter4j

In this tutorial we will see how to read twitter stream using java code. First create a utility class that will help you in creating connection to Twitter but in order to create connection you need to have an account with twitter so if you do not have an account with Twitter go to https://twitter.com/  and create your account. Once account is created go to https://apps.twitter.com/ and create an application with twitter api by clicking on “Create New App” button.

You need to provide following information. 

• Name
• Description
• Website
• Callback URL

Once this information is provided click on “Create your Twitter Application” button.  Once application is created you need to generate Access Token. Go to API Keys tab in your application and generate Access Token and Access Token secret. Note down following information somewhere (May be in notepad?)

• API Key
• API Secret
• Access Token
• Access Token Secret

Download API called Twitter4J from following location and add following jars from its lib folder in your Eclipse Java project so following is how your java project will look like.

You can ignore mongo-2.10.1.jar and packages and classes for now. We will use information that we saved for twitter application in our utility class as follows. (I have stored this values as public static final String variables in my utility class you can use your info similarly)

package com.techcielo.twitreader.util; import twitter4j.TwitterStream; import twitter4j.TwitterStreamFactory; import twitter4j.conf.ConfigurationBuilder; public class TwitterStreamBuilderUtil {                                 public static TwitterStream getStream(){                                 ConfigurationBuilder cb = new ConfigurationBuilder();                                 cb.setDebugEnabled(true);                                 cb.setOAuthConsumerKey(Constatnts.consumerkey);                                 cb.setOAuthConsumerSecret(Constatnts.consumerSecret);                                 cb.setOAuthAccessToken(Constatnts.accessToken);                                 cb.setOAuthAccessTokenSecret(Constatnts.accessTokenSecret);                                                                 return new TwitterStreamFactory(cb.build()).getInstance();                 } }

Also when you write this code ensure that you are using OAuth api and not OAuth2 API. Once this class is ready we will use it in our Stream reader class as follows. In our class we are reading all twits that has mention of keywords in String array keywords. While writing this tutorial I have used keywords that are currently trending on BBC news e.g. “Sochi”,”Ukraine”,”Whatsapp”.

package com.techcielo.twitreader.service; import twitter4j.FilterQuery; import twitter4j.StallWarning; import twitter4j.Status; import twitter4j.StatusDeletionNotice; import twitter4j.StatusListener; import twitter4j.TwitterStream; import com.techcielo.twitreader.bean.TwitterStreamBean; import com.techcielo.twitreader.util.TwitterStreamBuilderUtil; public class StreamReaderService {             public void readTwitterFeed() {               TwitterStream stream = TwitterStreamBuilderUtil.getStream();               StatusListener listener = new StatusListener() {                      @Override                      public void onException(Exception e) {                            System.out.println("Exception occured:" + e.getMessage());                            e.printStackTrace();                      }                      @Override                      public void onTrackLimitationNotice(int n) {                            System.out.println("Track limitation notice for " + n);                      }                      @Override                      public void onStatus(Status status) {                            System.out.println("Got twit:" + status.getText());                            TwitterStreamBean bean = new TwitterStreamBean();                            String username = status.getUser().getScreenName();                            bean.setUsername(username);                            long tweetId = status.getId();                            bean.setId(tweetId);                            bean.setInReplyUserName(status.getInReplyToScreenName());                            if (status != null && status.getRetweetedStatus() != null                                          && status.getRetweetedStatus().getUser() != null) {                                   bean.setRetwitUserName(status.getRetweetedStatus()                                                 .getUser().getScreenName());                            }                            String content = status.getText();                            bean.setContent(content);                      }                      @Override                      public void onStallWarning(StallWarning arg0) {                            System.out.println("Stall warning");                      }                      @Override                      public void onScrubGeo(long arg0, long arg1) {                            System.out.println("Scrub geo with:" + arg0 + ":" + arg1);                      }                      @Override                      public void onDeletionNotice(StatusDeletionNotice arg0) {                            System.out.println("Status deletion notice");                      }               };               FilterQuery qry = new FilterQuery();               String[] keywords = { "Sochi","Ukraine","Whatsapp" };               qry.track(keywords);               stream.addListener(listener);               stream.filter(qry);        } }

In this example I have used a bean class to store information retrieved from Twitter feed. Interested user can use it for Producer Consumer implementation ofr using this feed. When your run this code following will be output. You can ignore exception that occurs while your class is not able to connect to twitter and it retries for the same.

Done [Mon Feb 24 12:26:38 IST 2014]Establishing connection. [Mon Feb 24 12:26:48 IST 2014]Connection established. [Mon Feb 24 12:26:48 IST 2014]Receiving status stream. Got twit:RT @MaximEristavi: In Lutsk (Western Ukraine) local riot-policemen  on their knees publicly begged ppl for forgiveness. via live-stream htt… Got twit:A mi nunca me falló whatsapp, lo que me falló fue este pinche mundo. Got twit:RT @policia: Whatsapp sigue "on fire". Hoy, por un bulo q no es nuevo. Que se acaban las cuentas y reenvíes... (bla, bla) Ni caso! http://t… Got twit:Layanan Sempat Tumbang, Ini Penjelasan Pendiri WhatsApp: Ini menjadi kekurangan terbesar dan terpanjang kami, ... http://t.co/SNOmGkvSst Got twit:Whatsapp 9992511509 Got twit:RT @edwardlucas: Must-read from @uli_speck on what Merkel must do now for Ukraine http://t.co/Lx4FoxBxWW Got twit:Layanan Sempat Tumbang, Ini Penjelasan Pendiri WhatsApp: Ini menjadi kekurangan terbesar dan terpanjang kami, ... http://t.co/KynYog3HVa Got twit:RT @alanbaldwinf1: Sochi airport. Olympic Games? That's so yesterday. http://t.co/5rafgTJ5GZ Got twit:Layanan Sempat Tumbang, Ini Penjelasan Pendiri WhatsApp: Ini menjadi kekurangan terbesar dan terpanjang kami, ... http://t.co/sGzH2NDn10 Got twit:Holders Spain face Ukraine in Euro 2016 qualifying. http://t.co/HtTjnfW1WT #beINfootball http://t.co/k7yHJG5F9u Got twit:RT @JoleenStore: ?? ???? ???? ??

Another blog will show you how to read twitter Stream and send it to different channels in Spring Integration.