Wednesday, February 26, 2014

Kerberos Java Client: Code

This is the second post of the Kerberos Java Client series.
First post can be found here [1].

This post is on the java code that is used to connect to the Kerberized server using ssh, execute a command ('ls') there and get results ( read the output stream).

You need to provide the locations of the Kerberos configuration file and the Jaas configuration file as System properties. I've done it inside the class itself.

The code is pretty much self explanatory.

import com.jcraft.jsch.*;

import java.util.Properties;

public class JSCHKerberosConnector {
    public static void main() {

        String host = "";
        String user = "sachith";
        String  command = "ls -ltr";

        JSch jsch = new JSch();
        jsch.setLogger(new MyLogger());

        System.setProperty("", );
        System.setProperty("", );
        System.setProperty("", "false");
        //to enable kerberos debugging mode
        System.setProperty("", "true");

        try {

            Session session = jsch.getSession(user, host, 22);
            Properties config = new java.util.Properties();
            config.put("StrictHostKeyChecking", "no");


            Channel channel = session.openChannel("exec");
            ((ChannelExec) channel).setCommand( command);
            ((ChannelExec) channel).setErrStream(System.err);

            InputStream in = channel.getInputStream();
            byte[] tmp = new byte[1024];
               while (true) {
                while (in.available() > 0) {
                    int i =, 0, 1024);
                    if (i < 0) break;
                    System.out.print(new String(tmp, 0, i));
                if (channel.isClosed()) {
                    System.out.println("exit-status: " + channel.getExitStatus());
                try {
                } catch (Exception ee) {

        } catch (JSchException e) {
        } catch (IOException e) {
    //to log the jsch activity
    public static class MyLogger implements com.jcraft.jsch.Logger {
        static java.util.Hashtable name=new java.util.Hashtable();
            name.put(new Integer(DEBUG), "DEBUG: ");
            name.put(new Integer(INFO), "INFO: ");
            name.put(new Integer(WARN), "WARN: ");
            name.put(new Integer(ERROR), "ERROR: ");
            name.put(new Integer(FATAL), "FATAL: ");
        public boolean isEnabled(int level){
            return true;
        public void log(int level, String message){
            System.err.print(name.get(new Integer(level)));


Kerberos Java Client: Configuration

These posts will guide you through on using Java to talk to a kerberized server using the JCraft library [1].

This particular post will be on configuring the environment changes needed to make the Java Client work. The next post will be focussing on the java Client. 

Problem Scenario
This will be using the JSCH( of the JCraft) to SSH to the kerberos server using the Kerberos Ticket (TGT). It will execute a certain command on the kerberized server and read the output stream.
Note: This program uses a generated Kerberos ticket. it does not create one.

First of all make sure your computer is running a Kerberos client and you are able to ssh to the Kerberized server using the TGT using the terminal. It would look something like 
ssh username@REALM

If so that means you are good to proceed.

If you are doing this on a Mac, you need to do the following step, else skip this step
You have to set the KRB5CCNAME to point to a new Kerberos ticket Cache. Ex:
export KRB5CCNAME="/Users/swithana/krb5cc_swithana_022322"

This is the ticket cache that you should be pointing to in the login file( discussed below).

First run the command below to generate the kerberos ticket.
kinit username@REALM

You can list the tickets you have by using the 'klist' command.

Then you need to have the Kerberos configuration file in your local system ( usually at /etc/krb5.conf).
If you don't have it, create one.
Here's what a krb5.conf look like,
[domain_realm] = TEST.IU.EDU = TEST.IU.EDU

        default_realm = TEST.IU.EDU
        forwardable = TRUE
        TEST.IU.EDU = {
                kdc =

        default = FILE:/var/log/krb5.log

You also need to have a login configuration for JAAS[2]. This will specify which module to use to login, where the TGT is ...etc. Here's an example jaas.conf file. {


ticketCache: the path to the TGT.

After these changes you are good to go.

But sometimes Java is unable to decrypt the Kerberos Ticket since Java doesn't support AES256 out of the box due to some export control reasons.
To overcome that you need to manually modify the JRE and apply the Java Cryptography Extension (JCE)  [3] to the $JAVA_HOME/jre/lib/security directory.

Saturday, February 15, 2014

Mining Twitter Data using Python: Getting Started

Data Mining is a hot topic these days, and Twitter is being used heavily as a data source in various Data Mining applications. In this post I will introduce you to start mining twitter data with Python using the Tweepy module.
( I will not include the scientific module examples here( for mining,analysing ...etc). It's a basic guide to get the Twitter API setup)

Environment Setup

1. Install python ( MacOS comes with python installed)

2. Get a Twitter API key
    Go to, sign-in to twitter ( create an account if you don't already have one)
    Click the profile Icon ( top left) -> My Applications -> Create New App
    Provide the necessary data and it will create an application.
    Go to the application -> click on API Keys tab
    This will show you the necessary keys to authenticate your application using OAuth.

3. Install Tweepy
   Tweepy is a python library which supports the Twitter API
   Install in Mac:
pip install tweepy
sudo apt-get install python-tweepy
   Here's the github project :

Now you are ready to read some tweets!!

The code to get the twitter stream, ( insert your keys to this file)

from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener

#setting up the keys
consumer_key = ''
consumer_secret = ''
access_token = ''
access_secret = ''

class TweetListener(StreamListener):
    # A listener handles tweets are the received from the stream.
    #This is a basic listener that just prints received tweets to standard output

    def on_data(self, data):
        print data
        return True

    def on_error(self, status):
        print status

#printing all the tweets to the standard output
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)

stream = Stream(auth, TweetListener())

This prints the whole twitter stream filtered using the text "nba".

getting user info:

import tweepy

auth = OAuthHandler(consumer_key,consumer_secret)
api = tweepy.API(auth)

auth.set_access_token(access_token, access_secret)
twitterStream = Stream(auth,TweetListener())

user = api.get_user('sachithwithana')
print user.screen_name

This is a basic example to get you set up. Now you are ready to explore with the Twitter API.

I would recommend using the scikit-learn library for Machine Learning with Python.

Here's the Tweepy Documentation:

Tuesday, February 11, 2014

Apache Thrift: Securing the Cilent Server Communication using SSL

This is my second post on Apache Thrift. It's been a very informative ride so far with Apache Thrift.
In this post, I'm going to talk about securing oneway client-server communication using SSL.

The code is available at github [1]

We use the JDK provided keytool to generate the necessary certificates.

Concepts involved:

Public-key encryption ( SSL) : [2]

Keystore: Keystore contains all the private keys and their corresponding certificates with the public keys.

Truststore: Truststore contains the certificates that you TRUST.

Basically the server has its private key ( in the keystore). We need to create the corresponding certificate of that private key and add it to the TRUSTSTORE of the client ( ie public-key encryption).

Managing the keystore

To create these keys and keystores you need to go to the $JAVA_HOME/bin directory. There, you have a tool called the keytool.

1. Create the private key
keytool -genkeypair -alias mykey -keyalg RSA -validity 7 -keystore keystore.jks

2. Export the corresponding certificate
keytool -export -alias mykey -keystore keystore.jks -rfc -file certificate.cer

3. Import that certificate to the client truststore.
   Note that in this example, the server and the client are in the same host ( localhost), that's why we are   importing the certificates in the same host.

keytool -import -alias mykey -file certificate.cer -keystore truststore.jks


I have added the SSL secured code to my github project as well.
Set your keystore, truststore, and passwords in the respective property files found in /resources.
Then run the SecuredCalculatorTest.

Code changes from the first Post:

  • CalculatorClientService has a special init method for the SSL initiation ( secure_init()).
  • SecureCalculatorServer
  • SecureCalculatorClient

Adding SSL capability

These lines shows how to add the SSL capability to the server.

   //for the secure communication
   TSSLTransportFactory.TSSLTransportParameters params = new TSSLTransportFactory.TSSLTransportParameters();
   params.setKeyStore(serverKeyStore, keystorePassword);

   TServerSocket serverTransport = TSSLTransportFactory.getServerSocket(
        4030, 10000, InetAddress.getByName("localhost"), params);

These changes show the addition of SSL capability to the client.

    TSSLTransportFactory.TSSLTransportParameters params =
             new TSSLTransportFactory.TSSLTransportParameters()
    params.setTrustStore(clientTrustStore, truststorePassword);
    transport = TSSLTransportFactory.getClientSocket("localhost", 4030, 10000, params);

Notice that here, you don't have to open the transport.



Sunday, February 9, 2014

Apache Thrift: Getting started

Apache Thrift [1] is one of the most popular and innovative software framework I've used.

Thrift file ( IDL) defines the server interface and allows the server and the client to talk through RPC. That same thrift file can generate various server and/or client code enabling scalable cross-language development.

I've hosted the sample code on Github (Uses Maven, and surefire tests)   [2]

To get started, install thrift in your computer [3].

Then you need to create a "Thrift file"( it basically defines the interface, i:e Interface Definition).
Here's my simple Calculator.thrift file.

namespace java lk.swithana.calculator
service lk.swithana.calculator.server.CalculatorService
    i32 add  (1:i32 a, 2:i32 b),
    i32 sub  (1:i32 a, 2:i32 b),
    i32 div  (1:i32 a, 2:i32 b),
    i32 mult (1:i32 a, 2:i32 b)

This defines the calculator interface.
If you've installed Thrift correctly, you should run:

thrift --gen java Calculator.thrift 

This would generate the necessary Java Beans for the server and the client.

NOTE that it only creates the beans, now you have to create the server and the client using those beans.
I've already completed the server and the client in the code in github.

Refer to the CalculatorBasicTestCase java test class to see how it works.

Note: This is a VERY BASIC thrift file. You can/should add type definitions and other Thrift features to make it better. Here's a good guide on writing simple to complex thrift files.[4]