Wednesday, April 23, 2014

Kerberos Java Client: Adding Multi user support

I've written two blog posts on connecting to Kerberos clients using the JCraft library using Java.
In those posts, I've only allowed single user support by using a static login configuration file ( jaas.conf).

To allow multi user support, you have to provide the login configurations for Java programmatically, instead of setting it as a environment variable.
To do this we need to have a configuration object with the relevant settings that were in the login.conf file. We have to create that object extending the class.

Here's an example java class.

public class JaaSConfiguration   extends {
    private Map BASIC_JAAS_OPTIONS =
            new HashMap();

            new HashMap();

    private String ticketCache;

    // provide the ticket location in the constructor
    public JaaSConfiguration(String ticketCache) {
        this.ticketCache = ticketCache;
        System.out.println("TicketCache: "+ticketCache);

    private void init()
        USER_KERBEROS_OPTIONS.put("useDefaultCache", "true");
        USER_KERBEROS_OPTIONS.put("doNotPrompt", "true");
        USER_KERBEROS_OPTIONS.put("useTicketCache", "true");
        USER_KERBEROS_OPTIONS.put("debug", "true");
        USER_KERBEROS_OPTIONS.put("ticketCache", ticketCache);
        USER_KERBEROS_OPTIONS.put("renewTGT", "true");


    private AppConfigurationEntry USER_KERBEROS_LOGIN =
            new AppConfigurationEntry("",

    private AppConfigurationEntry[] SIMPLE_CONF =
            new AppConfigurationEntry[]{USER_KERBEROS_LOGIN};

    public AppConfigurationEntry[] getAppConfigurationEntry(String name) {
        return SIMPLE_CONF;


Now that you have the extended class, you have to specify before you start the session, where to look for the login configurations ( instead of setting it as a system variable) JaaSConfiguration(ticketCache));

notice that, in the constructor, you have to provide the ticket location of each user.

After that you are good to go.

Tuesday, March 11, 2014

SSH Agent Forwarding using Java

SSH Agent forwarding is a really useful way to use a chain of ssh commands without configuring public keys for each and every machine in the chain.
You just have to add your public key to all the nodes in the chain and it allows you to make ssh calls to any node while in a another node.
Here's a very good illustration of how it works ... [1]

First you should test it using the terminal.
You can do this by editing the config file inside the .ssh directory if you don't have one, create one

vi ~/.ssh/config

Example config file:
Host beast
   User swithana
   IdentityFile ~/.ssh/id_rsa
   ForwardAgent yes

Host iu10
   User swithana
   ForwardAgent yes
   IdentityFile ~/.ssh/id_rsa

With this file configured, you can ssh to the beast host machine and then do a ssh from inside the beast host machine to the iu10 without having to include the beast machine's public key in the iu10 machine. It can also be done the other way around ( through iu10 to beast)

Example commands
$swithana:~ ssh beast
$swithana@beast:~ ssh iu10

This would work because you've set the SSH Agent Forwarding to true.

Here's the Java code to copy a file from one host to another using SSH Agent Forwarding.( uses scp)
I have used the JCraft library [2]

public class SSHCommandExecutor {
    public static void main() {

        String host = "";
        String user = "swithana";

        //your paraphrase here
        String paraphrase = "***********";

        String command = "scp test.txt";

        //private key location
        String privateKey = "/Users/swithana/.ssh/id_rsa";

        JSch jsch = new JSch();
        jsch.setLogger(new MyLogger());

        try {
            jsch.addIdentity(privateKey, paraphrase);

            Session session = jsch.getSession(user, host, 22);
            Properties config = new java.util.Properties();
            config.put("StrictHostKeyChecking", "no");


            Channel channel = session.openChannel("exec");

            ((ChannelExec) channel).setCommand(command);

            // this is the key line that sets AgentForwading to true
            ((ChannelExec) channel).setAgentForwarding(true);

            ((ChannelExec) channel).setErrStream(System.err);



        } catch (JSchException e) {
        } catch (InterruptedException e) {

    public static class MyLogger implements com.jcraft.jsch.Logger {
        static java.util.Hashtable name = new java.util.Hashtable();

        static {
            name.put(new Integer(DEBUG), "DEBUG: ");
            name.put(new Integer(INFO), "INFO: ");
            name.put(new Integer(WARN), "WARN: ");
            name.put(new Integer(ERROR), "ERROR: ");
            name.put(new Integer(FATAL), "FATAL: ");

        public boolean isEnabled(int level) {
            return true;

        public void log(int level, String message) {
            System.err.print(name.get(new Integer(level)));


Wednesday, February 26, 2014

Kerberos Java Client: Code

This is the second post of the Kerberos Java Client series.
First post can be found here [1].

This post is on the java code that is used to connect to the Kerberized server using ssh, execute a command ('ls') there and get results ( read the output stream).

You need to provide the locations of the Kerberos configuration file and the Jaas configuration file as System properties. I've done it inside the class itself.

The code is pretty much self explanatory.

import com.jcraft.jsch.*;

import java.util.Properties;

public class JSCHKerberosConnector {
    public static void main() {

        String host = "";
        String user = "sachith";
        String  command = "ls -ltr";

        JSch jsch = new JSch();
        jsch.setLogger(new MyLogger());

        System.setProperty("", );
        System.setProperty("", );
        System.setProperty("", "false");
        //to enable kerberos debugging mode
        System.setProperty("", "true");

        try {

            Session session = jsch.getSession(user, host, 22);
            Properties config = new java.util.Properties();
            config.put("StrictHostKeyChecking", "no");


            Channel channel = session.openChannel("exec");
            ((ChannelExec) channel).setCommand( command);
            ((ChannelExec) channel).setErrStream(System.err);

            InputStream in = channel.getInputStream();
            byte[] tmp = new byte[1024];
               while (true) {
                while (in.available() > 0) {
                    int i =, 0, 1024);
                    if (i < 0) break;
                    System.out.print(new String(tmp, 0, i));
                if (channel.isClosed()) {
                    System.out.println("exit-status: " + channel.getExitStatus());
                try {
                } catch (Exception ee) {

        } catch (JSchException e) {
        } catch (IOException e) {
    //to log the jsch activity
    public static class MyLogger implements com.jcraft.jsch.Logger {
        static java.util.Hashtable name=new java.util.Hashtable();
            name.put(new Integer(DEBUG), "DEBUG: ");
            name.put(new Integer(INFO), "INFO: ");
            name.put(new Integer(WARN), "WARN: ");
            name.put(new Integer(ERROR), "ERROR: ");
            name.put(new Integer(FATAL), "FATAL: ");
        public boolean isEnabled(int level){
            return true;
        public void log(int level, String message){
            System.err.print(name.get(new Integer(level)));


Kerberos Java Client: Configuration

These posts will guide you through on using Java to talk to a kerberized server using the JCraft library [1].

This particular post will be on configuring the environment changes needed to make the Java Client work. The next post will be focussing on the java Client. 

Problem Scenario
This will be using the JSCH( of the JCraft) to SSH to the kerberos server using the Kerberos Ticket (TGT). It will execute a certain command on the kerberized server and read the output stream.
Note: This program uses a generated Kerberos ticket. it does not create one.

First of all make sure your computer is running a Kerberos client and you are able to ssh to the Kerberized server using the TGT using the terminal. It would look something like 
ssh username@REALM

If so that means you are good to proceed.

If you are doing this on a Mac, you need to do the following step, else skip this step
You have to set the KRB5CCNAME to point to a new Kerberos ticket Cache. Ex:
export KRB5CCNAME="/Users/swithana/krb5cc_swithana_022322"

This is the ticket cache that you should be pointing to in the login file( discussed below).

First run the command below to generate the kerberos ticket.
kinit username@REALM

You can list the tickets you have by using the 'klist' command.

Then you need to have the Kerberos configuration file in your local system ( usually at /etc/krb5.conf).
If you don't have it, create one.
Here's what a krb5.conf look like,
[domain_realm] = TEST.IU.EDU = TEST.IU.EDU

        default_realm = TEST.IU.EDU
        forwardable = TRUE
        TEST.IU.EDU = {
                kdc =

        default = FILE:/var/log/krb5.log

You also need to have a login configuration for JAAS[2]. This will specify which module to use to login, where the TGT is ...etc. Here's an example jaas.conf file. {


ticketCache: the path to the TGT.

After these changes you are good to go.

But sometimes Java is unable to decrypt the Kerberos Ticket since Java doesn't support AES256 out of the box due to some export control reasons.
To overcome that you need to manually modify the JRE and apply the Java Cryptography Extension (JCE)  [3] to the $JAVA_HOME/jre/lib/security directory.

Saturday, February 15, 2014

Mining Twitter Data using Python: Getting Started

Data Mining is a hot topic these days, and Twitter is being used heavily as a data source in various Data Mining applications. In this post I will introduce you to start mining twitter data with Python using the Tweepy module.
( I will not include the scientific module examples here( for mining,analysing ...etc). It's a basic guide to get the Twitter API setup)

Environment Setup

1. Install python ( MacOS comes with python installed)

2. Get a Twitter API key
    Go to, sign-in to twitter ( create an account if you don't already have one)
    Click the profile Icon ( top left) -> My Applications -> Create New App
    Provide the necessary data and it will create an application.
    Go to the application -> click on API Keys tab
    This will show you the necessary keys to authenticate your application using OAuth.

3. Install Tweepy
   Tweepy is a python library which supports the Twitter API
   Install in Mac:
pip install tweepy
sudo apt-get install python-tweepy
   Here's the github project :

Now you are ready to read some tweets!!

The code to get the twitter stream, ( insert your keys to this file)

from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener

#setting up the keys
consumer_key = ''
consumer_secret = ''
access_token = ''
access_secret = ''

class TweetListener(StreamListener):
    # A listener handles tweets are the received from the stream.
    #This is a basic listener that just prints received tweets to standard output

    def on_data(self, data):
        print data
        return True

    def on_error(self, status):
        print status

#printing all the tweets to the standard output
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)

stream = Stream(auth, TweetListener())

This prints the whole twitter stream filtered using the text "nba".

getting user info:

import tweepy

auth = OAuthHandler(consumer_key,consumer_secret)
api = tweepy.API(auth)

auth.set_access_token(access_token, access_secret)
twitterStream = Stream(auth,TweetListener())

user = api.get_user('sachithwithana')
print user.screen_name

This is a basic example to get you set up. Now you are ready to explore with the Twitter API.

I would recommend using the scikit-learn library for Machine Learning with Python.

Here's the Tweepy Documentation:

Tuesday, February 11, 2014

Apache Thrift: Securing the Cilent Server Communication using SSL

This is my second post on Apache Thrift. It's been a very informative ride so far with Apache Thrift.
In this post, I'm going to talk about securing oneway client-server communication using SSL.

The code is available at github [1]

We use the JDK provided keytool to generate the necessary certificates.

Concepts involved:

Public-key encryption ( SSL) : [2]

Keystore: Keystore contains all the private keys and their corresponding certificates with the public keys.

Truststore: Truststore contains the certificates that you TRUST.

Basically the server has its private key ( in the keystore). We need to create the corresponding certificate of that private key and add it to the TRUSTSTORE of the client ( ie public-key encryption).

Managing the keystore

To create these keys and keystores you need to go to the $JAVA_HOME/bin directory. There, you have a tool called the keytool.

1. Create the private key
keytool -genkeypair -alias mykey -keyalg RSA -validity 7 -keystore keystore.jks

2. Export the corresponding certificate
keytool -export -alias mykey -keystore keystore.jks -rfc -file certificate.cer

3. Import that certificate to the client truststore.
   Note that in this example, the server and the client are in the same host ( localhost), that's why we are   importing the certificates in the same host.

keytool -import -alias mykey -file certificate.cer -keystore truststore.jks


I have added the SSL secured code to my github project as well.
Set your keystore, truststore, and passwords in the respective property files found in /resources.
Then run the SecuredCalculatorTest.

Code changes from the first Post:

  • CalculatorClientService has a special init method for the SSL initiation ( secure_init()).
  • SecureCalculatorServer
  • SecureCalculatorClient

Adding SSL capability

These lines shows how to add the SSL capability to the server.

   //for the secure communication
   TSSLTransportFactory.TSSLTransportParameters params = new TSSLTransportFactory.TSSLTransportParameters();
   params.setKeyStore(serverKeyStore, keystorePassword);

   TServerSocket serverTransport = TSSLTransportFactory.getServerSocket(
        4030, 10000, InetAddress.getByName("localhost"), params);

These changes show the addition of SSL capability to the client.

    TSSLTransportFactory.TSSLTransportParameters params =
             new TSSLTransportFactory.TSSLTransportParameters()
    params.setTrustStore(clientTrustStore, truststorePassword);
    transport = TSSLTransportFactory.getClientSocket("localhost", 4030, 10000, params);

Notice that here, you don't have to open the transport.



Sunday, February 9, 2014

Apache Thrift: Getting started

Apache Thrift [1] is one of the most popular and innovative software framework I've used.

Thrift file ( IDL) defines the server interface and allows the server and the client to talk through RPC. That same thrift file can generate various server and/or client code enabling scalable cross-language development.

I've hosted the sample code on Github (Uses Maven, and surefire tests)   [2]

To get started, install thrift in your computer [3].

Then you need to create a "Thrift file"( it basically defines the interface, i:e Interface Definition).
Here's my simple Calculator.thrift file.

namespace java lk.swithana.calculator
service lk.swithana.calculator.server.CalculatorService
    i32 add  (1:i32 a, 2:i32 b),
    i32 sub  (1:i32 a, 2:i32 b),
    i32 div  (1:i32 a, 2:i32 b),
    i32 mult (1:i32 a, 2:i32 b)

This defines the calculator interface.
If you've installed Thrift correctly, you should run:

thrift --gen java Calculator.thrift 

This would generate the necessary Java Beans for the server and the client.

NOTE that it only creates the beans, now you have to create the server and the client using those beans.
I've already completed the server and the client in the code in github.

Refer to the CalculatorBasicTestCase java test class to see how it works.

Note: This is a VERY BASIC thrift file. You can/should add type definitions and other Thrift features to make it better. Here's a good guide on writing simple to complex thrift files.[4]


Saturday, January 11, 2014

Apache Airavata: Develop your own Science Gateways using Airavata

Let's say you want to create a Science Gateway to use the Computing resources available. How do you do this?

It's a tedious job to write the whole thing from scratch because you'll have to deal with the resources head on to create an interface to use the resources along with the gateway you'd want to develop.

The solution is that you can use Apache Airvata to do this task more conveniently. All you need to do is to develop your own gateway, connect it to an Airavata server using the Airavata API, configure the credentials of the resources you want to use, then you are good to go!!

How can I do this you asked?

checkout this wiki page which tackles this exact problem.

This is a simple guide on creating a Science Gateway using the Airavata API.