Tutorial

How Atlassian integrated with Google Apps Part 1

Editors note: This post was written by Richard Wallace of Atlassian. This is part one of a two-part series on Atlassians recent development for the Google Apps Marketplace. Atlassians JIRA Studio is a hosted software development suite, combining Subversion source control, issue tracking, wiki collaboration, code reviews and continuous integration. JIRA Studio is now integrated with Google Apps and is featured in the Google Apps Marketplace - a great tool for development teams using Google Apps. This series explores some of what Atlassian worked on, and what they learned along the way.

Part 1: Nine cool things you can do with the JIRA Studio Activity Bar

By now you have probably heard about the Google Apps Marketplace and JIRA Studio integration. If not, the short version is that Google has created a place for users of Google Apps to sign up for third party hosted SaaS offerings and have them fully integrated with their Google Apps domain. Google was kind enough to ask Atlassian if we would be interested in integrating our hosted agile development suite, JIRA Studio, for their initial offering, and we thought it would be an awesome opportunity. So, if you purchase JIRA Studio through your Google Apps account you get issue tracking and project management, a wiki, source code repository and review system, and (optionally) a continuous integration server configured out of the box, using the same user accounts as your Google Apps for authentication.

However, we decided that just stopping at Single Sign On (SSO) integration, while cool, wasnt enough. Google provides a rich set of APIs for accessing your Google Apps data, so we wanted to provide a better experience for anyone who purchased JIRA Studio. So, while a team in Sydney worked on the authentication integration bits, a second team worked on the "cool" features. A few of the more obvious cool things were Google Doc macros in the wiki, attaching Google Docs to JIRA issues and so on. But we wanted something that would really stand out.

The one Google App that we hadnt found a way to integrate yet was Google Talk. Someone suggested implementing a chat client similar to the one you get in Gmail that would appear on every page in Studio. We quickly came to realize it didnt need to be limited to just chat, it would also have tabs showing various bits of information from JIRA Studio and Google Apps. And so the JIRA Studio Activity Bar was born.


The feature we delivered does a lot. With the JIRA Studio Activity Bar a user can see:

1. all the recent activity going on in Studio - whos working on what issue, who committed what and how long ago
   issues assigned to you
2. what reviews you need to do
3. recent builds
4. favorite wiki pages
5. unread mail in your Gmail inbox
6. recent Google Docs
7. any upcoming events on your Google Calendar
8. and chat with anyone with a Google Talk or Jabber account

All without leaving the JIRA Studio environment.

Initial planning

Our initial estimate for implementing this feature was, "a lot of work". Im pretty sure we underestimated. The immediate question that jumped out at us and continued to bother us during development was, "How can we scale this?" Instant messaging is typically done with a persistent connection to the chat server. That way the server can "push" messages to clients when theyre sent. Thats what makes it "instant." To do the same with a browser wed need to use some Comet techniques for maintaining a connection between the browser and server. In Java apps, this typically means a thread will be blocked waiting for something to do. It was immediately obvious that this just wouldnt scale very well. For a Studio instance with 30-50 users, we could expect at least that many threads to be tied up all the time.

How do we scale?

So we started looking at how we could do non-blocking I/O in Java apps. Fortunately, most Java app servers have some form of non-blocking, asynchronous I/O capabilities these days. Unfortunately, they have different APIs for using those capabilities since Servlet 3.0 implementations arent yet wide spread. To make matters worse, we werent sure exactly what container we were going to wind up deploying to.

You see, JIRA Studio runs on Virtual Machines on Contegix hardware. This is to keep all the customer data separate and secure. Each Studio instance is allocated a specific amount of memory so that Contegix can properly load balance instances for maximum utilization of their hardware. Studio, before adding the Google Apps integrations, was already starting to get closer and closer to exceeding that memory limit.

How will it be deployed?

At first, we had hoped to develop the Activity Bar as a plugin to one of our apps. Because of the way servlets are wrapped in our plugin system this proved to be impossible as there is no way to access any of the container-specific methods for doing asynchronous I/O. Our next thought was to deploy it as a separate webapp. We could do that simply with a minimal web server like Jetty or Grizzly. We debated whether the app would meet the limited memory footprint even with the most minimal web server we could find, since the JVM by itself would take up 40-50MB of memory. The final thought was that we would deploy the Activity Bar webapp in one of the existing containers alongside JIRA, Confluence, Crowd or Bamboo. Even that decision was destined to change.

So, amidst all this uncertainty, which ironically couldnt be cleared up until we had something to deploy, what was a developer to do? Well, the same thing we always do. Abstract, abstract, abstract. Atmosphere was built exactly to solve this problem. By using its APIs you can run on any of the myriad of containers with asynchronous I/O support and not have to worry about the details. Even better, it integrates well with the Jersey library for building RESTful services, making it even easier to use! And to top it all off the developers are quick to patch and release bug fixes as they are found. We certainly couldnt have completed this project as quickly as we did without Atmosphere, so I want to give a big thanks to Jean-Francois for all his help.

Early days

And so we set out to develop the chat backend as a separate web application, to be run on an as-yet-undetermined container using Atmosphere to abstract away the details of doing asynchronous I/O. In pretty quick order we were able to get logging in to Google Talk, fetching the buddy list, message sending and receiving, and handling presence updates all working.

We naively did this using the HTTP streaming form of Comet techniques. This worked well for a while until we started doing some more tests and added error handling. At this point two nasty problems reared their ugly heads: 1) you couldnt really do any error handling if the connection failed, 2) if the initial connection succeeded but timed out after a while, there wasnt much we could do to detect that. This was due to the fact that the connection with the browser was maintained in an iframe. When the server needs to send a message to the browser, it simply spits out a <script> tag containing a call to a JavaScript function to handle it. Well, there is really no way of determining the HTTP responses status code of an iframe. That meant that when a connection problem occurred, there would be little we could do to handle it. As for the second problem, we might have created an onload event handler such that when it was invoked it would remove the iframe and create a new one to receive notifications. The two problems, combined, however, made us realize that HTTP streaming just isnt a great solution, so we set out to implement long polling instead.

Looking back, while HTTP streaming wasnt the best solution for us, it wasnt an entirely bad way to start. We avoided issues like storing notifications between polling requests that we otherwise would have had to deal with and we were able to get the UI development going with real live data while the migration to long polling was being done. It was also good because it made us confident that yes, this really would work and we could pull it off in the short time frame we had been given. While it wasnt the best solution, it was a good way to get started and the migration to long polling wasnt all that difficult or disruptive because we had a good foundation to work from.

Not too surprisingly the application tabs were the easiest bits to get working. For pulling data from JIRA, Confluence, Crucible and Bamboo we just had to request feeds for the data we wanted and parse and display them. Since all JIRA Studio apps come from http://yourcompany.jira.com, we didnt have to worry about any same origin policy issues and SSO took care of our authentication needs. The Google Apps data was a bit harder to come by because we did need to worry about the same origin policy and authentication.

As luck would have it, the other features that were being worked on as part of our integration work took care of all of that very nicely. To allow a user to see a list of Google Docs in Confluence and pick a document to attach to an issue, the integration team had developed plugins for the Atlassian applications that acted as proxies to the actual Google Data APIs. Authentication is handled using OAuth, which is setup automatically between your JIRA Studio instance and Google at the time you sign up. With those feeds available to the Activity Bar, it was a simple matter of fetching and parsing them to display your Google Docs, upcoming Google Calendar events and unread messages in your inbox.

By Richard Wallace, Atlassian

Now its just a matter of rolling it out, right? Not quite. Part 2 of this series talks through a few other hurdles we had to jump over (or crash through) to get ready for launch.