How do you retrieve test results from Bamboo via a REST call?

Sorry for the lack of updates; I was recently in a pretty bad biking accident and I've only recently returned to work. So!

I was asked by someone how they can retrieve test results somehow from Bamboo, because they have a separate, custom dashboard they use for displaying test results.

I know Atlassian has a REST API for all of their main products, is it possible to get test results back using it?
Yes, it is possible and not too difficult. The main caveat is that the smallest (and largest) group of test results you can retrieve is job sized.

Here's the latest documentation on Bamboo's API:
https://docs.atlassian.com/bamboo/REST/5.6-SNAPSHOT/
https://developer.atlassian.com/display/BAMBOODEV/Using+the+Bamboo+REST+APIs
https://developer.atlassian.com/display/BAMBOODEV/Bamboo+REST+Resources

To retrieve results, we'll use curl (though Python and many other tools are available) and xmllint to output the results in pretty handsome xml. (so if you're using Ubuntu (and who isn't?) "sudo apt-get install curl libxml2-utils" will fix you up)

Our example Bamboo server's base rest url is:

https://bamboo.company.com/rest/api/latest/

We can get build results back by adding

/result/projectkey-buildkey-job/buildnumber 

to the url.

Test results will be brief until we expand the test results, so we add

?expand=testResults.allTests 

to the end of that. If you're just looking for failures, or a slew of other filters, they have that too.

It defaults to xml output I believe, but you can output JSON format by adding .json to the end of the url, before the arguments. Word is that JSON will be deprecated at some point, but for now you can still get results back that way. I'll show how in a minute.

I don't want to specify the build number, I just want the latest results, and Atlassian supports "latest" as the build number. So, here's our complete url, using a made up plan with unit tests as our example:

https://bamboo.company.com/rest/api/latest/result/PLAN-STAGE-JOB/latest?expand=testResults.allTests

If you want JSON output, it would be this:

https://bamboo.company.com/rest/api/latest/result/PLAN-STAGE-JOB/latest.json?expand=testResults.allTests

Now, it outputs a huge block of unformatted xml by default. If you want to to look nice, we pass it through xmllint, by appending this to the end:

  | xmllint --format -

By default, anonymous is disabled in our install, so we need to pass authentication. The simplest example is using basic auth, so the complete command line would be this, using your username and password:

curl -u username:password https://bamboo.company.com/rest/api/latest/result/PLAN-STAGE-JOB/latest?expand=testResults.allTests | xmllint --format -

And it outputs something like this:

There's much more you can do beyond retrieving test results. You can also use this to kick off plans, get deployment results, etc. And, each release of Bamboo expands the REST API.

-Kelly G. Schoenhofen

Code Review Tools - Stash "versus" Crucible

We've been using Atlassian Crucible for code reviews, but Atlassian Stash becomes the scm/git server standard over the last 12 months, some of the teams are starting to use Pull Requests in Stash as their code review vehicle. As the Gospel According To Stash spreads, inevitably managers are asking "Hey, what is the deal? Is Crucible going away? Should we be switching to Stash Pull Requests?"

So, let's look at it. Some background - this is the state of the products of early 2014. We're on Crucible 3.3 (March 2014) and Stash 2.10 (Feb 2014) - there is a newer release of Stash out right now, and that's mentioned. We're also almost entirely git, but using other scm products is brought into play. 

Stash

• Stash reviews work best when all your changes can be encapsulated in single pull request from a single repository. 

Crucible

• Crucible reviews work best when your changes contain commits from multiple repos - Stash cannot span multiple repos in a single review. 
• Crucible also supports pulling in changes from multiple source control systems (cvs/svn/git/et al) & multiple scm servers (i.e. two different svn servers) in a single review - Stash code reviews have to be pinned to a pull request in Stash only (so git is your only scm choice), and only from a single repo local to the Stash server you create the request on. It's sort of a fine point that 99% of people wouldn't care about, but it needs to be pointed out if you require a certain flexibility. 
• Crucible will let you review any (set of) commit(s), Stash only lets you review pull requests. So if you wanted to do a formal review of an entire release, or maybe a point in time snapshot before you upload the changes to a public github repo -Crucible would be your tool. Stash cannot free-review commits. 

Conclusion

Crucible is a very established, stable product with a wide audience; it has a very active development team behind it but the audience it caters to is very broad. The Crucible platform probably won’t radically change in the future. Crucible has a long future in front of it. 

That said, Stash is a new product and has a very focused audience - git users only, and catering to the most popular git workflows. It also has very active development team behind it, but because the audience is so narrow compared to Crucible, the change velocity appears much higher. The upgrade to Stash (to 2.11 - the March 2014 release) I'm doing this weekend adds side by side diff, commit commenting outside of pull requests/code reviews, comments at the file level in a commit (for instance, the deletion of a file, or a binary file that can’t be parsed), and a much tighter integration in Jira. Pending pull-requests tied to an issue will now be visible in Jira after this weekend. 

Stash has evolved insanely quickly; Stash 2.0 came out last January, and Stash 2.11 came out a few weeks ago - that’s 11 major(ish) releases in 14 months. Many of the features in Stash that our development teams use on a daily basis simply weren’t there just 12 months ago. I can’t imagine what the product will look like 12 months from now.  

Pull Request code reviews really shine when you can embrace the idea; I highly recommend using them if it fits your team’s needs. If you've gotten this far and pull requests are an unfamiliar term, a quick refresher:

Pull requests are like saying "request to merge". 

1. You branch server side on any change - a feature, a bug, a hotfix. 
2. You make your changes, commit them and push your changes back up to the server.
3. Your Continuous Integration system shows your peers that your change builds & passes the automated test framework.
4. You create the Pull Request to merge it back into the parent branch.
1. Add appropriate reviewers. 
2. They review your change, if they comment & request changes you make those changes and this iterates until they approve the pull request. 
3. You complete the pull request and Stash automagically merges your changes back to the parent source line and deletes your branch.

If you do this, or could do this, pull request code reviews can be a very powerful thing. 

Tl;dr - Crucible & Stash are both awesome, use what works best for your team.

-Kelly Schoenhofen

Commit/Push Driven Builds Between Atlassian Stash & Bamboo

By default build systems poll scm every so often; every 180 seconds is probably the most common setting. Very quickly though, between your different systems - your build systems (Jenkins, Bamboo, TeamCity, etc), defect systems that keep an eye on commits like Jira, or code review tools like Fisheye/Crucible, you can be querying your git server several times a second, 24/7. 

To reduce that load and make things a bit more accurate (For instance, a best practice in continuous integration is to test every push individually; this helps pin down which push broke a build, decreased the code coverage, or increased the Coverity detected defect count. Polling every 3 minutes usually gets each push individually but not always, not even close.) git has had post-receive web hook scripting built in for some time. 
I use Stash as an enterprise git server, and digging into the raw folder structure of git (housed on the Stash server) is not a best practice. Not forbidden, just not recommended.  Luckfully, there is a post-receive webhook plugin for Stash.  

To enable commit-driven builds in Bamboo, it’s just two easy steps. 

Step #1) In Bamboo, switch your build trigger from “Repository Polling” to  “Repository triggers the build when changes are committed” (that rolls right off the tongue); you have to fill in the trigger IP address; leaving it blank disables repository triggering. Enter your Stash server's IP address here. The screenshot should explain the preceding text better than the text itself:

Step #2) In Stash, enable the post-receive webhook and give it the Bamboo url to call to trigger the build. Under the repo’s Settings, go to Hooks, Post-Receive WebHooks, and the url is:

https://bamboo/api/rest/updateAndBuild.action?buildKey=PROJECT-PLAN

Change the PROJECT to your project key, and PLAN to your plan key. You can get those values off the Bamboo page url you just left on the prior tab in your browser. See the picture below for a picture.

Caveat #1) If either the git server and Bamboo fail to communicate when the post-receive hook is called, Stash/git will not re-fire the hook until the next commit, so when using post-receive webhooks it’s considered a best practice to add a second build trigger to poll your git system every few hours for changes. That wasn’t in relation to Stash’s stability/availability, I think it’s just the general git ecosystem in general. So far I’ve never seen Bamboo<->Stash post-receive webhooks fail, but I don’t have a lot of datapoints yet. I am currently not adding a second build trigger (polling) as a backup until I start seeing data indicating relying on just webhooks isn’t robust enough. 

Caveat #2) Bamboo will compare the hash of the last scm pull to the notification hash, and if they aren’t different, it won’t trigger a build. So when setting this up and/or troubleshooting, your Bamboo build has to be on different commit than when the rest call is made or no build will occur (and no error will be posted).