Thursday, December 13, 2012

Linux App + ports below 1024 + not root = permission denied


Issue: you set your application on a Linux server (say Jira, Fisheye, Stash, Bamboo, etc) to port 80 and/or 443, and you get "permission denied" right away during startup. 

This has been a growing issue for the last couple years - it appears that sometime around the Linux 3.0 kernel release, they made the decision to deny port binding/usage below 1024 to applications not running as root. Example: a Tomcat based application can't use port 80 with running as root. I get some of their reasoning - it's harder to run a rogue website on port 80 on a box you've slightly compromised and steal people's data/identity/etc, if you don't have access to port 80 unless you're a super-privileged user. But it sure makes things difficult when your user base wants to type in http://application.company.com internally and it doesn't come up. 

Keeping an application on its default port (let's say port 1234 for simplicity's sake) has alot of advantages, for instance:

  • Future upgrades are a breeze, because you don't have to "fix" configuration files the upgrade overwrites. In fact, some upgrades detect your changes and if the migration of your changes fail, the upgrade can get very messy. (i.e. new features that need to be part of your server.xml file that you modified to use port 80 aren't written during the upgrade because it's too freaked out by your small change). 
  • Documentation & QA from the vendor is all based around a clean, out of the box install. When you start messing with ports, you start playing with fire. 
  • Staying with default ports means you can run your application as a non-root user; it's a really good idea in the first place, and as more corporations embrace Suse/Ubuntu/etc as viable application servers, security standards are being set & enforced, and not having access to root/sudo is becoming more common. 
  • Port proxying and reverse-proxies aren't always a panacea, because they kill gzip compression of the web stream and while they do their best, they are an imperfect software solution; you are going to take an increasingly performance hit the more web traffic you have to deconstruct, reconstruct and forward, and the whole deconstruct/reconstruct process isn't always transparent to the client or server ends; weird behavior or bugs can result. And if you call for support, the first thing they will have you do is bypass your proxy and reproduce the issue. And if the issue is from the proxying, they won't support you. 

But sometimes you have users (or management) that really want to type in http://app.company.com and have it work. 

What I've done in the past is use apache2 to setup a simple port redirection (not proxying) so users can type in http://app.company.com and it redirects them seamlessly to http://app.company.com:1234. 

Sample setup using a clean Ubuntu 12.04 LTS server, with an existing application listening for http requests on port 1234 and https requests on 1443:

1) Install apache2 (as in sudo apt-get install apache2)
2) Enable ssl - sudo a2enmod ssl
3) Edit the now-existing /etc/apache2/httpd.conf - 


NameVirtualHost *:80

<VirtualHost *:80>
   ServerName app.company.com
   DocumentRoot /home/appuser
   Redirect permanent / http://app.company.com:1234
</VirtualHost>

<VirtualHost *:443>
   ServerName app.company.com
   DocumentRoot /home/appuser
   Redirect permanent / https://app.company.com:1443
   SSLEngine on
   SSLCertificateFile /home/appuser/app.crt
   SSLCertificateKeyFile /home/appuser/app.key
</VirtualHost>


This works fairly well - it leaves the original ports intact for direct calls (i.e Jira needing to integrate with Stash or Fisheye - it will not work via a redirect), and users will automatically use the new port when they bookmark pages.  
---
However, let's say you need to run on port 80 & 443. Authbind is probably your best bet, I've never had success with setcap. I'll use Jira under Tomcat as an example, but you can adapt this to anything. 

The following steps assumes a nice clean Ubuntu 12.04 LTS server, a dedicated jira user and an existing install of Jira 5.2.

1) If you have anything using port 80/443 already, you'll have to remove it. In my prior example I used apache2 to do some redirects for user browsers; you might as well uninstall apache2 completely now. (i.e. sudo apt-get --purge remove apache2.2-common)

2) Install authbind (i.e. sudo apt-get install authbind)

3) Bind your ports to your dedicated Jira user with touch, chown and chmod. Here's port 80:

sudo touch /etc/authbind/byport/80
sudo chown jira /etc/authbind/byport/80
sudo chmod 755 /etc/authbind/byport/80

Repeat as needed for other ports like 443.

4) Disable ipv6 in Jira/Tomcat - this was not clear to me for quite while; I glossed over that authbind does not currently support IPv6, and I didn't realize ipv6 was the primary protocol in Tomcat when it's enabled on your nic. (i.e. add CATALINA_OPTS="-Djava.net.preferIPv4Stack=true" in the setenv.sh in your jira/bin folder). You cannot skip this step! Use ifconfig to see if Ipv6 is enabled; if it is, you have to disable it in Tomcat.

5) Don't forget to change your /jira/conf/server.xml to port 80 and/or port 443 instead of 8080 & 8443 respectively. To use Jira/Tomcat's basic http to https redirection, make sure your 80 block contains redirectPort="443" in it. If your users use http against anything but the exact url your SSL cert is for, they will get an ssl warning about a certificate mismatch. 

6) Reconfigure Jira to have authbind launch it with the --deep option so child spawns are covered. 

The simplest spot to set this is at your /etc/init.d/jira script, but if someone stops and starts Jira manually in the /jira/bin folder, then you will get permission denied on ports below 1024 and Jira won't be accessible. I got a little messier and edited the start-jira.sh script, so the /etc/init.d/jira script is covered (because it execs start-jira.sh) and other people who jump on your server and restart Jira don't have to know to start it with authbind --deep. Change your start-jira.sh to the following (my additions on line 4 & 6 - specifically the "authbind --deep" portion): 

if [ -z "$JIRA_USER" ] || [ $(id -un) == "$JIRA_USER" ]; then
    echo executing as current user and authbind
    if [ "$PRGRUNMODE" == "true" ] ; then
        exec authbind --deep $PRGDIR/catalina.sh run $@
    else
        exec authbind --deep $PRGDIR/startup.sh $@
    fi

Bonus 7) If you want to get extra fancy, so users get redirected from whatever http address they try your Jira instance with (say http://jira when it should be http://jira.company.com) to the correct, FQDN using https (so no ssl cert mismatch warnings), add the following rule to the /jira/WEB-INF/urlrewrite.xml instead of redirectPort="443" in the server.xml:

<rule>
   <name>Http Check</name>
   <condition type="scheme" operator="equal">^http$</condition>
   <from>^/(.*)</from>
   <to type="permanent-redirect" last="true">https://jira.company.com/$1</to>
</rule>

and then remove the redirectPort="443" from your server.xml. 

For brevity, I glossed over alot of bits that hopefully are trivial. Hopefully this helps other people get Jira (or any other product) working on ports below 1024, and not run with sudo/root. 

-Kelly Schoenhofen

No comments:

Post a Comment