NGINX Reverse Proxy for Gitlab with SSH proxy

 •  •  ☕️ Reading Time

Problem: cloud servers have great availability and fast bandwidth -- but storage comes at a premium. I initally wanted to put my GitLab server on my little cloud instance, but realized I would have to pay $30 a month to have it work relatively well. My home server doesn't have a fast connection out to the internet, but has an order of magnitude more storage in comparison to my cloud instance. Not to mention more CPU cores, RAM, faster storage... I want to leverage the fast connection of my cloud server for my website, but also serve my GitLab instance through my domain as well.

Solution: From a high level, it looks like this:

NGINX will still serve my website at advtech.ca, but gitlab will be served through a reverse proxy to my home server at gitlab.advtech.ca. This way I can use the fast connection of my cloud instance for serving my website, and gitlab can be slightly more sluggish. I can live with that.

1: HomeLab Prep

I'm assuming you're going to have GitLab already running in a VM, container, or whatever software execution environment suits your needs.

1.1: Dynamic DNS

One of the major hurdles with having a homelab is dealing with residential internet. Your ISP typically has your connection on a dynamic IP which can change. This isn't all that great if you make a DNS entry that points to that IP, only for it to change overnight.

Luckily there are solutions out there -- I personally like No-IP since they offer a free tier, and many modems have integration with No-IP. You can also run a No-IP daemon on your server if your modem doesn't offer dynamic DNS integration. This is great because if your public IP address changes No-IP will pick up that change and update their DNS servers almost instantly.

Eventually you will end up with some kind of link that refers to your home modem, this is good. Take note of this link since it's the main link in our reverse proxy chain.

1.2: Port Forward GitLab

Now you will want to expose your GitLab instance to the world wide web. On your modem, you will want to create the following entries:

1.3 Prep Gitlab + A word on Security

In your gitlab.rb config file, you will want to set these values:

external_url 'https://your.no-ip.address.com'
nginx['listen-port'] = 80
nginx['listen-https'] = false

Do you see the huge problem with this? this will work, but please don't do this unless you want the data being sent between your cloud instance and home server to be unencrypted. If this is something you can live with, then go ahead at your own risk. But sending unencrypted data over the internet isn't a smart idea.

So, in order to make this secure we will have to do some legwork. First, from gitlab.rb, set the following values

nginx['listen_port'] = 443
nginx['listen_https'] = true
nginx['ssl_certificate'] = "/etc/gitlab/ssl/cert1.pem"
nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/privkey1.pem"

This sets up GitLab to work over https, and use an arbitrary SSL certificate and key for its traffic. Wait - where does this cert come from? You will have to pull the SSL cert off of the server we are using as a reverse proxy. This way the reverse proxy server can encrypt traffic, and we can decrypt it because we have the keys. The process for this will vary based on your encryption provider, in the end you will end up with a cert and a cert key, these you can put in /etc/gitlab/ssl/ and point to them with your config file.

When all is said an done with either method you choose (please use the HTTPS method) run gitlab-ctl reconfigure

2: Cloud Prep

Related to 1.3, but if you decided to make your reverse proxy link encrypted with https (highly reccomend), you will want to pull your SSL certificate and key from your cloud instance. Alternatively there are many interestingg ways to synchronize certs between servers -- out of scope of this post, I advise some creative Googling if you want a fancy automated way to syncronize certs.

First, create a configuration file in /etc/nignx/sites-available/, call it gitlab-proxy or something that inspires you.

# Redirect http traffic to https
server {
    listen          80;
    server_name     gitlab.blah.com;
    return          301 https://$server_name$request_uri;
}

# Config block for the subdomain
server {
    listen                      443 ssl;
    server_name                 gitlab.blah.com;

    location / {
        proxy_read_timeout      300;
        proxy_connect_timeout   300;
        proxy_redirect          off;

        proxy_set_header        X-Forwarded-Proto $scheme;
        proxy_set_header        Host              $http_host;
        proxy_set_header        X-Real-IP         $remote_addr;
        proxy_set_header        X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header        X-Frame-Options   SAMEORIGIN;
        proxy_ssl_certificate /path/to/your/ssl/cert;
        proxy_ssl_certificate_key /path/to/your/ssl/key;

        proxy_pass              https://blah.no-ip.com;
    }
}

Symlink this newly created file against sites-enabled, ln -s /etc/nginx/sites-available/gitlab-proxy.com /etc/nginx/sites-enables/gitlab-proxy.com

Naturally, substitute in the certificate locations, the dynamic dns domain name that was set up earlier, and the name of your domain with ones that are configured and live on your server. Then, restart the nginx web server. You should now be able to navigate to gitlab.blah.com and everything should work! Except for ssh access...

2.2: NGINX SSH Proxy

When you set up port forwarding you would have assigned your external port to some arbitrary random number, like 6969. In your main nginx conf at /etc/nginx/nginx.conf, add the following:

stream {
        upstream ssh {
                server blah.no-ip.com:6969;
        }
        server {
                listen 6969;
                proxy_pass ssh;
        }
}

It's that easy! Streams in NGINX are a relatively new feature but a really interesting gone. note: you will have to use ssh over this arbitrary port (is this a dealbreaker? see 3).

3: Another Way?

If you chose to reverse proxy gitlab over https, there is another way! Git supports transactions over https, instead of using git clone [email protected], you can just use git clone https://gitlab.blah.ca to perform your git work over https. Then we don't have to worry about the ssh reverse proxy configuration in our cloud instance. Also, I personally reccomend the https route for git since some networks block port 22 (shocker) but since 443 is standard https, your git traffic won't be blocked.