Getting Started with Golang Google Cloud Functions

As of January 16, 2019, GCP’s Cloud Functions now natively support Golang! This post will briefly cover why this is a big deal and how one can begin to use Go in their cloud functions.

First, what are Cloud Functions? Essentially, Cloud Functions are analogous to AWS Lambda and Azure Functions. It’s a serverless and event-driven compute environment. The runtime manages deployment and resource scaling, so you can focus on application logic.

Similar to AWS Lambda, Cloud Functions are billed by invocations, memory/compute usage, and network bandwidth. For small-scale web apps, this tends to be cheaper than serving a traditional “full-time” web service.

Table of Contents

• Why Go?
• Getting Started: Setting up a GCP Project
• “Hello, World!” as a Service
• Designing for Serverless Execution
• Non-HTTP Triggers
• Conclusion
  • Appendix A: Pricing
  • Appendix B: Resource Customization
    • Memory
    • Maximum Duration
    • Local Storage

Why Go?

Node.js has long been the de facto language for writing serverless functions. However, now that both AWS and GCP support Go as a serverless runtime I predict that we’ll see a shift towards using Go in this domain.

• Speed - The Go community values speed. For computationally intensive serverless functions, Go might make more sense than Node.¹
• Language Features - Using Go gives you the benefits of a static type system and Go’s excellent concurrency support. Additionally, Go’s standard library has many helpful utilities for writing HTTP services out-of-the-box. This makes it easier to port existing Go web services to be serverless endpoints.
• Ecosystem - Go has a robust package ecosystem. Common serverless tasks like media encoding, numeric computation, and database access are all possible via a community-supported library.

There’s a common joke that Go is a language entirely designed around making API gateways. While this is facetious, there’s a kernel of truth: Go is ergonomic for writing HTTP services. One can write a fully-functional HTTP server in just a few lines of Go.

Fortunately, Cloud Functions takes full advantage of the interfaces exposed by the Go http module, allowing for rapid development of Go serverless endpoints.

Getting Started: Setting up a GCP Project

For our first Cloud Function, we’ll write and deploy a minimal function that responds “Hello, World!” to any incoming HTTP request.

1. Create a new GCP project from the Project Management page. It’s fine to reuse an existing project if you already have one.

2. Go to the Cloud Functions console. You may have to enable billing on your account.

3. Enable the Cloud Functions API.

4. Install the gcloud CLI tool from here. You can perform most of the steps of this tutorial using just the web interface, but it will be quicker to do most of the deployment steps from a CLI tool.

5. Run the following to get the latest version of the CLI, and install the Cloud Functions beta (which is needed to use Go). Note: You should do this even if you already have gcloud installed.

   gcloud components update
   gcloud components install beta
   

6. Now, we’ll point the gcloud client at our project. Set the active project to be the one you created in Step 1 with the following command:

   gcloud config set project <PROJECT_ID>
   

7. Now, run gcloud functions list to test that your gcloud client is authenticated correctly. If the response is Listed 0 items., then everything is working correctly.

“Hello, World!” as a Service

Let’s now write our “Hello, World” Cloud Function.

1. Start by creating and entering into a directory for the project.

   mkdir cloud-functions-golang
   cd cloud-functions-golang
   

2. Create a file named helloworld.go with the following contents.

   package helloworld
   
   import (
       "fmt"
       "net/http"
   )
   
   func HelloWorld(w http.ResponseWriter, r *http.Request) {
       fmt.Fprint(w, "Hello, World!")
   }
   

   If you’ve written web services in Go before, you’ll recognize that HelloWorld has the same type as HandlerFunc. In effect, we are writing a normal Go HTTP endpoint – the only difference being that our endpoint is managed and routed by Cloud Functions instead of by our own server package.

   Our Go function HelloWorld will be routed with each request made to the Cloud Function. Just like a normal Go web endpoint, we can send a response to an incoming request by writing to the http.ResponseWriter.

3. Now, we can create and deploy our function using the following command.

   gcloud functions deploy HelloWorld --runtime go111 --trigger-http
   

   Note that we’re specifying the go111 (Go 11.1) runtime, and we’re setting the function’s trigger to be HTTP. You should see something like the following as output:

   Deploying function (may take a while - up to 2 minutes)…done.
   availableMemoryMb: 256
   entryPoint: HelloWorld
   httpsTrigger:
     url: https://us-central1-<PROJECT_ID>.cloudfunctions.net/HelloWorld
   labels:
     deployment-tool: cli-gcloud
   …
   

4. If the deployment was successful, you should see a URL under httpsTrigger > url. This URL points to your newly created Cloud Function and works like an endpoint for a “normal” web service.

   Open this URL in a browser. You should see Hello, World! as the response! 🤩

A More Complex Example

For a more “realistic” example of how one might use Cloud Functions, we’ll implement a basic version of the sunrise-sunset.org API (which provides sunrise/sunset times for a given date and location).

We’ll be using schema to parse query string parameters and go-sunrise to get the sunrise/sunset time information.

Declaring Dependencies

For any non-trivial web service, we’ll need to depend on existing Go libraries. Cloud Functions uses go modules to define application dependencies. To declare the dependencies we need for our API, we’ll first initialize a mod.go file in our project directory with go mod init, and then go get our dependencies.

go mod init mydomain.me/helloworld
go get github.com/nathan-osman/go-sunrise github.com/gorilla/schema

Running cat mod.go should yield something like this:

module mydomain.me/helloworld

require (
    github.com/gorilla/schema v1.0.2
    github.com/nathan-osman/go-sunrise v0.0.0-20171121204956-7c449e7c690b
)

Note that upon deployment, Cloud Functions uploads your application code (and not a compiled binary) to a build server, and fetches dependencies remotely. Thus, you should not rely on any local dependencies that won’t be accessible by the Cloud Functions build server.

Implementing a Sunset/Sunrise API 🌞

Now, we can write our implementation for the Sunset/Sunrise API. Since we declared our dependencies in go.mod, we can include packages normally. Note that the http.Request we are passed in our handler function is mapped to the HTTP request that the Cloud Functions endpoint receives, so we’re able to do query string decoding just as if we were running the server ourselves.

Create a file called suntimes.go and add the following contents:

package helloworld

import (
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "reflect"
    "time"

    "github.com/gorilla/schema"
    "github.com/nathan-osman/go-sunrise"
)

type Request struct {
    Lat  float64   `schema:"lat,required"`
    Lon  float64   `schema:"lon,required"`
    Date time.Time `schema:"date,required"`
}

type Response struct {
    Sunset  time.Time `json:"sunset"`
    Sunrise time.Time `json:"sunrise"`
}

func SunTimes(w http.ResponseWriter, r *http.Request) {
    var decoder = schema.NewDecoder()
    decoder.RegisterConverter(time.Time{}, dateConverter)

    // Parse the request from query string
    var req Request
    if err := decoder.Decode(&req, r.URL.Query()); err != nil {
        // Report any parsing errors
        w.WriteHeader(http.StatusUnprocessableEntity)
        fmt.Fprintf(w, "Error: %s", err)
        return
    }

    // Perform sunrise/sunset calculation
    sunrise, sunset := sunrise.SunriseSunset(
        req.Lat, req.Lon,
        req.Date.Year(), req.Date.Month(), req.Date.Day(),
    )

    // Send response back to client as JSON
    w.WriteHeader(http.StatusOK)
    response := Response{sunset, sunrise}
    if err := json.NewEncoder(w).Encode(&response); err != nil {
        panic(err)
    }
}

func dateConverter(value string) reflect.Value {
    s, _ := time.Parse("2006-01-_2", value)
    return reflect.ValueOf(s)
}

Don’t worry if you don’t understand everything here. 🙂 All this function does is take a request (which comes in as query parameters on the request URL), performs a sunrise/sunset lookup, and returns a JSON-encoded response.

We’ll deploy our new function using a similar command to the HelloWorld example.

gcloud functions deploy SunTimes --runtime go111 --trigger-http

Again, take note of the deployed function’s URL. It should be of the form:

https://us-central1-<PROJECT_ID>.cloudfunctions.net/SunTimes

We can now construct a URL with query params to interact with our deployed API. Here’s an example that gets the sun times for Seattle, WA on 2019-01-01:

$ curl "https://us-central1-<PROJECT_ID>.cloudfunctions.net/SunTimes?lat=47.6&lon=122.3&date=2019-01-01"
{
    "sunset":"2019-01-01T08:08:47Z",
    "sunrise":"2018-12-31T23:38:58Z"
}

We can now get the sun rise/set times via an HTTP endpoint. Neat! 👍

Clean up

There are a few more subcommands of gcloud functions that are useful as you’re developing functions.

• gcloud functions list - 📃 Lists your currently deployed functions.
• gcloud functions describe <NAME> - 🤔 Prints out information about the chosen function. This is a useful way to find your function’s endpoint URL via the CLI.
• gcloud functions delete <NAME> - 🗑 Undeploys and deletes the chosen function.

To clean up, let’s delete the 2 functions we created:

gcloud functions delete HelloWorld
gcloud functions delete SunTimes

Designing for Serverless Execution

When working in a serverless environment, there are a couple of environmental limitations one should be aware of. It helps to have a mental model of what a deployed serverless function looks like.

To clarify terms, an invocation of a function is the process of running that function on a given request. An instance of a function is the actual worker container that services the function’s requests. For most purposes, you can consider a deployed function to behave like an ephemeral running instance of your code that may (or may not) be reused between invocations and may (or may not) have several instances running in parallel.

This leads to the following design constraints:

• 🔁 Functions should be (internally) stateless - It’s fine to depend on the state of an external database, but avoid using “local global” state as the runtime does not give you a guarantee of which instance of a function any particular invocation will run on.
• 🐛 Beware of global variables and memory leaks - Cloud Functions have a hard upper limit on the amount of amount of memory they can consume. Function instances may be reused between requests, so any memory that is allocated (but not released) between requests will continue to be allocated until the function runs out of memory. OOM errors can be difficult to debug, so its best to limit the amount of allocated memory that lives outside the scope of a request.
• 💾 Clean up any temporary data - If you make any calls to the file system, be sure to clean up. Cloud Functions use an in-memory filesystem, so failing to clean up will eventually result in OOMs.

Non-HTTP Triggers

Up to this point, we’ve focused only on triggering Cloud Functions via an HTTP endpoint. While this covers a large variety of use cases, there are other triggering mechanisms that can be useful.

Similar to AWS Lambda, Cloud Functions can be called in response to other events in the cloud. Of particular note are:

• Cloud Storage triggers - For example, you could watch a storage bucket for new video files and have a Cloud Function re-encode new files into a separate bucket.
• Cloud Pub/Sub Triggers triggers - For example, you could create a data ingest pipeline by publishing data to a Cloud Pub/Sub topic and have a Cloud Function do some transformation on that data, then either store the output in a database, or publish it to a separate Pub/Sub topic.

Here is a comprehensive list of all the supported trigger types.

Conclusion

To wrap up, we’ve seen how to write, deploy, and trigger Cloud Functions written in Go.

From the experience I’ve had so far with Go Cloud functions, I’ve been impressed with how easy it has been to drop-in existing Go service code into a serverless environment. I was particularly impressed with the ability to use existing HTTP handlers as Cloud Functions, which makes the process of migrating services relatively simple. I’ve also been quite happy with how straight-forward dependency management has been.

I hope this post encourages you to explore using Go for serverless functions. Both AWS and GCP now support Go functions, and it looks like Azure Function support is on the horizon. 😄👍

Code examples from this post can be found here in Github repo form.

Appendix A: Pricing

Price is an important consideration when picking a serverless function platform. All 3 primary serverless providers have pricing structures that are fairly similar: they all bill by charging a flat fee per function invocation with additional charges for memory consumption and function compute time.

All three major serverless platforms offer a generous pricing tier. As of 1/17/19, here’s how the free tiers stack up:

Appendix B: Resource Customization

Memory

Most serverless providers have a pricing model that is proportional to the amount of memory consumed multiplied by the time the function takes to execute.

For AWS Lambda and GCP Cloud Functions, you can explicitly set a maximum amount of memory. Azure Functions have a cap of 1.5GB, and dynamically price based on how much memory the function requests. For Lambda and CF, I’ve listed also the number of options for memory limit each platform provides.

Maximum Duration

Since serverless functions are intended to be ephemeral, most platforms place upper limits on the runtime of a function. This prevents you from using a function as a de facto VM. Depending on your use-case, however it may be useful to have a function run for several minutes (for example, if you are encoding media).

Local Storage

Most serverless platforms also afford a small amount of temporary filesystem space for a function to use. AWS Lambda and Azure Functions both provide a small (~500MB) amount of disk space. GCP Cloud Functions has an in-memory filesystem that functions can use – and thus any data written to the filesystem counts against the memory limit.