# Getting Started with Golang Google Cloud Functions As of January 16, 2019, GCP’s Cloud Functions now [natively support Golang](https://cloud.google.com/blog/products/application-development/cloud-functions-go-1-11-is-now-a-supported-language)! This post will briefly cover why this is a big deal and how one can begin to use Go in their cloud functions. First, what are [Cloud Functions](https://cloud.google.com/functions/)? Essentially, Cloud Functions are analogous to [AWS Lambda](https://aws.amazon.com/lambda/) and [Azure Functions](https://azure.microsoft.com/en-us/services/functions/). It’s a serverless and event-driven compute environment. The runtime manages deployment and resource scaling, so you can focus on application logic. Similar to AWS Lambda, Cloud Functions are billed by invocations, memory/compute usage, and network bandwidth. For small-scale web apps, this tends to be cheaper than serving a traditional “full-time” web service. **Table of Contents** - [Why Go?](#why-go) - [Getting Started: Setting up a GCP Project](#getting-started-setting-up-a-gcp-project) - ["Hello, World!" as a Service](#%22hello-world%22-as-a-service) - [Designing for Serverless Execution](#designing-for-serverless-execution) - [Non-HTTP Triggers](#non-http-triggers) - [Conclusion](#conclusion) - [Appendix A: Pricing](#appendix-a-pricing) - [Appendix B: Resource Customization](#appendix-b-resource-customization) - [Memory](#memory) - [Maximum Duration](#maximum-duration) - [Local Storage](#local-storage) ## Why Go? Node.js has long been the _de facto_ language for writing serverless functions. However, now that both AWS and GCP support Go as a serverless runtime I predict that we'll see a shift towards using Go in this domain. - **Speed** - The Go community values speed. For computationally intensive serverless functions, Go might make more sense than Node.[^1] - **Language Features** - Using Go gives you the benefits of a static type system and Go's excellent concurrency support. Additionally, Go's standard library has many helpful utilities for writing HTTP services out-of-the-box. This makes it easier to port existing Go web services to be serverless endpoints. - **Ecosystem** - Go has a robust package ecosystem. Common serverless tasks like media encoding, numeric computation, and database access are all possible via a community-supported library. There's a common joke that Go is a language entirely designed around making API gateways. While this is facetious, there's a kernel of truth: Go is _ergonomic_ for writing HTTP services. One can write a fully-functional HTTP server in just a [few lines](https://golang.org/pkg/net/http/) of Go. Fortunately, Cloud Functions takes full advantage of the interfaces exposed by the Go `http` module, allowing for rapid development of Go serverless endpoints. ## Getting Started: Setting up a GCP Project For our first Cloud Function, we'll write and deploy a minimal function that responds "Hello, World!" to any incoming HTTP request. 1. Create a new GCP project from the [Project Management](https://console.cloud.google.com/cloud-resource-manager) page. It's fine to reuse an existing project if you already have one. ![](create_project.png) 1. Go to the [Cloud Functions console](https://console.cloud.google.com/functions/list). You may have to enable billing on your account. 1. Enable the Cloud Functions API. ![](enable_api.png) 1. Install the `gcloud` CLI tool from [here](https://cloud.google.com/sdk/docs/). You can perform most of the steps of this tutorial using just the web interface, but it will be quicker to do most of the deployment steps from a CLI tool. 1. Run the following to get the latest version of the CLI, and install the Cloud Functions beta (which is needed to use Go). **Note:** You should do this even if you already have `gcloud` installed. ``` gcloud components update gcloud components install beta ``` 1. Now, we'll point the `gcloud` client at our project. Set the active project to be the one you created in Step 1 with the following command: ``` gcloud config set project ``` 1. Now, run `gcloud functions list` to test that your `gcloud` client is authenticated correctly. If the response is `Listed 0 items.`, then everything is working correctly. ## "Hello, World!" as a Service Let's now write our "Hello, World" Cloud Function. 1. Start by creating and entering into a directory for the project. ``` mkdir cloud-functions-golang cd cloud-functions-golang ``` 1. Create a file named `helloworld.go` with the following contents. ```golang package helloworld import ( "fmt" "net/http" ) func HelloWorld(w http.ResponseWriter, r *http.Request) { fmt.Fprint(w, "Hello, World!") } ``` If you've written web services in Go before, you'll recognize that `HelloWorld` has the same type as [HandlerFunc](https://golang.org/pkg/net/http/#HandlerFunc). In effect, we are writing a normal Go HTTP endpoint -- the only difference being that our endpoint is managed and routed by Cloud Functions instead of by our own server package. Our Go function `HelloWorld` will be routed with each request made to the Cloud Function. Just like a normal Go web endpoint, we can send a response to an incoming request by writing to the `http.ResponseWriter`. 1. Now, we can create and deploy our function using the following command. ``` gcloud functions deploy HelloWorld --runtime go111 --trigger-http ``` Note that we're specifying the `go111` (Go 11.1) runtime, and we're setting the function's trigger to be HTTP. You should see something like the following as output: ``` Deploying function (may take a while - up to 2 minutes)…done. availableMemoryMb: 256 entryPoint: HelloWorld httpsTrigger: url: https://us-central1-.cloudfunctions.net/HelloWorld labels: deployment-tool: cli-gcloud … ``` 1. If the deployment was successful, you should see a URL under `httpsTrigger > url`. This URL points to your newly created Cloud Function and works like an endpoint for a "normal" web service. Open this URL in a browser. You should see `Hello, World!` as the response! 🤩 ## A More Complex Example For a more "realistic" example of how one might use Cloud Functions, we'll implement a basic version of the [sunrise-sunset.org API](https://sunrise-sunset.org/api) (which provides sunrise/sunset times for a given date and location). We'll be using [schema](https://github.com/gorilla/schema) to parse query string parameters and [go-sunrise](https://github.com/nathan-osman/go-sunrise) to get the sunrise/sunset time information. ### Declaring Dependencies For any non-trivial web service, we'll need to depend on existing Go libraries. Cloud Functions uses [go modules](https://github.com/golang/go/wiki/Modules) to define application dependencies. To declare the dependencies we need for our API, we'll first initialize a `mod.go` file in our project directory with `go mod init`, and then `go get` our dependencies. ``` go mod init mydomain.me/helloworld go get github.com/nathan-osman/go-sunrise github.com/gorilla/schema ``` Running `cat mod.go` should yield something like this: ``` module mydomain.me/helloworld require ( github.com/gorilla/schema v1.0.2 github.com/nathan-osman/go-sunrise v0.0.0-20171121204956-7c449e7c690b ) ``` Note that upon deployment, Cloud Functions uploads your _application code_ (and not a compiled binary) to a build server, and fetches dependencies remotely. Thus, you should not rely on any local dependencies that won't be accessible by the Cloud Functions build server. ### Implementing a Sunset/Sunrise API 🌞 Now, we can write our implementation for the Sunset/Sunrise API. Since we declared our dependencies in `go.mod`, we can include packages normally. Note that the `http.Request` we are passed in our handler function is mapped to the HTTP request that the Cloud Functions endpoint receives, so we're able to do query string decoding just as if we were running the server ourselves. Create a file called `suntimes.go` and add the following contents: ```golang package helloworld import ( "encoding/json" "fmt" "log" "net/http" "reflect" "time" "github.com/gorilla/schema" "github.com/nathan-osman/go-sunrise" ) type Request struct { Lat float64 `schema:"lat,required"` Lon float64 `schema:"lon,required"` Date time.Time `schema:"date,required"` } type Response struct { Sunset time.Time `json:"sunset"` Sunrise time.Time `json:"sunrise"` } func SunTimes(w http.ResponseWriter, r *http.Request) { var decoder = schema.NewDecoder() decoder.RegisterConverter(time.Time{}, dateConverter) // Parse the request from query string var req Request if err := decoder.Decode(&req, r.URL.Query()); err != nil { // Report any parsing errors w.WriteHeader(http.StatusUnprocessableEntity) fmt.Fprintf(w, "Error: %s", err) return } // Perform sunrise/sunset calculation sunrise, sunset := sunrise.SunriseSunset( req.Lat, req.Lon, req.Date.Year(), req.Date.Month(), req.Date.Day(), ) // Send response back to client as JSON w.WriteHeader(http.StatusOK) response := Response{sunset, sunrise} if err := json.NewEncoder(w).Encode(&response); err != nil { panic(err) } } func dateConverter(value string) reflect.Value { s, _ := time.Parse("2006-01-_2", value) return reflect.ValueOf(s) } ``` Don't worry if you don't understand everything here. 🙂 All this function does is take a request (which comes in as [query parameters](https://en.wikipedia.org/wiki/Query_string) on the request URL), performs a sunrise/sunset lookup, and returns a JSON-encoded response. We'll deploy our new function using a similar command to the `HelloWorld` example. ``` gcloud functions deploy SunTimes --runtime go111 --trigger-http ``` Again, take note of the deployed function's URL. It should be of the form: https://us-central1-.cloudfunctions.net/SunTimes We can now construct a URL with query params to interact with our deployed API. Here's an example that gets the sun times for Seattle, WA on 2019-01-01: ```sh $ curl "https://us-central1-.cloudfunctions.net/SunTimes?lat=47.6&lon=122.3&date=2019-01-01" { "sunset":"2019-01-01T08:08:47Z", "sunrise":"2018-12-31T23:38:58Z" } ``` We can now get the sun rise/set times via an HTTP endpoint. Neat! 👍 ### Clean up There are a few more subcommands of `gcloud functions` that are useful as you're developing functions. - `gcloud functions list` - 📃 Lists your currently deployed functions. - `gcloud functions describe ` - 🤔 Prints out information about the chosen function. This is a useful way to find your function's endpoint URL via the CLI. - `gcloud functions delete ` - 🗑 Undeploys and deletes the chosen function. To clean up, let's delete the 2 functions we created: ``` gcloud functions delete HelloWorld gcloud functions delete SunTimes ``` ## Designing for Serverless Execution When working in a serverless environment, there are a couple of environmental limitations one should be aware of. It helps to have a mental model of what a deployed serverless function looks like. To clarify terms, an _invocation_ of a function is the process of running that function on a given request. An _instance_ of a function is the actual worker container that services the function's requests. For most purposes, you can consider a deployed function to behave like an ephemeral running _instance_ of your code that may (or may not) be reused between _invocations_ and may (or may not) have several _instances_ running in parallel. This leads to the following design constraints: - 🔁 **Functions should be (internally) stateless** - It's fine to depend on the state of an external database, but avoid using "local global" state as the runtime does not give you a guarantee of which instance of a function any particular invocation will run on. - 🐛 **Beware of global variables and memory leaks** - Cloud Functions have a hard upper limit on the amount of amount of memory they can consume. Function instances may be reused between requests, so any memory that is allocated (but not released) between requests will continue to be allocated until the function runs out of memory. OOM errors can be difficult to debug, so its best to limit the amount of allocated memory that lives outside the scope of a request. - 💾 **Clean up any temporary data** - If you make any calls to the file system, be sure to clean up. Cloud Functions use an in-memory filesystem, so failing to clean up will eventually result in OOMs. ## Non-HTTP Triggers Up to this point, we've focused only on triggering Cloud Functions via an HTTP endpoint. While this covers a large variety of use cases, there are other triggering mechanisms that can be useful. Similar to AWS Lambda, Cloud Functions can be called in response to other events in the cloud. Of particular note are: - **[Cloud Storage](https://cloud.google.com/functions/docs/calling/storage) triggers** - For example, you could watch a storage bucket for new video files and have a Cloud Function re-encode new files into a separate bucket. - **[Cloud Pub/Sub Triggers](https://cloud.google.com/functions/docs/calling/pubsub) triggers** - For example, you could create a data ingest pipeline by publishing data to a Cloud Pub/Sub topic and have a Cloud Function do some transformation on that data, then either store the output in a database, or publish it to a separate Pub/Sub topic. [Here](https://cloud.google.com/functions/docs/calling/) is a comprehensive list of all the supported trigger types. ## Conclusion To wrap up, we've seen how to write, deploy, and trigger Cloud Functions written in Go. From the experience I've had so far with Go Cloud functions, I've been impressed with how easy it has been to drop-in existing Go service code into a serverless environment. I was particularly impressed with the ability to use existing HTTP handlers as Cloud Functions, which makes the process of migrating services relatively simple. I've also been quite happy with how straight-forward dependency management has been. I hope this post encourages you to explore using Go for serverless functions. Both AWS and GCP now support Go functions, and it looks like Azure Function support is [on the horizon](https://github.com/Azure/azure-sdk-for-go/issues/894). 😄👍 _Code examples from this post can be found [here](https://github.com/bcongdon/getting-started-with-golang-cloud-functions) in Github repo form._ --- ### Appendix A: Pricing Price is an important consideration when picking a serverless function platform. All 3 primary serverless providers have pricing structures that are fairly similar: they all bill by charging a flat fee per function invocation with additional charges for memory consumption and function compute time. All three major serverless platforms offer a generous pricing tier. As of 1/17/19, here’s how the free tiers stack up: | Platform | Invocations | Compute Time (GB-s) | | ------------------- | ----------- | ------------------- | | Lambda[^5] | 1M | 400k | | Cloud Functions[^3] | 2M | 1,000k | | Azure Functions[^6] | 1M | 400k | ### Appendix B: Resource Customization #### Memory Most serverless providers have a pricing model that is proportional to the amount of memory consumed multiplied by the time the function takes to execute. For AWS Lambda and GCP Cloud Functions, you can explicitly set a maximum amount of memory. Azure Functions have a cap of 1.5GB, and dynamically price based on how much memory the function requests. For Lambda and CF, I've listed also the number of options for memory limit each platform provides. | Platform | Minimum | Maximum | # of Gradations | | ------------------- | ------- | ------- | --------------- | | Lambda[^5] | 128MB | 3008MB | 46 | | Cloud Functions[^3] | 128MB | 2018MB | 5 | | Azure Functions[^6] | 128MB | 1536MB | Dynamic | #### Maximum Duration Since serverless functions are intended to be ephemeral, most platforms place upper limits on the runtime of a function. This prevents you from using a function as a _de facto_ VM. Depending on your use-case, however it may be useful to have a function run for several minutes (for example, if you are encoding media). | Platform | Maximum | Default | | ------------------- | ------- | ------- | | Lambda[^7] | 15min | 3sec | | Cloud Functions[^4] | 9min | 1min | | Azure Functions[^8] | 10min | 5min | #### Local Storage Most serverless platforms also afford a small amount of temporary filesystem space for a function to use. AWS Lambda and Azure Functions both provide a small (\~500MB) amount of disk space. GCP Cloud Functions has an in-memory filesystem that functions can use -- and thus any data written to the filesystem counts against the memory limit. | Platform | Maximum | | ------------------- | ----------------------- | | Lambda[^7] | 500MB storage in `/tmp` | | Cloud Functions[^4] | In-memory filesystem | | Azure Functions | 500MB storage[^2] | [^1]: Python, with its excellent numerical computation libraries, would also be a strong contender -- if your use-case supports it. [^2]: This spec isn't advertised readily. [This](https://github.com/projectkudu/kudu/wiki/Understanding-the-Azure-App-Service-file-system) was the only close-to-authoritative information I could find about temporary storage for Azure functions. [^3]: [GCP Cloud Functions Pricing](https://cloud.google.com/functions/pricing-summary/) [^4]: [GCP Cloud Functions Execution Environment](https://cloud.google.com/functions/docs/concepts/exec#timeout) [^5]: [AWS Lambda Pricing](https://aws.amazon.com/lambda/pricing/) [^6]: [Azure Functions Pricing](https://azure.microsoft.com/en-us/pricing/details/functions/) [^7]: [AWS Lambda Limits](https://docs.aws.amazon.com/lambda/latest/dg/limits.html) [^8]: [Azure Functions scale and hosting](https://docs.microsoft.com/en-us/azure/azure-functions/functions-scale)