Deploying a Go web API on Kubernetes, part 1: The API part - Hacking Away

Deploying a Go web API on Kubernetes, part 1: The API part

Introduction

After having done some experiments with the Go programming, I really started to like the language, and decided a bold experiment: build a small web API, backed by a database and deploy the whole thing on a local kubernetes cluster.

Purpose

The purpose of this API is to store events. Events have a location, a description, a location and some sort of location. As you can see it is very simple. The source for this project, can be found on Github.

Prerequisites

For you to follow along this series you need the following:

• The Go compiler, which can be obtained here.
• An IDE. For this I usually use Visual Studio Code, although most of this small project I have done using Jetbrain’s excellent Goland IDE.
• The Docker desktop, which can be downloaded here. Make sure your machine is big enough.
• A docker account, because you will need to push images to the hub
• A local kubernetes installation, I tend to favour minikube.
• An installation of Postgres, also make sure you install PgAdmin because that will make life much easier for you
• Postman is also a valuable when you’re testing an API

Also some basic knowledge of Go (preferably with some knowledge of Gin and Gorm), Docker and Kubernetes would be nice.

When you have installed Postgres and pgAdmin create a database called ‘webevents’, where we will store our data

But no more talk now, let’s get started

Getting Started

Open your terminal or IDE in an empty directory where you want to build this project and type:

go mod init github.com/eventsweb

Now we can install Gin:

go get -u github.com/gin-gonic/gin

Now we can install GORM, our Object Relational Mapping framework, which makes working with the database very easy:

go get -u gorm.io/gorm
go get -u gorm.io/driver/postgres

The second line is needed to install the Postgres drivers.

The Model

Create a models folder, and in that a file called ‘event.go’:

package models

import "time"

type WebEvent struct {
	Id          uint `gorm:primaryKey,autoIncrement`
	Title       string
	Description string
	Location    string
	StartDate   *time.Time
	EndDate     *time.Time
}

A short explanation

• The WebEvent struct is part of the models package
• The Id field has two special annotations: primaryKey (because it is the primaryKey), and autoIncrement (that means each time a WebEvent is created, it gets a unique id)
• The rest of the fields are quite self-explanatory.

The database functions

Create a db directory, and in that directory create a file ‘db.go’. In this file we will add some database functions. Let us start with the preliminaries:

package db

import (
	"fmt"
	"gorm.io/driver/postgres"
	"gorm.io/gorm"
	"os"
)

Now we need a small utility to read environment variables, since both locally and later on in the Kubernetes cluster we will get data like the database host from the environment:

func getEnvironmentVariableWithDefault(key string, defaultValue string) string {
	currentValue := os.Getenv(key)
	if currentValue == "" {
		return defaultValue
	} else {
		return currentValue
	}
}

All this does, is request an environment variable, and if this is not available, it returns some default value.

Now we some way of constructing the ‘connectionstring’ for the database:

func ConstructDsn() string {
	host := getEnvironmentVariableWithDefault("host", "localhost")
	user := getEnvironmentVariableWithDefault("user", "postgres")
	password := getEnvironmentVariableWithDefault("password", "Secret")
	dbname := getEnvironmentVariableWithDefault("dbname", "webevents")
	port := getEnvironmentVariableWithDefault("port", "5432")

	dsn := fmt.Sprintf("host=%s user=%s password=%s dbname=%s port=%s", host, user, password, dbname, port)

	return dsn
}

This is simply matter of collecting the important values from the environment and stringing it together.

Now can build the connection:

func InitializeDatabaseConnection() (*gorm.DB, error) {

	dsn := ConstructDsn()
	db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
	//if err != nil {
	//		panic("Failed to connect to database")
	//}
	return db, err
}

Again straightforward:

1. We construct the Dsn
2. Then try to open the connection
3. Then return the connection and an error value.

Why not handle the error in the function proper? Because it is better if it is passed up so to say, so that logging logic can handle it, or a message can be shown to the user, which is not possible from this function.

The DbConnection Struct

In the same ‘db’ directory, add a ‘dbconnection.go’ file. First as usual the preliminaries:

package db

import (
	"eventsWeb/models"
	"fmt"
	"gorm.io/gorm"
	"sync"
)

Now we can define the struct:

type DbConnection struct {
	Connection *gorm.DB
}

All this struct have is a gorm.DB object which is a database connection.

Now we define three global variables:

var (
	connectionInstance *DbConnection
	once               sync.Once
	globalError        error
)

Since we want to make the connection a singleton, we define:

1. connectionInstance, which is the only connection instance
2. once of type sync.Once , a struct which makes threadsafe singletons possible.
3. The variable globalError is needed because once.Do does not return any value, and we still want to pass on errors.

Now we can define the GetConnection() function:

func GetConnection() (*DbConnection, error) {
	globalError = nil

	once.Do(func() {
		dbConnection, dbErr := InitializeDatabaseConnection()
		if dbErr != nil {
			globalError = dbErr
			connectionInstance = nil
			return
		}
		migrateError := dbConnection.AutoMigrate(&models.WebEvent{})
		if migrateError != nil {
			globalError = migrateError
			connectionInstance = nil
			return
		}
		connectionInstance = &DbConnection{
			Connection: dbConnection,
		}

	})
	return connectionInstance, globalError
}

Here a line by line explanation can help:

1. First we set the globalError to nil, after all, nothing has happened yet
2. once.Do is a function which is guaranteed to execute only once. It arguments is an anonymous function (hence the need for the globalError)
3. First we initialize, or try to, a databaseconnection. The globalError is set to the error, and the connectionInstance is set to nil, and we return from the anonymouse. (Mind you: this is from the inner anonymous function, not from GetConnection().
4. We do the same thing for the migrations.
5. If nothing went wrong we can instatiate the connectionInstance.
6. At the end of GetConnection() we can return the instance, and an error if there is one.

The API operations

In the main directory create a directory called ‘operations’, and in that create a file named ‘operations.go’. Again the preliminaries:

package operations

import (
	"eventsWeb/db"
	"eventsWeb/models"
	"github.com/gin-gonic/gin"
	"gorm.io/driver/postgres"
	"gorm.io/gorm"
	"net/http"
)

We need the definitions in ‘db’ and ‘models’, furthermore we need Gin to get some form of request context and to send back responses. And then of course we need Gorm to get at the database.

func ListEvents(c *gin.Context) {
	var webEvents []models.WebEvent
	db, err := db.GetConnection()
	if err != nil {
		c.JSON(http.StatusNotFound, err)
	}
	db.Connection.Find(&webEvents)
	c.JSON(http.StatusOK, webEvents)
}

Since this is more or less a template for the rest of the functions, we will do a line by line explanation:

1. First we define a slice of models.WebEvent, which is initially empty
2. Then we obtain, or try to obtain, a databaseconnection
3. If there is an error, we report a server error
4. Next we retrieve all the events from the database
5. Which we return as JSON.

Connection with this is the FindEvent method:

func FindEvent(c *gin.Context) {
	var webEvent models.WebEvent
	id := c.Param("id")
	db, err := db.GetConnection()
	if err != nil {
		c.JSON(http.StatusInternalServerError, err)
	}
	result := db.Connection.First(&webEvent, id)
	if result.Error != nil {
		c.JSON(http.StatusNotFound, nil)
	} else {
		c.JSON(http.StatusOK, webEvent)
	}
}

This follows more or less the same template as the previous function with some notable changes:

1. The id is a route parameter as we will see in main.go
2. The First method on the connection finds a record based on its primary key.

Before we can list or find anything, we must create:

func CreateEvent(c *gin.Context) {
	var webEvent models.WebEvent
	c.BindJSON(&webEvent)
	db, err := db.GetConnection()
	if err != nil {
		c.JSON(http.StatusInternalServerError, err)
	}
	db.Connection.Create(&webEvent)
	c.JSON(http.StatusCreated, webEvent)
}

Also more or less the same template, but with following change:

• The BindJSON binds the fields in a JSON object to the fields in a Go struct
• Create creates a record in the database.

To complete the CRUD operations, we can implement delete functionality:

func DeleteEvent(c *gin.Context) {
	id := c.Param("id")
	db, err := db.GetConnection()
	if err != nil {
		c.JSON(http.StatusInternalServerError, err)
	}
	result := db.Connection.Delete(&models.WebEvent{}, id)
	if result.Error != nil {
		c.JSON(http.StatusInternalServerError, nil)
	} else {
		c.JSON(http.StatusAccepted, gin.H{
			"message": "Deleted",
		})
	}
}

Again, after seeing the previous three functions, the code is self-explanatory

Putting it together

Create a ‘main.go’ file in the main directory. As usual we will start with the preliminaries:

package main

import (
	"eventsWeb/operations"
	"fmt"
	"github.com/gin-gonic/gin"
)

const version = "v1"
const groupName = "api"

Since we will want to version this api, and make clear that this is an api, I have included these constants. Their use will become apparent later.

Now the main function:

func main() {
	//now we can start serving

	router := gin.Default()
	group := router.Group(fmt.Sprintf("/%s/%s", groupName, version))
	{
		group.GET("/events", operations.ListEvents)
		group.GET("/event/:id", operations.FindEvent)
		group.POST("/create", operations.CreateEvent)
		group.DELETE("/delete/:id", operations.DeleteEvent)
	}

	router.Run()
}

Also here, some explanation is needed:

1. We start by spinning up a router, which will handle all the request.
2. Next we define a router group, basically, we define the prefix for all the requests
3. Then we define the operations using the HTTP verb, the route (with the route parameters as you can see) and the operation which belongs to the route.
4. After this is done, we run the router so our API is ready to receive requests.

Type:

go run . 

in the main directory, and try it out in Postman, first try create an entry using Postman, like this:

As you can see, there is no need to enter the Id, as it is autogenerated.

In a similar fashion you could get the events (http://localhost:8080/api/v1/events), find them and delete them.

Conclusion

Building a web api in Go was surprisingly easy, but I need to bear in mind that this is a very simplistic API.

Improvements could be:

1. Making it multithreaded
2. As a prerequisite to the previous point: build some sort of connection pooling

In the next part we will deploy both this API and the database in a single Kubernetes cluster.