Golang How to start using gRPC in devcontainer

gRPC is one of the nice tools when two applications need to be communicated with each other. It’s not hard to implement it for simple function calls. Let’s see how we can start gRPC.

You can check the complete code in my GitHub repository.

Create devcontainer for Golang

I created the Dockerfile for Golang.

FROM mcr.microsoft.com/devcontainers/go:0-1.20

# To format proto file with vscode-proto3 extension
RUN apt update && apt install clang-format --yes

# Install the compiler for protocol buffer
RUN curl -LO "https://github.com/protocolbuffers/protobuf/releases/download/v22.3/protoc-22.3-linux-x86_64.zip" && \
    unzip protoc-22.3-linux-x86_64.zip -d /usr/local && \
    rm protoc-22.3-linux-x86_64.zip

USER vscode

RUN go install mvdan.cc/gofumpt@v0.5.0 && \
    curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.52.1 && \
    # To generate gRPC files from proto for Golang
    go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28 && \
    go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2 && \
    export PATH="$PATH:$(go env GOPATH)/bin"

We need to define proto file but the formatter is not installed by default. We have to add a formatter. clang-format is used here because an extension called vscode-proto3 requires it.

The way how to install gRPC related things for Golang is actually described in the Quick Start page in the official site.

Create some Make commands

We need to regenerate files whenever we change something in proto file. We don’t want to type the long command every time. Let’s add shortcuts for them.

The directory structure looks as follows.

.
├── languages
│   ├── dart
│   └── go
│       ├── cmd
│       │   ├── client
│       │   │   └── main.go
│       │   └── server
│       │       └── main.go
│       ├── go.mod
│       ├── go.sum
│       ├── internal
│       │   ├── client
│       │   │   ├── client.go
│       │   │   └── resources
│       │   │       └── data.txt
│       │   ├── common
│       │   │   └── rpc.go
│       │   ├── proto <-- where auto generated files exist
│       │   │   ├── middle_grpc.pb.go
│       │   │   └── middle.pb.go
│       │   └── server
│       │       ├── dataupdater.go
│       │       ├── resources
│       │       │   ├── from_client
│       │       │   │   └── data.txt
│       │       │   └── test_file.txt
│       │       ├── server.go
│       │       └── userinput.go
│       └── Makefile
├── license
├── proto
│   └── middle.proto
└── README.md

Makefile exists in languages/go because I want to implement it in another language too. Proto files exist in proto directory.

The following Makefile can be used when the current directory is languages/go.

.PHONY: generate runServer runClient lint

generate:
    @protoc --go_out=./internal/proto --go_opt=paths=source_relative \
    --go-grpc_out=./internal/proto --go-grpc_opt=paths=source_relative \
    --proto_path=../../proto middle.proto

runServer:
    @go run ./cmd/server/main.go

runClient:
    @go run ./cmd/client/main.go

lint:
    @golangci-lint run

By defining generate command here, we can generate files from proto file by a simple command make generate. That’s useful.

Define a function without parameter in protobuf (Unary RPC)

Let’s define our first function in the proto file. Once we define the function in a proto file, the same function can be used in several languages. Our first function is ping.

// middle.proto
syntax = "proto3";

import "google/protobuf/timestamp.proto";

option go_package = "api-test/grpc/apitest";

service Middle {
  // Unary RPC
  rpc Ping(PingRequest) returns (PingResponse) {}
}

message PingRequest {}

message PingResponse {
  google.protobuf.Timestamp timestamp = 1;
}

What we need here is as follows.

Service name
Function name
Function parameters
Function return parameters

Service is a kind of class. Functions must be defined in service.

The function starts with rpc followed by the name, parameters, and return parameters. The parameter has to be defined with message keyword. It must be defined even though the function doesn't require any parameters.

The return parameter is timestamp here but of course, we can define whatever we want. The timestamp type can’t be used by default, so we need to import "google/protobuf/timestamp.proto".

Implement Ping for server side

gRPC is a middleman who does something under the hood between a server and a client. However, we have to implement our own behavior in our code for our business.

Let’s generate gRPC file by the following command that we defined above.

make generate

Then, the following two files are generated.

proto
├── middle_grpc.pb.go
└── middle.pb.go

The following code is auto generated.

//middle_grpc.pb.go

// MiddleServer is the server API for Middle service.
// All implementations must embed UnimplementedMiddleServer
// for forward compatibility
type MiddleServer interface {
    // Unary RPC
    Ping(context.Context, *PingRequest) (*PingResponse, error)
}

So, we have to implement it in our own code.

package server

import (
    "context"

    rpc "apitest/internal/proto"

    "google.golang.org/protobuf/types/known/timestamppb"
)

type GrpcCallHandler struct {
    rpc.UnimplementedMiddleServer
}

func NewGrpcCallHandler() *GrpcCallHandler {
    return &GrpcCallHandler{ }
}

func (s *GrpcCallHandler) Ping(ctx context.Context, req *rpc.PingRequest) (*rpc.PingResponse, error) {
    response := &rpc.PingResponse{
        Timestamp: timestamppb.Now(),
    }

    return response, nil
}

I wrapped the server interface because it’s often necessary to use other objects too. The ping function needs to return a timestamp. Therefore, the current time is assigned in the response object. It’s straightforward, isn’t it?

Run the server and register the gRPC functions

Once we implement the behavior on the server side, we have to run the server with the defined functions.

The implementation is quite simple. Look at the code below.

package main

import (
    "log"
    "net"
    "os"
    "os/signal"

    rpc "apitest/internal/proto"
    pb "apitest/internal/server"
    "google.golang.org/grpc"
)

func main() {
    port := ":8080"

    listener, err := net.Listen("tcp", port)
    if err != nil {
        log.Fatal(err)
    }

    server := grpc.NewServer()

    rpc.RegisterMiddleServer(server, pb.NewGrpcCallHandler())

    go func() {
        log.Println("start gRPC server")
        server.Serve(listener)
    }()

    // exit by ctrl + c
    quit := make(chan os.Signal, 1)
    signal.Notify(quit, os.Interrupt)
    <-quit

    log.Println("stopping gRPC server...")
    server.GracefulStop()
    log.Println("gRPC server stopped")
}

To listen on an address with a port number, we use net package. The second parameter of net.Listen() function is an address but it’s enough to specify a port because it runs on the server. It is localhost or 127.0.0.1. If we want to assign a random port number, it should be an empty string, “127.0.0.1:”, or “127.0.0.1:0”. This address becomes an endpoint.

Then, we establish a gRPC server with the specified behavior. The functions are defined in GrpcCallHandler, it must be registered with RegisterMiddleServer() function. To start the server, server.Serve() is called with listener that we defined above.

server := grpc.NewServer()

rpc.RegisterMiddleServer(server, pb.NewGrpcCallHandler())

go func() {
  log.Println("start gRPC server")
  server.Serve(listener)
}()

Go routine is not necessarily used if we don’t need to do another thing. The server.Serve() is a blocking call and thus it blocks the thread until it runs into an error.

Call Ping function from a client

What we need to do to call a gRPC function from a client-side is basically set the request parameter and call the function by using a connected session. It’s simple but we will implement it in a struct to make it handy.

As we saw in a server implementation, we define a new struct and implement client code in it.

package client

import (
    "context"
    "log"
    "time"

    rpc "apitest/internal/proto"

    "google.golang.org/grpc"
)

const (
    resourcePath = "./internal/client/resources/"
)

type MiddleMan struct {
    conn *grpc.ClientConn
}

func NewMiddleMan(conn *grpc.ClientConn) *MiddleMan {
    return &MiddleMan{
        conn: conn,
    }
}

func (m *MiddleMan) SendPing(ctx context.Context) {
    timeoutCtx, cancel := context.WithTimeout(ctx, time.Second)
    defer cancel()

    client := rpc.NewMiddleClient(m.conn)
    res, err := client.Ping(timeoutCtx, &rpc.PingRequest{})
    if err != nil {
        log.Printf("[ERROR] filed to ping: %v\n", err)
        return
    }

    log.Printf("Time: %s\n", res.GetTimestamp().AsTime().Format(time.RFC3339Nano))
}

Once it’s instantiated, the target function can be easily called. I highly recommend that you should specify the timeout. gRPC tries to connect until TCP tells that it fails.

gRPC Go: If the connection drops after sending a request, it will fail after TCP gives up retrying the request, which is about 15 minutes.
https://www.evanjones.ca/tcp-connection-timeouts.html

I actually ran into a connection problem that occurred only in a specific environment. I implemented reconnection logic for gRPC. I disconnected/connected a LAN cable between a gRPC server and a client. The reconnection logic worked if the client application was running on my laptop. However, the reconnection logic didn’t work well when the client application was running in the production environment. The connection was re-established after 15 minutes. To avoid this error, we should specify a reasonable timeout duration. The duration depends on the application.

Okay. After calling the function, we need to extract the result. The response has a timestamp property. We can directly access res.Timestamp but all properties have a getter function GetXxxxx(). Let’s check the implementation.

func (x *PingResponse) GetTimestamp() *timestamppb.Timestamp {
    if x != nil {
        return x.Timestamp
    }
    return nil
}

In some cases depending on the application, the property is not set. The property becomes nil in this case. Using a getter method is useful to avoid nil pointer error. We have to check if the value is not nil if the chained function expects a non-nil value. However, if the property is defined as a message it has some properties. In this case, it can be chained in the following way without a nil check.

response.GetItem().GetProduct().GetName()

If GetItem() is nil, the result is nil because both GetProduct() and GetName() return nil for nil value. If it’s a non-nil value, it returns a proper name. We can access the property without a nil check. That’s a nice point.

How to connect to a gRPC server from a client

Last but at least, we have to connect to a gRPC server. The code is as follows.

package main

import (
    "context"
    "log"
    "os"
    "os/signal"
    "time"

    "apitest/internal/client"

    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials/insecure"
)

const (
    serverHost = "localhost:8080"
)

func main() {
    grpcConn, err := grpc.Dial(serverHost, grpc.WithTransportCredentials(insecure.NewCredentials()))
    if err != nil {
        log.Fatalf("did not connect: %v", err)
    }

    defer grpcConn.Close()

    middleMan := client.NewMiddleMan(grpcConn)

    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()

    go func() {
        for i := 0; ; i++ {
            middleMan.SendPing(ctx)
            time.Sleep(time.Second)
        }
    }()

    // exit by ctrl + c
    quit := make(chan os.Signal, 1)
    signal.Notify(quit, os.Interrupt)
    <-quit
    log.Println("exit")
}

gRPC provides Dial() function to connect to a gRPC server. The second or later parameter can be used for options. There are many options but we use only one for this example. Without the parameter, it throws the following error.

grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials)

defer grpcConn.Close() should be written right after the error check to release the resources correctly.

Then, create a parent context passed to all the function call. Likewise, cancel() function should be written right after the creation if possible.

ctx, cancel := context.WithCancel(context.Background())
defer cancel()

By passing this context to the function call, we can easily stop all function processes without waiting for the result. This is useful for example when we want to update the client application without waiting for a long time to get the function result.

Function call test

We are ready to test gRPC. Run the server first.

$ make runServer 
2023/05/23 04:36:55 start gRPC server

Then, run the client. The ping is sent every second and the program ends when pressing ctrl + c.

$ make runClient
2023/05/23 04:48:42 Time: 2023-05-23T04:48:42.205676232Z
2023/05/23 04:48:43 Time: 2023-05-23T04:48:43.207411232Z
2023/05/23 04:48:44 Time: 2023-05-23T04:48:44.209415433Z
2023/05/23 04:48:45 Time: 2023-05-23T04:48:45.211344784Z
2023/05/23 04:48:46 Time: 2023-05-23T04:48:46.213832577Z
2023/05/23 04:48:47 Time: 2023-05-23T04:48:47.215616084Z
^C2023/05/23 04:48:47 exit
make: *** [Makefile:12: runClient] Error 1

Learn more about gRPC functions

We are ready to start gRPC in devcontainer. Let’s learn it more in this post.