Go (often referred to as GoLang, based on the original website: golang.org) has become one of the most popular programming languages, according to surveys performed by StackOverflow and open positions for Go Developers on LinkedIn (around 40000 in the United States alone). One of the main reasons for its growing popularity is that it offers the perfect starting point for developing high availability services.
At Ogury, we are developing an advertising engine that needs to be able to successfully respond to billions of requests on a daily basis from various integrations. Beside that, the services need to be very fast, often needing to respond in a tenth of a millisecond. Given this, choosing the right technology plays a critical role in our ability to meet these needs. This article provides insights into why we chose Go and how we leverage the language features.
Advantages of Go
We can use any programming language to develop different kinds of services. So why choose Go?
First, Go was designed to be simple but powerful, so it can be used for different kinds of modern software solutions. Syntax and semantics are pretty straightforward and therefore easy to learn (and fast). This simplicity and efficiency is one of the main pillars for development of high availability services. Maintenance is also a factor in choosing a language. Due to the structure of Go, it is also easier to maintain than other languages on the market.
Executable file, created by the Go compiler, is usually only a few megabytes, which is quite less in comparison with executable files created by other programming language compilers. Also compile time for very large projects is less than 30 seconds. Fast compile time is quite useful in situations when fast rebuild and redeployment is required. It also increases developer satisfaction since they do not have to waste time waiting for their code to be compiled.
It is quite easy and efficient to deploy a Go application. The process requires us to copy an executable file (and, in some cases, a local configuration file) to the server machine and run it. There is no need for any additional installations and dependencies which is not the case with other languages. For example, to successfully run a Java application we will need JVM (Java Virtual Machine), and this can increase resource costs of our software solution.
Additionally, one of the biggest advantages is the fact that Go is not object oriented. Without a large number of objects, memory consumption is less in comparison with other programming languages.
Go is heavily oriented to light threads, called goroutines, for better execution time. It is easy to create new goroutines, however, one must be careful as numerous routines can downgrade performance of a service.
One more thing that should be considered as a main advantage of Go is native support, through standard library, for server side applications. Also, with a fast growing community, a lot of well-developed and efficient third-party libraries (or packages if we want to use proper terms) are available. These libraries are useful “tools” for development of efficient services, and will be further discussed in this article.
Standard and third-party libraries
Most of the modern day services are based on REST principles and mainly use HTTP as a communication protocol. Go provides native support for this through the standard library.
The Standard library should be used whenever possible. It is bounded with language and comes with important guarantees:
These guarantees are enough for us to know that the standard library package will be efficient and appropriate for usage.
Efficient solutions can also be created with a combination of standard and third party libraries. There are no guarantees that some library will be relevant and maintained forever, therefore, it is important to explore and test potential improvements in advance. For example, if the library is popular, chances that it will be improved and maintained for a long period of time are higher.
It is important to note that Go supports (through standard or thirdparty libraries) other communication concepts like protocol buffers (gRPC) and message queuing protocols (Kafka and RabbitMQ). We use concepts that best suit our current requirements.
Monitoring and Alerting
In an ideal world, our application will work and be available 24/7, but as we all know, this is not always the case. Even if our solution has “repairing” mechanisms in place that can mitigate issues, errors can still occur.
Alerts must be triggered in cases when service stops working. For these cases, we can leverage a monitoring toolkit or out-of-the-box functionalities provided by deployment and management tools (like Kubernetes) to check the status of our service. Fun fact, Kubernetes source code is written in Go.
Solution
This topic is wide and we can go into small details for subjects described in this article and provide detailed solutions for each of them. We provided a couple of general guidelines and ideas.
Here at Ogury we are trying to follow them and invest time (when possible) into updating our services with the newest version of Go and libraries.
Selecting third party libraries with better performances
One of the most popular third party Go libraries for service applications is gin, HTTP web framework. According to the gin development team (official documentation), their solution is up to 40 times faster than similar solutions, which means better performance. As we all know better performance leads to faster response times and less problems.
Until last year we used the chi router. After we replaced chi with gin, we immediately saw 3-5% improvements in CPU and memory usage.
Go concurrency concepts for parallel inter-service communication
Often one service must communicate with multiple services. If we call on these services one by one, response time will increase. We can use the concurrency concepts of Go to call multiple services in parallel and significantly save time.
To be sure that responses from all services are received, we use the concept of WaitGroups before processing. WaitGroup will block execution until all related goroutines are completed. Basically we can observe it as a counter, with an initial value that corresponds with the number of related goroutines. When goroutine is completed, it will decrease the counter value. When the counter reaches zero, all goroutines have finished their tasks and “normal” execution flow can continue.
Adapter Pattern and Go
The entry point for our system is the service which should be able to handle requests from multiple sources. Besides our direct approach, sources for some of these requests are various header bidding integrations. In the flow, each integration has some specific parts, but also the common part which is the same for all of them. We are addressing this issue by using concepts of interfaces and adapter design patterns to implement this (Figure 1). However, we go one step further.
Figure 1 – Implementation of adapter pattern
If we had one singular service, for all integrations, we would burden the service with a lot of traffic. We observed each adapter as a different microservice, and by adding build tags, we built an adapter specific image and deployed it as an independent service. This way we divided traffic and reduced the load.
With different tags, we have a higher level of isolation for specific integrations. We can configure resources and scale up desired integrations separately. Integration with more traffic requires more resources and more running instances than the one with a low level of traffic. Additionally it is very convenient to find potential problems and errors with this implementation of the adapter pattern.
Monitoring with Go
Go supports (through third-party libraries) all popular monitoring toolkits. It is easy to integrate our service with a monitoring toolkit and use it to check metrics.
We use Prometheus, one of the most popular monitoring toolkits. With this, we can monitor standard metrics (like memory and CPU usage) or define custom ones. These metrics can be queried and visualised with Grafana (Figure 2):
Figure 2 – Metrics visualised with Grafana
In cases where a metric reaches a “problematic” value, we can trigger an alert and handle the problem as soon as possible. We use slack as the main channel of communication, so we have a dedicated alerting channel, where information, in the form of simple messages alert us in real-time.
Conclusion
As we’ve seen in this article, one single thing cannot guarantee that our service will be highly available. Instead, we must combine multiple technologies and approaches. The Go native advantages that we presented here have made our lives much easier and our services faster and more efficient.
In closing, we’ll leave you with one simple piece of advice – follow best practices and always monitor.