Let's Go Further: Advanced patterns for building APIs and web applications in Go

586 Pages·2021·7.664 MB·English

by  Alex Edwards

Contents
1. Introduction
1.1. Prerequisites
2. Getting Started
2.1. Project Setup and Skeleton Structure
2.2. A Basic HTTP Server
2.3. API Endpoints and RESTful Routing
3. Sending JSON Responses
3.1. Fixed-Format JSON
3.2. JSON Encoding
3.3. Encoding Structs
3.4. Formatting and Enveloping Responses
3.5. Advanced JSON Customization
3.6. Sending Error Messages
4. Parsing JSON Requests
4.1. JSON Decoding
4.2. Managing Bad Requests
4.3. Restricting Inputs
4.4. Custom JSON Decoding
4.5. Validating JSON Input
5. Database Setup and Configuration
5.1. Setting up PostgreSQL
5.2. Connecting to PostgreSQL
5.3. Configuring the Database Connection Pool
6. SQL Migrations
6.1. An Overview of SQL Migrations
6.2. Working with SQL Migrations
7. CRUD Operations
7.1. Setting up the Movie Model
7.2. Creating a New Movie
7.3. Fetching a Movie
7.4. Updating a Movie
7.5. Deleting a Movie
8. Advanced CRUD Operations
8.1. Handling Partial Updates
8.2. Optimistic Concurrency Control
8.3. Managing SQL Query Timeouts
9. Filtering, Sorting, and Pagination
9.1. Parsing Query String Parameters
9.2. Validating Query String Parameters
9.3. Listing Data
9.4. Filtering Lists
9.5. Full-Text Search
9.6. Sorting Lists
9.7. Paginating Lists
9.8. Returning Pagination Metadata
10. Structured Logging and Error Handling
10.1. Structured JSON Log Entries
10.2. Panic Recovery
11. Rate Limiting
11.1. Global Rate Limiting
11.2. IP-based Rate Limiting
11.3. Configuring the Rate Limiters
12. Graceful Shutdown
12.1. Sending Shutdown Signals
12.2. Intercepting Shutdown Signals
12.3. Executing the Shutdown
13. User Model Setup and Registration
13.1. Setting up the Users Database Table
13.2. Setting up the Users Model
13.3. Registering a User
14. Sending Emails
14.1. SMTP Server Setup
14.2. Creating Email Templates
14.3. Sending a Welcome Email
14.4. Sending Background Emails
14.5. Graceful Shutdown of Background Tasks
15. User Activation
15.1. Setting up the Tokens Database Table
15.2. Creating Secure Activation Tokens
15.3. Sending Activation Tokens
15.4. Activating a User
16. Authentication
16.1. Authentication Options
16.2. Generating Authentication Tokens
16.3. Authenticating Requests
17. Permission-based Authorization
17.1. Requiring User Activation
17.2. Setting up the Permissions Database Table
17.3. Setting up the Permissions Model
17.4. Checking Permissions
17.5. Granting Permissions
18. Cross Origin Requests
18.1. An Overview of CORS
18.2. Demonstrating the Same-Origin Policy
18.3. Simple CORS Requests
18.4. Preflight CORS Requests
19. Metrics
19.1. Exposing Metrics with Expvar
19.2. Creating Custom Metrics
19.3. Request-level Metrics
19.4. Recording HTTP Status Codes
20. Building, Versioning and Quality Control
20.1. Creating and Using Makefiles
20.2. Managing Environment Variables
20.3. Quality Controlling Code
20.4. Module Proxies and Vendoring
20.5. Building Binaries
20.6. Managing and Automating Version Numbers
21. Deployment and Hosting
21.1. Creating a Digital Ocean Droplet
21.2. Server Configuration and Installing Software
21.3. Deployment and Executing Migrations
21.4. Running the API as a Background Service
21.5. Using Caddy as a Reverse Proxy
22. Appendices
22.1. Managing Password Resets
22.2. Creating Additional Activation Tokens
22.3. Authentication with JSON Web Tokens
22.4. JSON Encoding Nuances
22.5. JSON Decoding Nuances
22.6. Request Context Timeouts
23. Feedback
Chapter 1.
Introduction
In this book we’re going to work through the start-to-finish build of an application called
Greenlight — a JSON API for retrieving and managing information about movies. You can
think of the core functionality as being a bit like the Open Movie Database API.
Ultimately, our Greenlight API will support the following endpoints and actions:
Method URL Pattern Action
GET /v1/healthcheck Show application health and version information
GET /v1/movies Show the details of all movies
POST /v1/movies Create a new movie
GET /v1/movies/:id Show the details of a specific movie
PATCH /v1/movies/:id Update the details of a specific movie
DELETE /v1/movies/:id Delete a specific movie
POST /v1/users Register a new user
PUT /v1/users/activated Activate a specific user
PUT /v1/users/password Update the password for a specific user
POST /v1/tokens/authentication Generate a new authentication token
POST /v1/tokens/password-reset Generate a new password-reset token
GET /debug/vars Display application metrics
To give you an idea of what the API will look like from a client’s point of view, by the end of
this book the GET /v1/movies/:id endpoint will return a response similar this:
$ curl -H "Authorization: Bearer RIDBIAE3AMMK57T6IAEBUGA7ZQ" localhost:4000/v1/movies/1
{
"movie": {
"id": 1,
"title": "Moana",
"year": 2016,
"runtime": "107 mins",
"genres": [
"animation",
"adventure"
],
"version": 1
}
}
Behind the scenes, we’ll use PostgreSQL as the database for persistently storing all the
data. And at the end of the book, we’ll deploy the finished API to a Linux server running on
Digital Ocean.
Conventions
In this book, code blocks are shown with a silver background like the snippet below. If the
code block is particularly long, parts that aren’t relevant may be replaced with an ellipsis.
To make it easy to follow along, most code blocks also have a title bar at the top indicating
the name of the file that we’re working on.
File: hello.go
package main
... // Indicates that some existing code has been omitted.
func sayHello() {
fmt.Println("Hello world!")
}
Terminal (command line) instructions are shown with a black background and generally
start with a dollar symbol. These commands should work on any Unix-based operating
system, including macOS and Linux. Sample output is shown in silver beneath the
command, like so:
$ echo "Hello world!"
Hello world!
If you’re using Windows, you should replace these commands with the DOS equivalent or
carry out the action via the normal Windows GUI.
Some chapters in this book end with an additional information section. These sections
contain information that isn’t relevant to our application build, but is still important (or
sometimes, just interesting) to know about.
About the author
Hey, I’m Alex Edwards, a full-stack web developer and author. I live near Innsbruck, Austria.
I’ve been working with Go for over 8 years, building production applications for myself and
commercial clients, and helping people all around the world improve their Go skills.
You can see more of my writing on my blog (where I publish detailed tutorials), some of my
open-source work on GitHub, and you can also follow me on Instagram and Twitter.
Copyright and disclaimer
Let’s Go Further. Copyright © 2021 Alex Edwards.
Last updated 2021-08-27 08:41:48 UTC. Version 1.1.0.
The Go gopher was designed by Renee French and is used under the Creative Commons 3.0
Attributions license. Cover gopher adapted from vectors by Egon Elbre.
The information provided within this book is for general informational purposes only. While
the author and publisher have made every effort to ensure that the accuracy of the
information within this book was correct at time of publication there are no representations or
warranties, express or implied, about the completeness, accuracy, reliability, suitability or
availability with respect to the information, products, services, or related graphics contained
in this book for any purpose. Any use of this information is at your own risk.
Chapter 1.1.
Prerequisites
Background knowledge
This book is written as a follow up to Let’s Go, and we’ll leverage a lot of the information and
code patterns from that book again here.
If you’ve already read and enjoyed Let’s Go, then this book should be a good fit for you and
the ideal next step in your learning. If you haven’t, then I highly recommend starting with
Let’s Go first — especially if you’re a newcomer to Go.
You can read this as a standalone book, but please be aware that it is somewhat advanced
— it doesn’t explain the fundamentals in detail, and some topics (like testing) don’t feature
at all because they were covered heavily in the previous book. But if you’re comfortable
using Go and already have a decent amount of experience behind you, then this book may
also be a good fit for you. Feel free to jump straight in.
Go 1.17
The information in this book is correct for the latest major release of Go (version 1.17), and
you should install this if you’d like to code-along with the application build.
If you’ve already got Go installed, you can check the version number from your terminal by
using the go version command. The output should look similar to this:
$ go version
go version go1.17 linux/amd64
If you need to upgrade your version of Go, then please go ahead and do that now. The
instructions for your operating system can be found here.
Other software
There are a few other bits of software that you should make sure are available on your
computer if you want to follow along fully. They are:
The curl tool for working with HTTP requests and responses from your terminal. On
MacOS and Linux machines it should be pre-installed or available in your software
repositories. Otherwise, you can download the latest version from here.
The hey tool for carrying out some basic load tests. So long as you have Go 1.16 or newer
on your computer, you can install hey with the go install command:
$ go install github.com/rakyll/hey@latest
The git version control system. Installation instructions for all operating systems can be
found here.
A web browser with good developer tools. I’ll be using Firefox in this book, but
Chromium, Chrome or Microsoft Edge will work too.
Your favorite text editor (cid:1)