[WIP] Add a program illustrating the use of a NAT instance (#162)

Add code for using a NAT instance, split into multiple files for manageability. Update documentation.

Signed-off-by: Scott Lowe <scott.lowe@scottlowe.org>
This commit is contained in:
Scott S. Lowe 2024-02-02 22:39:36 -07:00 committed by GitHub
parent 8bfa63866a
commit 7696da5b2e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 531 additions and 0 deletions

9
aws/README.md Normal file
View file

@ -0,0 +1,9 @@
# Learning Tools: AWS
Here you'll find a collection of tools and resources for learning about, experimenting with, or deepening your knowledge of AWS.
## Contents
**postman-aws-api**: This folder contains some JavaScript tests written for Postman, intended for use when interacting with AWS APIs using Postman.
**nat-instance-pulumi**: This folder has a Pulumi project for standing up a NAT instance to provide connectivity to EC2 instances on a private subnet (instead of a Managed NAT Gateway).

View file

@ -0,0 +1,3 @@
name: nat-instance-pulumi
runtime: go
description: A Go Pulumi program to use a NAT instance on AWS

View file

@ -0,0 +1,51 @@
# Using a NAT Instance for Private Subnet Connectivity
This [Pulumi](https://www.pulumi.com) project allows users to stand up and configure a NAT instance---instead of a Managed NAT Gateway---for internet connectivity from private subnets in a VPC. This Pulumi program was written in [Go](https://go.dev).
While not complex, the Pulumi program here does illustrate a few things that might be useful for newer users:
* How to structure Go code in Pulumi when splitting the code into multiple files for manageability
* Supporting both X86_64/AMD64- as well as ARM64-based configurations
* Dynamically looking up an AMI
* Creating an SSH key
## Contents
* `go.mod`: This file contains dependencies used by this Go program.
* `go.sum`: This file contains checksums for each of the direct and indirect dependencies. The checksum is used to validate that none of them has been modified.
* `main.go`: This Go file is the Pulumi program executed by the `pulumi` CLI. It calls `vpc.go` and `nat.go` to build out all the underlying infrastructure, then launches an Ubuntu-based EC2 instance in a private subnet. This instance can be used to verify connectivity is working through the NAT instance as expected.
* `nat.go`: This Go file contains a function (`buildNat`) that is called by `main.go` to build out the pieces for the NAT instance.
* `Pulumi.yaml`: This is the Pulumi project file.
* `README.md`: This file you're currently reading.
* `vpc.go`: This Go file contains a function (`buildInfrastructure`) that is called by `main.go` to create the VPC, subnets, and route tables. Routes for public subnets are also created here, but routes for private subnets are defined in `nat.go`.
## Instructions
These instructions assume you've already installed and configured Pulumi and all necessary dependencies (the AWS CLI and Go, for this example). Please refer to the Pulumi documentation for more details on installation or configuration.
1. Copy the contents of this directory down to a directory on your system, or clone the entire repository and then change into the directory where this section of the cloned repository resides.
1. Run `pulumi stack init` to create a new stack.
1. Run `pulumi config set aws:region <region-name>` to set the AWS region where the Pulumi program should create resources. _This is a required configuration value; CLI operations will fail if you don't set this value._
1. (Optional) Run `pulumi config set` to set configuration values that affect the behavior of the Pulumi program. The optional configuration values are:
* `architecture`: Set this to "amd64" or "arm64". The values "x86_64" and "x64" are also supported and will have the same effect as "amd64". The default value is "arm64".
* `versionname`: Set this to "bionic", "focal", or "jammy" to control the version of Ubuntu used in the EC2 instance. These version names correspond to the 18.04, 20.04, and 22.04 releases, respectively. The default value is "jammy".
1. Run `pulumi up` to instantiate the resources.
Once the resources are provisioned, you should be able to SSH to the NAT instance, or use the NAT instance as an SSH bastion host to get to the private instance. The private instance should have full Internet connectivity.
When you're finished, run `pulumi destroy` to tear down all the provisioned resources.
## License
This content is licensed under the MIT License.

View file

@ -0,0 +1,95 @@
module nat-instance-pulumi
go 1.21
require (
github.com/pulumi/pulumi-aws/sdk/v6 v6.18.1
github.com/pulumi/pulumi-awsx/sdk/v2 v2.4.0
github.com/pulumi/pulumi-tls/sdk/v4 v4.11.1
github.com/pulumi/pulumi/sdk/v3 v3.101.1
)
require (
dario.cat/mergo v1.0.0 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/ProtonMail/go-crypto v0.0.0-20230828082145-3c4c8a2d2371 // indirect
github.com/aead/chacha20 v0.0.0-20180709150244-8b13a72661da // indirect
github.com/agext/levenshtein v1.2.3 // indirect
github.com/apparentlymart/go-textseg/v13 v13.0.0 // indirect
github.com/atotto/clipboard v0.1.4 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/blang/semver v3.5.1+incompatible // indirect
github.com/charmbracelet/bubbles v0.16.1 // indirect
github.com/charmbracelet/bubbletea v0.24.2 // indirect
github.com/charmbracelet/lipgloss v0.7.1 // indirect
github.com/cheggaaa/pb v1.0.29 // indirect
github.com/cloudflare/circl v1.3.7 // indirect
github.com/containerd/console v1.0.4-0.20230313162750-1ae8d489ac81 // indirect
github.com/cyphar/filepath-securejoin v0.2.4 // indirect
github.com/djherbis/times v1.5.0 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.5.0 // indirect
github.com/go-git/go-git/v5 v5.11.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/glog v1.1.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/hcl/v2 v2.17.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/mattn/go-localereader v0.0.1 // indirect
github.com/mattn/go-runewidth v0.0.15 // indirect
github.com/mitchellh/go-ps v1.0.0 // indirect
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/reflow v0.3.0 // indirect
github.com/muesli/termenv v0.15.2 // indirect
github.com/opentracing/basictracer-go v1.1.0 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pgavlin/fx v0.1.6 // indirect
github.com/pjbgf/sha1cd v0.3.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pkg/term v1.1.0 // indirect
github.com/pulumi/appdash v0.0.0-20231130102222-75f619a67231 // indirect
github.com/pulumi/esc v0.6.2 // indirect
github.com/pulumi/pulumi-docker/sdk/v4 v4.4.3 // indirect
github.com/rivo/uniseg v0.4.4 // indirect
github.com/rogpeppe/go-internal v1.11.0 // indirect
github.com/sabhiram/go-gitignore v0.0.0-20210923224102-525f6e181f06 // indirect
github.com/santhosh-tekuri/jsonschema/v5 v5.0.0 // indirect
github.com/sergi/go-diff v1.3.1 // indirect
github.com/skeema/knownhosts v1.2.1 // indirect
github.com/spf13/cast v1.4.1 // indirect
github.com/spf13/cobra v1.7.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/texttheater/golang-levenshtein v1.0.1 // indirect
github.com/tweekmonster/luser v0.0.0-20161003172636-3fa38070dbd7 // indirect
github.com/uber/jaeger-client-go v2.30.0+incompatible // indirect
github.com/uber/jaeger-lib v2.4.1+incompatible // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/zclconf/go-cty v1.13.2 // indirect
go.uber.org/atomic v1.9.0 // indirect
golang.org/x/crypto v0.17.0 // indirect
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa // indirect
golang.org/x/mod v0.14.0 // indirect
golang.org/x/net v0.19.0 // indirect
golang.org/x/sync v0.5.0 // indirect
golang.org/x/sys v0.15.0 // indirect
golang.org/x/term v0.15.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/tools v0.15.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230731190214-cbb8c96f2d6d // indirect
google.golang.org/grpc v1.57.1 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
lukechampine.com/frand v1.4.2 // indirect
)

View file

@ -0,0 +1,133 @@
package main
import (
"fmt"
"log"
"github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
"github.com/pulumi/pulumi-tls/sdk/v4/go/tls"
"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
"github.com/pulumi/pulumi/sdk/v3/go/pulumi/config"
)
func main() {
pulumi.Run(func(ctx *pulumi.Context) error {
// Set up maps that are used later
versionMap := map[string]string{"jammy": "22.04", "focal": "20.04", "bionic": "18.04"}
typeMap := map[string]string{"amd64": "t3a.small", "arm64": "t4g.small", "x86_64": "t3a.small", "x64": "t3a.small"}
// Retrieve configuration values
instanceCpuArch, err := config.Try(ctx, "architecture")
if err != nil {
instanceCpuArch = "arm64"
}
instanceType, ok := typeMap[instanceCpuArch]
if !ok {
instanceCpuArch = "arm64"
instanceType = "t4g.small"
}
if instanceCpuArch == "x86_64" || instanceCpuArch == "x64" {
instanceCpuArch = "amd64"
}
// vpcNetworkCidr, err := config.Try(ctx, "networkcidr")
// if err != nil {
// vpcNetworkCidr = "10.0.0.0/16"
// }
versionName, err := config.Try(ctx, "version")
if err != nil {
versionName = "jammy"
}
versionNum, ok := versionMap[versionName]
if !ok {
versionName = "jammy"
versionNum = "22.04"
}
// Create an SSH key
natSshKey, err := tls.NewPrivateKey(ctx, "nat-ssh-key", &tls.PrivateKeyArgs{
Algorithm: pulumi.String("ED25519"),
})
if err != nil {
log.Printf("error creating SSH key: %s", err.Error())
}
// Create an AWS key pair
natKeyPair, err := ec2.NewKeyPair(ctx, "nat-key-pair", &ec2.KeyPairArgs{
PublicKey: natSshKey.PublicKeyOpenssh,
})
if err != nil {
log.Printf("error creating AWS key pair: %s", err.Error())
}
// Create a new VPC, subnets, route tables, and public route
// Private routes will be created later
buildInfrastructure(ctx)
// Create the NAT infrastructure
buildNat(ctx, natKeyPair.KeyName)
// Create a security group that we can use to connect to our instance
privateSg, err := ec2.NewSecurityGroup(ctx, "private-sg", &ec2.SecurityGroupArgs{
VpcId: idOfVpc,
Egress: ec2.SecurityGroupEgressArray{
ec2.SecurityGroupEgressArgs{
Protocol: pulumi.String("-1"),
FromPort: pulumi.Int(0),
ToPort: pulumi.Int(0),
CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
},
},
Ingress: ec2.SecurityGroupIngressArray{
ec2.SecurityGroupIngressArgs{
Protocol: pulumi.String("tcp"),
FromPort: pulumi.Int(22),
ToPort: pulumi.Int(22),
CidrBlocks: pulumi.StringArray{pulumi.String("10.0.0.0/16")},
},
},
})
if err != nil {
log.Printf("error creating security group: %s", err.Error())
}
// Get AMI ID for Ubuntu instance
amiName := fmt.Sprintf("ubuntu/images/hvm-ssd/ubuntu-%s-%s-%s-server*", versionName, versionNum, instanceCpuArch)
ubuntuAmi, err := ec2.LookupAmi(ctx, &ec2.LookupAmiArgs{
Owners: []string{"099720109477"},
MostRecent: pulumi.BoolRef(true),
Filters: []ec2.GetAmiFilter{
{Name: "name", Values: []string{amiName}},
{Name: "root-device-type", Values: []string{"ebs"}},
{Name: "virtualization-type", Values: []string{"hvm"}},
{Name: "architecture", Values: []string{instanceCpuArch}},
},
})
if err != nil {
log.Printf("error looking up Ubuntu AMI: %s", err.Error())
}
// Launch an instance using Ubuntu AMI
ubuntuInstance, err := ec2.NewInstance(ctx, "ubuntu-instance", &ec2.InstanceArgs{
Ami: pulumi.String(ubuntuAmi.Id),
InstanceType: pulumi.String(instanceType),
AssociatePublicIpAddress: pulumi.Bool(false),
KeyName: natKeyPair.KeyName,
SubnetId: privateSubnets[0],
VpcSecurityGroupIds: pulumi.StringArray{privateSg.ID()},
Tags: pulumi.StringMap{
"Name": pulumi.String("ubuntu-instance"),
},
})
if err != nil {
log.Printf("error launching instance: %s", err.Error())
}
// Export some values as stack outputs
ctx.Export("instanceId", ubuntuInstance.ID())
ctx.Export("natPublicIpAddress", natPubIpAddr)
ctx.Export("instancePrivateIpAddress", ubuntuInstance.PrivateIp)
ctx.Export("privateKey", natSshKey.PrivateKeyOpenssh)
return nil
})
}

View file

@ -0,0 +1,88 @@
package main
import (
"log"
"github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)
// Define variables needed outside the buildNat() function
var natPubIpAddr pulumi.StringInput
// Builds base infrastructure when called
func buildNat(ctx *pulumi.Context, key pulumi.StringInput) (err error) {
// Create a security group for the NAT instance
natSg, err := ec2.NewSecurityGroup(ctx, "nat-sg", &ec2.SecurityGroupArgs{
VpcId: idOfVpc,
Egress: ec2.SecurityGroupEgressArray{
ec2.SecurityGroupEgressArgs{
Protocol: pulumi.String("-1"),
FromPort: pulumi.Int(0),
ToPort: pulumi.Int(0),
CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
},
},
Ingress: ec2.SecurityGroupIngressArray{
ec2.SecurityGroupIngressArgs{
Protocol: pulumi.String("-1"),
FromPort: pulumi.Int(0),
ToPort: pulumi.Int(0),
CidrBlocks: pulumi.StringArray{pulumi.String("10.0.0.0/16")},
},
ec2.SecurityGroupIngressArgs{
Protocol: pulumi.String("tcp"),
FromPort: pulumi.Int(22),
ToPort: pulumi.Int(22),
CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
},
},
})
if err != nil {
log.Printf("error creating security group: %s", err.Error())
}
// Get AMI ID for the fck-nat instance
natAmi, err := ec2.LookupAmi(ctx, &ec2.LookupAmiArgs{
Owners: []string{"568608671756"},
MostRecent: pulumi.BoolRef(true),
Filters: []ec2.GetAmiFilter{
{Name: "name", Values: []string{"fck-nat-amzn2-*"}},
{Name: "root-device-type", Values: []string{"ebs"}},
{Name: "virtualization-type", Values: []string{"hvm"}},
{Name: "architecture", Values: []string{"arm64"}},
},
})
if err != nil {
log.Printf("error looking up NAT AMI: %s", err.Error())
}
// Launch a fck-nat instance
natInstance, err := ec2.NewInstance(ctx, "nat-instance", &ec2.InstanceArgs{
Ami: pulumi.String(natAmi.Id),
InstanceType: pulumi.String("t4g.nano"),
AssociatePublicIpAddress: pulumi.Bool(true),
KeyName: key,
SourceDestCheck: pulumi.BoolPtr(false),
SubnetId: publicSubnets[0],
VpcSecurityGroupIds: pulumi.StringArray{natSg.ID()},
Tags: pulumi.StringMap{
"Name": pulumi.String("nat-instance"),
},
})
if err != nil {
log.Printf("error launching instance: %s", err.Error())
}
// Set value of public variable
natPubIpAddr = natInstance.PublicIp
// Create a route in the route table for the private subnets
_, err = ec2.NewRoute(ctx, "nat-route", &ec2.RouteArgs{
DestinationCidrBlock: pulumi.String("0.0.0.0/0"),
NetworkInterfaceId: natInstance.PrimaryNetworkInterfaceId.ToStringOutput(),
RouteTableId: idPrivRoute,
})
// Return to the calling function
return nil
}

View file

@ -0,0 +1,152 @@
package main
import (
"fmt"
"github.com/pulumi/pulumi-aws/sdk/v6/go/aws"
"github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)
// Define variables needed outside the buildInfrastructure() function
var idOfVpc pulumi.StringInput
var publicSubnets pulumi.StringArray
var privateSubnets pulumi.StringArray
var idPubRoute pulumi.StringInput // ID of default route table
var idPrivRoute pulumi.StringInput // ID of NAT route table
var azNumber int // Number of AZs
// Builds base infrastructure when called
func buildInfrastructure(ctx *pulumi.Context) (err error) {
// Look up AZ information for configured region and gather details
desiredAzState := "available"
rawAzInfo, err := aws.GetAvailabilityZones(ctx, &aws.GetAvailabilityZonesArgs{
State: &desiredAzState,
})
if err != nil {
return err
}
numOfAzs := len(rawAzInfo.Names)
azNames := make([]string, pulumi.Int(numOfAzs))
for i := 0; i < numOfAzs; i++ {
azNames[i] = rawAzInfo.Names[i]
}
// Set value of public variable
azNumber = numOfAzs
// Create a new VPC and make the ID accessible outside the function
vpc, err := ec2.NewVpc(ctx, "vpc", &ec2.VpcArgs{
CidrBlock: pulumi.String("10.0.0.0/16"),
EnableDnsHostnames: pulumi.Bool(true),
EnableDnsSupport: pulumi.Bool(true),
Tags: pulumi.StringMap{
"project": pulumi.String("nat-instance-pulumi"),
},
})
if err != nil {
return err
}
// Set value of public variable
idOfVpc = vpc.ID()
// Create an Internet gateway
inetGw, err := ec2.NewInternetGateway(ctx, "inet-gw", &ec2.InternetGatewayArgs{
VpcId: vpc.ID(),
Tags: pulumi.StringMap{
"project": pulumi.String("nat-instance-pulumi"),
},
})
if err != nil {
return err
}
// Adopt the default route in the VPC
defRoute, err := ec2.NewDefaultRouteTable(ctx, "def-route-tbl", &ec2.DefaultRouteTableArgs{
DefaultRouteTableId: vpc.DefaultRouteTableId,
Tags: pulumi.StringMap{
"project": pulumi.String("nat-instance-pulumi"),
},
})
if err != nil {
return err
}
// Set value of public variable
idPubRoute = defRoute.ID()
// Associate gateway with default route
_, err = ec2.NewRoute(ctx, "inet-route", &ec2.RouteArgs{
RouteTableId: defRoute.ID(),
DestinationCidrBlock: pulumi.String("0.0.0.0/0"),
GatewayId: inetGw.ID(),
})
if err != nil {
return err
}
// Create public subnets
for i := 0; i < numOfAzs; i++ {
subnetAddr := i * 32
subnetCidrBlock := fmt.Sprintf("10.0.%d.0/22", subnetAddr)
subnet, err := ec2.NewSubnet(ctx, fmt.Sprintf("pub-subnet-%d", i), &ec2.SubnetArgs{
VpcId: vpc.ID(),
AvailabilityZone: pulumi.String(azNames[i]),
CidrBlock: pulumi.String(subnetCidrBlock),
MapPublicIpOnLaunch: pulumi.Bool(true),
Tags: pulumi.StringMap{
"project": pulumi.String("nat-instance-pulumi"),
},
})
if err != nil {
return err
}
// Add value to array
publicSubnets = append(publicSubnets, subnet.ID())
}
// Create a route for the NAT Gateway
natRoute, err := ec2.NewRouteTable(ctx, "nat-route-tbl", &ec2.RouteTableArgs{
VpcId: vpc.ID(),
Tags: pulumi.StringMap{
"project": pulumi.String("nat-instance-pulumi"),
},
})
if err != nil {
return err
}
// Set value of public variable
idPrivRoute = natRoute.ID()
// Create private subnets
for i := 0; i < numOfAzs; i++ {
subnetAddr := (i * 32) + 16
subnetCidrBlock := fmt.Sprintf("10.0.%d.0/22", subnetAddr)
subnet, err := ec2.NewSubnet(ctx, fmt.Sprintf("priv-subnet-%d", i), &ec2.SubnetArgs{
VpcId: vpc.ID(),
AvailabilityZone: pulumi.String(azNames[i]),
CidrBlock: pulumi.String(subnetCidrBlock),
MapPublicIpOnLaunch: pulumi.Bool(false),
Tags: pulumi.StringMap{
"project": pulumi.String("nat-instance-pulumi"),
},
})
if err != nil {
return err
}
// Add value to array
privateSubnets = append(privateSubnets, subnet.ID())
}
// Link private subnets to private route table
for i := 0; i < numOfAzs; i++ {
_, err := ec2.NewRouteTableAssociation(ctx, fmt.Sprintf("priv-rta-%d", i), &ec2.RouteTableAssociationArgs{
SubnetId: privateSubnets[i],
RouteTableId: natRoute.ID(),
})
if err != nil {
return err
}
}
// Return to the calling function
return nil
}