AWS offers the fairly excellent Cloudfront service, providing a solid caching proxy in front of your resources. It’s exceptionally good for static resources like CSS or Javascript, and even dynamic content that changes infrequently.
I consider it good design practice to ensure that AWS logs are shipped to a central logging account, providing a central location from which to build out logging infrastructure and tooling, instead of spreading it across multiple accounts in the organisation.
A single place to look for new insights, to try to understand what’s happening in the system? Good design, all around.
AWS Cloudfront supports logging its access requests to S3, like most AWS services. It also supports multiple accounts feeding in to the same S3 bucket, but, it’s not entirely obvious how to do that.
I recently spent some time digging into how to do this from Terraform, and I’d like to share how I solved this problem for multiple simultaneous accounts.
Bucket Policies and IAM
IAM is one of the more complicated but most important aspects of using AWS, and I regularly find myself writing new IAM policies that specifically lock down resource capabilities to ensure any misuse of the attached roles is limited.
However, this runs counter to how AWS Cloudfront distribution logging expects to work.
Instead of writing an S3 bucket policy that allows AWS Cloudfront to write to our target logging bucket, we instead need to grant s3:GetBucketACL and s3:PutBucketACL to each account that we want to be able to write logs.
This gives us a bucket policy that will look something like this:
data "aws_iam_policy_document" "logging_bucket_policy" {
statement {
actions = [
"s3:GetBucketACL",
"s3:PutBucketACL",
]
resources = [
"arn:aws:s3:::my-logging-bucket",
]
principals {
type = "AWS"
identifiers = [
"${data.aws_caller_identity.secondary.account_id}",
"${data.aws_caller_identity.tertiary.account_id}",
]
}
}
}
Recognising that I needed to let go and allow AWS Cloudfront to manage the bucket ACLs on its own was a major requirement to allowing logs to be written.
S3 Setup
At this point we create our S3 logging bucket:
resource "aws_s3_bucket" "logs" {
provider = "aws.primary"
bucket = "my-logging-bucket"
acl = "private"
policy = "${data.aws_iam_policy_document.bucket_policy.json}"
}
and we create the S3 buckets to serve our content:
resource "aws_s3_bucket" "server_secondary" {
provider = "aws.secondary"
bucket = "secondary-cloudfront-serve-bucket"
website {
index_document = "index.html"
}
policy = "${data.aws_iam_policy_document.read_secondary.json}"
}
and
resource "aws_s3_bucket" "server_tertiary" {
provider = "aws.tertiary"
bucket = "tertiary-cloudfront-serve-bucket"
website {
index_document = "index.html"
}
policy = "${data.aws_iam_policy_document.read_tertiary.json}"
}
But, we haven’t defined the bucket policies for either of those serve buckets yet — let’s do that next.
Bucket Policies and Cloudfront Origins
In order to ensure that access to our S3 bucket only goes through Cloudfront, we want to create Cloudfront Origin policies, that we attach to our buckets.
Origins
resource "aws_cloudfront_origin_access_identity" "s3_access_secondary" {
provider = "aws.secondary"
comment = "secondary identity"
}
and
resource "aws_cloudfront_origin_access_identity" "s3_access_tertiary" {
provider = "aws.tertiary"
comment = "tertiary identity"
}
Policies
Next, we can define our bucket policies.
These policies will allow the Cloudfront origin to read anything in our server bucket, and list out the buckets. This is enough permission to do everything we’ll need for a static site.
data "aws_iam_policy_document" "read_secondary" {
# Cloudfront can read anything
statement {
actions = ["s3:GetObject"]
resources = ["arn:aws:s3:::secondary-cloudfront-serve-bucket/*"]
principals {
type = "AWS"
identifiers = ["${aws_cloudfront_origin_access_identity.s3_access_secondary.iam_arn}"]
}
}
# Cloudfront can list the bucket
statement {
actions = ["s3:ListBucket"]
resources = ["arn:aws:s3:::secondary-cloudfront-serve-bucket"]
principals {
type = "AWS"
identifiers = ["${aws_cloudfront_origin_access_identity.s3_access_secondary.iam_arn}"]
}
}
}
and
data "aws_iam_policy_document" "bucket_policy_read_tertiary" {
# Cloudfront can read anything
statement {
actions = ["s3:GetObject"]
resources = ["arn:aws:s3:::tertiary-cloudfront-serve-bucket/*"]
principals {
type = "AWS"
identifiers = ["${aws_cloudfront_origin_access_identity.s3_access_tertiary.iam_arn}"]
}
}
# Cloudfront can list the bucket
statement {
actions = ["s3:ListBucket"]
resources = ["arn:aws:s3:::tertiary-cloudfront-serve-bucket"]
principals {
type = "AWS"
identifiers = ["${aws_cloudfront_origin_access_identity.s3_access_tertiary.iam_arn}"]
}
}
}
At this point, we’ve configured the entire chain for creating a Cloudfront distribution, that logs to our central primary account.
Let’s finally create the distributions.
Distributions
Setting up a Cloudfront distribution in Terraform has a lot of configuration options, and I recommend you read the documentation to see what options you might need.
The examples I’ve posted here are complete, but will need to be modified for your environment.
Once created these distributions will serve out of our S3 serve buckets in the secondary and tertiary accounts, while directing their logs - usefully prefixed, with logs-secondary and logs-tertiary, into our central primary account.
The distribution in the secondary account will be:
resource "aws_cloudfront_distribution" "s3_distribution_secondary" {
provider = "aws.secondary"
origin {
domain_name = "${aws_s3_bucket.server_secondary.bucket_domain_name}"
origin_id = "secondary_origin"
s3_origin_config {
origin_access_identity = "${aws_cloudfront_origin_access_identity.s3_access_secondary.cloudfront_access_identity_path}"
}
}
enabled = true
logging_config {
include_cookies = false
bucket = "${aws_s3_bucket.logs.bucket_domain_name}"
prefix = "secondary-cloudfront-logs"
}
default_cache_behavior {
allowed_methods = ["GET", "HEAD"]
cached_methods = ["GET", "HEAD"]
target_origin_id = "secondary_origin"
forwarded_values {
query_string = false
cookies {
forward = "none"
}
}
viewer_protocol_policy = "allow-all"
min_ttl = 0
default_ttl = 3600
max_ttl = 86400
compress = true
}
price_class = "PriceClass_All"
restrictions {
geo_restriction {
restriction_type = "none"
}
}
viewer_certificate {
cloudfront_default_certificate = true
}
}
and for tertiary:
resource "aws_cloudfront_distribution" "s3_distribution_tertiary" {
provider = "aws.tertiary"
origin {
domain_name = "${aws_s3_bucket.server_tertiary.bucket_domain_name}"
origin_id = "tertiary_origin"
s3_origin_config {
origin_access_identity = "${aws_cloudfront_origin_access_identity.s3_access_tertiary.cloudfront_access_identity_path}"
}
}
enabled = true
logging_config {
include_cookies = false
bucket = "${aws_s3_bucket.logs.bucket_domain_name}"
prefix = "logs-tertiary"
}
default_cache_behavior {
allowed_methods = ["GET", "HEAD"]
cached_methods = ["GET", "HEAD"]
target_origin_id = "tertiary_origin"
forwarded_values {
query_string = false
cookies {
forward = "none"
}
}
viewer_protocol_policy = "allow-all"
min_ttl = 0
default_ttl = 3600
max_ttl = 86400
compress = true
}
price_class = "PriceClass_All"
restrictions {
geo_restriction {
restriction_type = "none"
}
}
viewer_certificate {
cloudfront_default_certificate = true
}
}
The examples in this post can be found in the Eiara GitHub.