Can't use S3 backend with Terraform - missing credentials - terraform

Question

I have the most pedestrian of a Terraform sample:
# Configure AWS provider
provider "aws" {
region = "us-east-1"
access_key = "xxxxxxxxx"
secret_key = "yyyyyyyyyyy"
}
# Terraform configuration
terraform {
backend "s3" {
bucket = "terraform.example.com"
key = "85/182/terraform.tfstate"
region = "us-east-1"
}
}
When I run terraform init I receive the following (traced) response:
2018/08/14 14:19:13 [INFO] Terraform version: 0.11.7 41e50bd32a8825a84535e353c3674af8ce799161
2018/08/14 14:19:13 [INFO] Go runtime version: go1.10.1
2018/08/14 14:19:13 [INFO] CLI args: []string{"C:\\cygwin64\\usr\\local\\bin\\terraform.exe", "init"}
2018/08/14 14:19:13 [DEBUG] Attempting to open CLI config file: C:\Users\judall\AppData\Roaming\terraform.rc
2018/08/14 14:19:13 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2018/08/14 14:19:13 [INFO] CLI command args: []string{"init"}
2018/08/14 14:19:13 [DEBUG] command: loading backend config file: C:\cygwin64\home\judall\t2
2018/08/14 14:19:13 [DEBUG] command: no data state file found for backend config
Initializing the backend...
2018/08/14 14:19:13 [DEBUG] New state was assigned lineage "5113646b-318f-9612-5057-bc4803292c3a"
2018/08/14 14:19:13 [INFO] Building AWS region structure
2018/08/14 14:19:13 [INFO] Building AWS auth structure
2018/08/14 14:19:13 [INFO] Setting AWS metadata API timeout to 100ms
2018/08/14 14:19:13 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2018/08/14 14:19:13 [DEBUG] plugin: waiting for all plugin processes to complete...
Error configuring the backend "s3": No valid credential sources found for AWS Provider.
Please see https://terraform.io/docs/providers/aws/index.html for more information on
providing credentials for the AWS Provider
Please update the configuration in your Terraform files to fix this error
then run this command again.
I've been googling for hours on this. I've tried to use the 'profile' property - which yields slightly different trace logs, but the same end result. I've tried setting the AWS_ environment variables - with the same result.
I'm running terraform version 0.11.7. Any suggestions?

Answer 1

The provider configuration is independent from your backend configuration.
The credentials, you have configured in the provider block, are used to create your AWS related resources. For accessing S3 bucket as a storage for your remote state, you also need to provide credentials. This can be the same like in the config for your provider or can be completely different (with permissions only on this specific bucket for security reasons).
You can fix it by adding the credentials in the backend block:
# Terraform configuration
terraform {
backend "s3" {
bucket = "terraform.example.com"
key = "85/182/terraform.tfstate"
region = "us-east-1"
access_key = "xxxxxxxxx"
secret_key = "yyyyyyyyyyy"
}
}
Or you can create an AWS (default) profile in your home directory (Docs) and remove your credentials in your terraform code (preferred option, when you store your config in a version control system).

Answer 2

As pointed by #JimUdall in the comment, if you are re-running init on an updated backend configuration, you need to use -reconfigure for the updated config to apply the changed configuration.
terraform init -reconfigure