Intro

The cloud offers unlimited scalability at lower costs with the available resources. Companies enable their engineering teams to launch infrastructure as and when needed. However, since costs aren't shown to engineers before Cloud resources are deployed, the real cost implications are unknown. That causes higher than expected costs and ballooning budgets over time, which is a nightmare for CFOs considering Gartner estimated Cloud wasted spend exceeding $26.6 billion in 2021. Orgs Blaming dev teams or business units after the damage is done is pointless. What then, should they nuke everything after $$$ overspend? which one to kill first?

The usual suspects? Good ol’ Overprovisioning (~8.7billions) and idle resources ($14.5 billion).

Thus, the greatest challenge for every organization remains:
“How to proactively Reduce Cloud Wasted Spend?”

Today, we’ll explore an awesome solution that helps DevOps, SREs,& Devs continuously reduce their cloud costs.

This tool is called Infracost!
Table of contents

WHAT IS INFRACOST?

Infracost is a super cool tool that live calculates the cost of your Terraform resources on AWS, GCP or Azure before you even hit deploy. All it needs is your terraform files and its cli installed.

Infracost CLI fetches the prices from its Cloud Pricing API, that stores/updates the prices from AWS/Azure/GCP pricing APIs. Their GraphQL-based API gleans up to 3 million prices to return the cost .

VS Code extension: by far their slickest feature, where cost is displayed inside your tf code (See below)

Modules: Infracost can even scan modules within your project and pull all the pricing details in seconds.

How does Infracost work?

CLI Extracts cost-related parameters from parsing terraform files such as the instance type or disk size.
Retrieves prices from the Cloud Pricing API which returns the prices.
Calculates the monthly costs and outputs them in table, JSON or other formats.

No need to scroll long pricing pages, just pick a resource value in your *.tf file,& infracost will get its cost for you.

Syntax (basic): the format and out-file arguments are optional

 infracost breakdown --path myterraDir --format json --out-file infracost-base.json

Terraform variables can be set using --terraform-var-file or --terraform-var

I.Installation

Windows: Download and unzip the latest release

Note: In order to use the VS Code extension, you need to move the exe file into folder that’s in your `PATH`

I use pathed to add it’s folder in my system %PATH% environment variable permanently in a terminal

C:\> PathEd.exe add "C:\InfraCost_path\"

Linux/MacOS: The easiest way is to use the install script (in one line):

# Downloads the CLI based on your OS/arch and puts it in /usr/local/bin
curl -fsSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh| sh

II. Setup and authentication

C:\> infracost auth login
We're redirecting you to our log in page, please complete that,and return here to continue using Infracost.
Waiting...

In the login Page you have two options sign up via email or github

After this step your API key will be stored in credentials.yml file and infracost will be all set

# The API key was saved to C:\Users\brokedba\.config\infracost/credentials.yml
Your account has been authenticated.

`You are now ready to use infracost by navigating to any terraform project and running the cli command`

# check Authentication
C:\> infracost configure get api_key 
      ico-7XXXXXXXXXXXXXXXXXXXXX 

C:\>notepad  C:\Users\brokedba\.config\infracost/credentials.yml
version: "0.1"
api_key: ico-7XXXXXXXXXXXXXXXXXXXXX
pricing_api_endpoint: https://pricing.api.infracost.io

III. Use infracost

Let's see if we can use a simple terraform stack from my Github repo to check the cost of a vm compute
NEW #BlogPost #Azure #Terraform for dummies part III:
☑️☁️How to create a #VPC
☑️ static website (webserver) using Terraform
☑️+ few #tips
☑️+ little comparison #Azure VS #AWS providers.
https://t.co/3pRZ1q6EUk pic.twitter.com/oZHihY9Kun
— Kosseila.HD aka BrokeDBA (@BrokeDba) October 19, 2021
Clone the repository containing 3 of my terraform stacks (Azure/AWS/GCP) in your machine
```
$  git clone https://github.com/brokedba/terraform-examples.git
```

1. Azure stack

First navigate to my azure launch instance subdirectory (see blog)

$  cd  /terraform-examples/terraform-provider-azure/launch-instance/

Here's the content:

$  tree .
|-- cloud-init    ---> Subfolder containing cloudinit
|-- compute.tf    ---> Instance related terraform configuration
|-- outputs.tf    ---> displays the resources detail at the end of the deploy
|-- variables.tf  ---> Resource variables needed for the deploy   
|-- vnet.tf       ---> VNet used by the compute

Run Infracost

I will use one more option to include services that don’t incur costs using --show-skipped
```
 launch-instance/$ infracost breakdown –-path . --show-skipped
```

cost estimate of terraform config live before deploy.

Seems that our cheap static website will cost me USD$12/month

Of the 9 resources, 2 have a fixed pricing (vm compute & public ip) and one(os_disk) based on usage
7 are free which I wanted to display using --show-skipped option
Will talk later about usage based estimates

2. AWS

Same stack here but different provider (see blog)

$ cd  /terraform-examples/terraform-provider-aws/launch-instance/  
$ launch-instance/$ infracost breakdown --path . --show-skipped

USD$10/month for a t2.micro compute. Notice there is no fee for the public IP because it’s FREE is AWS.

3. GCP

Last supported cloud provider (see blog)

$  cd  /terraform-examples/terraform-provider-gcp/launch-instance/  
$  launch-instance/$ infracost breakdown --path . --show-skipped

USD$6.51/m for the same simple web instance stack . Again, no fees for the public IP nor usage based storage. Although GCP usually charges 0.004/hour for Static IPs.

Change Currency

What if I want to show the cost in CAD$ or Euro? Well, all you need is to run the below configure command.

# List your preferred ISO 4217 currency and run the configure command
$ Infracost configure set currency EUR

Check if your stack estimate currency has changed (you might need to restart Visual Code to see GUI change)

This image has an empty alt attribute; its file name is image-5.png

III. Features

CI/CD integration:

Add Infracost to your CI/CD pipeline, to see cost estimates in pull requests before making changes. It supports GitHub actions, GitLab CI, Circle CI, bitbucket, Jenkins & more

Infracost usage based estimate

Infracost distinguishes the price of a resource from its cost

Price is a per-unit value published by cloud vendors (fixed)

Cost is equal to (resource's price) x (its usage), see below AWS Lambda example

Name Quantity Unit Monthly Cost aws_lambda_function.hi ├─ Requests Cost depends on usage: $0.20 per 1M requests └─ Duration Cost depends on usage: $0.0000166667 per GB-seconds PROJECT TOTAL $0.00

There are two options for showing costs instead of prices:

Fetch usage from CloudWatch/cloud APIs: to narrow usage-based costs (AWS only).
Specify usage manually:

Use --sync-usage-file option to generate a new usage file
Edit the file with your usage estimates & run below command (more details here)
```
$ infracost breakdown --path . --usage-file infracost-usage.yml
```

Infracost Cloud

SaaS version which allows team lead visibility across all changes(i.e most expensive pull requests)
I had a free trial upon sign up & all my cost estimates were actually sent to my cloud account $Infracost Cloud dashboard showing pull request cost changes over the last 30 days$

Security and Privacy

Rest assured Infracost doesn't touch secrets and only parse what pricing API needs to return the cost.

Conclusion

There you have it, I promised I’ll blog about this amazing solution & it’s been very easy and fun to explore
This was longer than expected lol, but I hope it was enough information for you to dive into infracost
Bare I mind that I couldn’t list all the features which would require web scraping their whole website ;)
infracost diff shows diff of monthly costs between current and planned state (using usage.yml file)
Among Future features we might have

Supporting Oracle Cloud : please upvote the issue-911 in GitHub
Point to statefile and get the cost in a machine that doesn't even have terraform installed
Hiding and Sorting resources by cost
Integrate infracost with remote backend based tfsates (S3)

Friday, August 19, 2022

Terraform live cost estimate using infracost free plugin

Intro

The usual suspects? Good ol’ Overprovisioning (~8.7billions) and idle resources ($14.5 billion).

Thus, the greatest challenge for every organization remains: “How to proactively Reduce Cloud Wasted Spend?”

Today, we’ll explore an awesome solution that helps DevOps, SREs,& Devs continuously reduce their cloud costs.

This tool is called Infracost! Table of contents

WHAT IS INFRACOST?

Infracost is a super cool tool that live calculates the cost of your Terraform resources on AWS, GCP or Azure before you even hit deploy. All it needs is your terraform files and its cli installed.

Infracost CLI fetches the prices from its Cloud Pricing API, that stores/updates the prices from AWS/Azure/GCP pricing APIs. Their GraphQL-based API gleans up to 3 million prices to return the cost .

VS Code extension: by far their slickest feature, where cost is displayed inside your tf code (See below)

Modules: Infracost can even scan modules within your project and pull all the pricing details in seconds.

How does Infracost work?

CLI Extracts cost-related parameters from parsing terraform files such as the instance type or disk size.

Retrieves prices from the Cloud Pricing API which returns the prices.

Calculates the monthly costs and outputs them in table, JSON or other formats.

No need to scroll long pricing pages, just pick a resource value in your *.tf file,& infracost will get its cost for you. Syntax (basic): the format and out-file arguments are optional

Terraform variables can be set using --terraform-var-file or --terraform-var

I.Installation

Windows: Download and unzip the latest release

Note: In order to use the VS Code extension, you need to move the exe file into folder that’s in your PATH

I use pathed to add it’s folder in my system %PATH% environment variable permanently in a terminal

Linux/MacOS: The easiest way is to use the install script (in one line):

II. Setup and authentication

Register for a free API key,used by the CLI to authenticate/retrieve prices from our Cloud Pricing API

In the login Page you have two options sign up via email or github

After this step your API key will be stored in credentials.yml file and infracost will be all set

You are now ready to use infracost by navigating to any terraform project and running the cli command

III. Use infracost

Let's see if we can use a simple terraform stack from my Github repo to check the cost of a vm compute

Clone the repository containing 3 of my terraform stacks (Azure/AWS/GCP) in your machine

1. Azure stack

First navigate to my azure launch instance subdirectory (see blog)

Here's the content:

Run Infracost

I will use one more option to include services that don’t incur costs using --show-skipped

Seems that our cheap static website will cost me USD$12/month

Of the 9 resources, 2 have a fixed pricing (vm compute & public ip) and one(os_disk) based on usage

7 are free which I wanted to display using --show-skipped option

Will talk later about usage based estimates

2. AWS

Same stack here but different provider (see blog)

USD$10/month for a t2.micro compute. Notice there is no fee for the public IP because it’s FREE is AWS.

3. GCP

Last supported cloud provider (see blog)

USD$6.51/m for the same simple web instance stack . Again, no fees for the public IP nor usage based storage. Although GCP usually charges 0.004/hour for Static IPs.

Change Currency

What if I want to show the cost in CAD$ or Euro? Well, all you need is to run the below configure command.

Check if your stack estimate currency has changed (you might need to restart Visual Code to see GUI change)

III. Features

Add Infracost to your CI/CD pipeline, to see cost estimates in pull requests before making changes. It supports GitHub actions, GitLab CI, Circle CI, bitbucket, Jenkins & more

Infracost distinguishes the price of a resource from its cost

Price is a per-unit value published by cloud vendors (fixed)

Cost is equal to (resource's price) x (its usage), see below AWS Lambda example

There are two options for showing costs instead of prices:

Fetch usage from CloudWatch/cloud APIs: to narrow usage-based costs (AWS only).

Specify usage manually:

Use --sync-usage-file option to generate a new usage file

Edit the file with your usage estimates & run below command (more details here)

SaaS version which allows team lead visibility across all changes(i.e most expensive pull requests)

I had a free trial upon sign up & all my cost estimates were actually sent to my cloud account

Security and Privacy

Rest assured Infracost doesn't touch secrets and only parse what pricing API needs to return the cost.

Conclusion

There you have it, I promised I’ll blog about this amazing solution & it’s been very easy and fun to explore

This was longer than expected lol, but I hope it was enough information for you to dive into infracost

Bare I mind that I couldn’t list all the features which would require web scraping their whole website ;)

infracost diff shows diff of monthly costs between current and planned state (using usage.yml file)

Among Future features we might have

Supporting Oracle Cloud : please upvote the issue-911 in GitHub

Point to statefile and get the cost in a machine that doesn't even have terraform installed

Hiding and Sorting resources by cost

Integrate infracost with remote backend based tfsates (S3)

Thank you for reading

No comments:

Post a Comment

Thus, the greatest challenge for every organization remains:
“How to proactively Reduce Cloud Wasted Spend?”

This tool is called Infracost!
Table of contents

No need to scroll long pricing pages, just pick a resource value in your *.tf file,& infracost will get its cost for you.

Syntax (basic): the format and out-file arguments are optional

Note: In order to use the VS Code extension, you need to move the exe file into folder that’s in your `PATH`

`You are now ready to use infracost by navigating to any terraform project and running the cli command`

I had a free trial upon sign up & all my cost estimates were actually sent to my cloud account $Infracost Cloud dashboard showing pull request cost changes over the last 30 days$