Single AWS Cloud Costing
By default, when adding AWS accounts as clouds in Morpheus, no costing data is displayed unless you configured the COSTING setting in the Advanced Options.
When setting the COSTING field to Costing, Morpheus will use data it retrieves from the account using Cost Explorer and our own additional logic to help provide cost details for the cloud. This default costing is referred to as Metered, and the costs are not as precise as they could be.
We can verify that the cloud is using Metered costing by navigating to Operations > Costing > Invoices and locating a Cloud type invoice for the cloud and clicking on the Invoice ID link.
Note that if the cloud was just added, an invoice may not be available immediately. It may take up to an hour to generate an initial invoice
To ensure a more accurate costing is displayed, the AWS cloud can be configured to use a Cost and Usage Report (CUR) from AWS. The AWS CUR is extremely accurate and displays all costs line by line in a large CSV. Morpheus is able to both create these for you but also ingest them to ensure the costs are as accurate as possible, using the same data AWS is. When Morpheus ingests the Costing Report, this will enable Actual costing, as opposed to Metered. To create a Costing Report via Morpheus and enable Actual costing, follow these steps:
- Edit/Add an AWS cloud (
Infrastructure > Clouds > Add/Edit) - In the
Detailssection
a. Be sure to choose theRegionwith workloads you want to inventory
b. Enter the appropriate user credentials that has access to create CUR files and S3 buckets, along with permissions to inventory compute, S3, etc.
c.Inventorycan be left disabled, as the Global region won’t need to sync VPCs - Ensure the
COSTINGfield is set to Costing in theAdvanced Optionssection (towards the bottom of the list) - Select Create New from the
COSTING REPORTfield in theAdvanced Optionssection
a. Any previously created CURs will be shown in the drop down as well. If a report is configured correctly, they can be chosen instead of creating a new report. Needed settings can be found in the v4.2.2 and Below documentation - Enter a unique costing report name, if others exist, in the
COSTING REPORT NAMEfield - Enter a value for the
COSTING FOLDER(also known as the prefix in the AWS CUR) - Select Create New from the
COSTING BUCKETto create a new S3 bucket
a. Alternatively, you can choose an existing S3 bucket but you must ensure the user configured on the cloud has access to the S3 bucket to read the CUR data - Enter a globally unique S3 bucket name in the
COSTING BUCKET NAMEfield - Choose a region for the S3 bucket
- Click the
Save Changes/Nextbutton to create the report and S3 bucket
In the AWS console, we can see the report was created by navigating to Billing > Cost & usage reports:
Same with the S3 bucket by navigating to the S3 service:
Morpheus will automatically apply the appropriate policies to the bucket created and will automatically ingest the data from the CUR created
Note that the CUR file updates in AWS once per day. When the CUR is initially created, there will be no cost data, just a file to test programmatic access. Until this data is generated by AWS, Morpheus will not show costing data. Additional AWS CUR Info
Once the CUR data has populated into the bucket, Morpheus will be able to ingest the data and present it in the cloud or in the invoices. Morpheus will sync the costing data automatically at intervals but these can be started by navigating to cloud Infrastructure > Clouds > [click the cloud link] and then click the REFRESH button and choose Costing
Nothing needs to be changed on this popup.
If no PERIOD is chosen, then it will refresh all of the data available. If a single month is chosen, only that month will be refreshed with the data available. Rebuild will purge all previous data and then sync all the data again for period selected.
After initiating a refresh of costing, the status can be monitored by navigating to Infrastructure > Clouds > [click the cloud link]
After the Costing Status returns ok, the cost data will be displayed below:
Which we can see matches up to the AWS Console:
We can verify that the cloud is using Actual costing now by navigating to Operations > Costing > Invoices and locating a Cloud type invoice for the cloud and clicking on the Invoice ID link.
AWS Organization Cloud Costing
Update 12/01/2023: Instead of using the Global (Cost Aggregator Only) region as described below, the All Regions region is now available and recommended. Similar to Global (Cost Aggregator Only), the All Regions will sync costing for the No Region and the Global Region. The below configuration is still valid but the All Regions region would be the recommended choice going forward.
The previous steps detailed how to setup costing on an individual cloud, which could be performed on each cloud added. When adding a new Morpheus cloud for the same AWS account, just a different region, you would see the previous Costing Report and could select that instead of performing all of the steps.
However, many organizations are using AWS Organizations, which allows cloud administrators to create additional AWS accounts that are linked to the AWS Organization. The above process can become tedious if there are hundreds of accounts. That is where the AWS Global (Cost Aggregator Only) region is powerful.
Initially, when enabling AWS Organizations, the first account becomes the root or management account, where all costs can be consolidated to, if consolidated billing is enabled. This means that single management account has the cost information for all the child AWS accounts that would be created in the organization. We can now add this management account as a cloud in Morpheus, using the Global region, which will allow us to create a single Costing Report and all costing data for all AWS accounts in the organization will be available. This eliminates choosing a report on each cloud and the additional network overhead of each cloud syncing the costing data.
The process is very similar to the process seen in the Single AWS Cloud Costing section above. We’ll add our AWS management account, which contains all of the consolidated billing, choose the Global (Cost Aggregator Only) region, and create a Costing Report. Later, additional child AWS accounts added as Clouds in Morpheus will automatically received costing information from that single sync.
Note that you still need to enable at least Costing in the COSTING field in the Advanced Options of the child cloud, so the costing functionality is enabled. Costing and Reservations can also be enabled, which will additionally enable Reservations and Savings Plans recommendations, if the configured user has access IAM to Cost Explorer.
Steps to adding the AWS Global (Cost Aggregator Only) cloud:
- Add an AWS cloud (
Infrastructure > Clouds > Add) - In the
Detailssection
a. Be sure to choose theRegionGlobal (Costing Aggregator Only)
b. Enter the appropriate user credentials that has access to create CUR files and S3 buckets
c.Inventorycan be left as Off, as the Global region won’t need to sync VPCs - Ensure the
COSTINGfield is set to Costing and Reservations in theAdvanced Optionssection (towards the bottom of the list) - Select Create New from the
COSTING REPORTfield in theAdvanced Optionssection
a. Any previously created CURs will be shown in the drop down as well. If a report is configured correctly, they can be chosen instead of creating a new report. Needed settings can be found in the v4.2.2 and Below documentation - Enter a unique costing report name, if others exist, in the
COSTING REPORT NAMEfield - Enter a value for the
COSTING FOLDER(also known as the prefix in the AWS CUR) - Select Create New from the
COSTING BUCKETto create a new S3 bucket
a. Alternatively, you can choose an existing S3 bucket but you must ensure the user configured on the cloud has access to the S3 bucket to read the CUR data - Enter a globally unique S3 bucket name in the
COSTING BUCKET NAMEfield - Choose a region for the S3 bucket
- Click the
Save Changes/Nextbutton to create the report and S3 bucket
After adding the Global (Cost Aggregator Only), a costing refresh can be performed, as mentioned in the previous section. Once performed, we can see some cost data appearing in the Global cost aggregator, which are the No Region and global costs. These are costs that are not tied to a region, such as Route53, upfront costs, etc. These also match what is seen in the AWS Console.
Now that the Global (Cost Aggregator Only) region has been added, any additional child AWS account added as clouds in Morpheus will have costs assigned automatically if Costing or Costing and Reservations is enabled on the COSTING field in the Advanced Options section.
Reminder: The COSTING REPORT does not need to be configured on the child clouds added, since the costs are sync’d from the global cloud.
Finally, either start a Costing refresh on the Global (Cost Aggregator Only) cloud or wait for the next time it runs on its own and the child cloud(s) will show precise costs and invoices will be Actual as well.
