March 19, 2025

What is data ingestion and why is it important?

Data ingestion is a simple term for a complex set of actions. It is more than just collecting data from various sources — it includes normalizing, formatting and correlating differences into a single coherent data set. 

Cloud Service Providers (CSPs) are constantly evolving their cloud cost and usage data, what information is provided and how you access it. For example, the AWS Cost and Usage Report (CUR) can contain millions of rows and easily exceed the capacity of most desktop spreadsheets. When a new service is introduced, CUR fields can be added or formats changed. When you are using multiple CSPs, keeping up with the different formats and changes is a daunting task.

Adding to the complexity of the CSP-provided data, your business likely has important metadata to add context, such as business unit, cost center, owner, etc. Your metadata is a constantly evolving landscape as well. 

How can CloudHealth help?

CloudHealth seamlessly integrates with AWS, Azure, GCP, Oracle and Alibaba Cloud. If you are using a CSP that CloudHealth doesn't already integrate with, our "Bring Your Own Data" feature (currently in beta) can be used to integrate it with your other CSP data to give you a true view of your cloud environments. CloudHealth can connect to your VMware Aria Operations installations to include your private on-premises cloud data. 

CloudHealth can utilize data from third-party platforms such as Dynatrace, DataDog, New Relic and Chef. If your organization relies on Kubernetes, CloudHealth can collect CPU and memory metrics and include them as well. 

Let the CloudHealth team worry about keeping up with these challenges so you can focus on working to reach your FinOps goals, like getting this data to the right people so they can make the right decisions. 

How do you take data ingestion to the next level?

Once the "easy part" of getting CSP data ingested is done, adding your business-specific metadata is the next crucial step. At CloudHealth, we hear from many customers that they need to include information from their CMDB that is not part of the CSP resource tagging. 

You can leverage CloudHealth's custom tags for this integration. Custom tags gives you the ability to enhance resource metadata outside the CSP environment. By combining CSP data with your CMDB metadata, you will have a complete, robust view of your cloud resources and their cost, usage and management details. 

How can CloudHealth Professional Services help?

The CloudHealth Professional Services team can help architect a custom integration with CMDB. We have worked with many customers to integrate the CMDBs within ServiceNow itself or using native CSP tooling (e.g., AWS lambda, instances, etc.). An example of a high-level design for ServiceNow implementation might look something like this:

CloudHealth Professional Services works with you to build scalable and repeatable processes that maximize the existing tools and resources that you already have.

Interested in having a conversation about how we can help? Reach out to us today to schedule a conversation!