[Work_Item] Add SKU categorization hierarchy
Opened this issue · 9 comments
1. Problem Statement *
Describe the problem, issue, use case, or opportunity that this work item addresses.
Include practitioner quotes illustrating real examples a) of questions being asked by practitioners and b) value unlocked by answering these questions, if available.
- What is the problem?: Explain the context and why it needs resolution.
- Impact: Describe how the problem affects users, systems, or the project.
The most fundamental thing we are describing in FOCUS is the SKU. Yet as-is, the only thing we know about the thing we're explicitly being charged for is 2 IDs that need to be looked up in another system. And how those IDs map to the provider's price list is not clear since each provider uses different terms for those attributes. This makes relying on SkuId and SkuPriceId very difficult.
Use cases:
- High-level analysis of SKU usage across a large organization.
NOTE: This is one of the most common ways Microsoft customers look at costs across enterprise billing accounts today.
- Facilitates commitment discount utilization and coverage conversations because it helps identify the charges that may be eligible.
2. Objective *
State the objective of this work item. What outcome is expected?
- Success Criteria: Define how success will be measured (e.g. metrics and KPIs).
Since SKU attributes are the most common way leaders throughout an organization look at their cloud usage, there needs to be a human-friendly categorization of SKUs. At a bare minimum, we need a SkuCategory that is the same as ServiceCategory (since there will be a one-to-one mapping in some cases). A SkuSubcategory would also be helpful, but having a normalized column for this might require a significant effort.
Beyond normalized columns, FOCUS needs to allow practitioners to categorize their SKUs. Each of the main providers have a logical way to organize SKUs already so this is mainly about identifying clear names to describe a taxonomy of products. As an example, AWS has product_group and product_family in the CUR, GCP has SKU group as a concept outside of the cost data, and Microsoft has MeterCategory and MeterSubcategory. In some cases, providers merge multiple concepts into these attributes because there isn't a very well-defined structure that can support such a broad catalog of services. Having a predefined structure that is more open would go a long way to making it easier to understand what SKUs are being used across an expansive account, especially when those SKUs span services and need to be measured for things like commitment discounts.
There's already a somewhat common taxonomy for product categorization that exists that we could leverage as a starting point. You can find many different websites that discuss product categorization, but as a starting point, see https://catsy.com/log/product-categorization.
3. Supporting Documentation *
Include links to supporting documents such as:
- Data Examples: [Link to data or relevant files; DO NOT share proprietary information]
- Related Use Cases or Discussion Documents: [Link to discussion]
- PRs or Other References: [Link to relevant references]
See Microsoft ServiceFamily, MeterCategory, MeterSubcategory.
Related to #315
SkuCategory would likely mimic ServiceCategory, since every usage and purchase charge would map to a SKU. For instance:
- Compute
- Networking
- Storage
SkuSubcategory may differ from ServiceSubcategory. This would need more discussion.
For examples of the provider-specific SKU hierarchy, refer to the proposed SKU categorization appendix in the potato branch.
4. Proposed Solution / Approach
Outline any proposed solutions, approaches, or potential paths forward. Do not submit detailed solutions; please keep suggestions high-level.
- Initial Ideas: Describe potential solution paths, tools, or technologies.
- Considerations: Include any constraints, dependencies, or risks.
- Feasibility: Include any information that helps quantify feasibility, such as perceived level of effort to augment the spec, or existing fields in current data generator exports.
- Benchmarks: Are there established best practices for solving this problem available to practitioners today (e.g. mappings from existing CSP exports that are widely used)?
Add normalized and provider-specific columns for SKU categorization. Normalized columns would be similar to ServiceCategory and ServiceSubcategory but provider-specific columns would need to support more levels to account for the variations in types of related products. Note this is not a "type", but a grouping. A "type" is an attribute of a thing and not the grouping those "types" are categorized into.
5. Epic or Theme Association
This section will be completed by the Maintainers.
- Epic: [Epic Name]
- Theme: [Theme Name, if applicable]
TBD
6. Stakeholders *
List the main stakeholders for this issue.
- Primary Stakeholder: [Name/Role]
- Other Involved Parties: [Names/Roles]
TBD
@shawnalpay to port to Work Item issue template.
Summary Members' call on Oct 17:
#608 [FEEDBACK]: SKU Categorization Columns
Primary Issue: Practitioners have requested SKU categorization columns to help organize and differentiate various SKUs at different levels of granularity.
Core Problem: The current specification lacks sufficient categorization for SKUs, making it difficult to analyze or group SKUs effectively, especially in large multi-cloud environments. This is particularly important for normalized data as well as provider-specific attributes.
Divergent Views: Some members favored a simple categorization schema, while others argued for more granular, provider-specific categories. There were also discussions about maintaining consistency while supporting varied provider taxonomies.
Final Agreement: The group agreed to move forward with both normalized and provider-specific SKU categorization columns. This will help practitioners categorize and analyze SKUs across different cloud providers while maintaining provider-specific details where necessary.
Action Items:
- [Members-#608] Shawn @shawnalpay /Michael @flanakin : Draft a detailed proposal for both normalized and provider-specific SKU categorization columns, including examples.
[Members-#608] Joaquin @jpradocueva : Ensure the issue is formatted according to the new work item template for stakeholder review.
@flanakin Per our discussion last week, I've converted this issue into the Work Item format! A couple things:
- That catsy.com link doesn't resolve; I suspect they changed their site map since you authored the original item eight million years ago. Do you have a different URL that explains it?
- It would be extremely helpful to have some sample data for stakeholders to see. A quick dump of examples across providers would be great. For example, I know that GCP has a seven-level taxonomy. @kk09v, any chance I could ask you to help with GCP?
I'm going to add back the needs examples label because it will be very helpful to our discussion with stakeholders (and amongst ourselves) if we can see some real-world sample data.
Here's just one example. GCP SKU 026-B8EB-D1EF has a multi-level product taxonomy of:
GCP -> Compute -> Hyperdisk -> Hyperdisk Storage Pools -> Balanced -> IOPS -> Standard
Having a spreadsheet of many such examples (it doesn't have to be a full dump of all SKUs; say at least 1000?) will help drive understanding and prioritization.
Presented and discussed in the 11/1 FUG NA Meeting. Interest from attendees Eli Lilly, Nissan Americas, both adopters, in getting this breakdown of category by meter/service Type.
@flanakin @shawnalpay Is this the link to the blog that was suggested as a starting point?
https://catsy.com/blog/product-categorization/
Notes from the Maintainers' call on November 4:
**Context:**The addition of a SKU taxonomy would help standardize how SKUs are categorized across cloud services, improving resource grouping and cost tracking. This taxonomy must balance the need for detail with general applicability across diverse provider SKUs.
Level of Effort Required: Very High — Creating a universal SKU taxonomy is complex and requires input from multiple stakeholders to ensure it accommodates varied provider structures.
Comments from Members' call on November 7:
#608: The group analyzed this issue, and focused on SKU categorization, hierarchy, and taxonomy. TF-3 prioritized SKU ID as the most critical element for the current release.