Concept and low fidelity wireframes for DX score
Closed this issue · 14 comments
Description
We want to encourage API owners in taking into account developer experience for their APIs. One of the ways to enourage this would be an incentive system, which gamifies the progress of adding e.g. information, documentation and media to the API profile.
Goal
- Define and document which parts of the API profile enhance the developer experience.
- Consider how to gamify this and how to create a sense of progress
- Also, think of easy ways to show completeness (percentage, points etc. ) and to go from viewing score / completeness to improving the score.
- Benchmark other solutions (like LinkedIn profile completion).
- Consider both short term and long term vision.
- Create at least low fidelity wireframes for current profile.
See also the EPIC description and links from the comments for #2297
Wireframes
-
API Users can see DX score of an API on the API profile heading.
It is represented with a Conical flask with a number associated with it.
The number represents DX value of an API
-
API owners can identify different fields where DX score is applicable throughout the API profile.
The DX specific field has conical flask appearing beside the field name.
On clicking the flask icon, a tooltip appears.
The tooltip contains messages to notify how a specific field can be used to improve DX of the API.
-
When a new DX score is achieved, it appears as a notification.
the total dx point also appears
-
At the bottom part of Details tab, a new section "API DX" appears.
This section contains some logo indicating different aspects of the API and its profile (e.g. General Information, Documentation, Performance, Related Media, etc.).
Initially the logos are disabled.
When all subfields under each criteria logo are filled up, the logo becomes green.
-
(depending on the placement: either Details Tab or Settings) API owners should be able to see how much score they earn on filling up a specific field.
The field names along with allocated DX score should be visible to API owner.
The fields that are completed will appear green.
The rest will appear as grey.
-
In Brand Setting, we can add a new section, where site admin can decide the DX score for each specific fields.
Why API Developer Experience Matters More Than Ever
Can we implement checking the following criteria with a monitoring system/existing proxy in APInf?
This seems to be a good idea on to rank an API or its completeness
Some advanced level things can be considered as well:
- existence of multiple data source
- improving filtering
- searching
- checking API rate
- Ability to return list of items in once call
A good documentation indicates how well an API is managed.
So if API owner provides code snippets, SDKs, etc. for consumers to try out, this could lead to a good DX and API profile can be updated accordingly/
A quick start guide to show how to use the API to build different use-cases is always helpful for API consumer. Since we have Oembded contents to add external media, adding a reference to such tutorial can also attribute to DX and the profile can b updated accordingly.
"Begin with a Quick Start Guide and follow with tutorials showing them how to implement interesting use cases with your API. Provide sample snippets using the SDKs in as many languages as possible, so that developers can simply copy and paste the code and communicate with your API with minimum effort."
An API-First approach is a good way to increase DX. This aspect can be achieved by:
- allowing an API to be available from the very beginning
(this ensures branding, open innovation between end users and company, etc.)
From issue #2297 the initial DX can be achieved the the following existing APInf features for API profile
- title
- description
- documentation
- backlog
Not sure with the estimate, but we can say about 80% completeness of API profile if the above fields are filled. The eventual progress happens when API owner starts adding advanced criteria in the API profile.
N.B: Initially an API is added by defining name, description, endpoint and lifecycle status. So if these fields are defined accordingly, we can give the profile a 60% progress rate.
- Documentation
- Clear and Up-To-Date
- logically organized sections
- clear API reference
- Getting Started
- How-Tos
- Quick Starts
- Ease of Use
- reducing non-coding setup requirements. (creating accounts, apps, signing up for a paid plan, or talking to a sales person)
- SDKs in multiple language. Make them open source so communitycan submit pull requests (Ruby, Python, NodeJS, PHP, and Java)
- Ease of Debugging
- API Dashboard
- Easy to Get Help
- Dedicated API Suport Team
- GitHub and StackOverflow issues
Badges ca be associated with API profile to create motivations among API developers to remain in ranking by providing good DX
https://openbadgefactory.com/fi/
https://www.discendum.com/etusivu/
Proposing two phase development of DX:
-
Using existing APInf Features
-
Integrating newly developed features as future development strategy
-
existing APInf Features:
- on providing complete information when Adding an API (Name, Description, API end point and lifecycle status), profile will be 60% complete.
All of the above fields need to be filled in order to get this progress.
On providing only mandatory information, profile can be shown as 55% complete.
A badge can be associated with this progress (name and look needs to be decided later). Badge will be displayed only when profile has 60% progress. - When API owner is redirected to the API profile, this notification can appear immediately (consider how FB games notified when you achieve a new rank or reach a level).
- Badges can appear to both API owner and consumers.
But the progress should be visible only to API owner. - Further progress can be added when Ower adds a swagger document (75%) and starts addng backlog to profile (80%). On getting 80% progress, API profile gets another badge after wards.
- After wards profile progress becomes a bit slower to achieve and it involves setting up advanced settings. (e.g. adding a proxy)
Considering the existing features in APInf
We should have a new feature for API owner to know how much the profile is complete by comparing points given for adding a specific field.
This feature should include a list of APInf features for API profiles and numbers associated with them when a particular field is filled.
The fields can be categorized as completed and incomplete.
Similar to this article, the points distributed can be:
General
- Description = 4
- API Logo = 4
- Metadata = 4
Advanced
- Lifecycle Status = 5
- Swagger Documentation = 50 (I'll add a related issue about it)
- Quick start tutorial = 10
- API connected to an Organization = 5
- Backlog (atleast 1) = 5
- Public/Private visibility (with API profile getting point on being public) = 4
- Link of apps/ services developed on top of the API = 5
Technical
- SDK Generation (min. 5 popular languages) = 5
- Adding an end point to monitor = 5
- Enabling Key requirement = 5
- Rate limit mode = 5
- API Performance monitoring = 5 (I'll add a related issue about it)
The total DX score for an API profile will be 100. Once an API profile reaches this number, it can be considered as a good API.
To Maintain the quality of the API, we also need to consider the following scenarios:
- How well is the API documentation structured? Does it include well define objects? does it have use cases? when was the last update made?
- When the API profile has been updated last (in terms of adding related media, new backlogs, changing lifecycle, etc.)
- What is the consistency of the API performance? (number of successful API calls, deviation of response time, etc.)
- Response time of API owners to a particular feedback/issue posted by API consumers
- If GitHub is associated with the API development, how many issues are opened/closed? What were their time period? etc.
To keep the feature simple, I am suggesting not to incorporate gamification aspect right at this moment.
Gamification sounds good, but our primary focus should be to implement a good DX framework for APInf. And that would require careful considerations. However, we can show a pseudo gamification concept
@bajiat @kyyberi @philippeluickx looking for your feedback and suggestions.
My opinion comes a bit late, but i don't like the scores. Setting scores for different fields looks complicated to me. Might be tempting for the young generation who likes mobile games, but what about others?
Did we consider some bar as maybe in LinkedIn, which shows that your "profile" is 70% ready and which fields you should still fill in to get optimized results?
@Laura-Sampo A percentage can always be an option although that would be numeric as well.
After a discussion with @philippeluickx and @kyyberi we thought to propose a DX score which would be 100 in total. This can be incroporated in UI (e.g. "70 out of 100") if everyone decides about it.
However, I am more into choosing a number.
The reason behind is that, in LinkedIn, when you keep on adding information on profile, it gets updated and doesn't fall usually.
But what if your API had a high DX score in the beginning and you decided not to take care of it? (e.g. updating documentation, keeping API performance consistent, going back to consumer queries, etc.) the DX doesn't remain same.
If we think about near future implementation, we can always add features to measure and assign appropriate DX to a profile.
I understand your concern here, considering both are profiles. But if you put an API profile and a Professional profile in LinkedIn side by side, you'll see that the context of use is different.
after a discussion with @bajiat we propose the following:
- on any lifecycle status, the API profile DX should be updated. On changing the lifecycle phase next time, the DX score will not be affected. Only removing the status would decrease the DX.
- Quick Start as OEMBD content might be too complicated to decide. Perhaps having a related media should add some points to DX score. but it should not be as high as 10.
- DX score for SDK should be part of the swagger documentation score, not something separate on its own.
This is swagger codegen is always available when there is a valid swagger file. - On disabling API key requirement, the given score will be deducted. (-5) [reason, if we had a customer and a consumer dashboard, they could follow the API calls for a particular API where they were using the related API key]
- need use cases for "DX score on rate limiting"
- API performance monitring should have a higher max score (ex: 30). This should be determined from the threshold value of performance at a particular period, API responses (HTTP statu codes for example), API response time, etc. At worst the score can also be negative, in such case,Overall DX value would decrease.
@philippeluickx @kyyberi looking for your feedback.
question what happens to the DX score of an API that has been deprecated? Should we freeze the scoring? change the DX icon color?
Deprecated does not have dx score.
how we should consider users' feedback in rating an API or looking at the response rate from API owners to user's questions.