Product Manager Speak: Deconstructing Ratings in India and the U.S.

This article has been authored by Kriti Mehra from UpGrad and Andrew Kaufteil from Cooper and was originally published on Cooper.

India and the United States have a lot in common.

Both countries are characterized by growing and diverse economies, and inhabited by passionate and entrepreneurial citizens.

In FY16, the US was the world’s largest economy, and India was the world’s fastest growing major economy, at a 7.6% GDP growth rate. In particular, the design and tech sectors of both countries are growing, as are our partnerships with one another. For example, India is the largest market for Uber outside the United States.

Walking around Bangalore or Hyderabad, you see logos of Facebook, Apple, AirBnB sewn onto backpacks, making you feel like you’ve been transported to Silicon Valley. Given our growing convergence, it’s important to understand our similarities as well as differences. One aspect of our cultures that are a bit different are ratings.


Image source: BBC

Rating Inflation

On a recent trip to India, Andrew Kaufteil, Senior Director of Growth & Partnerships for Cooper, ordered an Uber and noticed the assigned driver partner had a rating of 4.2. He had never seen an Uber rating so low. This scared him and he immediately canceled the request. He asked himself: what kind of a maniac could have a 4.2, and why was this person not fired?

Just a few months ago, he had looked at his Uber passenger rating (how he had been rated by drivers), and had been disappointed to see a 4.81. How could Uber keep a driver partner with a 4.2?!

He re-ordered, and his next driver partner had a 4.31. He needed to get to his destination, so even though the rating was low, he pursued the ride. He was surprised that his driver was courteous and competent, and the car was clean; a clear “5” in his estimation. In the following weeks, he identified lower overall ratings than he was used to in the States.

Two approaches to ratings seem prevalent in India. One is an empathetic approach in which the consumer views delays in pick-up or speaking on the phone as minor annoyances and gives a 5 rating so that the driver is not penalized. The other is more value-based, where a 4 rating is associated with meeting expectations and a 5 is associated with exceeding expectations.

In the US, if service is competent and nothing goes wrong, a perfect rating — a 5 in the Uber example — is assumed. In India, consumers have a greater expectation of value. A perfect rating demands extravagant service.

Image source: Uber

This reality has broad implications for business. For example, many companies use Net Promoter Scores (NPS) to measure customer engagement with a product or service. However, given the cultural considerations explained above, companies with more Indian customers will likely yield a lower NPS than companies with American customers.

This is problematic for benchmarking customer loyalty to Indian versus American companies, or American companies catering to large groups of Indian customers, or vice-versa. In order for the score to be relevant, it must be normalized.
Moving forward, understanding your customer’s relationship with your product or service will require comprehensive design research to unlock customer goals, behaviors, and motivations, coupled with sophisticated data science modeling, taking into account where the rater is located, or their cultural background.

Additionally, India uses ratings in different ways than the United States. Take for instance, Andrew presenting at Agile India 2017. When he looked at his profile leading up to the conference, he observed a rating of 210. He was jarred by the rawness of the rating, and embarrassed when he saw that another speaker at the conference, Evan Leybourn had 998 points. From the ConfEngine website, here is an explanation of the “points system” for the conference:

  • You earn 30 bonus points on your first activity in every new conference.
  • You earn 1 point when others like your proposal.
  • You spend 1 point when you like others’ proposal.
  • You spend 10 points when you propose a session.
  • You earn 100 points when your proposal is selected.

According to Kriti Mehra, raw points systems are widely used in India to create a sense of transparency and competitiveness that prompts individuals and companies to increase their engagement with web platforms.

Many US-based sites reward increased engagement differently; they hide their methodology or relate points to levels, labels, and/or titles to increase engagement with web platforms. For example, hyper-engaged users of Yelp are labeled “Yelp Elite,” though criteria and points for inclusion in Yelp Elite are shrouded and seemingly subjective. On TripAdvisor, a user’s points correspond with different “levels” (1-6).

Image source: TripAdvisor

The nuances of American and Indian approaches to rating are a reflection of different cultural contexts. It will be interesting to see how these systems evolve over time.

Will Indian sites soon have winners and losers?
Or will US-based sites adopt raw numerical scores?

Uncovering these differences will allow companies to design products and services for diverse customer bases, and to learn new approaches to engagement driven by design thinking and big data.

Study Product Management Courses online from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.

Featured Program for you: Design Thinking Certification Program from Duke CE

How to improve my product’s ratings on Playstore?

If your product has a low rating on Playstore, there are some very simple steps you can take to rectify the situation. One of the most important of these is to address negative reviews. Go through all the negative reviews which have been posted, and respond to each one, asking the reviewer to contact you personally. In case technical issues have been reported, get in touch with your developers and make sure that these are sorted out at the earliest. In the meantime, bolster the reviewer’s experience by taking solving their issues and convincing them to post a positive review. Ask your happy customers to post positive reviews and offer rewards in the form of discounts.

Are ratings a true measure of a product’s success or failure?

The answer is yes and no. Just because a product has been given too many positive ratings, does not mean that it is successful. A lot of organisations often tie up with agencies that may help them with inflated ratings, all of which may be untrue. Unfortunately, there would be no way to verify this. It is better to deep dive and find out if your customers share the negative opinions of the reviewers. If they do, then you know that you have a problem. If not, then you should look at strategies to improve your ratings by asking existing customers to give good reviews, and addressing negative reviews through dialogue.

How to decide when ratings of a product have been inflated?

Check the online reviews of the product. If most of them seem negative, and report similar issues, then it could be a sign that the company has been ignoring teething issues reported by genuine customers and is relying on marketing tactics to inflate their ratings. Next, analyse the good reviews of the product. If they sound repetitive, claim benefits and services which the product simply doesn’t offer, then it is again, a very good possibility that these reviews are fake. Lastly, experience the product yourself from the customer’s point of view. If you are dissatisfied with the experience, it is possible that most customers would also feel that way, indicating that the ratings have been inflated.

Want to share this article?

Build Cutting-Edge Products

UpGrad's Product Management Certification Program
Learn More

Leave a comment

Your email address will not be published. Required fields are marked *

Our Popular Product Management Course

Get Free Consultation

Leave a comment

Your email address will not be published. Required fields are marked *

Get Free career counselling from upGrad experts!
Book a session with an industry professional today!
No Thanks
Let's do it
Get Free career counselling from upGrad experts!
Book a Session with an industry professional today!
Let's do it
No Thanks