[BreachExchange] 5-Star Ratings – Just How Vulnerable Is That Shiny New Application?

[Inga Goddijn]

https://www.riskbasedsecurity.com/2018/10/5-star-ratings-just-how-vulnerable-is-that-shiny-new-application/

Star-based ratings are everywhere you look these days. From hotel and
restaurant reviews to doctors and lawyers, practically every service and
seller imaginable is subject to some sort of performance score. These
rating systems are so familiar in fact that they have become the de facto
shorthand for making fast judgements about the quality of a product or
service. Despite – or perhaps because of – their prevalence, the basis for
how these ratings are developed is often overlooked, which can lead to
dubious scoring or ratings of questionable validity.

Consider some of the most common examples of 5-star ratings.

   - On the more rigorous end of the spectrum, the NHTSA (National Highway
   Traffic Safety Administration) <https://www.nhtsa.gov/ratings> provides
   5-star crash ratings for automobiles.  You’ve probably seen the crash
   dummies and the slow-motion videos that the NHTSA uses to assess how a
   given automobile make and model does in certain crash conditions. These
   ratings are widely respected thanks in large part to the meticulous testing
   process and are used by organizations and individuals alike to make
   informed decisions.
   - On the opposite end of the spectrum, you’ll find 5-star ratings on
   many popular retail buying sites which are typically based on customer
   reviews.  Are they helpful? Yes, somewhat. Are they based on input from
   professional analysts who specialize in assessing the quality of that
   wireless keyboard or fleece jacket you’re eyeing?  Probably not.
   - Then there is the muddy middle ground. Hotels and restaurants have
   been assessed using 5-star rating systems for a long time. Official ones
   like those from Forbes <https://www.forbestravelguide.com/about>, AAA
   <https://www.aaa.com/diamonds/diamond-ratings-definitions.html>, and
   Michelin
   <https://guide.michelin.com/us/washington-dc/the-inspection-process> are
   based on well-defined and established criteria.  They are grounded in data
   about the hotel and restaurant industries with common measurements used to
   assess the various establishments being reviewed.  Then there are the
   crowdsourced scores, largely based on aggregated customer feedback which
   can be quite subjective.

What does this tell us? That all ratings can bring some value to the table,
but it is the more exacting and objective systems that can provide deeper
insight into a product or service. When it comes to assessing the risk
associated with using a piece of software or selecting a vendor for your
next project, it’s all the more important to look to a thorough rating
system. One such system is an objective comparison based on data for known
vulnerabilities, using criteria such as:

   - How often vulnerabilities get released
   - How exploitable (easy v. hard) the vulnerabilities are
   - How much damage can an exploit cause

This is a helpful way to assess software investments. After all, even the
slickest application can quickly lose its luster if it requires constant
patching and puts a drain on already tight resources.

Risk Based Security’s VulnDB <https://vulndb.cyberriskanalytics.com/> provides
this objective vulnerability intelligence for vendors and their products.
Our expert research team assesses vulnerabilities for risk and
exploitability. Our proprietary model calculates 5-star ratings for each
software product and then aggregates 5-star ratings for each vendor based
on their portfolio of products, using the history for all the
vulnerabilities to derive the rating.

What can you use this information for?

*Vulnerability Evaluation:* You can evaluate software products for
vulnerability quality. For example, here’s ratings data for a screen
sharing tool with a 5-star rating of 2.5.  Major factors in that rating are
the number of vulnerabilities over the last 11 months and the fact that
several vulnerabilities can be exploited remotely.

*Product Comparisons:* Another use for 5-star ratings is product
comparisons. Below is a comparison of three similar products.  The first
two are on the lower half of the 5-star rating scale, mostly likely due to
the large number of vulnerabilities and relatively short amount of time
between when new vulnerabilities are disclosed.  The third product has a
very high 5-star rating, and even though the average CVSS scores are
similar to the others, vulnerabilities occur on a much less frequent basis
(only one every 174 days).

5-star ratings are available for many  products and services. When these
ratings are based on objective data, they are useful aids for making
informed decisions free of bias. The 5-Star ratings included in VulnDB
<https://vulndb.cyberriskanalytics.com/> provide meaningful insight into
the performance of products and vendors over time. Whether used on their
own or in combination with other objective performance measurements,
VulnDB’s 5-Star ratings can provide a powerful tool for selecting your next
service provider.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.riskbasedsecurity.com/pipermail/breachexchange/attachments/20181105/c4fdcc28/attachment.html>