What most product reviewers overlook

The world is filled with products. We have all had products that cost quite a bit of money that have disappointed, that we have regretted spending our hard-earned money on. We have all had products that turned out to be surprisingly good too. The problem is, how do you sort the bad from the good?  The question then becomes, how to get mostly or ideally only good quality products?

 

Compounding this problem, there is the ever increasing amount of marketing directed towards the end consumer. I dislike marketing because it makes terrible products or terrible value products seem like good quality products.

 

Reviews from other people and experts seems like the obvious solution. They have proliferated especially with the Internet. No matter where you look online these days, there are lots of reviews. Individual websites have reviews, while major sites where you can purchase also review. There are also personal blog websites, like mine that cover specific topics.

 

I often find product reviews disagreeable. It is not based on agreement that I find this flaw, but on quality of content.  They vary widely in quality and perspective. The problem of course becomes, who do you trust? There is an overflow of information. The question becomes one of credibility. You do not have the time to review every single review. Actually, even if you were to find some sort of consensus, it may not reflect the actual quality of product that you are buying.

 

Types of Reviews

I will go over the types of reviews. Reviews come from all sorts of places. I will explore the flaws from different sources.

 

A general rule of the thumb is that the more expensive a product is, the more time you should spend carefully thinking about the product and of course, researching.

 

Formal Review Websites

Formal review websites generally have people who are very passionate in the topic they are reviewing about. For example, being a technology geek, I very frequently visit many technology websites. Anandtech is the best known, but there are many others. I like Anandtech because the writers, although I often disagree with their opinions are usually very knowledgeable. Sadly, even amongst the larger websites, not all of them have writers who are experts. In general they are all very enthusiastic about the topic at hand, but deep technical knowledge is not assured in all review websites.

 

The big weakness is that they are dependent on high traffic and on manufacturer’s advertisements to survive. These large formal review websites are basically places where a small group of people are trying to maintain their living through website traffic and advertising revenue. Meanwhile, the companies that send free samples to these websites see them as an extension of their marketing arm. Remember that these are the same manufacturers that are trying to get these websites to review that are paying for their advertisements. The problem is that the reviewers cannot call out too many products spades, even if there are lots of spades in the industry  because they would lose their advertising revenue. That does not mean that you should hate formal review websites or regard everything they type as wrong, but you do need a sense of professional skepticism.

 

One of the things that I admire most deeply about reviewers is when they are willing to take a stand for their journalistic integrity, even if they are punished for it. An example of a website that was “blacklisted” was HardOCP. The companies that blacklisted HardOCP were EVGA and Corsair. I’d like to go into the details, because both companies are known for being great for their customer support.

  • EVGA blacklisted HardOCP after it detailed the poor quality of its post Sandy Bridge motherboards and the poor quality of its power supplies. Anyways, long story short, EVGA for its X58 Classified motherboard had an outstanding team and Peter Tan (known as “Shamino”), who was one of the top overclockers before his retirement design the motherboard for them. They lost Shamino, who left to join the Taiwanese motherboard giant Asus, while their design team left for Sapphire. EVGA’s motherboards have never since recovered. The P67 and X79 Sandy Bridge boards were disastrous. The Haswell (Z87, Z97, and X99) have improved somewhat, but EVGA is still behind in terms of BIOS compared to the likes of Asus, Gigabyte, MSI, and Asrock.  HardOCP was given a blacklist for calling out on several occasions the poor quality of EVGA’s products, such as its power supplies.  For now, I will continue to recommend some of their mouse models, but I have become hesitant about their other products.
  • Corsair is a company that I am reluctant to criticize because they do have truly outstanding support (and I do recommend their brand for it to my friends). However, they had a view at Corsair that HardOCP was out to get them. The reason why was because they ranked Corsair Power Supplies as mediocre. Corsair when it started selling power supplies (after diversifying from a maker of RAM, which was its roots in the enthusiast PC community). For the first few years, they were very successful, mostly because they pursued high quality OEMs (Corsair does not manufacture its own power supplies, they contract other brands to do so), particularly the companies Seasonic and Flextronics. However, since the 2010s, other brands have overtaken them, offering what I (along with many review websites such as HardOCP and many of my colleagues in the computer enthusiast community) consider to be better power supplies at lower prices. Corsair products nowadays do often carry quite a bit of a price premium (although you can find deals of course, like every other brand). Anyways, HardOCP in its reviews showed that Corsair products were often surpassed by the competition, priced uncompetitively, and although of good quality, were not the absolute best money could buy, nor a very good value due to the price premium. Corsair needless to say did not like that and blacklisted HardOCP. The only thing I will say here is that I admire Corsair as a company, I really do, especially its outstanding support, but I cannot help but feel like this was a terrible business move. That said, I will continue to recommend Corsair’s cases, which are very well designed, and at least when they go on sale, their power supplies.

First, I would like to offer kudos to HardOCP for covering such these products from otherwise outstanding companies objectively. The other is that I don’t recommend blacklisting these companies. Every company has its bad and good products. Let us use EVGA as an example. Their 780Ti Kingpin GPU was the top of the 780Ti GPUs, but the 980 Kingpin turned out to be a disaster due to lack of BIOS support. Another might be their mice – the Torq x10 is a pretty terrible mouse, while the Torq x5 Optical 6400 DPI is a great value and very highly recommended. Now, let’s discuss power supplies since I criticized EVGA earlier on this front. Some of EVGA’s power supplies that are made by the company Super Flower, from their Leadex platform are amongst the best power supplies around, perhaps rivaled only by Delta and the top power supplies from Seasonic. I would also like to note that HardOCP should be praised for remaining objective despite the fact that it has been denied review samples when EVGA does offer good quality supplies that it acknowledges that the power supply is indeed outstanding.

 

Another problem that many reviewers have is that they are under tight deadlines and have to complete their reviews to drive traffic. That means that their reviews will have to be done with just a couple of weeks with the product. That can be difficult because of the deadlines. Speaking with several product reviewers, for major product launches, they often stay up late to meet the deadlines. The bigger problem for you the end user is that the reviewers do not have a lot of long-term experience with the product. The end result is that they can overvalue first impressions and undervalue how well a product ages. Likewise, another big challenge is that with time and more people, there is the likelihood to find out more problems with a problem.

 

Do not be surprised too if the most visible websites are the least credible. Polygon is an example of a website that looks credible, but in reality, it is more or less, the public relations mouthpiece of the gaming industry.

 

Anyways, for you the reader, one tough thing to do is to find the reviews that are credible. I think that the majority of reviewers are honest people trying to earn a living and not trying to screw you into buying a product that they would consider of poor quality, but remember the limitations I have highlighted above. The other is that the Internet can be a source of a lot of drama and that you need to try to rise above to see who actually has as good product or within a company, which products are good and which ones are to be avoided.

 

Also, if you do not know, try asking a question. Most review websites generally have a forum thread or comments section that you can ask. Often if it is a good question and well written, the original authour might respond to you.

 

Reviews on places that sell

You can find reviews everywhere on websites that sell products. Amazon is perhaps the best known example, but Newegg too is another example. There are so many websites out there that it is beyond the scope of this article to list them all.

 

These are reviews that are posted by end users, like you and I. The big advantage is that they may have had the time to carefully work with the product with time and/or update their reviews.  They may also be experts or people with a lot of hard-won experience in a field. You can often learn insights from people that are not in the official reviews because they have had a longer time to work with a product. The other advantage is that you can, for more popular products, have a large sample size.

 

Drawbacks include:

  • Variable technical knowledge: There are a lot of users out there who think they know what they are doing but who really do not. To use an example, when 64 bit operating systems started to become mainstream around 2006, there were many reviews from people who were running 4GB or more RAM and who had 32 bit operating systems who blamed the RAM rather than their own lack of technical knowledge. You have to be careful when reading reviews to see if the person knows what they are taking about. If you are new to a topic, that requires some research on your own.
  • User bias: I love looking at reviews for controversial books on Amazon. One thing that I have noticed is that there are often many 4-5 star reviews and then a lot of 1 star reviews. You can tell that the reason is because of the agreement with the book.  Similarly with any products, people are going to have favorite brands, and their own biases. This can colour their perception of product quality.
  • Paid reviewers: Yes, companies do have paid reviewers that write fake positive reviews. It happens on product websites and apparently the job review website Glassdoor too has faced pressure from employees of certain companies to give their company a positive review. Apparently sometimes media does it as well. To give an example, Fox News has been known to use paid trolls to attack people who do not agree with their blog posts. All of this makes it hard to get objective information because you have to be extra critical of positive reviews.

 

Typically reviews are then sorted by their helpfulness to the user. You can rate reviews as well. I generally when I look at reviews for products look at the top reviews that rate the product very highly (4-5 star), then carefully consider the more critical reviews (1-3 stars). Look very carefully at what the user said and how they used the product. Sometimes, you will find that a product has been rated 4-5 stars by people who have not owned it for a long-time and 1-3 stars for the more long-term owners. There are many products that are good at giving good appearances, but do not last over time.

 

Amazon and several other websites also have threads too allowing you to ask the original reviewer questions. If they are using a product in a manner similar to you, or you don’t know something about a product, it helps to ask. I would only do so though for outstanding reviews.

 

Anyways, as you can see, there are ups and downs to end-user reviews compared to formal published reviews.

 

Blogs and Forums

I put this as a separate category because often blogs have more detail than reviews written on review websites. They are after all maintaining a dedicated website towards the topic. Similarly, people who participate actively on a specific forum about a topic are usually very knowledgeable, experienced, and passionate about what they participate in. They would have to be, otherwise, they would not be participating on the forums on a very specific topic. They have similar flaws to product reviews, but their own benefits as well.

 

You have to filter if you use these websites. The big thing to watch out for here are the trolls and the people who are brand loyal. In some ways, this is very similar to reviews on Amazon and similar websites. There can often be lots of Dunning-Kruger types here as well, who think they know a lot but really don’t. In fact your biggest difficulty will be sorting out the people who really know what they are doing and who are really objective, from the fans. There are usually lots of people who are very knowledgeable about a given topic, in many cases, with more knowledge than formal written reviews. However since there are both beginners, fans who are interested in actively dis-informing due to their brand allegiances, and extremely knowledgeable people, you have to filter.

 

How do you filter? Look at their post history and reputation. Typically good users will have many posts, may have received many points (typically websites have reward systems, like “thanks”, “likes”, “useful”, and awards for particularly outstanding members). Look to at what other members say about a member. Eventually if you stay at a forum long enough, you get a sense for who the experts are and who the fanboys are. For blogs, read through the user’s other posts. What do they say about a given topic? Do they have a great deal of expertise?

 

Also look at the website. For example, an Apple fan forum is unlikely to have opinions extremely critical of Apple products. The same holds true for any brand. I generally prefer to go to large established websites with many users that are not loyal to any particular brand.  I then filter for who I consider to be the most knowledgeable users and look up what they have to say about the product. Look too at the quality of the average poster. Are they low-knowledge and very fanboyish or are they highly knowledgeable and more reasonable in their posts? The quality of the community will define your experience at a forum. If the website is truly outstanding, it may be a good idea to create an account and be active there.

 

Blogs and forums also have the advantage of having long-term owners, much like product reviews. The difference is that the signal to noise ratio is going to be somewhat better because you are dealing with people who are likely passionate about the category they are reviewing. In any typical blog, there will be “Owner’s Clubs” of certain products. Those are especially important because they are from posters who have had the product for long periods of time. I put Owner’s Clubs as my most important source of information, ahead of even formal review websites, save when the writer of the formal review website is a noted expert. The reason why is because such clubs usually have a very good signal to noise ratio.

 

Essentially it is the benefits of the typical form on steroids. They offer:

  • Users that have owned the product for a very long time.
  • The user has a lot of experience with the product and/or a lot of technical knowledge in the field in question.
  • Owning a product for a long time will get a more realistic picture of what the product’s limitations are and what the problems t hat short reviews have not covered are.
  • They are also usually very eager to help people or they would not be on a forum related to the topic.

 

Blog posts too from highly knowledgeable people fall into the same category. You would not blog about something extensively unless you felt a strong opinion or that it was worthy of spending lots of time to write.

 

In both cases, you can participate too. Bloggers are usually very open to people asking good questions. Similarly Owner’s Clubs and good forums are worthy of asking questions in the Owner’s Club thread or creating your own thread. I think that this is personally one of the best ways to get information about a new product.

 

Other Considerations

 

If you intend to use a product in a particular way, unless you see information online from someone that uses the product in the exact same way that you do, you are taking a risk.

 

For expensive products, you should also research the manufacturer’s history. Do they provide good technical support? Do they have a history of honouring their warranty or looking for excuses to worm out? I have blacklisted many vendors for their poor business practices. You do not want to be screwed over in case anything ever goes wrong.

 

For review websites, you can often also look at their testing methodology. Often it is as important as the end result. Since I raised the issue earlier about HardOCP, is an example from HardOCP. They also raise the impacts of consequences in their articles. Generally speaking, the more stringent the testing, the better. It will mean that a higher percentage of products fail and that only a handful of truly good products gain a recommendation. This can be hard for review websites because remember, product manufacturers are their sponsors too. This is one area where forums and personal reviews have an advantage.

 

The other key consideration is time. Obviously you do not have unlimited time. Remember, the larger the consequences of the purchase, the more you should spend time carefully researching. The reason of course is that the consequences of a bad purchase are that much larger. Where do you research if you have limited time? I’d recommend skimming over the top review sites and going straight to the conclusions. Go through the top 2-3 positive and critical reviews on Amazon. Then go  to the best forum you can find and browse the Owner’s Club. This will yield the best results for the time spent.

 

Conclusions – The Key Takeaway

There is a lot of information and reviews about products out there.

 

Our worst fear is that we get the bad product that costs a lot of money and we lose a lot of time/money/effort on a terrible product. The problem is finding out which products are actually good and which are not. It becomes compounded by the fact that there is lots of bad information out there.

 

To be successful at this, you have to have a good sense of critical thinking and the ability to understand what is credible information.  Filtering can be difficult, time consuming, but you will get better. Look for experts who know what they are talking about, and for the top Owner’s Clubs or good blog posts identifying flaws.

 

My Goals for my Reviews

I’m not going to have the time to do many reviews, unfortunately.

 

In running my website and starting to do product reviews (which I plan to do in the coming years of my blogging), I plan for my product reviews to focus on the long-term ownership perspective. I will sometimes give previews, which are shorter, and more focused on my initial impressions. Then after that, I will give reviews, which will be longer, focus on problems that may not have been discovered, and try to discuss whether there are any workarounds. I have not yet finalized everything, so I expect that some things will change.

 

This website is still relatively young, so I am trying to finalize my plans for it at this point, but I don’t think that I much will change on how I review.

 

I will not be the perfect reviewer, even according to my own ideals. First because of time and monetary constraints, I cannot review as often as I would like. Good content unfortunately takes a lot of time, which is something that we are all limited in. The other is because I do not know everything about a product. I will try to make an effort to acknowledge my limitations for reviews.

Leave a Comment

Your email address will not be published. Required fields are marked *