How We Test

By Blain Reinkensmeyer | January 21st, 2020

arrow_drop_down arrow_drop_up

Our mission at StockBrokers.com is simple: provide thorough and unbiased reviews of online brokers.

For the StockBrokers.com tenth best online brokers review published in January 2020, a total of 3,540 data points were collected over six months and used to score brokers. This makes the site home to the largest independent database on the web covering the online broker industry.

We also take pride in StockBrokers.com data quality. Every broker that participates in the annual review must complete an in-depth data profile and provide executive time for an annual update meeting. We then audit each data point to ensure its accuracy. 

This rigorous data validation process yields an error rate of less than .001% each year, providing site visitors quality data they can trust.

Testing Devices

All testing is conducted on the devices listed below, using a hardwired, 50 MB Internet connection. Mobile device testing is conducted using an 802.11n wireless WiFi connection; 4G connections are used when WiFi is not available. Unless noted, all websites and web-based platforms are tested using the latest version of the Google Chrome browser.

Desktop: Dell XPS 8930 desktop with three ASUS 27″ LED Monitors, Intel Core i7-8700 Processor at 3.20 GHz, Nvidia GeForce GTX 1070, and 16 GB RAM. Windows 10 Pro operating system.

Tablet: iPad Air 2 with the most current iOS.

Smartphone: iPhone XS with the most current iOS.

Review Categories, Variables

For the StockBrokers.com 2020 Online Broker Review, all U.S. equity broker participants were assessed on 236 different variables across 8 core areas: Commissions & Fees, Customer Service, Platforms & Tools, Research, Ease of Use, Offering of Investments, Education, and Mobile Trading.

The variable counts of each category were as follows: 

Review Category Variables
Commissions & Fees 22
Offering of Investments 36
Platforms & Tools 53
Research 54
Customer Service 9
Mobile Trading 42
Education 19
Ease of Use 1

Awards

StockBrokers.com presented “Best in Class” awards to equity brokers across all eight core areas as well as additional categories Beginners, Options Trading, Futures Trading, Active Trading, Professional Trading, Order Execution, IRA Accounts, Banking Services, and Overall. A “Best in Class” designation means finishing in the top five brokers for that category.

The following individual Industry awards were also distributed:

  • #1 Innovation
  • #1 International Trading
  • #1 Investor App
  • #1 Trader App
  • #1 Desktop Platform
  • #1 Web Platform
  • #1 Platform Technology
  • #1 Client Dashboard
  • #1 Trader Community
  • #1 New Tool
  • #1 ESG Investing
  • #1 Overall Client Experience

Category Scoring Methodology

While most categories are scored using our default method of combining Yes / No points (Variable score) with a 1 - 10 Opinion score, scoring does vary. One category is scored with 100% Yes / No points, one is scored with 100% opinion, and two others are entirely custom.

  • Commissions & Fees - Custom
  • Platforms & Tools - Yes/No with Opinion
  • Research - Yes/No with Opinion
  • Mobile Trading - Yes/No with Opinion
  • Customer Service - Custom
  • Offering of Investments - Yes/No Only
  • Education - Yes/No with Opinion
  • Ease of Use - Opinion Only

Each individual category score is also translated into a star rating. Specifically:

  • 95% - 100% = 5 Stars
  • 85% - 94.99% = 4.5 Stars
  • 75% - 84.99% = 4 Stars
  • 65% - 74.99% = 3.5 Stars
  • 55% - 64.99% = 3 Stars
  • 45% - 54.99% = 2.5 Stars
  • 35% - 44.99% = 2 Stars
  • 25% - 34.99% = 1.5 Stars
  • 15% - 24.99% = 1 Stars
  • 5% - 14.99% = 0.5 Stars
  • 0% - 4.99% = 0 Stars

How We Score: Yes/No with Opinion

First, points are awarded for each “Yes” answer. The number of points awarded for each "Yes" varies by variable depending on its significance. Next, the total points earned from “Yes” answers is divided by the total points possible to calculate a final Yes/No score, also known as a Variable score.

Example: Broker XYZ has nine of ten features for category Research and earns 14 out of a possible 15 total points (remember, points allotted to each individual variable varies). Thus, Broker XYZ earns a Variable score of 93.33% for category Research.

Next, the editorial team gives an opinion score to each broker for the category. Opinion scores range from 1 – 10, with one being “very bad” and ten being “very good”. Half points are used, e.g. "9.5", so scoring isn't restricted to just whole numbers. Note: While the editorial team does take into consideration the Variable score, the Opinion score is more focused on the editorial view of how well the broker performs overall as compared to the industry standard and its closest competitors.

Example: Broker XYZ does a great job with its Research offering, exceeding the industry standard. Not only are a variety of tools available to conduct research, but they are rich with depth and also easy to use. The editorial team awards Broker XYZ a score of "10" for its Opinion score, which equates to 100%.

Finally, to determine a final percentage score for the category, the Variable score and Opinion score are independently weighted and added together. This score is then also translated to a star rating. Weightings vary by category depending on the complexity of the category, e.g. 70% Variable, 30% Opinion.

Example: Broker XYZ earned a Variable score of 93.33% and a Opinion score of 100%. For category Research, the Variable score is 60% of the total and the Opinion score is 40%. Thus, Broker XYZ earns a final score of 96% for Research, which is calculated as "(0.9333*.6)+(1*.4)". Translated into a star rating, Broker XYZ earns 5 Stars for Research.

How We Score: Custom

Both Commissions & Fees and Customer Service apply separate custom scoring formulas due to their complexity.

Commissions & Fees

First, multiple scenarios are assessed for equity trades, options trades, and margin rates. For example, the cost of placing a 500-share market order or the cost of placing a two-legged one-contract options spread, or the margin rate for a $50,000 balance. Brokers are awarded points depending on how low cost they are relative to their peers.

Next, multiple monthly pricing scenarios are assessed for different investor types. For example, the net monthly cost to place one stock trade (500 shares per), representative of a casual investor, or the monthly cost to place 10 stock trades (500 shares per) and 5 options trades (10 contracts per), representative of an active trader. Brokers are awarded points depending on how low cost they are relative to their peers.

Thereafter, all points are tallied to determine a total Scenarios score. Separately, scoring is then conducted to assess Order Execution quality, which applies a Yes/No with Opinion methodology (see above).

Finally, to determine a final percentage score for the category, the Scenarios score and the Order Execution score are independently weighted and added together. Note: While there is currently no universal metric to assess order execution quality, the significance of execution quality is as important as ever given the reliance on PFOF as a source of revenue for brokers that charge $0 for trades. All said, Order Execution made up 10% of the final Commissions & Fees category weighting in 2020. Read our guide to understanding order execution quality.

This final score is then also translated to a star rating.

Customer Service

For the past six years, StockBrokers.com has partnered with customer experience research group Confero to conduct phone tests from locations across the United States.

For the 2020 Review, 280 customer service tests were conducted (20 per broker) over an eight week period during the months of September through November 2019. Note: Since Robinhood does not offer phone support, it was excluded from testing.

Confero agents used one of ten unique scenario templates provided by StockBrokers.com (on average, three questions are asked) to assess each broker. To ensure fairness as well as gather the highest quality data, twenty separate agents were used and each agent used one single template to test all 14 brokers. Thus, twenty agents conducted 15 tests each (14 brokers), for a total of 280 tests.

To score each test, three separate buckets are assessed: 1. Time to connect 2. The ability of each broker representative to professionally (and thoroughly) answer each question and 3. A Net Promoter Score for the overall support experience. Response times are 15% of each test score, representative assessments are 35%, and Net Promoter Scores make up the remaining 50%.

Once all individual test scores were compiled, they were combined to determine an overall score for Customer Service. This final score is then also translated to a star rating. View infographic.

Overall Score Calculation

All eight core categories are factored in to compile the Overall ranking for each broker. The weightings of each category were as follows (view the Final Rankings): 

Review Category Weighting
Commissions & Fees 18%
Offering of Investments 15%
Platforms & Tools 15%
Research 15%
Customer Service 15%
Mobile Trading 12%
Education 7%
Ease of Use 3%

Read Next

arrow_drop_upBack to top