How We Test

By Blain Reinkensmeyer | February 18th, 2019

arrow_drop_down arrow_drop_up

Our mission at StockBrokers.com is simple: provide thorough and unbiased reviews of online brokers.

For the StockBrokers.com ninth best online brokers review published in February 2019, a total of 4,544 data points were collected over six months and used to score brokers. This makes the site home to the largest independent database on the web covering the online broker industry.

We also take pride in StockBrokers.com data quality. Every broker that participates in the annual review must complete an in-depth data profile and provide executive time for an annual update meeting. We then audit each data point to ensure its accuracy. 

This rigorous data validation process yields an error rate of less than .001% each year, providing site visitors quality data they can trust.

Testing Devices

All testing is conducted on the devices listed below, using a hardwired, 50 MB Internet connection. Mobile device testing is conducted using an 802.11n wireless WiFi connection; 4G connections are used when WiFi is not available. Unless noted, all websites and web-based platforms are tested using the latest version of the Google Chrome browser.

Desktop: Six ASUS 24″ LED Monitors, Intel Core i7-3770 Processor, 8 GB Corsair Vengeance DDR3 1600 MHz memory, 2 AMD Radeon HD 7770 video cards, and a 120 GB ADATA S510 SSD Hard Drive. Windows 10 operating system.

Tablet: iPad Air 2 with the most current iOS. Safari Browser. 

Smartphone: iPhone XS with the most current iOS. Safari Browser.

Review Categories, Variables

For the StockBrokers.com 2019 Online Broker Review, all U.S. equity broker participants were assessed on 284 different variables across 9 core areas: Commissions & Fees, Customer Service, Platforms & Tools, Research, Ease of Use, Offering of Investments, Education, Mobile Trading, and Banking. Note: Banking is excluded from the Overall rating. 

The variable counts of each category were as follows: 

Review Category Variables
Commissions & Fees 33
Offering of Investments 40
Platforms & Tools 59
Research 60
Customer Service 9
Mobile Trading 50
Education 19
Ease of Use 1
Banking 13

Awards

StockBrokers.com presented “Best in Class” awards to equity brokers across all nine core areas as well as additional categories Options Trading, Active Trading, Beginner Investors, Professionals, and Overall. A “Best in Class” designation means finishing in the top five brokers for that category.

The following individual Industry awards were also distributed:

  • #1 Broker Innovation
  • #1 International Trading
  • #1 Investor App
  • #1 Trader App
  • #1 Desktop Platform
  • #1 Web Based Platform
  • #1 Platform Technology
  • #1 Client Dashboard
  • #1 Overall Client Experience
  • #1 Trader Community
  • #1 New Tool
  • #1 Order Execution

Category Scoring Methodology

While most categories are scored using our default method of combining Yes / No points (Variable score) with a 1 - 10 Opinion score, scoring does vary. One category is scored with 100% Yes / No points, one is scored with 100% opinion, and two others are entirely custom.

  • Commissions & Fees - Custom
  • Platforms & Tools - Yes/No with Opinion
  • Research - Yes/No with Opinion
  • Mobile Trading - Yes/No with Opinion
  • Customer Service - Custom
  • Offering of Investments - Yes/No Only
  • Education - Yes/No with Opinion
  • Ease of Use - Opinion Only

Each individual category score is also translated into a star rating. Specifically:

  • 0% - 4.99% = 0 Stars
  • 5% - 14.99% = 0.5 Stars
  • 15% - 24.99% = 1 Stars
  • 25% - 34.99% = 1.5 Stars
  • 35% - 44.99% = 2 Stars
  • 45% - 54.99% = 2.5 Stars
  • 55% - 64.99% = 3 Stars
  • 65% - 74.99% = 3.5 Stars
  • 75% - 84.99% = 4 Stars
  • 85% - 94.99% = 4.5 Stars
  • 95% - 100% = 5 Stars

How We Score: Yes/No with Opinion

First, points are awarded for each “Yes” answer. The number of points awarded for each "Yes" varies by variable depending on its significance. Next, the total points earned from “Yes” answers is divided by the total points possible to calculate a final Yes/No score, also known as a Variable score.

Example: Broker XYZ has nine of ten features for category Research and earns 14 out of a possible 15 total points (remember, points allotted to each individual variable varies). Thus, Broker XYZ earns a Variable score of 93.33% for category Research.

Next, the editorial team gives an opinion score to each broker for the category. Opinion scores range from 1 – 10, with one being “very bad” and ten being “very good”. Half points are used, e.g. "9.5", so scoring isn't restricted to just whole numbers. Note: While the editorial team does take into consideration the Variable score, the Opinion score is more focused on the editorial view of how well the broker performs overall as compared to the industry standard and its closest competitors.

Example: Broker XYZ does a great job with its Research offering, exceeding the industry standard. Not only are a variety of tools available to conduct research, but they are rich with depth and also easy to use. The editorial team awards Broker XYZ a score of "10" for its Opinion score, which equates to 100%.

Finally, to determine a final percentage score for the category, the Variable score and Opinion score are independently weighted and added together. This score is then also translated to a star rating. Weightings vary by category depending on the complexity of the category, e.g. 70% Variable, 30% Opinion.

Example: Broker XYZ earned a Variable score of 93.33% and a Opinion score of 100%. For category Research, the Variable score is 60% of the total and the Opinion score is 40%. Thus, Broker XYZ earns a final score of 96% for Research, which is calculated as "(0.9333*.6)+(1*.4)". Translated into a star rating, Broker XYZ earns 5 Stars for Research.

How We Score: Custom

Both Commissions & Fees and Customer Service apply separate custom scoring formulas due to their complexity.

Commissions & Fees

First, multiple scenarios are assessed for equity trades, options trades, and margin rates. For example, the cost of placing a 500-share market order or the cost of placing a two-legged one-contract options spread, or the margin rate for a $50,000 balance. Brokers are awarded points depending on how low cost they are relative to their peers.

Next, multiple monthly pricing scenarios are assessed for different investor types. For example, the net monthly cost to place one stock trade (500 shares per), representative of a casual investor, or the monthly cost to place 10 stock trades (500 shares per) and 5 options trades (10 contracts per), representative of an active trader. Brokers are awarded points depending on how low cost they are relative to their peers.

Thereafter, all points are tallied to determine a total Scenarios score. Separately, scoring is then conducted to assess Order Execution quality, which applies a Yes/No with Opinion methodology (see above).

Finally, to determine a final percentage score for the category, the Scenarios score and the Order Execution score are independently weighted and added together. Note: While there is currently no universal metric to assess order execution quality, the significance of execution quality is as important as ever given the reliance on PFOF as a source of revenue for brokers that charge $0 for trades. All said, Order Execution made up less than 25% of the final Commissions & Fees category weighting in 2019. Read our guide to understanding order execution quality.

This final score is then also translated to a star rating.

Customer Service

For the past five years, StockBrokers.com has partnered with customer experience research group Confero to conduct phone tests from locations across the United States.

For the 2019 Review, 300 customer service tests were conducted (20 per broker) over a ten week period during the months of October through December 2018. Note: Since Robinhood does not offer phone support, it was excluded from testing.

Confero agents used one of ten unique scenario templates provided by StockBrokers.com (on average, three questions are asked) to assess each broker. To ensure fairness as well as gather the highest quality data, twenty separate agents were used and each agent used one single template to test all 15 brokers. Thus, twenty agents conducted 15 tests each (15 brokers), for a total of 300 tests.

To score each test, three separate buckets are assessed: 1. Time to connect 2. The ability of each broker representative to professionally (and thoroughly) answer each question and 3. A Net Promoter Score for the overall support experience. Response times are 15% of each test score, representative assessments are 35%, and Net Promoter Scores make up the remaining 50%.

Once all individual test scores were compiled, they were combined to determine an overall score for Customer Service. This final score is then also translated to a star rating. View infographic.

Overall Score Calculation

All nine primary categories, except for Banking, are factored in to compile the Overall ranking for each broker. The weightings of each category were as follows: 

Review Category Weighting
Commissions & Fees 18%
Offering of Investments 15%
Platforms & Tools 15%
Research 15%
Customer Service 15%
Mobile Trading 12%
Education 7%
Ease of Use 3%
Banking 0%

2019 Rankings

Scored on 4,544 data points, here's the final Overall ranking for 2019.

StockBrokers.com online stock broker rankings 2019

Read Next

arrow_drop_upBack to top