menu

How We Test

By Brandon Reinkensmeyer | February 21st, 2017

arrow_drop_down arrow_drop_up

Our mission at StockBrokers.com is simple: provide thorough and unbiased reviews of online brokers.

For the StockBrokers.com 2017 Online Broker Review, published in February 2017, a total of 5,277 data points were collected over six months and used to score brokers. This makes the site home to the largest independent database on the web covering the online broker industry.

We also take pride in StockBrokers.com data quality because it is 100% verified by the industry itself. Every broker that participates in the annual review must complete an in-depth data profile and provide executive time for an annual update meeting. We then validate each data point to ensure its accuracy. 

This rigorous data validation process yields an error rate of less than .001% each year, providing site visitors quality data that they can trust. As a symbol of our focus on quality data, we unveiled the StockBrokers.com Verified Quality Data seal in 2016, which is displayed throughout the site.

Testing Devices

All testing is conducted on the devices listed below. Desktop testing is conducted using a hardwired, 50 MB Internet connection. Mobile device testing is conducted using an 802.11n wireless Wi-Fi connection; 4G connections are used when WiFi is not available. Unless noted, all websites and web-based platforms are tested using the latest version of the Google Chrome browser.

Desktop: Six ASUS 24″ LED Monitors, Intel Core i7-3770 Processor, 8 GB Corsair Vengeance DDR3 1600 MHz memory, 2 AMD Radeon HD 7770 video cards, and a 120 GB ADATA S510 SSD Hard Drive. Windows 10 operating system.

Tablet: iPad Air 2 with most current iOS. Safari Browser. 

Smartphone: iPhone 6s with most current iOS. Safari Browser.

Review Categories

For the StockBrokers.com 2017 Online Broker Review, all U.S. equity broker participants were assessed on 308 different variables across 10 core areas: Commissions & Fees, Customer Service, Platforms & Tools, Research, Ease of Use, Offering of Investments, Education, Mobile Trading, Order Execution, and Banking. 


All ten primary categories, except for Banking, are factored in to compile the Overall ranking for each broker. The weightings of each category, alongside their respective variable counts, were as follows: 


For our 2017 Robo-Advisor Review, we established four core categories for rating and review: Investment Approach, Cost, Customer Service, and Tools. Each section was extensively tested utilizing identical methods and processes to how we conduct our equity broker reviews.

Review Category Weighting Variables
Commissions & Fees 16% 29
Offering of Investments 16% 40
Platforms & Tools 16% 58
Research 16% 51
Customer Service 11.5% 26
Mobile Trading 11.5% 52
Education 6% 20
Ease of Use 4% 9
Order Execution 3% 10
Banking 0% 13

Category Scoring

StockBrokers.com presented “Best in Class” awards to equity brokers across all ten core areas as well as additional categories such as Options Trading, Active Trading, New Investors, and Overall. A “Best in Class” designation means finishing in the top five brokers for that category. Individual Industry awards were also distributed.

A breakdown of how the categories are assessed is below. For detailed commentary and analysis on all categories, view our Best Brokers section. For example, Best Brokers for Mobile Trading 2017.


Yes/No with Opinion

First, points are awarded for each “Yes” answer. The number of points awarded depends on the variable. Next, the total points earned from “Yes” answers is then divided by the total points possible to calculate a final Yes/No score.

To determine a final score for the category, this Yes/No score is then combined with an opinion score (1 – 10, with one being “very bad” and ten being “very good”). The weighting of the Yes/No score as compared to the opinion score varies for each category. For example, 50% Yes/No, 50% Opinion.

Categories assessed in this manor include:  Offering of Investments, Platforms & Tools, Research, Mobile Trading, Education, Ease of Use, Order Execution, and Banking. Note, Offering of Investments does not incorporate an opinion score. Also, in assessing Ease of Use, brokers that are designated as “platform only” are 100% opinion weighted.


Customized

Both Commissions & Fees and Customer Service apply separate scoring rubrics due to their complexity.

To score Commissions & Fees, first, multiple scenarios are assessed for both equity trades, options trades, and margin rates. For example, the cost to place a 500-share market order, or the cost to place a two legged one contract options spread, or the margin rate for a $50,000 balance. Brokers are awarded points depending on how cheap they are relative to their peers.

Next, multiple scenarios are assessed for different investor types on a monthly basis, incorporating any miscellaneous account fees, platform charges, and active trader rates. For example, the net monthly cost to place 10 stock trades (500 shares per), or the monthly cost to place 100 stock trades (500 shares per) and 50 options trades (10 contracts per).

Finally, all scenarios are independently weighted and points are combined to determine an overall score for Commissions & Fees.

To score Customer Service, StockBrokers.com partners with customer experience research group Confero to conduct phone, email, and live chat tests from locations throughout the United States. For our 2017 Online Broker Review, 349 customer service tests were conducted; phone was weighted highest followed by live chat and email, respectively.

Confero agents use one unique template provided by StockBrokers.com (on average, three questions are asked) to assess each broker. Agents are only allowed to conduct one round of tests for each method to ensure the highest quality, and most diverse data, is collected.

To score each test, tests are broken into three separate groups and scored: the time to connect with each support representative, the ability of each representative to professionally (and thoroughly) answer each question, alongside a Net Promoter Score of the overall experience. Response times are 15% of each test score, representative assessment are 35%, and Net Promoter Scores makes up the remaining 50%.

Once all individual tests scores are compiled, they are combined to determine an overall score for each testing method. Lastly, these final method scores are weighted and combined to determine an overall score for Customer Service.