How We Test

Brandon Reinkensmeyer

Monday, November 28th, 2016

Our mission at is to provide accurate and widespread data for every online broker we track. As of February, 2016, is home to the largest independent database on the web covering the US online brokerage industry. In our 2016 Review, we assessed 295 separate variables across ten core categories for each broker.

With this data, we were able to thoroughly evaluate each broker and provide visitors with accurate and easy to interpret star ratings for topics ranging from Commissions & Fees to Customer Service. Alongside star ratings, data are displayed within each broker’s review page and are also integrated into research tools such as the Compare Tool and Screener Tool.

Live broker testing


The annual Broker Review is the backbone for validating and verifying the data we have on file. Preparation for the review begins in August, brokers are contacted in September, and the review process runs approximately until publishing in February. During the review, we spend ample time re-validating data points ourselves through extensive live testing. More importantly, each broker participant independently verifies every data point we have on file for them. Data verification is one of several requirements for inclusion in the review.

After publishing in mid-February, our team monitors, updates, and maintains the data sets for each broker until the next annual review cycle begins in August. Thanks to this rigorous verification process, maintains an error rate of less than .001% for brokers included in the annual review each year.


All of our testing is conducted on the devices listed below. All desktop testing is done on a hardwired, 50MB Internet connection. Mobile device testing is conducted using an 802.11n wireless Wi-Fi connection, as well as 4G connections. Unless noted, websites alongside web-based platforms are tested using the latest version of Google Chrome.

Desktop: Professional trading computer, with six 24-inch monitors running Windows 10 OS
Tablets: iPad Air 2 with the latest iOS, Safari Browser | Samsung Nexus 10, with the latest Android OS
Smartphones: iPhone 6s with the latest iOS, Safari Browser

iPhone broker apps


We assess each broker according to ten primary categories: Commissions & Fees, Customer Service, Platforms & Tools, Research, Ease of Use, Offering of Investments, Education, Mobile Trading, Order Execution, and Banking. Also included are three additional categories that focus on specific investor segments: Active Trading, Options Trading, and New Investors.

All ten primary categories, with the exception of Banking, are factored in to compile the Overall ranking for each broker. The weightings of each category within the Overall category, along with their individual variable counts, are as follows:

Commissions & Fees (15.5%) - 29 total variables
Offering of Investments (15.5%) - 42 total variables
Platforms & Tools (15.5%) - 52 total variables
Research (15.5%) - 50 total variables
Customer Service (10%) - 26 total variables
Mobile Trading (10%) - 48 total variables
Ease of Use (10%) - 7 total variables
Education (6%) - 20 total variables
Order Execution (2%) - 8 total variables


To assess these categories, each category is broken into two parts: a variable assessment and an opinion score. Each variable is assigned a point value earned for a “Yes” result. With variable assessments complete, the total points earned are divided by the total points possible to determine a final variable score. The variable score makes up a predetermined percent of the category’s overall score. Next, a 1 – 10 opinion score (one being “very bad” and 10 being “very good”) is applied. The opinion score is then weighted based on the remaining percent of the category’s overall score. Combined with the variable score, a final percent tally is computed and a star rating is determined. 


To assess Commissions & Fees, the category is broken into five groups, which are all weighted independently: equity trade costs, option single-legged trade costs, option multi-legged trade costs, margin rates, and monthly scenario costs. Each variable (we refer to them as scenarios) falls into one of these groups. 

For each scenario, brokers are ranked 1 – 13 based on the cheapest to the most expensive, respectively. Depending on their ranking, they earn either 2, 4, 6, 8, or 10 points, with 10 being the top score. Points are then summed together and divided by the total possible to determine a score for that group. It is important to note that each scenario can be custom weighted. For example, the frequency of traders placing a 100-share order is far higher than those that place 5,000-share orders. Thus, a 100-share trade is weighted higher than a 5,000-share trade.

With all five groups scored, weightings are applied for each group and a final overall score is determined for the category. This score is then converted into a star rating.


To assess Customer Service, scoring is based on actual live support tests conducted via phone, email, and live chat. Phone is weighted the highest, with email being second highest, and live chat being third. If live chat is not offered, the weighting of phone and email increases, respectively. If live chat is not offered, the weighting of phone and email increases to 72.2% and 27.8%, respectively.

All phone testing is conducted in partnership with Confero, a professional private shopping group whose clients include Coca-Cola and Shell. For the 2016 Review, Confero made over 100 phone calls to branch offices and support numbers across the United States using templates and scoring rubrics. With email and live chat testing included, a total of over 300 customer support inquiries were made.

Each category (phone, email, live chat) has a custom scoring rubric. Each rubric is broken into a time element, yes/no questions, and a net promoter score. The time element focuses on how long it takes to be connected to a rep (phone, live chat) or receive a response (email). The yes/no section includes six questions (live chat) or seven questions (phone, email), which the tester answers. Yes/no examples include “Did the rep use your name?” or “Did the rep answer all your questions?” Finally, a net promoter score of 1-10 is given on the overall experience of that particular call, email, or live chat. 

Once the individual scores are compiled, they are then combined to determine an overall score for that test category (phone, email, live chat). With the test category scores in, scores are weighted by category, a final score is determined, and a star rating is determined.


To assess Ease of Use, the same standard formula applied to Offering of Investments, Platforms & Tools, Research, Education, Mobile Trading, and Banking is followed. However, if a broker is a platform-only brokerage, e.g., TradeStation or OptionsHouse, the variables are excluded and the opinion score makes up 100% of the category rating.


To assess Order Execution, the same standard formula applied to Offering of Investments, Platforms & Tools, Research, Education, Mobile Trading, and Banking is followed. However, opinion scores are not included, and variable scoring makes up 100% of the category rating. To learn more about testing for this category, see Best Brokers for Order Execution 2016.

Back to top