Distorted Statistics and Performance Tests: Part 3

  • September 27, 2017

    This part of the series explains Type I and Type II errors as a basis for understanding why the analyst’s conclusion in Part 2 is wrong.

    Advanced financial researchers begin by setting up a null hypothesis. The null hypothesis is a statement that the researcher seeks to disprove or fail to disprove. Notice that most good research methods do not seek to include truths, they seek to exclude falsehoods. These researchers should not claim to know the truth, they can only rule out those things that evidence says are false.

    For example, an investigator testing to see whether or not an investment strategy works will construct a null hypothesis stating that the strategy does not work. Consider two ancient scholars debating whether the sun or the moon is bigger. They both believe that the moon is very slightly closer to the earth than the sun. The scholar named “Sun” believes that the sun is bigger and the scholar named “Moon” believes that the moon is bigger. Their charts of the paths indicate that a solar eclipse is imminent. Each sets up a null hypothesis that they seek to refute.

    Scholar Sun sets up a null hypothesis (that he seeks to disprove) that the moon is bigger. He will refute the null hypothesis if the moon does not fully block out the sun. Scholar Moon sets up a null hypothesis (that she seeks to disprove) that the sun is bigger. She will refute the null hypothesis if the moon fully bocks out the sun.

    Here is the amazing result: Scholar Sun fails to refute the hypothesis that the moon is bigger and Scholar Moon refutes the hypothesis that the sun is bigger. The both get it wrong! They both get it wrong because their model was based on a faulty premise – that the distance of the two objects from the earth was similar. But this is an important point: a researcher’s conclusions are only as good as the model on which the test is based.

    Scholar Sun’s test results in a Type I error, also known as a false positive, which is when the researcher mistakenly rejects a true hypothesis. Scholar Moon’s test results in a Type II error, also known as a false negative, which is when the researcher mistakenly fails to reject a hypothesis when it is false. This example may seem to be special or even ridiculous due to its obviously erroneous assumption. But it represents the core problem in investment studies. That problem is discussed in Part IV.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>



1256 contracts, 2010, 401K, adp report, Advisors, Alexandra Twin, alternative investments, annuity, asset allocation, asset classes, asset management, Associated Press, away, banks, Barrons Top 100, BCA, BCA. Biltmore Capital, Ben Bernanke, Best Week, bias, Biltmore, Biltmore Capital, Biltmore Capital Advisors, bond market, Brand, Brand Recognition, Business, Call, CFP Princeton, changes in market price, Chief Investment Officer, closed end funds, CNBC, CNBC Halftime Report, CNN, CNN Money, CNNMoney, CNNMoney.com, collar stock, college grads, Consumer Confidence Report, correlations, Covered, Covered Call Options, credit ratings, customized investment strategies, Dean, donald chambers, Dow Jones, dr don chambers, drop, Early, easy trading, economic policy, economy, Edge, efficiency, election, end, equilibrium, equity trading, etf, etf trading, ETFs, euro, Europe, Europe debt, European debt crisis, family office approach, federal income tax, federal investment income tax, Financial, financial advisor, financial crises, financial distress, financial management, financial security, financial strategies, Ford, Fox, Fox Business, fund manager, futures contracts, Gains, General Motors, Global, global market, Global stocks, groupon, halftime report, Herbert Lash, high income tax, home country, Home Sales, housing market, IDEAS, income tax, income taxes, initial public offering, Instant View, investment bias, investment income, investment returns, investment risk, investment strategies, Investors, IPOs, IRA, Jilian Mincer, Jonathan Cheng, KANA INAGAKI, key, large losses, long risk exposure, Los Angeles Times, make money, making investment money, making money last, Manufacture, Manufacturing, Market, market news, market price, market volatility, Markets, marketwatch, MICHELE MAATOUK, Mike Miliard, Molly Vernon, money, Money Manager, money managers, money strategy, municipal bond interest, municipal bonds, nassau club, New Jersey Advisors, New York Times, NJ, NJ advisors, NJ financial advisors, NJ money manager, NJ wealth advisor, NJ Wealth Advisors, obamacare, oil slide, Options, options strategies, Outlook, outperformance, Pending, personal financial services, personal risk analysis, Play, portfolio manager, Potfolio Manager, Princeton, Princeton Advisors, Princeton asset management, princeton financial advisors, Princeton Money Managers, Princeton wealth advisor, private wealth management, rally, recession, Recognition, registered investment advisor, Registered Investment Advisory Firm, retirement, retirement strategy, return, returns, Reuters, reward, RIA, RIA Princeton, Risk, risk exposure, riskier, rmd, Roth IRA, safer, saving taxes, savings, SEC-registered, Seeking Alpha, short risk exposure, Shudder, signals, skype, social security, social security benefits rules, star ledger, state income tax, Stephen Bernard, Stock, stock market, stock market returns, stock market winners, stock option strategies, stock price, stock prices, stock research, stock strategy, Stock Volatility, Stocks, structured notes, Stuart Day School, swine flu, tail risk, Tax Advantaged Investments, tax free investments, tax savings, tax strategy, taxes, taxes for social security, taxes on investment income, The Dean, The Wall Street Journal, Thomson, Thomson Reuters, Tick, Tim Ralph, Timothy Ralph, Today, Tony Roth, Trading, tricks, Tyler, tyler vernon, U.S., U.S. dollar gains, U.S. Stock, unemployment, USA, USA Today, VIX, volatility, Wall Street, wealth management, wealth manager, wells fargo, world market, worth, Yahoo, Yahoo Financial,