Consider this. A broad market index is an average. The index averages all of it's constituents. As of today, the S&P 500 index has gained 202% over 10 years. It started at 1362.33 and is now at 4114.26 so it has tripled in value. How many investors have tripled their portfolio in the last 10 years? Damn few have done it. And yet, this is an average. Where are the above average people? Not everyone can be below average, by definition.
Why does virtually no one achieve the average returns? I will answer with a classic question: "Where Are the Customers' Yachts?" The other answer to this age old question is: "look in the mirror"
There are a few people that equal or beat the averages. BUT......they are totally ignored by the media and are derided as IDIOTS by the majority of investors/traders/speculators.
I think its because no one buys and holds for 3 years. Finding a good entry point, on the right stock, and NOT cashing out sometime prior to tripling, ya know?
To answer the question: It goes to the big guns who manipulate the market and to the government in the form of short term capital gains tax. As to why "most" do not meet or beat the averages... I suspect those who buy and hold for a long time are busy living their best life and we are not aware of them. The modest winners instead of the blowhards who do not beat the averages, if you will.