- an increase in migrations over time from the S&P MidCap Index
- an overall increase in the market’s ability to provide liquidity to index changes
What about the fact that there are more activities of additions/deletions to the S&P 500 given how much more competition there is today versus decades ago?
You will see anywhere from 15-20 additions in recent years compared to 3-5 additions 30 years ago. Their figure 1 demonstrates this well:
That to me just says the stock added to the S&P 500 has less time to actually accumulate said wealth to stay competitive on the top 500 list while the 1% of companies continue to reap the effect of it. Maybe that has something to do with the similar benefits of an index fund, the less activity, the more money you accumulate due to time in the market.
Once something becomes "common knowledge" in the stock market, it looses all predictive power. In this example, you could trade on the index inclusion because it observably happened, but market participants didn't respond.
Now, everyone expects the effect to happen, so it's just another thing 'baked in' to pricing.
The returns are on a short period. From the paper: Our main interest is the total return, defined as the cumulative market-adjusted return from the last trading day before the announcement to the first trading day after the implementation.
This has nothing to do with the "time in the market" as you mention above.
What happens when everyone (or some sufficiently large portion of the population) index invests? I've heard it said that this becomes a problem, for whatever reason.
In the limit, the market becomes completely detached from reality.
In practice, the market will become increasingly irrational, leading to an increasing arbitrage opportunity for active investors. There may be a period where active investment gives a better ROI, even when accounting for fees; but in the long term, the amount of passive investment should reach an equilibrium where the benefits active investment is equal to its overhead.
> There may be a period where active investment gives a better ROI, even when accounting for fees;
Where would those gains come from? It can't be that the index funds are matching the market and the active investors are doing better. Someone has to be doing worse in that situation.
I think the implication is that the retail index investors will end up doing worse.
One thought experiment is if being part of an index confers additional perceived value, the addition or removal from it index is an Arbitrage opportunity.
Imagine trading against someone who will reliably pay a premium for an index of "top 10 stocks". Price will spike for a stock that goes from #11 to 10, and drop for every stock that goes from 10 to 11.
Index investors will overpay for a newly listed stock and that money will go to an active investor that held it before the listing. Inversely, index investors will lose money whenever a stock is removed from the index.
If you only have one index, going from position 11 to 10 increases the value of the stock. But if you have a lot of indices, all these effects should average out to basically zero, since that stock might be in thousands of indices, and a drop out from a single index will only have a marginal effect.
I think that's correct, but it is the opposite case as the hypothetical that we are discussing. The problem with everyone index investing is that there is not an infinitely diverse pool of indices. If most people index invested into a small number of indexes, they become targets for people to exploit their predictable Behavior.
This is a long way of saying that a limited number of indices is an assumption of the hypothetical.
It would break the market if everyone invested in indexes because nobody is paying attention to fundamentals.
If there are a huge number of indexes and people are diligently picking between them, that is basically the same thing as being active investors.
This is my thought as well, that the alpha would increase until it's enticing enough for active investors, but then I thought, most people are lazy and won't want to actively invest, so will this alpha ever decrease to an equilibrium state?
The more likely outcome is an increase in actively managed funds. Individuals will still contribute their money into the fund just like with index investing. However, instead of simply tracking an index, the fund would have a (team of) professional investors deciding how to invest on an ongoing basis, in exchange for a management fee. These exist today and are no more difficult to invest in than index funds. Of course, these also have higher overhead than index funds and, at the moment. Currently they are not popular because their performance relatively to an index does not justify their increased fees. If alpha gets high enough, this will stop being true and slowly the popular wisdom will update to saying that actively managed funds give a better ROI.
There is also the Wall Street Bets class of active investors. In the current market, I'm not convinced they are actually helping. But, in a world where no one else is activly investing, they are probably better than nothing.
The vast majority of the money in the stock market is not from individual investors, but from hedge funds, large asset management companies, etc. And these hyenas will be happy to jump on any potential way of making more out of their investment. Equilibrium will be reached almost instantly the moment they decide it's worth it.
- Concentration of ‘voting power’:
if more investors flock to index funds, a smaller group of fund managers will have more voting power on behalf of their clients.
Theoretically, a ‘concentration of power’ could give [few] fund managers a lot more responsibility than they should otherwise have; that’s a lot of livelihoods to hold be responsible for.
- Increased crorrelation:
if a [large enough] portion of the population invests in index funds, the returns of those funds will become more correlated with each other; a ripple could become a tsunami.
The concentration of voting power is an issue, but not because of the "increased responsibility." Large funds like Vanguard and BlackRock end up owning huge portions of many different companies. These companies are often direct competitors with one another, e.g. Vanguard owns over $62 million in American Airlines and $136 million in United. This creates incentives for anti-competitive practices.
The more people passively index, the more the market becomes sensitive to small changes by those using an active strategy.
Think of index fund investments as increasing gain on an amplifier — the more gain, the higher small signals can get boosted. An index fund is basically following the active investors and putting more weight behind each of their decisions. More passive investment money means a higher amplification factor.
If you go to a restaurant, point to the person next to you, and say "I'll have what they're having" then you'll get a meal of some sort. You may be happy with it, but who chose it? The person next to you. What if everyone does this? Who decides the initial meal that will propagate to all customers? One person.
When everyone is index investing, then active investing becomes incredibly easy because there is no competition. It's a problem that solves itself. The more passive investors there are, the more incentive there is to actively invest.
According to the article (and the discussed research) this was only a minor factor:
"The researchers also explored the possibility that index changes are more predictable today than in the past, leading sophisticated market participants to trade ahead of the events. They concluded that this played only a minor role in their findings, however."
The two main factors being:
"...an increase in migrations over time from the S&P MidCap 400 index, and an overall increase in the market's ability to provide liquidity to those investors seeking to buy and sell around the time of index changes.
The additional liquidity has 'made it easier for everyone to trade, and as a result, the prices move less.' says Dr. Greenwood.
And with more stocks migrating to the S&P 500 from the S&P MidCap 400, midcap funds are selling as S&P 500 funds are buying, leading to 'a wash' in demand, he says.
This factor is actually lessening over time. The BlackRocks and Vanguards of the world have dozens, even hundreds, of index funds - all with different benchmarks and goals. When a stock is "rebalanced" out of one fund, it ends up being added to some other fund that the same company manages. So the transfer occurs as a notation on the internal books; no actual arbitrage-able trade actually happens. This is even happening in the ESG space - we are starting to see fund "pairs" - one fund tracks the companies that meet the specific rules of that fund, and the other tracks the companies that do not.
Note that this isn't perfect - index fund A has more assets under management than index fund B, etc. But as index investing grows, the actual investor inflows and outflows are what dominate, not rebalancing moves.
> This is even happening in the ESG space - we are starting to see fund "pairs" - one fund tracks the companies that meet the specific rules of that fund, and the other tracks the companies that do not.
This is very interesting. Do you have any examples of these funds?
There is an easy to understand white paper about this concept at CRSP’s website. Their particular ESG indexes aren’t investable, but it seems to be only a matter of time given that all their others are.
The last 10 to 20 years have seen a bullish round of momentum stocks negatively correlated to value loaded stocks.
Value has consistently lost money in the last decade except for short times during momentum crashes in 2018 and 2023.
Either your definition of value is inconsistent with the rest of the world, or your analysis is biased.
Considering your sentence:
> an algorithm to mass-analyze all the stocks on the US market
I guess you reinvented some variation of"low vol" or "small vs big" signal.
If you want to say you've beaten the S&P, you would have to show me a long short, beta neutral portfolio that does so using the same investment universe in a point in time manner.
The issue with backtesting this sort of thing is the data:
- Was survivorship bias accounted for? If you don't have a list of stocks that includes dead issues (bankrupt, merged, or closed) then all your value stocks will be the ones that had a bad period and then miraculously recovered. Your backtest would buy those and make money, while not seeing the ones that had a bad period and then went to zero. This also means you have to track whether the stock was tradable, whether due to the exchange announcing a freeze, or the market just seizing up as participants decided not to risk much. If your strategy does shorting this is going to cause even more pain as you look for historical hard-to-borrow data.
- Was the data backfilled? This is the hardest one to drag out of data sales people, because they often just don't get it. If I have a company that reports its Q2 earnings about six weeks into Q3, is the data labelled as coming out in Q2? Because if it is, I'm backtesting on future information. Also firms that are in trouble may also have timeliness issues in getting their numbers out.
The real reason why the index effect is gone is actually simple. For years now there's been a bunch of traders who build their models around the effect. Their methods are common sense: screen the market for firms that are not yet in the index, and rank them according to how likely they are to get included. Buy a lot of what you're certain will get in, a bit less if you're less certain, and so on. The effect of this is to give you a smoothed out effect so that you can't just hop on board right before inclusion anymore.
It also seems likely that the big index investors actually hire a few of these experts to optimize their purchases. In fact I had a chat with a guy who did this yesterday. It turns out to follow an index, you don't need to buy the whole index. Stocks tend to move together so you can cover most of the principal components with just a tiny amount of stocks. You're not going to do this by buying the little stocks at the bottom of the index, you buy the big ones at the top.
Makes perfect sense. There were massive increases in certain SP500 stocks, but the majority goes nowhere.
On the whole, if you have the financials of all companies, and apply certain variations of the cigar-butt strategy, you get out ahead. This not only makes sense logically but also works in backtesting.
Value investing is a wide term that includes many strategies. Some of them have made quite a lot of money, very consistently.
The burden of proof is on the person claiming no form of value investing will have passed a backtest over the last decade. That’s you, not the other person.
> The burden of proof is on the person claiming no form of value investing will have passed a backtest over the last decade
Says who?
In my book the burden of proof is on the argument being the less in line with general consensus.
Value being a bad investment in the last decade is not an argument, it's a fact. Value is not something you find, it is a defined factor with academic consensus on the definition. You can have some leaway and build a portfolio highly correlated to value, but that only goes so far.
Claiming to have found a good value signal is then akin to claiming being able to doing consistently good factor timing on value.
This is a feat the top hedge funds on the planet would be weary to claim. Allow me to be skeptical when someone on HN claims to have done so with "basic value indicators" and found something that "prints money".
An algorithm could be anything, right? Maybe they analyzed the middle school report cards of executives and majority shareholders, and tied that to value over time in a way that the S&P 500 did. Maybe there's some telling data point that people have never looked at before.
I agree that predicting future price based on the current price has likely been tried to death by billions of hedge funds, but who knows, maybe they found something new that nobody has ever noticed. And, you can come up with the best stock trading algorithm in history and still not have the capital necessary to execute it, so it's possible that portfolio results don't matter.
I'm not saying anything is likely here, merely that we have nowhere near enough data to even speculate or offer advice.
> Maybe they analyzed the middle school report cards of executives and majority shareholders, and tied that to value over time in a way that the S&P 500 did.
You are talking about ad-hoc signals here. Of course there are a lot of them that beat the benchmark, it's the basis of active management.
But we're not talking about any signal here, we're talking about "value" - a signal so well known, studied and persistent we call it a "factor", and often decompose beta/sector/country adjusted residual returns _just_ based on a set of 4/5 of these factors.
I think he says one thing to the crowd and does another thing in private.
I remember him taking about discounted cash flow calculations and what maybe, supposedly, he and Munger use as the discount rate, but as soon as you apply this calculation to all 3000+ publicly traded US securities, you find that most of them are completely overvalued and have been for years. You can adjust the discount rate, but it doesn’t help in practice.
Anyone with a computer and the patience to parse financial reports can do this, so of course people have done it.
The problem is there aren’t many good investments actually there anymore. Period.
If you’ve been investing the last few years and have made returns as all of us collectively have, you know those returns definitely aren’t coming from real gains.
They’ve almost entirely been speculative returns.
I was so jaded calculating this stuff and seeing marketable securities histories across various sectors that I came to realize all the free lunch has already been eaten.
People are searching among the crumbs now.
It’s just entirely an entrenched practice. There’s too many people in that game.
I’m also a fan of the textbook guidelines from Security Analysis and The Intelligent Investor, but of course using those specific measurements also yield nothing worse investing in at the present date, and haven’t for years.[1]
Everything is way more expensive than it should be valued at, and I suspect a part of it, too, is simply that a majority of shares are on the market and have no reason to be sold lower than what inflated values are presently passing for.
It’s like owning a house knowing people need a place to live.
People need securities for retirement. There’s only also many that exist.
> I think he says one thing to the crowd and does another thing in private.
He doesn't do anything in private as Berkshire has so much money all their moves are public and well known. The advice is true but fits another era, read his autobiography.
> People are searching among the crumbs now.
Yeah, the cigarette butts. He had ~$10 million by age 30. Incidentally he amassed this as the the son of a congressman, had a rich auntie, and was in very good social standing in a small wealthy town (Omaha). And it was 1960.
By 1960, Buffett operated seven investment partnerships. He asked one of his partners, a doctor, to find ten other doctors willing to invest $10,000 each in his partnership. Eventually, eleven agreed, and Buffett pooled their money with a mere $100 original investment of his own.
I think it was his biographer, Alice Schroeder, who said that Buffett has never revealed his start to finish analysis for an investment. He's only talked about guidelines and principles, but it's like one thing to know F = m*a vs. how to design a bridge. Also, a lot of his money came from what was basically insider trading as a 20-something (chatting up CEO's and trading from that information).
Yep. He’s revealed in vague passing some guidelines for, say for instance, a discount rate to use, but it’s vague enough only to use as a rough estimate such that you arrive at a possible ballpark.
It also doesn’t help that DCF calculations are incredibly sensitive with respect to interest rates that had for some time been historically low.
It might be easier to calculate these days, but my findings from years ago informed me that these issues were beyond reading financial reports.
There’s a bigger problem of there simply just not being enough attractive companies at even fair rates.
Statistically, you can only buy earnings at so high of a price to meaningfully make a worthwhile return, and for a while now, those prices have been too high in comparison to the risk free rate.
I’m sure this has since changed due to current interest rates, but I haven’t bother to look simply because of the first problem.
There's too much money in the game. There's more capital invested than there are things to invest in at a reasonable rate of real returns. It needs to be clawed back but it's hard an unpopular to do. What we effectively need is a temporary effective progressive wealth tax that alters the exponent of wealth distribution, and that tax just needs to go "nowhere" instead of being reinvested in government programs. Carefully though because such an action has lots of side effects. The problem is whenever you propose such a thing there will be a long list of people with a long and varied list of reasons why this or something similar is insane. It is A) complicated and B) poorly understood so the discourse is usually just useless, which is reason #1 why it is hard.
>They’ve almost entirely been speculative returns.
While that may be true to some extent (especially with the more popular stocks like, let's say... Tesla, the P/E ratio is through the roof, although it got a bit better in the last year or two), but I feel that stocks that pay dividends do have actual gains - in the form of dividends.
I personally prefer dividend stocks, because with the growth stocks you have to sell to get any profits or anything back, and I don't like to sell. Also, when the stuck is moving down, they usually still do pay dividends, which allows you to buy more of the stock at an even lower price.
In my opinion, it's much harder to inflate the price of the stock through the dividends, because those actually affect the company. They need to have that money to be able to pay it out to the shareholders.
The price of something is what people are willing to pay for it. Those formulas that you use to conclude their value is speculative are prescriptive and mostly antiquated. It is almost impossible to separate "real" from speculative value.
> It really isn’t, and the math is quite literally comparable to calculating the coupon of a bond.
There is no math that claims the price of a stock should be some form of discounted future cashflows since at least the 90s.
Since then, all models have been multifactor ones, with fundamentals (quality, value) being only one fragment of the
market, sector and country residualized stock returns.
What `epups` is explaining to you is the basis of behavioral economics, which is the de facto current leading theory since its foundations were Nobel prized 20 years ago...
Ok, then you use your math to buy stocks and I'll use my definition, which is the one accepted by everyone else. I'll gladly be here with the idiots making bank, buying my overpriced index funds while you wait for them to return to their platonic price.
Reductio ad absurdum: would you spend one million US dollars to make one dollar a year? No, that's clearly foolish.
Would you spend one US dollar to make a million a year? Of course, but what such investments would ever exist?
And so instruments which bare gains over time have some value, which is calculated by the means of cost of investment to returns over time.
If there are a market of such instruments, they can and will be in competition with each other.
This "things are only what people value them at" stuff is for the birds. Those other people you're talking about are actually calculating their investments. Others with such mentality will follow, not knowing what they're doing, and taking missteps when the goods are already gone.
What you think is "accepted by everyone else" is what you're seeing at face value. There are reasons behind actions. Those actions in and of themself have no meaning without context.
The mathematics of investment have never, ever changed in any meaningful way. The history of debts, interest rates, and returns have always fluctuated, but the math hasn't.
The problem is that the future is uncertain, and price is guided by feelings and impressions. People think Tesla is a great company for a variety of reasons that have nothing to do with math. They trust Elon Musk, they think Tesla has a technological advantage, etc. Of course a lot of people also look at balance sheets and hard facts before making a purchase. The price of Tesla's stock is composed of the coordinated actions of all these people. It turns out that adhering to your narrow idea of value does not help you accurately predict the price of a stock, or even the market as a whole for that matter.
The easiest way to make money is to convince the government to print more of it, and then take a percentage cut of the distribution and growth of the money supply. The financial industry has been doing this for decades.
To everyone asking for more detail - don't bother. I don't mean to sound mean, OP, but if you have alpha ("several basic Buffett-style value strategies seem to beat the S&P 500 handily"), go start a prop shop and make millions of dollars. You can turn huge profits with even miniscule alpha.
The problem is, armchair traders typically commit a few classical blunders:
1. Accidentally used future information to make trading decisions in backtesting. Obviously a crystal ball would make for GREAT alpha. And while you might think it's easy to avoid doing this, there can be some very subtle oopsies that are not apparent at all to even a somewhat sophisticated analyst that will catch you out. This was a common oopsie I would see our analysts do, sometimes only even becoming evident after the research team had vetted and idea and they hand it off to me to implement and I'd be sitting there in the data flow and have to tell them "yeah this data item you used, it's latency sensitive, did you make sure you were using the appropriate network path when you keyed into the dict or did you just hardcode the common one - because if you hardcoded then you're using a crystal ball for all the other network paths".
2. Assuming that if you see a price, you can trade at that price. Even HFTs can't just go get any ole' price they see.
2.5. A tangent to 2 is that your own trading activity can impact prices. Not important if your alpha is long term but if it's short term then this can be significant.
3. Ignored trading fees, and I don't mean brokerage fees - I mean SEC fees, etc that are usually very small for most retail trades so most brokerages don't even show you them (especially because depending on exactly how they executed your trade they might be different than you'd get if you just sent them to an exchange).
4. Not taking into account tail risks. Look at how many retail investors got blown the fuck up when XIV died [0]
Trading is hard. There's literally armies of ridiculously sophisticated actors as your market participant peers. I always used to joke that the federal budget is to defense spending as our nation's brainpower is to finance. (We probably throw WAY too much %-wise at it, but damn do we get a world class product as a result.)
This Quandl company has a terrible website. They don't say what they offer and how much it costs, all is behind either creating an account or talking to their sales department. I don't understand how they suppossedly have 400K users.
They can't count on it, but they may still exploit it or attempt to. It's just less likely to be a significantly better strategy than others which tend to selling rapidly after acquisition, which is why this was profitable.
The article isn't referencing index investing as a strategy; it's talking about a strategy in which, for a while, you could make money by buying specific stocks when they met the criteria to be newly added to an index. That appears to no longer work, because markets have become more efficient and the value is already priced in.
People thought they could beat the market by timing buys of stocks that were likely to become part of indexes.
If there was any alpha in this practice, it’s been sucked out of the market at this point. The anticipation of this has already been priced in, so there’s no remaining advantage in trying this strategy.
In theory an economy could expand more in private and small businesses and not help index funds. I could see this happening if there were a lot of corporate taxes based on business size.
I think but am not sure that the economy can expand by shifting more “income” from capital to wages which would be bad for stocks. Think 1950s.
If the expanding economy is caused by a lot of disruption from new businesses it could also be bad to hold indices weighted towards incumbents, this seems the most likely to happen of the three cases. The disrupters don’t necessarily need to be big, just enough of a threat that investors think future growth or earnings of incumbents are threatened.
The book Adaptive Markets has a different theory, of markets driven by behavioral economic principles based on human psychology rather than assuming everyone is a Homo Economicus, a fully rational and efficient actor in the market.
I would guess that you could still make some profit, since once the stock is added to S&P500 it will raise at least a bit, since many people do invest in the index, but I would guess that this change is less noticeable now that many more people are looking out for stocks that are going to be added or are close to be added to the index.
I'm also guessing that many have algorithms that check for stocks that fulfill the criteria to be added to the index.
Don't even ask the question. The answer is yes, it's priced in. Think Amazon will beat the next earnings? That's already been priced in. You work at the drive thru for Mickey D's and found out that the burgers are made of human meat? Priced in. You think insiders don't already know that? The market is an all powerful, all encompassing being that knows the very inner workings of your subconscious before you were even born. Your very existence was priced in decades ago when the market was valuing Standard Oil's expected future earnings based on population growth that would lead to your birth, what age you would get a car, how many times you would drive your car every week, how many times you take the bus/train, etc. Anything you can think of has already been priced in, even the things you aren't thinking of. You have no original thoughts. Your consciousness is just an illusion, a product of the omniscent market. Free will is a myth. The market sees all, knows all and will be there from the beginning of time until the end of the universe (the market has already priced in the heat death of the universe). So please, before you make a post on wsb asking whether AAPL has priced in earpods 11 sales or whatever, know that it has already been priced in and don't ask such a dumb fucking question again.
Would the gain be actually zero? What I was pointing to was that you might still get a bit, not a lot, but it would still be a positive change, even though it might be a rounding error for many.
If the expected gain was nonzero, bigger players than you would already be exploiting this opportunity up to the point where the expected gain drops to zero.
- an increase in migrations over time from the S&P MidCap Index
- an overall increase in the market’s ability to provide liquidity to index changes
What about the fact that there are more activities of additions/deletions to the S&P 500 given how much more competition there is today versus decades ago?
You will see anywhere from 15-20 additions in recent years compared to 3-5 additions 30 years ago. Their figure 1 demonstrates this well:
https://www.nber.org/system/files/working_papers/w30748/w307...
That to me just says the stock added to the S&P 500 has less time to actually accumulate said wealth to stay competitive on the top 500 list while the 1% of companies continue to reap the effect of it. Maybe that has something to do with the similar benefits of an index fund, the less activity, the more money you accumulate due to time in the market.