Wealthy Americans have less doubt about market rally, economy, but still fear stock investing

The U.S. stock market recovered from the March bottom so fast that many wealthy investors are not sure what move to make next. Millionaire investors are more upbeat about the market and economy than they were one quarter ago. The Dow Jones Industrial Average and S&P 500 just completed three consecutive weeks of gains. But as coronavirus cases have continued to increase in July and polling has showed increased support for Democratic presidential candidate Joe Biden, many among affluent investors remain unconvinced that they should continue to add risk in stocks. 

“From a sentiment perspective, people are still not sure it will last, still trying to figure it out and the last few months things have really risen considerably, the risk assets,” said Mike Loewengart, chief investment officer at E-Trade Financial‘s capital management unit. “Given what’s transpired they are still optimistic, but cautious.”

Bullishness among investors with $1 million or more in a brokerage account they self-manage was up 13% from last quarter, according to the latest E-Trade Financial investor survey, from 41% to 54%. And these investors are more likely to remain convinced in the strength of the rise in U.S. stocks, with 43% saying the third quarter will end in the green for the U.S. market. The percentage that think stocks will drop fell from 57% last quarter to 42% now. But the percentage of wealthy investors who said they would be moving out of cash and into new positions dropped, from 24% to 7%, according to the E-Trade survey. 

The TIGER 21 investment club for ultra-affluent investors has seen cash levels among its members hit an all-time high in June. These investors have tended to prioritize private equity and real estate over the stock market, with only 21% of assets in U.S. stocks. However, Michael Sonnenfeldt, founder of TIGER 21, said something “remarkable” happened in a June poll of its members when they reported raising cash levels up to 19%.

“I can tell you statistically that this is the largest, fastest change in asset allocation in more than a decade we’ve been doing it,” Sonnenfeldt said.

These ultra-wealthy investors have never had less than 11% in cash going back to the financial crisis, but he added, “Nothing forced them to have more or less cash and in just three months of Covid they have gone from 12% to 19%, like a four-sigma event.”

Loewengart said spikes in new cases across country are weighing on investors, but the market continues to rise,  creating a contradictory set of information for investors to digest. “They see the market continues to run and it is still being conducive for risk assets because of the policy response, but they are seeing a health crisis roar along. It’s no clear picture and we know when they don’t have a clear picture, they default to caution,” Loewengart said.

The gains investors made as the stock market rebounded from its nadir in late March — when all of the gains from the Trump presidency were wiped out — to being back near a record high in the Dow, has resulted in many hitting the pause button.

“Between February and April 15, we were net buyers of equities in a huge way,” said Bruce Weininger, principal and senior financial advisor at Chicago-based wealth management firm Kovitz.

For investors whose portfolios had been rebalanced to target allocations after the selloff and sharp rebound by late April, Kovitz has “been doing more selling than buying,” Weininger said. That’s not a market call, but just long-term investing discipline, which compels the firm to buy low and sell high. 

“The clients I am talking to the most are worried and they are very confused, seeing that the market has gone way up, and there doesn’t seem to be justification from the economy, where things aren’t great,” said Michael Prendergast, director at New York City-based Altfest Personal Wealth Management. 

More from Invest in You
What Gen Z investors learned by opening robo-investing accounts
Suze Orman: How to keep your retirement on track during Covid-19
No money? No market knowledge? How to ditch all the excuses and start investing

This week, J.P. Morgan chairman and CEO Jamie Dimon provided a warning about the future path for the U.S. economy.

The message Altfest has been conveying to clients is to expect volatility will remain in the short-term. “They are happy the market has gone up, but not sure it will last,” Prendergast said. “We have clients talking about reducing allocations because they don’t feel confident …  some just keeping cash on the sidelines because they are really unsure,” he said.

The majority of millionaire respondents to the E-Trade survey (56%) said they were making no changes to their portfolio allocations this quarter, up from 42% in Q2. And amid a stock market rebound that has been notable for the surge in trading from younger investors and new day traders on platforms like Robinhood, there has been a notable decrease in millionaires who say their risk tolerance has increased as a result of the pandemic, with only 16% of those managing more than $1 million in investments indicating they are taking on more risk. That is versus 28% of the general investing ($10,000 or more in a brokerage account) universe that cited greater risk appetite because of Covid-19.

The E-Trade Financial quarterly survey was conducted July 1 to July 9 and included a sample of about 900 self-directed active investors with at least $10,000 in assets — the millionaire data subset provided exclusively to CNBC includes 167 investors with $1 million or more of investable assets.

Weininger said while a long-term view of markets and investing decision-making is always No. 1, sentiment among clients is a factor right now, and it ranges from “cautious to fearful.”

“I don’t have more than a handful who I would say are bullish about things,” the CFP at the $5 billion wealth management firm Kovitz said. Concerns that the market is pricing in a V-shaped recovery, which is looking less likely based on the recent Covid-19 new case trends, have led to a lack of investor conviction about the U.S. stock market’s valuation.

Forty-six percent of investors in the E-Trade survey said they were worried about the presidential election’s impact on their portfolio, more than those worried about a recession (35%) or the coronavirus (35%). Both of those fear factors fell by double-digit percentages quarter over quarter. 

Loewengart said the election fears are a reflection of how important federal policy, from the Trump tax cuts to the Fed and the recent Covid-19 stimulus, has been to the stock market. However, he and other experts said it is the one big fear that investors should throw out.

“When we consider who is better for the economy, the GOP or Democratic Party, there is no clear-cut answer. Market performance relative to elections, there is no clear-cut decision,” the E-Trade official said.

Some investors are looking at the likelihood of the Democratic Party sweeping all three branches of government and taking a more cautious stance as a result. Weininger said that is a real phenomenon, but it is the easiest to dismiss as an investor. 

“Don’t let politics get in the way of money,” he said. “There is no evidence Republicans are better for investments. … If you let politics drive investment decisions you will have bad results. We’re happy to take on that argument, but on valuation, not a lot looks cheap to us.”

At TIGER 21, election uncertainty comes up in “virtually every meeting,” Sonnenfeldt said. There are real issues with the potential to influence short-term market performance, from changes in personal, corporate and estate taxes to climate policy with radical differences and income disparity, but betting on a presidential outcome makes less sense than hedging, he said. Betting on an election outcome implies taking much more risk, and the wealthy investors he knows look at that as a “foolish” bet.

Weininger said even though he can come up with a long list of reasons why the market is overvalued and pricing in a “too rosy scenario,” when you look at the Federal Reserve taking interest rates back to near zero, that alone does justify higher market value. “If you are using rates that are half of what they were even two years ago, and a third of what they were 10 years ago, to discount future cash flows, it is perfectly reasonable to assign higher values to these businesses if you believe rates will stay lower for longer,” he said.

Growth stocks trading at the prices evidenced by the top S&P 500 technology holdings assume rates remain very low, and if the rate environment changes, “there  is nothing that says Microsoft can keep growing at the same rate if interest rates are up two to three percent,” said Weininger. “Growth stocks are more affected by good and bad changes in interest rates.” 

This is a bigger element for investors to watch than who is president, said Loewengart. “If lenders throughout the world, the ones who hold our debt, think our revenue is not significant enough to service our debt load, that would put additional pressure on policy makers to raise taxes, and if we see that pressure on rates, that to me would be the bigger force to changing fiscal policy than who is in office,” the E-Trade official said. “We’ve seen the deficit expand considerably … it could create a situation where investors and lenders to the U.S. have pause.”

A change in the rate environment is among the mid-term fears that market experts are now weighing more heavily than an election or hard-to–evaluate Covid-19 vaccine race. They know that the Fed control over short-term rates is not absolute control over the rate environment. “Intermediate and long-term rates are driven by investors being willing to buy those bonds, locking in 2 percent for 20 years as the best thing they can do. If and when investors get scared about inflation … it can happen quickly,” Weininger said.

That would mean trouble for stocks. “That active printing of money to service debt, if needed, is inflationary at some point. … The risk of high rates could meaningfully affect valuations of equities and the multiples people are willing to pay,” he said.

The E-Trade survey found majority support for only two sectors of the market: health care and technology.  Seventy-one percent of millionaire investors said they expect health-care stocks to perform the best this quarter, while conviction in tech stocks surged back, from 39% of investors who said in Q2 tech stocks would do best, to 59% this quarter. These were the only two sectors where the survey found majority belief that a strong rise will continue.

Part of this is likely performance-chasing and there is reason to be cautious on tech stocks given the run.

Financial advisors say Apple makes the best case for a value stock among the trillion-dollar tech leaders, and the massive run in top tech stocks can be seen in Warren Buffett’s stock-picking universe, with Apple now representing 40% of Berkshire Hathaway‘s equities portfolio.

Loewengart said health-care has always been a defensive sector play for knowledgeable investors and now it looks like technology is assuming a similar role, with a perceived defensive nature. That is a change from recent years when tech was more of a momentum play, but the revenue the top tech companies continue to generate and the performance during the pandemic are leading to a change in investor perception.

Advisors do worry about too much faith in tech leading to overly rich valuations.

“Don’t get me started on how much indexes are distorted by the top five companies. It’s crazy,” Weininger said, referring to the just-under one-quarter of the S&P 500 represented by a handful of top technology stocks — Apple, Microsoft, Amazon, Facebook and Alphabet. 

Year-to-date, ETFs that track the market capitalization weighted S&P 500 Index, like SPY,  are back up by roughly 1%, while the S&P 500 index funds that use an equal weight approach for all holdings, tracked by ETFs like RSP, are down more than 7% in 2020.

“The difference in return is crazy, so without those top 5 stocks the return potential is very low,” Weininger said.

As a long-term value investor, he is more interested in contrarian plays in the current market environment, in sectors including travel and in names like Expedia, Booking Holdings and Disney, strong brands that have been decimated by being in the wrong businesses during the pandemic. 

Investors should be mindful of performance chasing in hot sectors. “People tend to like what’s doing well, so health care and tech is easy and feels good,” Weininger said. “The true value is in areas everyone else loves to hate.”

At Altfest, for investors who are putting more money to work in stocks, the firm is making investments in two, three or even four tranches over time to lessen the risk of a sharp reversal in stocks immediately hitting portfolio values. “You don’t want to do it in one fell swoop,” Prendergast said. “We are not advocating to just invest in the market all at once because anything can happen in the short- term.”  

He said that approach is dictated by the way the market is reacting to snippets of news, both on the positive and negative side, such as a vaccine trial at an early stage sending the market up, or more Covid-19 cases in the South and West and other hotspots pressuring stocks.

“But this is short-term,” he said. “Portfolio success is not short-term, it’s not six months from now or a year from now. Things will be choppy over the next year or more, with the election, strained relations with China, and lots of uncertainty. But it won’t last forever.”

As overseas markets in Europe and Asia have made progress against the coronavirus while the U.S. situation worsens, investors are more interested in overseas stocks. The E-Trade survey finds a 10% increase among investors saying the health of stocks overseas appeals to them this quarter (up from 19% in Q2 to 29% of millionaire investors in Q3). There was also a marked decrease in wealthy investors disagreeing with the view that international markets are attractive, down from 58% to 39% of investors, quarter over quarter.  

“We are seeing that valuation of international equities is more attractive and the opportunity for growth in developed overseas and emerging markets is better than the U.S., which is expensive,” Prendergast said.

Concerns about the global economic slowdown and the continued global spread of coronavirus mean there will be short-term volatility all over the world, according to the Altfest financial planner, but over the medium to long-term there is opportunity for catch-up in international equities because the spread in valuations between the U.S. and emerging markets, in particular, is wide.

“Obviously there are country to country dynamics in the coronavirus pandemic, but it does speak to the broader resilience on a global basis,” said E-Trade’s Loewengart. “And when you couple that with the dynamics in the market, you see that investors recognize U.S. markets moved fast and perhaps even got ahead of themselves. … International has lagged the U.S. for some time now.”

Millionaires do not expect the return from recession to economic expansion to happen quickly. The E-Trade survey finds 55% of investors saying it will take at least one year for the U.S. economy to recover from the pandemic.

TIGER 21’s Sonnenfeldt said the wealthy continue to ascribe to the Buffett view that is undiminished as far as a long-term bet on America, but right now the bar is much higher for taking action. “These people don’t want to miss the opportunity of a lifetime, but they don’t want to reach for a deal with marginal benefits either. … The gap between Main Street and Wall Street has never been bigger, and we now understand that the stock market reflects not all economic activity, but corporate activity,” Sonnenfeldt added. “More people are out of work and yet technology stocks up.”

This dynamic has shined a light on the new economy and the technology sector may be able to keep growing due to its scaling phenomenon, he said. Stocks like Zoom Video Communications have shown how the crisis can accelerate the pace of development in tech. But the political turmoil and the lack of resolution over Covid-19 will remain among the factors that lead the wealthy to conclude there are more bumps in the road ahead, or worse. “It’s business as usual as long as the business is more bullet-proof than ever. They want more security to protect on the downside,” he said.

The wealthy are concerned about unwinding fiscal stimulus, rebuilding industries where there will be a long list of winners and losers, and from a societal point of view, a high level of concern about income inequality and polarization. ”There is no V-shaped recovery in our members’ minds,” Sonnenfeldt said.

For investors who went to cash and then missed the market rebound entirely, it’s too late now to add more risk in stocks, and that’s not because of concerns about market valuations. It’s about the investors themselves. “When you get back in, you should have a higher percentage in boring fixed income, because you mistated your true risk tolerance if you couldn’t sustain the negative reality,” Weininger said. “It means you had too much in the stock market to begin with. Don’t go back to what you had before late March.”

SIGN UP: Money 101 is an 8-week learning course to financial freedom, delivered weekly to your inbox.

CHECK OUT: How I became a millionaire in my 30s and retired early by learning to invest during the 2008 recession via Grow with Acorns+CNBC.

Disclosure: NBCUniversal and Comcast Ventures are investors in Acorns.