We met Israel Gonzalez in a remote warehouse in Brooklyn under the roaring Gowanus Expressway. Seated on the matted floor not far from a massive tire, the tattooed fitness pro had just led four kids through a rigorous CrossFit workout. This had always been his mission; a lifelong resident of the neighborhood, Mr. Gonzalez opened his gym in the hopes of giving back to his community.

Mr. Gonzalez’ gym, CrossFit 718, first opened in 2010 in a different location—six blocks away at Fifth Avenue and 22nd Street, near Greenwood Cemetery.

Situated next to residential buildings, the gym ; neighbors called 9-1-1 and 3-1-1 about the loud noise and vibrations caused by CrossFit athletes yelling, flipping tires and dropping heavy weights.

Even though Mr. Gonzalez said he never violated any city noise ordinances, police were nevertheless showing up at the gym on a daily basis. It was disruptive and embarrassing.

At CrossFit 718’s original location, Mr. Gonzalez also tried to keep his neighbors happy. He lowered the volume of his music, outlawed tire-flipping, and spent $2,000 on floor padding.

But residents kept complaining. At CrossFit Virtuosity, neighbors called the city’s Environmental Protection Agency to complain about a storm drain that spewed water onto the sidewalk in the back of the gym. Over at CrossFit 718, residents took issue with Mr. Gonzalez’s refuse disposal.

“The neighbors were using the police like the Gestapo,” he argues. “Any time you put your garbage out the wrong way—boom, boom, boom, the cops were at your door.”

After a few years, Mr. Gonzalez and his wife, Tammyli, made the collective decision to move out of respect for their neighbors; after all, many of them were people Mr. Gonzalez had known all his life, and it was “disheartening” that they objected to his gym. Last August, they moved to an industrial space, surrounded by other warehouses.

“Thank God we can make as much noise as we want,” he said.

Over at CrossFit Virtuosity, Ms. Orme eventually learned that developers wanted to convert her gym into a condo building. That, combined with the noise complaints, prompted her to move in 2013 to a commercial space near the BQE. There’s a jiu jitsu school on one side and a usually-empty art storage facility on the other, neither of which have complained in the year-and-a-half since she’s been there.

It’s not just in New York that CrossFit gyms move to new digs following community opposition.

In Austin, Texas, Valerie Hunt is currently searching for a new space for her gym, BVM CrossFit. The gym is located in a commercial zone, but residents nearby are complaining they can hear people grunting and dropping weights. They’ve taken the matter to City Council, as well as launched a vicious attack against the gym.

“I had to get my phone number changed,” Ms. Hunt told the Observer. “The texting turned really ugly—a lot of cussing [and] ‘We hate you’.”

Ms. Hunt believes angry neighbors signed up her CrossFit gym for dozens of magazine subscriptions.

“We had like 80 magazines in our name,” she said.

Though Ms. Hunt said Austin’s City Council confirmed BVM hadn’t violated any noise ordinances, the neighbors’ complaints led the city to , as opposed to a personal improvement service. That means Ms. Hunt must find a new space and convince her landlord to let her end her lease early.

Ms. Hunt isn’t totally opposed to packing up and moving.

“I’m a nice person—I don’t like thinking I’m making someone else’s life not good,” she said. “I also don’t want—I would like for us to be able to operate and not be harassed.”

When faced with complaints, not all CrossFit gyms opt to relocate; some, instead, gird their well-built loins—even if it costs hundreds of thousands of dollars.

Residents of a Chelsea condo building have long complained about BRICK New York, the CrossFit gym located directly underneath them. At the end of 2012, they sued the gym for applying too late for an arcane permit from the Board of Standards and Appeals (called a physical cultural establishment permit, it was originally designed to ensure massage parlors weren’t secretly operating as brothels).

We asked Mr. Perelmutter what other advice he’d give to people opening CrossFit gyms in New York City.

“Open in Omaha!” he said.