Jersey City Mayor Steven Fulop says he ordered the city to take down the memorial to 27-year-old Lawrence Campbell on Monday night.
Fulop says he won't let the residents who put up the mural pretend they express the views of the entire city.
The memorial had included candles, balloons, empty liquor bottles and message of love from friends scrawled on T-shirts taped to a brick wall.
Police say Campbell gunned down Officer Melvin Santiago early Sunday as he responded to an armed robbery call at a 24-hour pharmacy.
Rest in Peace