The gunman, a former doctor at the hospital, killed himself in the attack.
The shooting began at about 14:55 local time (18:55 GMT) at the 1,000-bed Bronx-Lebanon Hospital.
Mayor Bill de Blasio said the shooting was not an act of terrorism, but rather workplace-related.
He said the attack had been a “horrific situation in the middle of a place that people associate with care and comfort”.
Several of the injured are “fighting for their lives,” he added.
Police said the suspect was wearing a white medical coat when found. An assault rifle was also discovered nearby, which a local politician separately said appeared to be a military-grade M16 rifle.
Police Commissioner James O’Neill said the attack began on the 16th floor and all the victims were shot on the 16th and 17th floors.
The gunman has not been officially named but police sources told US media he was Henry Bello, 45, a former family-medicine doctor at the hospital. Sources quoted by NBC said he had resigned in 2015 in lieu of termination.
Mr O’Neill said the gunman had tried to set himself on fire and died of a self-inflicted wound.
Messages on social media spoke of doctors and nurses barricading themselves inside the building in the Mount Hope district.
One patient in the radiology department, Felix Puno, tweeted: “Building is in complete shut down, I was in the middle of getting an X-ray when security alerted us to the active shooter situation.”
Garry Trimble, whose fiancée works at the hospital, said security was not good enough.
He said: “I can walk through the back door with an employee. If the employee opens the door, I can walk in. I think every hospital should have one police officer at each entrance. They only ever do something when something happens.”
Bronx-Lebanon is a private, not-for-profit hospital that has been operating for 120 years.
Shootings at hospitals are not common, but there have been several such instances in recent years.
In 2015, a man entered a Boston hospital and asked for a cardiologist by name, shooting him dead when he arrived. During the investigation, it emerged that the man’s mother had previously been a patient at the hospital.
In July 2016, another man opened fire in a patient’s room at a Florida medical centre, killing an elderly woman and a hospital worker. The suspect was later deemed to suffer from mental health issues, casting doubt over his competency to stand trial.
In July last year, a patient at a Berlin hospital shot a doctor before turning the gun on himself. The city had also seen a shooting outside another hospital earlier in the year, in which no-one was killed.