Local MP Torsten Bell is encouraging residents to avoid the area as thick black smoke can be seen billowing into the sky.
The MP says it appears to be coming from the City Church area.
He said in a post on Facebook: “I’m concerned there may be a large fire in central Swansea in the city church area – I’d recommend staying away until it’s clear what’s going on.
“It sounds like fire engines are on the way. I’m contacting the emergency services and council and will be following closely as the situation develops.
“Stay safe.”