One of the things I like most about Mombasa is that public transport is available and allowed into almost all parts of the city. Taxis are available but are an expensive way of moving around. Ubers are barely used by locals too.
Better option: Use tuk-tuks, also known as rickshaws in other countries. They move literally throughout the whole city dropping and picking passengers. To hail one you just have to flag it down.
Safety: Tuk-tuks are generally safe, I've barely heard of any security incidences involving tuk-tuks. However, drivers will probably try to overcharge slightly especially if they realize you're a newbie in town.
TIP: Always ask a local how much Tuktuk transport costs to your destination before hailing for one, that way you have a fairly rough estimate of the amount the driver will charge. It shouldn't be more than Ksh 300 ($3) for any destination within town.