Should you prioritize completing a project or writing clean code?

When I write software, I tend to care more about code quality than some of my colleagues. In addition to submitting new code, I refactor the existing code (e.g. eliminate duplicate code, simplify the implementation, e.t.c).

Refactoring existing code takes time, as a result my work pace is quite slow. Sometimes I miss the project completion deadline, or request more time to complete the project than usual, due to which my boss is unsatisfied.

Even when I submit refactored code to the system, two weeks later other colleagues modify it by quickly adding new features, and the code becomes messy again.

So I'm wondering, up to what extent code refactoring is worth doing in a situation like this? What are your experiences?