no, it’s not wrong to have a broken and contrite heart. quite the contrary

The sacrifice acceptable to God[d] is a broken spirit;
a broken and contrite heart, O God, you will not despise.

Psalm 51:17

After we’ve sinned, and I refer not just to “great transgression” (Psalm 19:13), but to all sin, when we confess and repent we’re often told to forget about it. That our sins have been removed as far as the east is from the west (Psalm 103:12) which is true upon genuine confession and repentance. But if we’re to take seriously the great penitent Psalm, 51, then we need to accept the fact that what pleases God is not only acceptance of God’s forgiveness to us in Christ, but also God’s acceptance and I would even say pleasure in a broken and contrite heart over our sin.

That doesn’t mean we should wallow in our sin, or hate ourselves. We hate what we’ve done, and not merely the consequences. If we hate only the consequences, then we certainly don’t have a broken and contrite spirit. For most sins the consequences are only a reminder that we haven’t arrived in this life, and that we do well to be more and more humble. For some sins the consequences may be greater along with the realization that there’s work for us to do to be rid of our tendencies without ever thinking we’ll come to the place in this life when we’re actually above the possibility of falling again.

No, with thanksgiving and praise to God we accept God’s forgiveness upon our confession of sin. But we also take our sins seriously, out of love for God and others allowing our hearts to be broken. So that our lives following will become different. Hating what we’ve done, and making amends as best we can with an entirely different life. Letting God, as the psalmist, probably David in Psalm 51 aspires, to do God’s work of thoroughly cleansing us on the inside, so that our hearts might be inclined in love toward righteousness and justice, wanting to avoid all wrong.

In and through Jesus.