It's crucial to know the difference. Interest is compensatory, paid to the government for the delay in receiving tax money. A penalty, however, is punitive and is imposed to punish a taxpayer for breaking the law, such as through tax evasion or fraud.