A federal judge in the United States has rejected Irish horseracing figure JP McManus’s bid to recover the $5.22 million that he claimed the U.S. government withheld as tax from his backgammon winnings.

Last Friday, U.S. Court of Federal Claims Judge Nancy Firestone denied the Irish businessman’s request to order the Internal Revenue Service (IRS) to hand over the money that his fellow player, Israeli-born investor Alec Gores, withheld and paid the tax agency to cover potential income tax liabilities, according to the Irish Times.

The story went like this: McManus won the money in 2012, but he claimed that Gores mistakenly sent $5.22 million to the IRS as withholding. So the businessman filed a non-resident U.S. federal income tax return, in which he emphasized that he is entitled to a refund because of a tax treaty between the U.S. and Ireland.

Fast forward to 2014, when Forbes reported that the IRS had already approved the Irish man’s refund claim in August 2013. But the claim was remanded to another department for review, and then silence. So with the IRS failing to act on the claim, McManus resorted to a lawsuit.