It kinda depends where you want to go with this. Unity is a good choice if you're starting out, the barrier to entry is lower and if it's specifically
game development that interests you then you can get results sooner or give it up if it turns out not to be to your liking. On the other hand, C# is a general purpose programming language that you could use to make any utility type program, not just games. But you may get frustrated with the slower progress to a playable result.
C# and XNA are both free. You can get Visual Studio Express edition for free (just have to sign up with an email address). Microsoft's website is retarded so it's not obvious how to get hold of these tools.
http://www.microsoft.com/visualstudio/e ... 10-express - download C#
http://www.microsoft.com/en-us/download ... x?id=23714 - download for XNA Game Studio 4
If you're starting at ground zero, I recommend you pick up a book and work your way through it. There's really great content on the web, but having a structured course that you can follow will make it faster and less painful to get a grip on how to write code. Unfortunately I can't recommend a book for either C# or Unity as I haven't read any beginner's books in those areas. Maybe someone else can pitch in with suggestions.