Windows has had IPv6 support included by default since 2006 (i.e. Vista onwards), so the missing piece of the puzzle was my wireless router (a Linksys WRT320N). Unfortunately, the built-in firmware doesn’t support IPv6. (Source: Linksys devices that support IPv6.)
So, I investigated open source alternatives. There are a few different firmware projects out there, which all seem to be based on Linux. According to the OpenWRT wiki, it isn’t supported on the WRT320N. However, the WRT320 is listed in the dd-wrt router database, so I chose that instead. JP Hellemons wrote about this in 2010 (How I upgraded my Linksys WRT320N to DD-WRT v24); he also checked Tomato and HyperWRT, and neither of those were compatible. However, apparently the NoUSB edition of Tomato USB does does support the WRT320N.
Just to forewarn anyone else who’s in a similar position, this isn’t a simple process. Here’s a good (valid) rant about how complex it is. I heard a good phrase a while ago: “Open source software is only free if your time is worthless.” I.e. if you assume that your time is valuable, consider how long it will take you to get a system working. Is it worth paying money to save yourself some time? For instance, in this case I could replace my router with a different model that has IPv6 support built in. (You will still need to invest some time in learning any system, but maybe you could reduce that from a day to an hour.)
In brief, I (eventually) got the router working fine with dd-wrt over IPv4. IPv6 took a bit longer; I’ve elaborated on that in another post (Native IPv6 in dd-wrt).