GeezerVisonThe World Wide Web, restored to 800 pixels of glory. |
| Home|About|Gaming cabinet|Archives|How it works|Starter sites |
GeezerVison is showing mail.python.org — [Python-Dev] urllib unicode handling [Python-Dev] urllib unicode handlingTom Pinckney thomaspinckney3 at gmail.comWed May 7 04:06:01 CEST 2008
Hi, While trying to use urllib in python 2.5.1 to HTTP GET content from various web sites, I've run into a problem with urllib.quote (and .quote_plus): they don't accept unicode strings. I see that this is an issue that has been discussed before: see this thread: http://mail.python.org/pipermail/python-dev/2006-July/067248.html especially this post: http://mail.python.org/pipermail/python-dev/2006-July/067335.html While I don't really want to re-open a can of worms, it seems that the current implementation of urllib.quote and urllib.quote_plus is painfully incompatible with how the web (circa 2008) actually works. While the standards may say there is no official way to represent unicode strings in URLs, in practice the world uses UTF-8 quite heavily. For example, I found the following URLs in Google pretty quickly by looking for percent encoded utf-8 encoded accented e's. http://www.last.fm/music/Jos%C3%A9+Gonz%C3%A1lez http://en.wikipedia.org/wiki/Joseph_Fouch%C3%A9 http://apps.facebook.com/ilike/artist/Jos%C3%A9+Gonz%C3%A1lez/track/Stay+In+The+Shade?apv=1 While in theory UTF-8 is not a standard, sites like Last.fm, Facebook and Wikipedia seem to have embraced it (as have pretty much all other major web sites). As with HTML, there is what the standard says and what the actual browsers have to accept in order to work in the real world. urllib.urlencode already converts unicode characters to their UTF-8 representation before percent encoding them. Why not urllib.quote and urllib.quote_plus? Thanks for any thoughts on this, Tom
More information about the Python-Dev mailing list |
| You are visitor 01402520 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |