TITLE Beautiful Soup: We called him Tortoise because he taught us. META Description Beautiful Soup: a library designed for screen-scraping HTML and XML.

You didn't write that awful page. You're just trying to get some data out of it. Beautiful Soup is here to help. Since 2004, it's been saving programmers hours or days of work on quick-turnaround screen scraping projects.

Beautiful Soup

"A tremendous boon." -- Python411 Podcast

New! Try out the beta release of Beautiful Soup 4. (Last updated February 8, 2012)
easy_install beautifulsoup4 or pip install beautifulsoup4 or download a tarball.
Here's the BS4 documentation.
If you want to help me finish the beta, here's what I'm looking for.

[ Download | Documentation | Source | Discussion group ]

If Beautiful Soup has saved you a lot of time and money, the best way to pay me back is to check out Constellation Games, my sci-fi novel about alien video games.
You can read the first two chapters for free, and the full novel starts at 5 USD. Thanks!

If you have questions, send them to the discussion group.

Beautiful Soup is a Python library designed for quick turnaround projects like screen-scraping. Three features make it powerful:

  1. Beautiful Soup provides a few simple methods and Pythonic idioms for navigating, searching, and modifying a parse tree: a toolkit for dissecting a document and extracting what you need. It doesn't take much code to write an application
  2. Beautiful Soup automatically converts incoming documents to Unicode and outgoing documents to UTF-8. You don't have to think about encodings, unless the document doesn't specify an encoding and Beautiful Soup can't autodetect one. Then you just have to specify the original encoding.
  3. Beautiful Soup sits on top of popular Python parsers like lxml and html5lib, allowing you to try out different parsing strategies or trade speed for flexibility.

Beautiful Soup parses anything you give it, and does the tree traversal stuff for you. You can tell it "Find all the links", or "Find all the links of class externalLink", or "Find all the links whose urls match "foo.com", or "Find the table heading that's got bold text, then give me that text."

Valuable data that was once locked up in poorly-designed websites is now within your reach. Projects that would have taken hours take only minutes with Beautiful Soup.

Interested? Read more.

Download Beautiful Soup

Version 3.2.0 is the current release (November 21, 2010), but it's about to be replaced by Beautiful Soup 4 (February 8, 2012).

If you're using Python 3.x, you must use Beautiful Soup 4. If you like trying new things, I recommend you give the beta a spin. Otherwise, I recommend you use Beautiful Soup 3.2 until the beta period ends.

Beautiful Soup is licensed under the same terms as Python itself, so you can drop the bs4/ directory into almost any Python application (or into your library path) and start using it immediately. You can also install it with easy_install BeautifulSoup (for Beautiful Soup 3) or easy_install beautifulsoup4 (for the Beautiful Soup 4 beta).

Development

Development happens at Launchpad. You can get the source code or file bugs.