You didn't write that awful page. You're just trying to get some data out of it. Beautiful Soup is here to help. Since 2004, it's been saving programmers hours or days of work on quick-turnaround screen scraping projects.
"A tremendous boon." -- Python411 PodcastTool Safety, a short zine I wrote about what I learned about software development from working on Beautiful Soup. Thanks!
If you have questions, send them to the discussion group. If you find a bug, file it.
Beautiful Soup is a Python library designed for quick turnaround projects like screen-scraping. Three features make it powerful:
Beautiful Soup parses anything you give it, and does the tree traversal stuff for you. You can tell it "Find all the links", or "Find all the links of class externalLink", or "Find all the links whose urls match "foo.com", or "Find the table heading that's got bold text, then give me that text."
Valuable data that was once locked up in poorly-designed websites is now within your reach. Projects that would have taken hours take only minutes with Beautiful Soup.
Interested? Read more.
Download Beautiful Soup
The current release is Beautiful Soup
4.6.3 (August 12, 2018). You can install Beautiful Soup 4 with
pip install beautifulsoup4.
In Debian and Ubuntu, Beautiful Soup is available as the
python-bs4 package (for Python 2) or the
python3-bs4 package (for Python 3). In Fedora it's
available as the
Beautiful Soup is licensed under the MIT license, so you can also
download the tarball, drop the
bs4/ directory into almost
any Python application (or into your library path) and start using it
immediately. (If you want to do this under Python 3, you will need to
manually convert the code using
Beautiful Soup 4 works on both Python 2 (2.7+) and Python 3.
Beautiful Soup 3 was the official release line of Beautiful Soup from May 2006 to March 2012. It is considered stable, and only critical security bugs will be fixed. Here's the Beautiful Soup 3 documentation.
Beautiful Soup 3 works only under Python 2.x. It is licensed under the same license as Python itself.
The current release of Beautiful Soup 3 is 3.2.1 (February 16,
2012). You can install Beautiful Soup 3 with
BeautifulSoup. It's also available as
python-beautifulsoup in Debian and Ubuntu, and as
python-BeautifulSoup in Fedora.
You can also download the tarball and use
BeautifulSoup.py in your project directly.
Hall of Fame
Over the years, Beautiful Soup has been used in hundreds of different projects. There's no way I can list them all, but I want to highlight a few high-profile projects. Beautiful Soup isn't what makes these projects interesting, but it did make their completion easier:
If you've used Beautiful Soup in a project you'd like me to know about, please do send email to me or the discussion group.
Development happens at Launchpad. You can get the source code or file bugs.
This document (source) is part of Crummy, the webspace of Leonard Richardson (contact information). It was last modified on Sunday, August 12 2018, 16:48:15 Nowhere Standard Time and last built on Tuesday, October 16 2018, 19:00:01 Nowhere Standard Time.