Headless browser based on WebKit
This tool will help you make your AJAX applications crawlable.
The entire project is under 50 lines of code so you won't have any trouble understanding it. It's based on Qt4 so it should work on all operating systems. There's also a small example written in PHP, but you're free to use it however you want.
There's one small thing that you should know about this headless browser. It will output everything as UTF-8. Most of the time this works like a charm, but there are also a lot of pages that don't use any encoding, or set one encoding in the headers, but the content has another. Most browsers try to guess what encoding are using the pages you make it open, and that's ok with most pages, but there are some exceptions. And that's why I don't want to play the same game of trying to guess the encoding. Instead I'm leaving that part to you, dear developer :)
This project relies heavily on Socket-IPC. To get this project and the required submodules you'll need git. Make sure to run this:
git clone git://github.com/alexandernst/headless_browser.git headless_browser cd headless_browser git submodule init git submodule update