Automated fare query engine
I'm not sure airlines will be happy about this but...
I find some airline websites a pain-in-the-a**e to use when trying to scan different dates and routes for flights in which I might be interested in buying. I know there are websites like skyscanner.net but they don't really do what I'm looking for. Furthermore, I've found that the prices quoted on Skyscanner can often be quite unreliable, even when marked as being from just 18 hours ago.
I'm pondering trying to write a bit of code that allows one to simulate the webform post action to the airline website, which would then return some HTML that I can save to my hard disk and parse automatically later.
I know the recommended approach is to screen-scrape rather than parse HTML / XML, but at the moment, I'm interested in how to simulate repeatedly the form post action (with different dates included for example).
Is anyone able to point me in the right direction as to how to go about coding this up ?