Webbots, Spiders, and Screen Scrapers: A Guide to Developing Internet Agents with PHP/CURL

· No Starch Press
4.1
7 reviews
Ebook
306
Pages

About this ebook

The Internet is bigger and better than what a mere browser allows. Webbots, Spiders, and Screen Scrapers is for programmers and businesspeople who want to take full advantage of the vast resources available on the Web. There's no reason to let browsers limit your online experience--especially when you can easily automate online tasks to suit your individual needs.

Learn how to write webbots and spiders that do all this and more:
-Programmatically download entire websites
-Effectively parse data from web pages
-Manage cookies
-Decode encrypted files
-Automate form submissions
-Send and receive email
-Send SMS alerts to your cell phone
-Unlock password-protected websites
-Automatically bid in online auctions
-Exchange data with FTP and NNTP servers

Sample projects using standard code libraries reinforce these new skills. You'll learn how to create your own webbots and spiders that track online prices, aggregate different data sources into a single web page, and archive the online data you just can't live without.

You'll learn inside information from an experienced webbot developer on how and when to write stealthy webbots that mimic human behavior, tips for developing fault-tolerant designs, and various methods for launching and scheduling webbots. You'll also get advice on how to write webbots and spiders that respect website owner property rights, plus techniques for shielding websites from unwanted robots.

Some tasks are just too tedious--or too important!--to leave to humans. Once you've automated your online life, you'll never let a browser limit the way you use the Internet again.

Ratings and reviews

4.1
7 reviews
A Google user
May 2, 2012
Webbots, Spiders, and Screen Scrapers is first book I have read on Web robots. Book is very good for beginners and gives moderate understanding of subject. Book assumes that reader should have basic understanding of PHP which is good keeping the topic it is covering. Though Author covers library written by him but that makes book good for beginners and does not bog down the first timers to web robots. Schrenk also covers legal issues which may arise while designing and writing web robots which is fair warning to any developer. Book also has its web site which gives latest information about subject. This book certainly be in by book shelf for long.

About the author

Michael Schrenk uses webbots and data-driven web applications to create competitive advantages for businesses. He has written for Computerworld and Web Techniques magazines and has taught courses on Web usability and Internet marketing. He has also given presentations on intelligent Web agents and online corporate intelligence at the DEFCON hacker's convention.

Reading information

Smartphones and tablets
Install the Google Play Books app for Android and iPad/iPhone. It syncs automatically with your account and allows you to read online or offline wherever you are.
Laptops and computers
You can listen to audiobooks purchased on Google Play using your computer's web browser.
eReaders and other devices
To read on e-ink devices like Kobo eReaders, you'll need to download a file and transfer it to your device. Follow the detailed Help Center instructions to transfer the files to supported eReaders.